07 Jun 06-07
Best laid plans...
Well, that didn’t go as planned.
Moving a large web application from one datacenter to another is never an easy task, and sometimes you learn that the hard way.
We messed up, and we apologize for the downtime. We know that our shop owners and their customers rely on us to provide them with a reliable, stable e-commerce platform, and today we have failed to provide that to them.
I’d like to personally apologize to everyone who has been affected today. I am JadedPixel’s system administrator and I’m sad to say that this happened on my watch.
We are catching heat for this in our support email right now, and rightly so. Believe me when I tell you that we are taking this incident as a giant “lesson learned” and we will be improving just about every aspect of how we do this kind of maintenance in the future.
Thank you, Paul.
Comments
June 07, 2007 01:55PM EDT
Are you no longer hosted at Joyent? If not, any special reason why you left?
June 07, 2007 01:55PM EDT
Despite our announcement we never actually switched to joyent for technical reasons.
June 07, 2007 01:55PM EDT
OK so I totally understand a move like this having issues. I’ve done them before and know how it goes. How about some info on what happened and why. Also I am curious about the non-event the Joyent move what and where you have just moved (if it actually happened)
June 07, 2007 01:55PM EDT
Will we get to know where are you moving to now? Rackspace ?
June 07, 2007 01:55PM EDT
Hi Guys,
I only noticed this due to a customer not being able to complete his order.
Just letting you know that the https://checkout.shopify.com/carts/17292/497306d916db57da5682094b7245e0b6 is down both in Safari / Firefox.
Firefox error :
The connection was interrupted
The connection to checkout.shopify.com was interrupted while the page was loading.
The site could be temporarily unavailable or too busy. Try again in a fewmoments. If you are unable to load any pages, check your computer’s network
connection. If your computer or network is protected by a firewall or proxy, make sure
that Firefox is permitted to access the Web.
Cheers,
Marcus
June 07, 2007 01:55PM EDT
The problem which prevented this move was that of character encoding. Texts which would work fine on shopify right now would look broken on the new server farm if the text contained any special characters such as umlauts or chinese alphabet. The tracked down a way to convert the data correctly and will re-attempt the move on sunday.
The reason for the non move to joyent was simply that joyents service is based around virtualization and “slices” of computers but shopify is too complex an application for such an installation and needs dedicated hardware which worries about nothing else. This became apparent days before the move and thats why we canceled it. Instead we invested and build a completely dedicated datacenter in Toronto ( Priority Colo ) which is tailor made to shopify’s requirements ( performance, security ).