What Does Your Webserver Do When a User Hits Refresh?

Your web application is likely rendering requests when the requesting client has already disconnected. Eric Wong helped us devise a patch for the Unicorn webserver that will test the client connection before calling the application, effectively dropping disconnected requests before wasting app server rendering time.

The Flash Sale

A common traffic pattern we see at Shopify is the flash sale, where a product will be discounted heavily or only available for a very short period of time. Our customer's flash sales can cause traffic spikes an order of magnitude above our typical traffic rate.

This blog post highlights one of the problems dealing with these traffic surges that we solved during our preparation for the holiday shopping season.

In a flash sale scenario, with our app servers under high load, response time grows.  As our response time increases, customers attempting to buy items will hit refresh in frustration.  This was causing a snowball effect that would contribute to reduced availability.

Connection Queues 

Each of our application servers run Nginx in front of many Unicorn workers running our Rails application.  When Nginx receives a request, it opens a queued connection on the shared socket that is used to communicate with Unicorn.  The Unicorn workers work off requests in the order they're placed on the socket’s connection backlog.  

The worker process looks something like:

The second step takes the bulk majority of time of processing a request.  Under load, the queue of pending requests sitting on the UNIX socket from Nginx grows until it reaches maximum capacity (SOMAXCONN).  When the queue reaches capacity, Nginx will immediately return a 502 to incoming requests as it has nowhere to queue the connection.

Pending Requests

While the app worker is busy rendering a request, the pending requests in the socket backlog represent users waiting for a result.  If a users hits refresh, their browser closes the current connection and their new connection enters the end of the queue (or nginx returns a 502 if the queue is full).  So what happens when the application server gets to the user's original request in the queue?

Nginx and HTTP 499

The HTTP 499 response code is not part of the HTTP standard.  Nginx logs this response code when a user disconnects before the application returned a result.  Check your logs - an abundance of 499s is a good indication that your application is too slow or over capacity, as people are disconnecting instead of waiting for a response.  Your Nginx logs will always have some 499s due to clients disconnecting before even a quick request finishes.

HTTP 200 vs HTTP 499 Responses During a Flash Sale

When Nginx logs an HTTP 499 it also closes the downstream connection to the application, but it is up to the application to detect the closed connection before wasting time rendering a page for a client who already disconnected.

Detecting Closed Sockets

With the asynchronous nature of sockets, detecting a closed connection isn't straightforward.  Your options are:

  • Call select() on the socket.  If a connection is closed, it will return as "data available" but a subsequent read() call will fail.
  • Attempt to write to the socket.

Unfortunately it is typical for web applications to find out the client socket is closed only after spending the time and resources rendering the page, when it attempts to write the response.  This is what our Rails application was doing.  The net effect was that for every time a user pressed refresh, we would render that page, even if the user had already disconnected.  This would cause a snowball effect until eventually our app workers were doing little but rendering pages and throwing them away and our service was effectively down.

What we wanted to do was test the connection before calling the application, so we could filter out closed sockets and avoid wasting time.  The first detection option above is not great: select() requires a timeout, and generally select() with even the shortest timeout will take a fraction of a millisecond to complete.  So we went with the second solution:  Write something to the socket to test it, before calling the application.  This is typically the best way to deal with resources anyways: just attempt to use them and there will be an error if there’s something in the way.  Unicorn was already acting that way, just not until after wasting time rendering the page.

Just write an 'H'

Thankfully all HTTP responses start with "HTTP/1.1", so (rather cheekily) our patch to Unicorn writes this string to test the connection before calling the application.  If writing to the socket fails, Unicorn moves on to process the next request and only a trivial amount of time is spent dealing with the closed connection.

Eric Wong merged this change into Unicorn master and soon after released Unicorn V4.5.0.  To use this feature you must add 'check_client_connection true' to your Unicorn configuration.


Continue reading article ›

Shopify for Designers Workshops 2013


In 2012 we went on the road and delivered five "Shopify for Designers" in the UK and USA. Not only were they a lot of fun but the feedback was very positive. If you missed out last year we have good news.

Over the last couple of months we have been hard at work putting together a full global programme of workshops and meet ups for 2013. We'll be visiting cities in the USA, Canada and the UK. Exciting times.

Continue reading article ›

Introducing the Super Debugger: A Wireless, Real-Time Debugger for iOS Apps


LLDB is the current state of the art for iOS debugging, but it’s clunky and cumbersome and doesn’t work well with objects. It really doesn't feel very different from gdb. It's a solid tool but it requires breakpoints, and although you can integrate with Objective C apps, it's not really built for it. Dealing with objects is cumbersome, and it's hard to see your changes.

This is where Super Debugger comes in. It's a new tool for rapidly exploring your objects in an iOS app whether they're running on an iPhone, iPad, or the iOS Simulator, and it's available today on Github. Check over the included readme to see what it can do in detail.

Today we're going to run through a demonstration of an included app called Debug Me.

  1. Clone the superdb repository locally to your Mac and change into the directory.

    git clone https://github.com/Shopify/superdb.git
        cd superdb
  2. Open the included workspace file, SuperDebug.xcworkspace, select the Debug Me target and Build and Run it for your iOS device or the Simulator. Make sure the device is on the same wifi network as your Mac.

  3. Go back to Xcode and change to the Super Debug target. This is the Mac app that you'll use to talk to your iOS app. Build and Run this app.

  4. In Super Debug, you'll see a window with a list of running, debuggable apps. Find Debug Me in the list (hint: it's probably the only one!) and double click it. This will open up the shell view where you can send messages to the objects in your app, all without setting a single break point.

  5. Now let's follow the instructions shown to us by the Debug Me app.

  6. In the Mac app, issue the command .self (note the leading dot). This updates the self pointer, which will execute a Block in the App delegate that returns whatever we want to be pointed to by the variable self. In this case (and in most cases), we want self to point to the current view controller. For Debug Me, that means it points to our instance of DBMEViewController after we issue this command.

  7. Now that our pointer is set up, we can send a message to that pointer. Type self redView layer setMasksToBounds:YES. This sends a chain of messages in F-Script syntax. In Objective C, it would look like [[[self redView] layer] setMasksToBounds:YES]. Here we omit the braces because of our syntax.

    We do use parentheis sometimes, when passing the result of a message send would be ambiguous, for example something like this in Objective C: [view setBackgroundColor:[UIColor purpleColor]] would be view setBackgroundColor:(UIColor purpleColor) in our syntax.

  8. The previous step has no visible result, so lets make a change. Type self redView layer setCornerRadius:15 and see the red view get nice rounded corners!

  9. Now for the impressive part. Move your mouse over the number 15 and see it highlight. Now click and drag left or right, and see the view's corner radius update in real time. Awesome, huh?

That should be enough to give you a taste of this brand new debugger. Interact with your objects in real-time. Iterate instantly. No more build, compile, wait. It's now Run, Test, Change. Fork the project on Github and get started today.

Continue reading article ›

New Feature - Multiple Tracking Number Support in Fulfillments

Sometimes when you place an order through a fulfillment service, such as Amazon your order may be fulfilled from several fulfillment centers.  Unfortunately this information would not be provided to the customer which could lead to some confusion.

We've now added support for multiple tracking numbers in a single fulfillment, which you can use immediately in your Shipment confirmation templates.

You can start using these in your templates right away

The tracking details for these items are as follows:
{% for tracking_number in fulfillment.tracking_numbers %}
  {{ tracking_number }}
{% endfor %}

For shipping status on these items:
{% for tracking_url in fulfillment.tracking_urls %}
  {{ tracking_url }}
{% endfor %}

Continue reading article ›

New Feature - Orders Export Includes Payment Reference

The Export Orders feature in your Shop's Administration page will now include the Payment Reference used to cross-reference transactions to the gateway.

The Payment Method has been added to the end of each line of the Order export file:

This change will be put into production on December 19, 2012 at 12:00 EST. If you have any custom processing of the Orders CSV file that Shopify generates, please ensure that you have updated it to handle this new field.

Continue reading article ›

Tis' the Season to get a Better Job


Do you want to be a Shopify Guru? Our support services will soon be going 24/7, so we’re hiring more Gurus to help out our merchants around the clock.

While we’re not huge fans of traditional interviews here at Shopify, we are huge fans of parties. So instead of inviting you in for a ho-hum one-on-one, we’re hosting a big bash for potential Shopify Gurus on December 13th from 7-10pm. Specific location is TBA, but it will be in Ottawa, Canada.

You’ll get to hang out with fun people who are interested in working at Shopify, chat about the potential job, have a few laughs – and a few drinks! You can meet the people who might become your colleagues, and get a great feel for the job, the space, and Shopify’s culture.

But wait! Before you jump on the party bus, you should probably know exactly what a Shopify Guru is:

Shopify Guru
[shaw-puh-fahy goo-roo]
1. A rare, interesting character who gets a kick out of helping Shopify’s customers get their stores up and running.  A guru is comfortable on the phone and can type like a maniac. In the wild, gurus are often spotted laughing at or telling a great joke, as they have a naturally keen sense of humour.

Space is limited, so this party is invite-only. If you’re interested in attending, please enter your info in the provided fields at the bottom of one of our 3 Guru job postings: daytime, weekend and evenings and overnight

Mention in your application that you want to attend our Guru party on December 13th, and we’ll be in touch!

Continue reading article ›

API update - status code change

Hello developers!

We are changing the status code returned when an API client has hit its call limit. Presently, we use 503 Service Unavailable to indicate that a client has reached the call limit. 

As of November 30th, we will begin using status code 429 Too Many Requests (RFC 6585). 

Please ensure that your app can handle this change to guarantee expected functionality.

If you encounter any API issues or have any questions, you can visit the tech forums. Additionally, please be sure to monitor your email inboxes for further communications regarding this issue. You can also watch the Technology blog and @shopifyapi for updates.

Continue reading article ›

Meetups and Hackathons - come see us in real life!


Shopify meetups (Ottawa) - November 20, 2012

Come to Shopify on November 20th to meet with Shopify merchants, partners, and our own developers, designers, and gurus. This will be a great chance for developers to network with merchants for app ideation synergy, and designers for great app branding justice. Be sure to register soon, spaces are filling up fast!

Windows 8: Pure Imagination (Toronto) - November 24, 2012

We're hitting the road! Come join the Shopify Developer Advocates as we adventure to Toronto to hack on Windows 8 apps. Windows 8: Pure Imagination is an all-night hackathon being held at Ryerson by our friends at Microsoft, and promises to be lots of fun. Again, spots are filling up fast, so register now to ensure your spot!

Hope to see you all this month!

Continue reading article ›

API updates

Hello developers! We have a number of API changes happening this month. Please take a few moments to familiarize yourself with the updates to ensure that you know whether you're affected.

The updates

Customer API updates - http://api.shopify.com/customer.html
We've made some exciting additions to the Customer API - you can now create customer user accounts for a shop via the API! Check out the API docs for details.

Cart API bulk endpoint removal
We have removed the endpoint for bulk cart retrieval, and are moving towards complete removal of the Cart API in the near future. Please ensure that you are using webhooks over these endpoints. 

Checkout API - http://api.shopify.com/checkout.html
We are moving closer to the permanent implementation date of the Checkout API, happening November 20th. This impacts apps using the Cart or Order APIs, such as abandoned order apps. Get the details on this update here

Need help?

If you encounter any issues or have any questions, you can visit the tech forums or post to Stack Overflow using the 'shopify' tag. Also be sure to continue watching this space, as well as @shopifyapi on Twitter for updates.

Continue reading article ›

Shopify Desktop Theme Editor 1.0 Release

The new Shopify Desktop Theme Editor allows any Mac user, regardless of choice in text editor, to easily keep a local copy of all of their theme files in sync with their shop. These files can then be edited natively using any text editor, and those changes automatically and seamlessly sent back to your store. The app is a super simple install; just grab the download [here] and drag it to your Applications folder. You're then ready to open up the app, add and authenticate your shop, and get started editting your theme files locally!

One of the main benefits of the new Desktop Theme Editor is the ability to use any editor to make changes to local, synchronized copies of theme files. Whether you use Textmate, Sublime Text, vim, or even emacs, you'll be right at home. Your changes will be automatically and magically synced to your store. You can even use a service like Dropbox to make your theme files available on any computer connected to your Dropbox account.

The Desktop Theme Editor lets you know when your theme files have been synced so you know everything's working right. Never worry again about changes not being synced or your connection failing. If you're working offline, you can manually sync the files when you come back online and get a notification when that's done.

If you're a developer or a merchant working on multiple stores, The Desktop Theme Editor can handle that too. There's no limit to how many shop's you can connect. It's great for contract work as a Shopify Partner or Expert, as well as developers in the App Store and Theme Store to see their changes across multiple test shops with different themes. If you only run an individual shop but use multiple themes (desktop and mobile, for example), you can edit multiple themes per shop too.

A massive thank you to Mitch Amihod who did the development of this app – he's done an amazing job.

You can download The Desktop Theme Editor from the Shopify App Store here.

Continue reading article ›

Create an online store in minutesTry Shopify Free