API / Liquid

App devs everywhere, rejoice! Shopify is pleased to announce that we’re replacing our current API authentication system with one based on OAuth 2.0.

Anyone who knows anything about webapps will tell you that OAuth 2.0 is quickly becoming the de-facto standard for authenticating users against an API. Facebook, Foursquare, Google, Github, and other big names are already using it. We’ve been working on our own implementation here at Shopify for a while now, so we’re really pleased to finally make it public.

Why Use Oauth2?

Our previous authentication scheme was pretty good, but it lacked in some key areas. Primarily, it was completely custom and so the first thing a new dev would often have to do was implement our auth mechanism in their favourite environment. By implementing OAuth 2.0, developers can tie into existing auth solutions that implement the spec for them. We hope this makes getting stuck into the Shopify API much easier.

Secondly, we’re adding the ability to request granular access scope for your application. If you’ve ever used a Facebook or Twitter app, you’ve already seen this in action. We’ve divided up the API into a number of categories and apps can request read or write access to these individually. Now merchants won’t have to trust a product-related app with all their customer info for example. The available scopes are as follows:

• Content: Blog posts, pages, comments, etc.
• Themes: Themes and theme assets.
• Products: Everything product-related. Images, variants, etc.
• Customers: Customers and customer groups.
• Orders: Orders, transactions, and fulfillments
• Script Tags
• Shipping: Coming Soon!

Related to this, you can also change the scopes you ask for as your app functionality evolves. You’ll need merchants to approve the changes of course, but it’s a lot more flexible. A full scope reference is available in our documentation.

What This Change Means

If you’re the owner of an app that uses the old auth system (i.e. everyone at the moment), don’t worry! The old system isn’t going to be going away any time soon. We're phasing it out eventually, but there will be lots and lots of notice before we do that.

One thing that is going to change is that new apps are going to be required to use Oauth 2.0 starting soon. We’re  leaving both options up for a short while so that the authors of our various community API clients have time to update their libraries but after that it’s Oauth all the way down.

Resources

We’ve updated our technical docs with new instructions on how to get up and running with the new auth scheme. Read this page throughly before you get started.

There’s already an updated version of the shopify_api gem available that works with Oauth 2.0. We’re working on converting the shopify_app gem as well as bringing our other official adapters up to date. There’s also an omniauth solution available here.

Feedback

We’d love to hear what you think of the new changes. You can leave a comment on this blog post or head over to our developer mailing list and discuss it there.

HackVan, the Vancouver edition of HackDays -- the cross-Canada hackfest in which developers of all skill levels are challenged to use one or more of the sponsors' APIs and build a cool new app in a day -- takes place today! The apps will be judged by a panel of distinguished people from the Vancouver tech scene, and if you were at last year's HackVan, you know the prizes are pretty sweet.

Here's a compressed version of today's itinerary: At 9:30 a.m. Pacific time, each of the sponsors will present their APIs, after which the developers will form teams (if they haven't already) and come up with ideas for an app they can build by the deadline, which is around 5:00 p.m.. At 5:30, teams will present their apps to the judges, who will then deliberate and announce the top teams and honorable mentions at 7:30.

We know that coming up with an app idea on short notice isn't easy, especially when you combine it with forming a team with people you might not know, using APIs that may be unfamiliar to you and trying to get something working in a day (well, part of a day, actually). In order to make life easier (and with the not-so-ulterior motive of getting you to use the Shopify API at HackVan), we've come up with a list of apps that you might want to try building, whether you're at HackVan or "just want to play the home edition".

Product Comparison

 

Customers often have a general idea of what they want to buy. It gets tricky when it comes down to choosing a specific item, especially once they've narrowed down their choices. The harder it is for a customer to compare his or her narrowed-down choices, the more likely he or she is going to give up and not make a purchase.

This app idea is an attempt to solve that problem. It would allow customers to select two (or possibly more) items and see them side by side, perhaps along with the description and price of item. If you want to get fancy, you could highlight something about an item, such as which one is the cheaper one, or which one was bought by more customers.

For this app, you'll definitely use the Shopify API's product resource, which lets you retrieve information about one, several or all products in a shop. Each product object has properties that you'll want to use in this app, such as:

• the body_html property, which contains the description of the item, complete with HTML formatting
• the variants array, which holds info about the different variations of the product. One of the properties of each variant object is price, which you'll need, especially if you're highlighting price differences
• the images array, which contains image objects. Each image object has a property called src which specifies the location of a product image stored within Shopify.
• If you're taking on the additional challenge of showing which product is the more popular one, you'll want to use the orders resource, which you can use to get a list of all orders placed at the shop. You would then comb through the orders to count purchased products in order to determine which one was more popular.  

User Surveys

Just after their customers' money, the thing that shopowners want is customer feedback. Without this information, they're just guessing at what their customers want and what's working for them and what isn't. 

One way many shopowners can get this feedback is by sending surveys to customers who've recently bought something from their shops. Keep in mind that you don't want this survey to be too long, because customers will simply ignore it, or worse, choose not to go to that shop again!

You'll want to use the Shopify API's order resource for this one. You can use it to get information about one, several or all orders placed at the shop. One of the properties of an order object is email, which is the email address of the customer who placed the order. You might also want to use other properties, such as the line_items array to get an idea of what they customer bought and the total_price property to determine how much he or she spent (and possibly tailor the survey, based on these values).

Naturally, you'll only want to send the survey to people whose orders have already been shipped. Check the order's fulfillment_status!

Follow-up Email

In a world where there are so many places online to buy stuff, customer service can set you apart. Good customer service and treating the customer well creates return customers. One aspect of good customer service is the follow-up, which can often just be a quick email saying, "hey, thanks for shopping with us".

If you're new to the Shopify API or a developer just getting started, you might want to try developing this app. It should email customers who've recently made a purchase at the shop a quick "thank you" note. As an added bonus, you might want to include a discount code in the email to entice the customer to use the shop again.

As with the "User Surveys" app mentioned above, you'll want to use the Shopify API's order resource for this app. Each order object has an email property containing the email address of the person who made the purchase, a line_items array containing the items they purchased, and fulfillment_status property to tell you if the order has been completely shipped.

Store Status via Voice

Shopowners want to stay on top of their stores' current status. They check their shops' admin pages constantly, seeing if new orders have come in, if any items are out of stock, what's getting shipped, and so on.

There are times when a shopowner can't check his or her shop's admin pages visually -- while driving or walking about. To solve this problem, we suggest an app that uses voice technology (such as the great technology from our friends at Twilio) to report highlights and alerts about their shop status. Two important pieces of information that shopowners would like to know are: orders that have just come in, and if an item has become out of stock.

You can check stock of levels of each item using the Shopify API's product resource. Each product object has a variants array, and each variant object has two properties: inventory_policy, which tells you if Shopify tracks the stock levels of that variant, and inventory_quantity, which tells you how many of those items are in stock if Shopify tracks its stock levels.

To see which orders have recently come in, use the Shopify API's order resource.

Customer Image/Text Uploads

Some shopowners want to sell customizable items. This app would allow them to upload an image or customer text, which would then be incorporated into the item's design, such as putting the customer's picture or name on a t-shirt, mug or other item.

For this app, you'll definitely want to put some kind of annotation in an order object, probably in the note field.

Minimum Quota

In some cases, a shopowner doesn't want to fulfill an order for an item until s/he gets a minimum number of orders for that item. This often happens in cases where the item is hard to come by or if the wholesaler for that item has a similar quota.

The idea behind this app is to not fulfill any orders for specific items until a predetermined number of orders of that item appear in the shop. Only when that number has been reached or exceeded will those orders be fulfilled.

You'll definitely want to use the Shopify API's order resource for this one.

In Shopify shops, a product can have multiple photos and multiple variants. Wouldn't it be nice if the current big photo was one that corresponded to the variant the customer currently had selected? Tetsuro Takara, one of the guys on our Guru team, has written an article showing how you can rig up your shop’s template to do just that. It’s a clever, subtle trick, but I think it’s one that your customers will like.

[ This article also appears in Global Nerdy. ]

Creative Commons photo by paul (dex). Click to see the original.

HackDays -- the gatherings that bring together Canada's brightest geeks to build projects in a single day -- kicks off 2012 with a session in Toronto on Saturday, February 12th. This one's an interesting one, as it's a part of the extended Social Media Week, the global multi-city conference taking place from February 13th through 17th. Here's a quick explanation of what Social Media Week, and this year's theme, Empowering Change Through Collaboration, are all about:

In keeping with the idea behind Social Media Week and HackDays' spirit of building stuff with readily-available APIs, the theme of this Toronto HackDay is "Apps for Good". Take the APIs -- including Shopify's ecommerce API -- to build applications that can change the world around us for the better. You'll have a day to do it, and the best apps, as determined by a panel of judges, will be awarded prizes (the prizes are usually pretty sweet: we've handed out MacBook Airs and iPads to the winners).

Shopify will be there! More specifically, I'll be there, representing Shopify, along with some other Shopifolks, to help run the event, help participants with the Shopify API and generally provide moral support to everyone there. We'll also be sponsoring the event and providing a prize for the best use of our API.

If you're looking to take on a hacking challenge and put your skills to the test, meet your fellow geeks and build an app for the greater good, HackDays Toronto is for you! Sign up here (registration is free for hackers building apps), and do it soon -- space is limited!

[ This article also appears in Global Nerdy. ]

Webhooks are brilliant when you’re running an app that needs up-to-date information from a third party. They’re simple to set up and really easy to consume.

Through working with our third-party developer community here at Shopify, we’ve identified some common problems and caveats that need to be considered when using webhooks. Best practices, if you will.

When Should I Be Using Webhooks?

Let’s start with the basics. The obvious case for webhooks is when you need to act on specific events. In Shopify, this includes actions like an order being placed, a product price changing, etc. If you would otherwise have to poll for data, you should be using webhooks.

Another common use-case we’ve seen is when you’re dealing with data that isn’t easily searchable though the API you’re dealing with. Shopify offers several filters on our index requests, but there’s a fair amount of secondary or implied data that isn’t directly covered by these. Re-requesting the entire product catalog of a store whenever you want to search by SKU or grabbing the entire order history when you need to find all shipping addresses in a particular city is highly inefficient. Fortunately some forward planning and webhooks can help.

Let’s use searching for product SKUs on Shopify as an example:

The first thing you should do is grab a copy of the store’s product catalog using the standard REST interface. This may take several successive requests if there’s a large number of products. You then persist this using your favourite local storage solution.

Then you can register a webhook on the product/updated event that captures changes and updates your local copy accordingly. Bam, now you have a fully searchable up-to-date product catalog that you can transform or filter any way you please.

How Should I Handle Webhook Requests?

There’s no official spec for webhooks, so the way they’re served and managed is up to the originating service. At Shopify we’ve identified two key issues:

• Ensuring delivery/Detecting failure
• Protecting our system

To this end, we’ve implemented a 10-second timeout period and a retry period for subscriptions. We wait 10 seconds for a response to each request, and if there isn’t one or we get an error, we retry the connection several times over the next 48 hours.

If you’re receiving a Shopify webhook, the most important thing to do is respond quickly. There have been several historical occurrences of apps that do some lengthy processing when they receive a webhook that triggers the timeout. This has led to situations where webhooks were removed from functioning apps. Oops!

To make sure that apps don’t accidentally run over the timeout limit, we now recommend that apps defer processing until after the response has been sent. In Rails, Delayed Jobs are perfect for this.

What Do I Do if Everything Blows Up?

This one is a key component of good software design in general, but I think it’s worth mentioning here as the scope is beyond the usual recommendations about data validation and handling failures gracefully.

Imagine the worst case scenario: Your hosting centre exploded and your app has been offline for more than 48 hours. Ouch. It’s back on its feet now, but you’ve missed a pile of data that was sent to you in the meantime. Not only that, but Shopify has cancelled your webhooks because you weren’t responding for an extended period of time.

How do you catch up? Let’s tackle the problems in order of importance.

Getting your webhook subscriptions back should be straightforward as your app already the code that registered them in the first place. If you know for sure that they’re gone you can just re-run that and you’ll be good to go. One thing I’d suggest is adding a quick check that fetches all the existing webhooks and only registers the ones that you need.

Importing the missing data is trickier. The best way to get it back is to build a harness that fetches data from the time period you were down for and feeds it into the webhook processing code one object at a time. The only caveat is that you’ll need the processing code to be sufficiently decoupled from the request handlers that you can call it separately.

Webhooks Sound Magic, Where Can I Learn More?

We have a comprehensive wiki page on webhooks as well as technical documentation on how to manage webhooks in your app.

There’s also a good chunk of helpful threads on our Developer Mailing List.

This is just a quick update to let you know that yes, we know that the Shopify developer documentation needs work. There’s a fair bit of information there, but it could stand some improvement. There’s some missing information, it could be organized better, there are parts of it that are confusing and there need to be examples in languages and frameworks other than Ruby and Rails.

This update is also here to let you know that we’re actively working on it, bit by bit, every day. As I write this, David Underwood and are are working on a wholesale reorganization of the developer sections of the wiki and clear writeups of all the API resources, including explanations of the parameters they expect and the attributes they return as well as how they relate to other resources and what effects they have on shops. We’re also working on more example code, in more languages.

If you’ve got comments, questions and suggestions about the docs or what we’re doing with them, please let us know -- feel free to leave a comment or drop me a line.

[ This article also appears in Global Nerdy. ]

Shopify and Twilio are Holding a Contest!

Shopify, the ecommerce platform that makes it easy to set up online shops, is teaming up with Twilio, the platform for voice-over-phone, voice-online and SMS messaging, to create a developer contest where you’re challenged to write an app that makes use of both platforms.

Perhaps it’s an app that phones customers or sends them an SMS message when their order has been shipped. Or voice notification to a bricks-and-mortar store that tells the staff that a customer is coming to pick up their order. With a little creativity, some coding and our APIs, you can write an app that’s both useful and cool enough to win one of our prizes…

The Prizes

First prize is from Shopify: it’s Edward from Shopify’s preferred rig! A hot-rodded 11-inch Macbook Air with 256GB solid-state drive and 1.8 GHz CPU along with a gigantic 27-inch Apple Cinema Display. Small and light for when you’re on the go, a nice big screen for when you’re at home, the office of wherever you decide to keep the display and plenty of horsepower to build apps.

Second prize comes from Twilio and it’s a robot hacker’s dream: a stacked LEGO Mindstorms swag bag with the NXT 2.0, a bundle of touch, light and sound sensors, and a Bluetooth dongle to keep your creations connected. I, for one, welcome our new LEGO robot overlords.

Third prize is a joint Shopify/Twilio effort: a Kindle Fire, the newest multi-touch full-color hotness from the world of ebook readers.

The Rules

We try to keep things simple – it’s about building cool apps, not being buried in regulations!

• Entries must be submitted by 11:59 PM PST Sunday December 11th.
• Entries must use Twilio and Shopify APIs
• Entry cannot be an app that has previously won a Twilio Developer Contest
• Bonus points for submission to the Twilio App Gallery and the Shopify App Store
• Extra bonus points for use of Twilio Client

Okay, How Do I Get In on This Action?

• Find out more about the contest on the Twilio blog.
• Start working on an app!
• Check out the Shopify API documentation and the Twilio docs.
• Enter the contest using by using this form.
• Ask questions on the Twilio forums.
• Ask questions on the Shopify API Google Group.
• Email the Twilio help email address.
• Email me at joey@shopify.com.

This article also appears in Global Nerdy.

Over the past few days we've rolled out several new features to liquid templates. These are the additions:

for...else condition

Currently it's pretty common to see liquid markup with a for loop nested in an if condition like this:

{% if cart.item_count > 0 %}
  {% for item in cart.items %}
    <!-- display items -->
  {% endfor %}
{% else %}
  no items in your cart.
{% endif %}

Now, that markup can be simplified to this:

{% for item in cart.items %}
  <!-- display items -->
{% else %}
  no items in your cart.
{% endfor %}

If the object that the for tag is looping through is empty, the optional else condition will be used instead.

split filter

The new split filter makes it easy to display parts of a string. For example, to have a different product description on the collection and product pages:

In collection.liquid:

{{ product.description | split: '<!-- split -->' | first }}

In product.liquid:

{{ product.description | split: '<!-- split -->' | last }}

New Variables

We have also exposed several new variables to liquid templates.

The cart is now accessible to the json filter, which allows you to power all sorts of Javascript logic easily.

{{ cart | json }}

We have also exposed customer tags, so if a customer is logged in, you can display contextualized information.

{% if shop.customer_accounts_enabled and customer and customer.tags contains 'Wholesale' %}
  <p>Use the discount code WHOLESALE at checkout. That'll give you 40% off. Cheers!</p>
{% endif %}

Last month we announced the removal of our global API limit, which was hampering the growth of some of our larger apps. Today I have some more good news: The per-shop limit has been raised from 300 calls every 10 minutes to 500 calls every 5 minutes. This works out to a 3(-and-a-bit)-fold increase over the previous number, and should accommodate the vast majority of apps.

The best part about this announcement is that the change has already been rolled out, and you should see the new numbers appearing in the HTTP_X_SHOPIFY_SHOP_API_CALL_LIMIT header in our API responses.

We're in the process of updating our docs to reflect this change, but if you see any references to the old numbers on the wiki feel free to swap them out for the new ones or just ping me about it if you're lazy :)

 

App developers, rejoice! The bonds binding your applications have been broken (somewhat)!

I'm pleased to announce that we've removed the global limit on API calls for applications. Previously, this is what our limits looked like:

• 300 calls per store per 10 minutes
  
• 3000 calls globally per 10 minutes
  

You can probably see the problem: As soon as an app has more than ten installs, they're effectively limited by the global limit rather than the per store limit. As an app grows this global limit becomes increasingly tight and there's only so much efficient usage can do to mitigate that. Shopify is supposed to be all about fostering growth, so we've decided to ditch the global limits. Here's what the restrictions look like now:

• 300 calls per store per 10 minutes

That's it. No more global limit. Apps are now free to grow their install base without having to worry about hitting a barrier. Hurrah!

Implementation Details

There are two stages to the rollout of this change. The first has already taken place: The HTTP_X_SHOPIFY_API_CALL_LIMIT header will now always return 0/3000 regardless of the number of calls you make.

The followup change, which we're rolling out shortly, will be to remove this header all together. If your app relies on it being there,  you need to update your code. This is being done primarily to avoid confusion amongst new users of the API but also to keep the API clean and free of junk code.

Keep an eye on the API Announcements mailing list for info about this and future changes. Good hunting!