Popular

How the Apps Team Gets Things Done

  Pictured above is the team at Shopify to which I belong. It's the Apps Team, and while…

 

Pictured above is the team at Shopify to which I belong. It's the Apps Team, and while it may be small, it takes on the company's most ambitious projects: the Shopify App Store, Shopify Experts, Shopify Partners and Shopify Fund as well as the company's business development and developer advocacy efforts. It's our team's job to take the Shopify platform and see how far we can take it.

As a small team charged with a lot of responsibilities, we have to do things in a way that maximize the effect our actions have. Over the past year, we've worked out a number of ways of doing this, some gained from experience, others from experimentation. They've remained what's called "tacit knowledge" -- practiced by the team but not written down or formally codified in an operations manual -- until team leader Harley Finkelstein, our Chief Platform Officer, collected them into a set of slides.

The way we get things done boils down to the general principles listed below. Your team may not be like ours, but I'm sharing these principles because you might find at least some of them useful:

  • Act like an owner. You don't "just work here", you own a piece of a company and have a stake in its success. Work as if your livelihood, career and reputation were riding on it, because as an owner, it is! Be entrepreneurial and own your domain: if you have an idea and it lines up with the company's goals, make that idea happen.
  • Know what to work on and what things to ship. While owners have the freedom to work on and ship whatever they like, they also work in the real world. 80% of what makes the company go is often achieved by doing the most important work first, which typically makes up 20% of the available tasks. Sometimes these tasks can be tedious and feel like drudgery, but if they're what makes things happen for our customers and their customers, they've got to be done, and with the highest priority.
  • Done is better than perfect, or "the best" is the enemy of "the good". Perfectionism is a form of procrastination. It assumes that time is an infinite resource, that other tasks can wait while you add "just one more touch" and that "perfect" is attainable. You have to be able to make the call and say "done" at some point. A good feature that our customers use and enjoy is infinitely better than a perfect one that "will be available soon". As they say at Apple, "Real artists ship".
  • Have high standards. While done is better than perfect, good still remains better than bad.
  • It's okay to fail; just fail gracefully. The only sure-fire way to not fail is to not do anything. Since we can't do that and remain in business, never mind take the company to the heights we want to, we have to accept failure as part and parcel of trying. Sometimes we'll make mistakes, other times we'll do things right and still our best-laid plans won't work because of circumstances outside our control. The trick is to learn from failure and make sure our failures aren't fatal. As our CEO Tobi likes to say: "If I'm not failing every now and again, I'm not trying hard enough."
  • Communicate good news quickly, communicate bad news ever more so. The first part is easy: it takes no effort to tell the team your project is a success. It's a good thing to do so; good news bolsters the team and success often breeds more success. However, a combination of pride and fear (and in some companies, a "cover your ass" culture) makes it difficult to tell the team that you're having trouble or that something's not working out. It's best to tackle problems as soon as possible, while they're still small and manageable, and the best way to do this is to communicate bad news as quickly as possible -- remember, it's okay to fail.
  • Understand and respect the makers' and managers' schedules. As Paul Graham wrote in his essay, Maker's Schedule, Manager's Schedule, makers and managers operate by different schedules. Managers' days are determined by their appointment calendars, which divide the days into hours and even half-hours, and things like meetings fit into the manager's schedule easily. Makers, on the other hand, do things in half-day or even full-days blocks, and things like meetings are disruptive. Some of the team operate on a maker's schedule, other operate on a manager's schedule, and many of us switch between the two, depending on what day it is and what tasks they have on that day. Know who operates on which schedule (and when), and understand and respect those schedules.
  • Operate lean and mean. We're made up of multi-talented, capable, autonomous, ambitious go-getters, and that means we don't have to operate like a big, lumbering beast. Unless the circumstances are unusual, there really should be 2 people maximum per deal or project. Meetings and calls should be kept to 30 minutes or less, not counting brainstorming or design pow-wows. And full-on meetings aren't always necessary: you should be able to "just pop by" anyone's office or desk or call them up on Skype.
  • Update often. Because we operate lean, means and independently, communication is vital. Keep your teammates apprised of your progress! 
  • Draw the owl. In the end, that's what you're trying to do...

[ This article also appears in Global Nerdy. ]

Defining Churn Rate (no really, this actually requires an entire blog post)

If you go to three different analysts looking for a definition of "churn rate," they will all agree…

Old-style woodcut of a woman with a butter churn

If you go to three different analysts looking for a definition of "churn rate," they will all agree that it's an important metric and that the definition is self evident. Then they will go ahead and give you three different definitions. And as they share their definitions with each other they all have the same response: why is everyone else making this so complicated?

How can it be so confusing? All I want to know is how quickly my users are cancelling their service.

Unfortunately, churn rate is actually an extremely important metric. Why? To put it in modern startup terminology, churn rate is an ideal actionable (non-vanity) metric. As Eric Ries describes in his classic blog post, understanding your company metrics at a customer level is key to understanding the effects of your actions. It is your customer level metrics that will inform you if you are currently on a path to profitability, or if your current profits are sustainable. And at the heart of customer level metrics is churn rate: the measurement of the likelihood of your customer to become an ex-customer.

Let me take you through the recent stumbling process we went through at Shopify and share with you the definition we ended up with. (Or you can jump straight to the answer.)

The accountants' dream

As I say, a churn rate seems simple. Since it's just a rate all you need is a numerator and a denominator, right? So why not pick the obvious numerator and denominator.

\frac{(number\,of\,churns\,during\,period)}{(number\,of\,customers\,at\,beginning\,of\,period)}  

The problem here is that the [number of churns over period] value is affected by the entire period but the [number of customers at beginning of period] value is a snapshot from the beginning of the period. This might not have much impact if new customers only make up a small percentage of your user base but for a company that's growing this can lead to some major misinterpretations.

Consider you are calculating your churn rate for July and August. Let's say in July you started with 10,000 customers, lost 500 of them (5% of 10,000), gained 5,000 but lost 125 of those (you only lose 2.5% of the 5,000 because you gain those 5,000 over the course of the month). Now in August you start with 14,375 customers and lost a similar amount of 719 (5% of 14,375), gained another 5,000 and again lost 125 of them. So July's churn rate would be 6.25% and August's would be 5.87%.  That's a shift of 38 basis points for the exact same behaviour in both months (here the standard deviation is about 20~25 basis points depending on how you calculate it). You would have told yourself that your churn rate is improving when nothing has changed. Misreporting your performance is a bad problem to have.

And beyond that, what if July had been a dead month with only 100 new customers, 2 of which churned, and then August picked up again.  Then July would have a churn rate of 5.02% and August would have a churn rate of 6.3%. This is because this definition of churn rate is directly influenced by the number of new customers you acquire. But the whole point of a churn rate is to understand churn behaviour normalized for growth and size.

The accountants' adjusted dream

Since we had a classic problem in financial analysis in our previous rate, let us try the classic solution.



Great, this seems to solve the problem of the number of new customers affecting our measurement of churn: according to this definition in either of the scenarios above we always have a churn rate of 5.1% (that is where July was dead or not). Our fluctuations due to artifacting has disappeared.

So let's go on and consider the next natural question: what is the churn rate for the quarter? Well, if we say September is like August and gains another 5,000 customers, 125 of which it loses, and it lost 927 of your original customers, you end up with 22,480 at the end of the month. So if you use this definition you end up with a churn rate for the quarter of 15.5%?! That's a pretty radical shift from 5.1%.

But of course I should calm down. My quarter has three months in it so I just have to divide my 15.5% by three, right? And I have a reasonable number of 5.2%. However, let's consider that dead July again where I only gained 100 customers. I still had a July, August, September churn rate of 5.1% but now my quarterly churn is 4.6%?!

This is because my formula has a cooked in assumption that the churns are evenly spread out. If your data breaks this assumption you will get results that no longer make sense. Unfortunately you can't insist your customers follow your assumptions so that your equations are satisfied (I'm just imagining telling a customer "I'm sorry, we're going to have to churn you today because we haven't had enough churns to satisfy our linearly distribution assumption").

A good ratio metric needs to be able to expand and shrink in the length of period that it measures. You want to be able to see how your current month churn rate compares to the churn rate of the quarter or the YTD churn rate. You also should be able to change your view to be able to see a weekly churn rate that provides better resolution but still with comparable values. And even a daily churn rate. (Of course the more narrow your window the more volatility you should expect.)

Even going from 28 days in February to 31 days in March is enough of an increase in days to create results that will suggest churn is increasing even though customer behaviour hasn't changed.

(As an aside I once suggested that we stop reporting figures for actual months and instead chop up the year into 30 day periods and only report figures for these 30 day Shopify months. I didn't really get an answer as much as a I got a look that said did we really just hire this guy? I needed to remind myself that analysis needs to work within the world that we are in if it is to be useful; not an idealized version of the world. The globe isn't a sphere. The market isn't efficient. And months aren't all 30 days.)

We also tried:


which basically has the same problems but to different degrees. Instead we had to let this dream die and consider alternatives.

The predictive modelers' fancy

From a predictive modeling background my priority was how can I define a churn rate so that it might be useful for making predictions. The most straight forward way to do this is to find a churn rate r so that if you have the number of customers for today n then r*n is a prediction for the number of customers that will have churned sometime in the next 30 days (this is actually not a very good way to make this sort of prediction). To do this you might take the weighted average of rate of people who churn within 30 days for every day in your period. That is:


or



where the weights are



This seems to solve all of our past problems. If customer behaviour remains unchanged then this churn rate will remain consistent. And it can happily return a churn rate for a month, a week, or a day all in nice comparable numbers.

However, this number hindered by the fact that it is neither current or timely. By timely I mean that at the end of August the most recent churn rates you can report are for periods that end August 1st. You have to wait until the end of September before you can report August's churn rate. At worst you want to be able to report a month's rate only two or three days after the month ends. Maybe this is just a perception problem that can be solved by reporting July's churn rate for August; ie change the recognition date. This opens other problems and doesn't resolve the issue of not being current.

By current I mean that there are churn events that have occurred that can't be reflected in your most recent churn rate. It is possible for a surge in cancellations to not be captured in the churn rate until weeks later. This is a major problem. If your main measure for churn isn't able to notify you of a major change in churn behaviour shortly after the change then it is not performing one of its primary functions.

But really, these are all problems with the fact this metric makes it too hard to understand what is going on. That is when you say this is the churn rate for August 24th to the 31st one expects that this number is an aggregate of churn behaviour during that week. Instead, churn behaviour during that week and most of the month of August is reflected in that number. But only some of that churn behaviour from August. Which means it is very hard to understand how this number relates to when anything else is happening in your business. You get funky effects like churn rate dropping just before you put into practice a new retention strategy.

So while it is nice that this version of churn rate has some predictive utility it fails on so many other accounts that it no longer seems so fancy.

Where we ended up

Between these two ideas is where we ended up:


or


where the weights are



We've found that resolves all the issues we've had above: it's current, it's timely, it produces comparable results for different period lengths, and an increase or decrease in this churn rate reflects an actual change in churn behaviour for your measured period.

This isn't the best number to use to multiply your current customer count by to get an estimate of how many of your current customers will churn in the next 30 days. However, as demonstrated in a post over at Custora, this isn't a good practice anyhow.

What this metric is useful for is keeping track of changes in customer churn behaviour while giving a rough estimate of what percentage of your customers will leave in the next 30 days.

And finally, what makes this calculation actually something we can use is the fact that the components are all reconciliable numbers that we were already recording. While I am clearly a fan of rigor I believe that your final result should follow the 37signals advise of of "make it easy." All you have to know is how many customers you have each day and the number of cancellations for each day. Of course "customers" and "cancellations" are definitions that also need to be sorted out.

Epilogue

One thing that I did not mention but needs to be kept in mind: for virtually all businesses, new customers will have a higher churn rate than mature customers. But what this means is that some form of segmentation is necessary to have a useful churn rate. For example you may want to only report the churn rate for customers who have been around for at least 90 days. Or you may want separate churn rates for all sorts of demographics and tenure. The aforementioned post by Custora has a great discussion of this that goes into greater detail.

If you don't apply some form of segmentation in your reporting you will find that your churn rate increases whenever your ratio of new customers to mature customers increases; even though it may be that the churn rates of both new and mature customers are dropping.

No aggregate is ever going to perfectly communicate a particular customer behaviour: an aggregate is lossy compression after all. But what you want to avoid are aggregates that hide big news, tell you something has changed when everything is still the same, or leave you with the opposite impression of what is actually happening. And remember, a change in aggregate will almost never tell you the entire story. What it tells you is there is a story to be told and now you need to find it.

Webhook Testing Made Easy

Webhooks are fantastic. We use them here at Shopify to notify API users of all sorts of important…

Webhooks are fantastic. We use them here at Shopify to notify API users of all sorts of important events. Order creation, product modification, and even app uninstallation all cause webhooks to be fired. They're a really neat way to avoid the problem of polling, which is annoying for app developers and API providers alike.

The trouble with Webhooks is that you need a publicly visible URL to handle them. Unlike client-side redirects, webhooks originate directly from the server. This means that you can't use localhost as an endpoint in your testing environment as the API server would effectively be calling itself. Bummer.

Fortunately, there are a couple of tools that make working with webhooks during development much easier. Let me introduce you to PostCatcher and LocalTunnel.

PostCatcher

PostCatcher is a brand new webapp that was created as an entry for last week's Node Knockout. Shopify's own Steven Soroka and Nick Small are on the judging panel this year, and this app caught their eye.

The app generates a unique url that you can use as a webhook endpoint and displays any POST requests sent to it for you to examine. As you might expect from a js contest, the ui is extremely slick and renders all the requests in real-time as they come in. This is really useful in the early stages of developing an app as you can see the general shape and structure of any webhooks you need without writing a single line of code. On the flip side, API developers can use it to test their own service in a real-world environment.

 

The thing I really like about PostCatcher over similar apps like PostBin is that I can sign in using github and keep track of all the catchers I've created. No more copy/pasting urls to a text file to avloid losing them. Hooray!

LocalTunnel

LocalTunnel is a Ruby gem + webapp sponsored by Twilio that allows you to expose a given port on your local machine to the world through a url on their site. Setup is really easy (provided you have ruby and rubygems installed) and once it's installed you just start it from the console with the port you want to forward and share the url it spits out.

 

From then on that url will point to your local machine so you can register the address as a webhook endpoint and get any incoming requests piped right to your machine. My previous solution was endless deployments to Heroku every time I made a small change to my code, which was a real pain in the arse. Compared to that, LocalTunnel was a godsend.

Alternatives

Whilst PostCatcher and LocalTunnel are currently my top choices for testing webhooks, they're by no means the only party in town. I've already mentioned PostBin, but LocalTunnel also has a contender in LocalNode (another Node KO entry). The latter boasts wider integration (you don't need ruby) as well as permanent url redirects but setup is more complicated as you have to add a static html file to your web server.

If there are other services, apps, or tricks that you use to test webhooks when developing apps, call them out in the comments! I'd love to hear what I've missed in this space.

How we use git at Shopify

Photo by Paul Hart A little while back, Rodrigo Flores posted to the plataformatec blog, A (successful) git branching…


Photo by Paul Hart

A little while back, Rodrigo Flores posted to the plataformatec blog, A (successful) git branching model, where he talks about the git workflow they've been using on some projects. I thought this was a great post and decided to do something similar explaining the git workflow that we use at Shopify.

Preface

Git is an incredibly powerful tool that can be used in many different ways. I don't believe there is a 'correct' workflow for using git, just many different options that work for particular situations and people. The workflow that I am going to describe won't work for everyone, and not everyone at Shopify uses git in the same way - you have to modify and massage it to shape your needs and the way you work. I don't consider myself an expert with git, but am comfortable enough with the tool to handle just about everything I might need to do. If there's anything I can't figure out, James MacAulay is our resident git expert in the office and always willing to help out.

Okay, lets get down to business.

Setup

When working on a project each developer and designer first forks the repository that they want to work on. Forking a repository is really simple, Github even has a guide if you need some help with it. A fork is basically your own copy of the repository, that you can change without affecting anyone else. We use GitHub for all of our projects so it makes managing the forks really easy. All the work is done on your fork of the repository and only gets pulled into the main repository after it has been fully tested, code reviewed, etc. We also use the concept of feature branches to make it easy to switch between tasks and to share the work with other colleagues. A branch is kind of like a fork within your own repository, you can have many branches within your forked repository for each of the tasks you're working on. Your checkout of a project should be setup with a couple of remotes and branches to get started.

Remotes:

  • origin - This is a remote pointing to your clone of the project and added by default when you do 'git clone'.
  • mainline - This is a remote pointing to the main repository for the project. We use this remote to keep up to date and push to the main repository.

Branches:

  • production - This is the production branch of the main repository (or mainline). This is the code that is ready to be deployed to production.
  • staging - Contains the code that is being run on the staging server (we have two that developers can use). Before a feature is considered 'finished' it must be tested on one of the staging servers, which mirrors the production environment.
  • master - Contains completed features that can be deployed.

So how do we set all this up? These couple of git commands should take care of it:
git clone git@github.com:jduff/project.git
git remote add mainline git@github.com:Shopify/project.git
Keeping a project up to date is also really easy, you just pull from mainline.
git checkout master
git pull --rebase mainline master
I know what you're thinking, what the heck is that 'rebase' doing in there? Well, you don't really need it, but it helps to use it in case you've merged a new feature that you haven't pushed yet. This keeps the history all tidy with the changes you made on top of what is already in master instead of creating an additional "merge" commit when there's a conflict.

Day To Day Usage

So how does all of this work day to day? Here it is, step by step:
git checkout master
git checkout -b add_awesome # Feature branches, remember
# Do some work, listen to a lightning talk, more work
git commit -m "Creating an awesome feature"
Mainline master moves pretty fast so we should keep our feature branch up to date
git checkout master
git pull --rebase mainline master

git checkout add_awesome
git rebase master
Everything is finished, test it out on staging!
git push -f mainline add_awesome:staging
# This blows away what is currently being staged, make sure staging isn't already in use!
Staging is cool...code review...high fives all around, ship it!

It's always easier to release a feature if master is up to date and you've rebased your branch. See above for how to 'keep our feature branch up to date'. We also make sure to squash all the commits down as much as possible before merging the feature into master. You can do this with the rebase command:
# Rebase the last 5 commits
git rebase -i HEAD~5
Now we can merge the feature into the master branch:
git checkout master
git merge add_awesome
git push mainline master
And if you want your code to go out to production right away you have a couple more steps:
git checkout master
git pull mainline master # Make sure you're up to date with everything

git checkout production
git merge master
git push mainline production
# Ask Ops for a deploy
That's about it. It might seem like a lot to get a hang of at the start but it really works well and keeps the main repository clear of merge commits so it's easy to read and revert if required. I personally really like the idea of feature branches and rebasing as often as possible, it makes it super easy to switch tasks and keeps merge conflicts to a minimum. I almost never have conflicts because I rebase a couple of times a day.

A Few More git Tips

I've got a couple more tips that might help you out in your day to day git usage.
# Unstage the last commit
git reset HEAD~1
git reset HEAD^ # Same as above

# Remove the last commit from history (don't do this if the commit has been pushed to a remote)
git reset --hard HEAD~1

# Interactive rebase is awesome!
git rebase -i HEAD~4
git rebase -i HEAD^^^^ # Same as above

# Change the last commit message, or add staged files to the last commit
git commit --amend

# Reverses the commit 1b9b50a if it introduced a bug
git revert 1b9b50a

# Track down a bug, HEAD is bad but 5 commits back it was good
git bisect start HEAD HEAD~5

Conclusion

So there you have it, that's how we use git at Shopify. I don't know about everyone else, but once I got going I found this workflow (particularly the feature branches) to work very well. That doesn't mean this is the only way to use git, like I said earlier it is an incredibly powerful tool and you have to find a way that works well for you and your team. I do hope that this might serve as a starting point for your own git workflow and maybe provide a little insight into how we work here at Shopify.

Our tools and the way we use them are constantly evolving so I would love to hear about how you use git to see if we might be able to improve our own workflow. Let us know in the comments or better yet, write you own blog post and drop us the link!


StatsD at Shopify

Here at Shopify, we like data. One of the many tools in our data toolbox is StatsD. We've…

Here at Shopify, we like data. One of the many tools in our data toolbox is StatsD. We've been using StatsD in production at Shopify for many months now, consistently sending multiple events to our StatsD instance on every request.

What is StatsD good for?

In my experience, there are two things that StatsD really excels at. First, getting a high level overview of some custom piece of data. We use NewRelic to tell us about the performance of our apps. NewRelic provides a great overview of our performance as a whole, even down to which of our controller actions are slowest, and though it has an API for custom instrumentation I've never used it. For custom metrics we're using StatsD.

We use lots of memcached, and one metric we track with StatsD is cache hits vs. cache misses on our frontend. On every request that hits a cacheable action we send an event to StatsD to record a hit or miss. 

Caching Baseline (Green: cache hits, Blue: cache misses)



Note: The graphs in this article were generated by Graphite, the real-time graphing system that StatsD runs on top of.

As an example of how this is useful, we recently added some data to a cache key that wasn't properly converted to a string, so that piece of the key was appearing to be unique far more often than it was. The net result was more cache misses than usual. Looking at our NewRelic data we could see that performance was affected, but it was difficult to see exactly where. The response time from our memcached servers was still good, the response time from the app was still good, but our number of cache misses had doubled, our number of cache hits had halved, and overall user-facing performance was down.

A problem



It wasn't until we looked at our StatsD graphs that we fully understood the problem. Looking at our caching trends over time we could clearly see that on a specific date something was introduced that was affecting caching negatively. With a specific date we were able to track down the git commit and fix the issue. Keeping an eye on our StatsD graphs we immediately saw the behaviour return to the normal trend.

Return to Baseline


The second thing that StatsD excels at is proving assumptions. When we're writing code we're constantly making assumptions. Assumptions about how our web app may be used, assumptions about how often an interaction will be performed, assumptions about how fast a particular operation may be, assumptions about how successful a particular operation may be. Using StatsD it becomes trivial to get real data about this stuff.

For instance, we push a lot of products to Google Product Search on behalf of our customers. There was a point where I was seeing an abnormally high number of failures returned from Google when we were posting these products via their API. My first assumption was that something was wrong at the protocol level and most of our API requests were failing. I could have done some digging around in the database to get an idea of how many failures we were getting, cross referenced with how many products we were trying to publish and how frequently, etc. But using our StatsD client (see below) I was able add a simple success/failure metric to give me a high level overview of the issue. Looking at the graph from StatsD I could see that my assumption was wrong, so I was able to eliminate that line of thinking.

statsd-instrument

We were excited about StatsD as soon as we read Etsy's announcement. We wrote our own client and began using it immediately. Today we're releasing that client. It's been in use in production since then and has been stalwartly collecting data for us. On an average request we're sending ~5 events to StatsD and we don't see a performance hit. We're actually using StatsD to record the raw number of requests we handle over time.

statsd-instrument provides some basic helpers for sending data to StatsD, but we don't typically use those directly. We definitely didn't want to litter our application with instrumentation details so we wrote metaprogramming methods that allow us to inject that instrumentation where it's needed. Using those methods we have managed to keep all of our instrumentation contained to one file in our config/initializers folder. Check out the README for the full API or pull down the statsd-instrument rubygem to use it.

A sample of our instrumentation shows how to use the library and the metaprogramming methods:

# Liquid
Liquid::Template.extend StatsD::Instrument
Liquid::Template.statsd_measure :parse, 'Liquid.Template.parse'
Liquid::Template.statsd_measure :render, 'Liquid.Template.render'

# Google Base
GoogleBase.extend StatsD::Instrument
GoogleBase.statsd_count_success :update_products!, 'GoogleBase.update_products'

# Webhooks
WebhookJob.extend StatsD::Instrument
WebhookJob.statsd_count_success :perform, 'Webhook.perform'

That being said, there are a few places where we do make use of the helpers directly (sans metaprogramming), still within the confines of our instrumentation initializer:

ShopAreaController.after_filter do
  StatsD.increment 'Storefront.requests', 1, 0.1

  return unless request.env['cacheable.cache']

  if request.env['cacheable.miss']
    StatsD.increment 'Storefront.cache.miss'
  elsif request.env['cacheable.store'] == 'client'
    StatsD.increment 'Storefront.cache.hit_client'
  elsif request.env['cacheable.store'] == 'server'
    StatsD.increment 'Storefront.cache.hit_server'
  end
end

Today we're recording metrics on everything from the time it takes to parse and render Liquid templates, how often our Webhooks are succeeding, performance of our search server, average response times from the many payment gateways we support, success/failure of user logins, and more.

As I mentioned, we have many tools in our data toolbox, and StatsD is a low-friction way to easily collect and inspect metrics. Check out statsd-instrument on github.

New Liquid Template Features

I just pushed three new features for your liquid templating pleasure: Raw tag Web developers are using client-site…

I just pushed three new features for your liquid templating pleasure:

Raw tag

Web developers are using client-site templates like Mustache and jQuery Templates more and more these days, but they use the same curly braces as liquid, so it's hard to mix them into liquid templates. There is now a solution to this:

{% raw %}
  <h1>{{ title }}</h1>
{% endraw %}

The raw tag will render its contents verbatim, so this is the result:

<h1>{{ title }}</h1>

Read all about it on the wiki.

link.active

link.active is a new boolean that allows you to easily tell if the current page is the same as a link in a linklist.

{% for link in linklists.main-menu.links %}
<li>
   <a href="{{ link.url }}"{% if link.active %} class="active"{% endif %}>{{ link.title }}</a>
</li>
{% endfor %}

There are some nuances to it, so make sure you read the documentation on the wiki.

Alternate views

Did you know that you can make multiple versions of a template? Previously they were only selectable on a per-product basis. But now you can pick an alternate template using the url. For example, visiting http://store.myshopify.com/products/cool-kicks?view=quick will use product.quick.liquid. (Take a look at it in action on the collection page)

This works for all templates: appending ?view=foo to the URL will use template-name.foo.liquid. You can create alternate templates by clicking 'add new template' in the Template Editor.

Why developers should be force-fed state machines

This post is meant to create more awareness about state machines in the web application developer crowd. If…

This post is meant to create more awareness about state machines in the web application developer crowd. If you don’t know what state machines are, please read up on them first. Wikipedia is a good place to start, as always.

State machines are awesome

The main reason for using state machines is to help the design process. It is much easier to figure out all the possible edge conditions by drawing out the state machine on paper. This will make sure that your application will have less bugs and less undefined behavior. Also, it clearly defines which parts of the internal state of your object are exposed as external API.

Moreover, state machines have decades of math and CS research behind them about analyzing them, simplifying them, and much more. Once you realize that in management state machines are called business processes, you'll find a wealth of information and tools at your disposal.

Recognizing the state machine pattern

Most web applications contain several examples of state machines, including accounts and subscriptions, invoices, orders, blog posts, and many more. The problem is that you might not necessarily think of them as state machines while designing your application. Therefore, it is good to have some indicators to recognize them early on. The easiest way is to look at your data model:

  • Adding a state or status field to your model is the most obvious sign of a state machine.
  • Boolean fields are usually also a good indication, like published, or paid. Also timestamps that can have a NULL value like published_at and paid_at are a usable sign.
  • Finally, having records that are only valid for a given period in time, like subscriptions with a start and end date.

When you decide that a state machine is the way to go for your problem at hand, there are many tools available to help you implement it. For Ruby on Rails, we have the excellent gem state_machine which should cover virtually all of your state machine needs.

Keeping the transition history

Now that you are using state machines for modelling, the next thing you will want to do is keeping track of all the state transitions over time. When you are starting out, you may be only interested in the current state of an object, but at some point the transition history will be an invaluable source of information. It allows you to answer all kinds of questions, like: “How long on average does it take for an account to upgrade?”, “How long does it take to get a draft blog post published?”, or “Which invoices are waiting for an initial payment the longest?”. In short, it gives you great insight on your users' behavior.

When your state machine is acyclic (i.e. it is not possible to return to a previous state) the simplest way to keep track of the transitions is to add a timestamp field for every possible state (e.g. confirmed_atpublished_atpaid_at). Simply set these fields to the current time whenever a transition to the given state occurs.

However, it is often possible to revisit the same state multiple times. In that case, simply adding fields to your model won’t do the trick because you will be overwriting them. Instead, add a log table in which all the state transitions will be logged. Fields that you probably want to include are the timestamp, the old state, the new state, and the event that caused the transition.

For Ruby and Rails, Jesse Storimer and I have developed the Ruby gem state_machine-audit_trail to track this history for you. It can be used in unison with the state_machine gem.

Deleting records?

In some cases, you may be tempted to delete state machine records from your database. However, you should never do this. For accountability and completeness of your history alone, it is a good practice to never delete records. Instead of removing it, add an error state for any reason you would have wanted to delete a record. A spam account? Don’t delete, set to the spam state. A fraudulent order? Don’t delete, set to the fraud state.

This allows you to keep track of these problems over time, like: how many accounts are spam, or how long it takes on average to see that an order is fraudulent.

In conclusion

Hopefully, reading this text has made you more aware of state machines and you will be applying them more often when developing a web application. Disclaimer: like any technique, state machines can be overused. Developer discretion is advised.

Start your free 14 day trial!Create your store now

Create an online store in minutesTry Shopify Free