According to 37 Designs, 48% of online shoppers fail to purchase because of usability roadblocks.
Spending money on advertising can bring visitors in through the door — but investing your money in usability can make your store more useful to them (and continue shopping as a result). After all, the easiest thing in the world for any visitor to do is to leave your website.
You might have started your store with a product that solved a problem for yourself. And to your delight, other people felt the same way. But even though your customers might find your products useful, they might be very different from you in other ways. For example, they could be younger, not comfortable with websites, or have different ways of communicating.
That’s why it’s important to understand how your visitors, buyers, and customers actually browse your ecommerce store. For example, they might buy more stuff because of your very informative header, or clear navigation — or they might be browsing in spite of them
There’s no better way to learn how visitors, buyers, and customers behave at your ecommerce store than to simply watch them. The formal term for this simple concept is a “field study.” Of course, it’s never as simple as “just watching.” Watching for what, exactly? What specific questions would you like answered? What would you like to learn?
Before even planning for a field study, you should identify your curiosities. For example:
- Do you want to learn how to improve conversion rates?
- Do you want to better identify high intent visitors?
- Do you want to create greater loyalty?
- Do you want to streamline user performance and minimize the time required to perform certain key tasks?
- Do you want to plan for a new product launch, or identify opportunities to improve customer experience?
The list could go on forever, but ideally, these are questions that have been raised by an analytics audit. You can pick more than one question, but make sure you know which ones will drive the greatest impact (and which provide the most profitable ROI).
That also means you’ll know which ones can be deprioritized or left out if time or budget don’t permit. Generally, the rule is the more questions you have, the lower the quality of insight per question. Once you figure out what specifically you and your team want to learn out of this field study, it’s time to get cracking.
What Is a Usability Field Study?
In ecommerce, depending on what you’re testing, usability field studies could take place wherever a visitor would actually browse your store. It could be on her iPad in her Lay-Z-Boy, on his phone on the bus, on her old HP desktop in her study. The key is to observe how users would behave in a real environment.
Although usability field studies may sound foreign and fancy, you don’t need to hire a team of anthropologists to conduct them. Instead, your design team will be your researchers and will lead and coordinate the field studies.
It could also make sense to enlist members of your marketing and development teams, particularly if your objectives involve them (e.g., cheaper customer acquisition, better prioritization of engineering resources). And while customer knowledge is typically a design responsibility, marketers and engineers can contribute with different perspectives, experiences, and solutions.
It’s best to conduct a field study to explore or identify problems before actually implementing the changes. Otherwise, if you conduct a field study simply to validate a completed design that you’d just paid for, you might not be able to implement the deep changes that field studies require.
With the first few field study sessions, the researchers might only watch users during the field study.
But as researchers start understanding the user, they might ask some questions and conduct interviews with users.
Eventually, as they come up with solutions to pain points or prototype features to explore, they might let the users try them and watch what happens.
You’re now growing faint at the prospect of flights for your researchers (and bizarrely excited by racking up the ensuing flight points). Hold that trigger finger.
There’s a slight tweak to traditional field study that you can implement in order to make sure you talk to customers from all around the world...
Without having to set even a toe outside of your offices.
What Is Remote User Testing? Why Remote?
Remote user testing studies are exactly what they sound like — your users and researchers are in two different places. Your users would be wherever they would browse your store, and your researchers in their offices observing remotely.
There are a few reasons why these are the most relevant for most ecommerce companies:
- Given that you run an ecommerce store, your customers aren’t necessarily restricted to one region of the world. It will be taxing on time, finances, and human resources to travel to even a small sample of them.
- Your product is digital. Your store is the main thing you’re testing — and users constantly use a computer or phone to access it. As such, they’re immersed in the digital world, and their physical environment might not affect them as much (unless there are connectivity issues).
Remote user testing also brings in the element of time. Will your researchers be joining users in the same remote space at the same time? Or, will the researchers have the user participate in the study on their own schedule, and recorded the session and results for later review?
As Nielsen Norman Group highlights, moderated sessions enable researchers to ask more detailed questions during the session, and to support the user in case there’s confusion.
Obviously, even with moderated remote user testing, the experience and effect won’t be entirely the same as users and researchers being in the same room. Your researchers won’t be able to see anything the cameras and mics don’t capture. They might miss some peripheral elements of the environment.
If you conduct an unmoderated remote user testing field study, you can use tools like webcams, screen recording, and audio recording to capture data. Most importantly, you should encourage your participant to think out loud, so that your researchers can understand what’s going in the participant’s mind. The Nielsen Norman Group calls thinking aloud the #1 usability tool, and likens it to:
“a window on the soul, letting you discover what users really think about your design. In particular, you hear their misconceptions, which usually turn into actionable redesign recommendations: when users misinterpret design elements, you need to change them. Even better, you usually learn why users guess wrong about some parts of the UI and why they find others easy to use.”
With users thinking out loud during a moderated or unmoderated field study, you should have ample context into what they’re thinking and why they’re behaving a certain way.
Once you’ve put together an idea of how you want your researchers to conduct these remote field studies, you can start looking at the other crucial question. Who should you recruit to participate?
Who Should You Recruit to Participate?
When you’re recruiting, your sample of users should cover everyone you want your ecommerce store to target — visitors, buyers, customers, influencers, decision makers (e.g., spouses, parents, etc...).
If there are decision makers or influencers who aren’t as comfortable with ecommerce (e.g., grandma buying a gift), make sure you include at least a person who isn’t as fluent with computers and standardized UI/UX (see tip #7). You also want to make sure you don’t oversample a certain type of user, like regular customers, or eager participants.
Remember, the point is not to make the field study go smoothly — it’s to learn about how people actually use your website.
Be prepared to spend a little bit of money recruiting and incentivizing users to participate. The Nielsen Norman Group found the average per-user cost to be $171, but with variations depending on location and profession.
That might sound expensive, but remember that recruiting is an investment. For example, you could create a segment of customers for people who are willing to participate in future tests or ongoing testing.
Companies like UserTesting.com help with recruiting test participants and recording their screens and mics. (If you’re really curious, try a free 5-minute version of their service). You could also try services like Try My UI, UserBob, Userbrain, UserFeel.com, and Userlytics.
Alternatively, if that cost is a dealbreaker, you could also reach out to your current customers to participate in the test in exchange for an incentive (e.g., discounts, gift certificates, etc.) to keep costs down.
What Should You Observe and Ask?
If you’re conducting a moderated remote field study, you’ll want your researchers to stay quiet (especially in the early stages) so that they don’t bias the users.
Be cynical with what the users report — instead, watch what they do. You can identify prospects for usability improvements when you see and hear users’ sticking points, frustrations, or even the breaking points when they give up.
You should record the user's face and the computer or mobile screen at the same time. That synchronization will enable you to correlate changes in facial expressions and emotion as users interact with different elements on the site.
Screen and webcam recording can be useful, but there are also advanced software tools that can do this in real time, like UserZoom or Affectiva. However, they can get very expensive (UserZoom starts at $25,000/year) and are best for more experienced teams.
When people answer questions, most of them will — consciously or subconsciously — mold the truth to be closer to what they think you want to hear, what “should” have happened, or what’s socially acceptable.
If a user is sharing feedback after an unmoderated field study, just remember not to take all their recollections at face value. Memory is malleable, and many users can’t remember details at all. As Jakob Neilsen writes:
In reporting what they do remember, people rationalize their behavior. Countless times I have heard statements like "I would have seen the button if it had been bigger." Maybe. All we know is that the user didn't see the button.
As your researchers start understanding usability and problems more, they can start asking questions in subsequent sessions with users. The Nielsen Norman Group shares four rules for asking questions:
- Ask open-ended questions. Skip the questions that can be answered with "yes" or "no." In other words, ignore the “how”, and ask questions that start with W (who/what/when/where/why).
- Avoid leading questions. It might be natural to improvise leading questions without realizing it, so try to prepare as much as possible prior. Take extra time to think before you speak, and consider what you say.
- Speak in simple language. Don’t use confusing words and jargon.
- Don’t point out specific issues or areas you want to emphasize. The second you do, they’ll notice it much more and it’ll bias their experience. It’ll also tweak their recollection (even more than it would’ve been).
If you’d prefer to have a user fill out surveys, make sure that they can answer questions immediately after the test. This information will be more reliable because their memory is fresh, and because they’ll actually fill out the survey.
When users ask you and your researchers questions, you must figure out how to respond. Was it a real question, a rhetorical one, or were they just thinking out loud?
You can respond by repeating what they said (as a question, not mockery), encouraging them with a non-threatening question (e.g., “What would you do if you were really doing this on your own?”), or forming slow, incomplete, sentences and letting the user complete the sentence. The Nielsen Norman Group calls these three techniques Echo, Boomerang, and Columbo, and share audio examples of how these sound.
Sometimes it makes sense to probe further, but most of the time it probably makes sense to err on the side of silence. You should use that time to figure out if these user’s comments — or even just sounds like “hm,” and “err,” — are enough to indicate that they’re confused, or if they’re just trying to fill awkward silences.
How Do You Make Users Comfortable?
I know I’m being a broken record here, but this is of utmost importance: make sure that users know they’re encouraged to think aloud.
Build their confidence quickly. When users are first getting started, they’re going to feel the pressure to “perform well” — no matter how many times you tell them you’re not evaluating them.
As such, you should build a quick, small, task into the beginning of the session. This small win will help them get more comfortable with thinking aloud. The Nielsen Norman Group shares an example of a simple task (tip #189), “In today’s session, you’ll be working with Acme’s website. Go ahead find the site as you normally do.”
If you really want to make it a good experience for the user, make sure you and your researchers familiarize yourselves with the “Participant Bill of Rights” (starts right before tip #184).
Before users start the test, reassure them you’ll protect their identity and that nobody will be judging them (including themselves). Let them know how you’ll protect their identity by emphasizing anonymity, and making sure their names are separated from data. That way, they’ll know that no matter how silly or dumb their mistakes sound, it’s very unlikely anyone will remember.
How Do You Create a Happy Ending for the User?
Wow! The field study session is already almost over. Don’t rest on your laurels just yet, though. The peak-end rule, made famous by famed psychologist Daniel Kahneman, states that our memories of an event are an average of the high point of an experience, and the ending of it. It’s very important to finish strong and on a high note.
Your incentive shouldn’t be your only expression of gratitude and thanks. Wrap things up by saying thank you if you’re in a live moderated remote field study session, or in an email through an unmoderated one. Ask if they’d like to be involved in future studies or if they could provide referrals.
People appreciate compliments (sometimes, even more than money). Regardless of who gave you the insights and feedback you needed, thank them all for their energy and time.
They could’ve been anywhere on the web, but they spent it with you.
Final Thoughts
The concept of field studies sounds simple enough. But, as with all jobs involving other people, there are actually a lot of nuances to executing on it well.
Make sure you know what you want to learn from the field study, recruit a representative sample, observe closely and ask sparsely, make everyone comfortable, and show them how much you appreciate their time and energy.
And if you’re ever thinking twice about spending money, remember that advertising is an expense, and usability is an investment.
