Manscaped, the men’s grooming brand, scaled to $300 million in three years across 39 countries thanks, in part, to user testing.
For founder Paul Tran, the disconnect between what investors assumed men wanted and what men really wanted was the white space. So he validated his hypothesis in the cheapest way possible— by asking his friends.
“Each iteration was driven by customer feedback,” says Paul.
And he’s not alone in betting on it. The global user testing platform market is forecasted to reach $6.3 billion by 2033, with retail and ecommerce among the fastest-growing end-user segments.
This guide covers what user testing is and methods to conduct tests for your product or store.
What is user testing?
User testing is the practice of watching real people use your store, product, or prototype to find out what works, what confuses them, and what makes them quit.
Participants attempt specific tasks like completing checkout or choosing a size, while a researcher observes. The output is often qualitative: it’s what shoppers do, say, and misunderstand, expressed in their own words.
The research method sits inside the broader category of user research, alongside surveys, interviews, and analytics.
Note: User testing, the methodology, is not the same as UserTesting.com, the commercial platform. UserTesting is one of several tools that help you recruit participants and record sessions, alongside Maze, Lookback, UserBrain, Useberry, and others.
User testing vs. usability testing vs. user acceptance testing (UAT)
Each one tests a different thing at a different stage of building or shipping a product.
| User testing | Usability testing | User acceptance testing (UAT) | |
|---|---|---|---|
| What does it answer? | Does this product, idea, or experience meet what real people want, need, and understand? | Can people complete specific tasks on this design without friction or confusion? | Does the final build meet the documented requirements and work as intended in real conditions? |
| When to run it | Before, during, and after design; especially valuable in early discovery, before committing to a direction. | After a design or prototype exists, and before development is finalized; repeat after major changes. | At the end, after development is complete and before public launch or release. |
User testing
A user test provides a broad look at a product. In completing a user test, you might ask questions like whether the product has a useful purpose in a customer’s life and what emotions the user associates with the product.
User tests help teams develop products that meet customer expectations. User testing methods include surveys, interviews, focus groups, observational studies, and usability testing. User testing can be moderated or unmoderated, meaning you may or may not need someone supervising test participants.
As freelance UX consultant Jesmond Allen says, generative user research—testing before you start designing—is “brilliant for ensuring you make decisions based on facts rather than assumptions.”
Usability testing
Usability tests are a subset of user testing, focused specifically on how easy or difficult it is to complete various tasks with a product. As such, usability tests are largely moderated or supervised by product research team members.
A combination of user and usability testing helped Lindsay McCormick turn her eco-friendly personal hygiene product brand, Bite, into a smashing success. “In all of my preparation and research, I wasn’t thinking, ‘Is this a good business idea?’” Lindsay says.
Instead, she asked herself: “How can I make the best tablet possible?” Lindsay’s goal was to make toothpaste in tablet form, and testing was a key part of her development process.
User acceptance testing (UAT)
UAT is the final check before a product, feature, or update goes live. The test is run with people from the intended audience—often internal stakeholders, beta customers, or a representative sample of end users, who confirm whether the build does what it’s meant to do in real conditions.
UAT acts as a structured pass-fail review against specific requirements, run before deployment to catch anything that fell short between design and delivery.
Why is user testing important?
User testing provides several key benefits for your product:
- Meeting user expectations
- Identifying unexpected issues
- Lowering development costs
- Validating willingness to pay
- Reducing post-launch failure risk
Meeting user expectations
If you’re releasing a new iteration of an existing product, user testers may have high expectations. A new offering needs to meet or exceed those expectations. A user test can let you know if you’re on the right track.
Jesmond says that it’s not enough to know what your customers want to buy: “How, exactly, do your customers narrow down their options from your wide range of excellent widgets?
“If you run research to discover this, you can prioritize your designs, showcasing the different widget features in your users’ journeys at the point at which they are most helpful.”
Read: User Journey Maps: What They Are and How To Make Them
Identifying unexpected issues
A series of user tests can reveal usability problems designers didn’t expect. You can use test data to see if people stay happy with your product over time or if it drops off as people spend more time with it. The sooner you catch these types of problems, the cheaper the fix will be.
This kind of discovery is hard to do from the inside. As Lindsay says, the work is to “find the confusion that a customer would have with your product and figure out how to close that gap, and have it make sense to them.”
Lindsay built her eco-friendly toothpaste tablet brand into a multimillion-dollar business by treating that confusion gap as the primary thing to test for.
Lowering development costs
A user test can keep product development process costs in check, because it can identify problems before you make the product.
If you’re testing a physical product, you could make a prototype, ask people to interact with it, and collect feedback. If you’re making a digital product, like a website or an app, you might want to start with paper prototypes (paper drawings that simulate a desktop or mobile screen) and only go full code if the research indicates that people will like it.
Across more than 21,000 product launches studied by NIQ BASES, products that disappoint customers at launch have a roughly 5% chance of succeeding in the market. That’s a lot of money to spend on something that might not get a second chance.
Validating willingness to pay
There’s a difference between what people say they’d pay for something and what they actually pay when their card comes out. This is part of determining product-market fit.
As Erika Hall, director of strategy at Mule Design and author of Just Enough Research, says, price is “a signal to your client’s customers about quality, intended use, and intended audience. This is why one brand of dental floss is $9 on a luxury wellness website and another is $1.79 at a chain drugstore.”
Erika further says “perceived value depends on context,” and user testing can surface the context shoppers bring to a purchase.
Reducing post-launch failure risk
Not every product flop is a demand problem. Sometimes the product is what people want, but the execution—the packaging design, the price, the page copy, the way the experience is framed—gets in the way. User testing helps separate the two before launch.
The way to do it is in small rounds, not one large study at the end. As Jakob Nielsen says in his foundational NN/g work, most usability problems surface within the first five users you test, and you’ll catch more issues by running multiple small rounds than by testing one big group once.
Types and methods of user testing
You can use several different methodologies to gather insights:
- Exploratory testing
- Comparative testing
- Usability testing
- Surveys and interviews
- Diary studies
- First-click testing
- Card sorting
- Moderated vs. unmoderated testing
- Remote vs. in-person testing
- Quantitative vs. qualitative testing
Exploratory testing
Exploratory testing is a largely unmoderated method used during the early stages of the product development life cycle. Participants interact with a product that feels natural, and the resulting data reveals user expectations, behaviors, and needs that inform product design decisions.
Yale’s UX team describes the method as letting users “freely engage with a product or interface without specific instructions,” which makes it especially useful for uncovering unexpected behavior.
This is the work that happens before design starts. As Jesmond says, generative—or discovery—research “happens before you embark on design work. It’s about really understanding your users’ needs and thought processes before you start designing for them.”
It can be run remotely, with participants using the product in their own environment. This makes it useful for testing websites or digital products with wide audiences, because it widens the recruitable pool and lets testers spend real time with the product before forming a verdict.
Comparative testing
Also known as A/B testing, comparative testing pits two or more product versions against each other. Two user groups receive different versions of the same product and report their respective findings.
This method can point to which product version is more effective or user-friendly.
For Shopify store owners,Rollouts, launched in the Winter ’26 Edition, brings comparative testing natively into the admin. You can stage theme changes, set a traffic percentage, and watch how each version performs against your live store before committing to a full launch.
Usability testing
Usability, or task-based, testing gives participants specific tasks to complete while interacting with a product. This test lets designers see how well their product helps users accomplish these tasks.
For example, in large-scale usability testing across 19 ecommerce sites, including Amazon, Best Buy, Ikea, Macy’s, and Zappos, Baymard Institute recorded task completion rates as low as 10% to 30% when participants were asked to find fairly common products: a sleeping bag for cold weather, a spring jacket, or a camera with a case.
Surveys and interviews
You don’t always have to interact with a product in surveys and interviews. Instead, you could ask people what they expect from your product. You can use each answer to help your team create products that customers will love.
The two methods answer different kinds of questions. “If you want to know how many of your customers have children living at home, a survey might be good,” says Erika. “But if you want to know the top three barriers to purchasing your products, interviews are a better choice."
Diary studies
A diary study involves users recording their interactions with a product over time. This method is particularly useful for products that are used regularly, like toothpaste, because the user’s experience is captured over time rather than just during testing.
According to the Nielsen Norman Group, diary studies are “useful for exploring a wide variety of research questions about long-term experiences and repetitive activities,” which is what makes them a poor fit for one-off transactions but a strong fit for products with ongoing usage, like subscriptions.
First-click testing
A website builder uses first-click testing to determine how useful and intuitive the structure is. The idea is to show participants a website prototype and ask them where they’d click first to accomplish a task. The first click they make is analyzed to see if it gets them closer to their goal.
The goal is to match the website structure with users’ intuition.
In a canonical study by Bob Bailey and Cari Wolfson, first conducted on CDC.gov in 2006, users who clicked correctly on their first click completed the task 87% of the time.
But users whose first click was wrong succeeded only 46% of the time.
Card sorting
Card sorting is a user testing method for site structure and information architecture. Participants group items by categories, menu labels, and content topics based on how they feel makes sense to them.
The method, the Nielsen Norman Group says, “can help your users find the information on your site more easily” by surfacing the gap between how the team that built the site organizes content and how shoppers expect to find it.
In an open card sort, participants create and name the groups themselves; you learn what categories shoppers naturally form and what they’d call them. NN/g recommends open card sorting as the default approach because it gives you a holistic view of users’ mental models.
In a closed card sort, you give participants pre-defined categories and ask them to sort items into your existing structure; useful for validating a taxonomy you’ve already built, though tree testing is often a stronger fit for that.
Moderated vs. unmoderated testing
In moderated testing, a researcher runs the session live, either in person or over video, guiding participants through tasks, asking follow-up questions, and probing when something interesting happens. The output is depth, showing what shoppers did and why they did it.
A common moderated approach is think-aloud testing, where participants narrate their thoughts as they use the product. Jakob calls it “the number one usability tool” for surfacing the assumptions, hesitations, and misreadings that drive behavior.
In unmoderated testing, participants complete tasks on their own using a platform like Maze, Lyssna, or UserTesting.
The trade-off is scale and speed—you can run dozens of sessions in the time it takes to schedule one moderated study, and the data is more easily quantifiable. But the cost is depth, because you see what shoppers did, but not always why.
Remote vs. in-person testing
In remote testing, participants complete sessions from their own devices in their own environments, using a platform that records screen, audio, and sometimes video.
The advantages are scale, cost, and authenticity. You can recruit globally, run more sessions for less money, and watch shoppers interact with your store the same way they normally would.
In-person testing brings participants into a controlled setting like a lab or an office, with a moderator running the session face to face. But the cost is higher once you budget for travel and facilities; the recruitable pool is narrower; and the data takes longer to gather.
Nielsen Norman Group makes a specific case for in-person testing, arguing it earns its cost through building rapport with participants, minimizing distractions, observing subtle non-verbal cues, providing a full sensory experience, and accommodating multiple devices and platforms in one session.
Quantitative vs. qualitative testing
Quantitative testing measures what shoppers do, at scale. This includes, but isn’t limited to, completion rates, time on task, error rates, click paths, conversion lifts; anything that produces a number you can track across hundreds or thousands of sessions.
Qualitative testing measures why shoppers do what they do. This includes interviews, moderated usability sessions, think-aloud studies, and open-ended survey questions; anything that produces narrative, reasoning, or emotional response rather than a number.
Jakob, the pioneering UX researcher who founded the discount usability movement, argues that companies should prioritize qualitative research over quantitative research.
“Knowing that 73% of users failed to complete a purchase is a useless statistic for a designer,” he says. “Knowing why they failed is everything.”
Nielsen recommends a 90/10 split of UX research budgets toward qualitative for mature teams, and closer to 100% qualitative for organizations still building a research practice.
Read: Market Research Guide: How To Conduct Market Research
How to conduct user testing
Jakob says that at its core, user testing has two components: “(1) watch users while they (2) perform tasks.”
Here’s a step-by-step method to use in your own user testing:
1. Define your objectives and write a user testing plan
Identify what you want to measure, such as usability optimization, user preferences, or task efficiency. Then document the objectives, participants, tasks, and metrics in a one-page document before recruitment begins. The plan is also what you’ll send to stakeholders, so everyone agrees on what success looks like before the first session runs.
2. Select a user testing method
Consider whether you want your tests to be moderated or unmoderated, in-person or remote, and long-term or short-term. For instance, long-term user testing might best match with diary studies, while short-term testing might benefit from an A/B test.
3. Design the test
Develop the questions and tasks that participants will respond to or perform during the test. These should align with your objectives and mimic real-world usage. Name the metrics you’ll track, such as the time it takes to complete a task or how often a participant uses an aspect of the product.
4. Recruit participants
Identify and recruit participants who match your target audience. If you’ve created buyer personas for your product or service, make sure your test subjects match those personas. Nielsen recommends five to eight participants per user group for qualitative studies, and at least 40 for quantitative ones.
5. Conduct the test
Make sure the participants understand the tasks and are comfortable with them before running the user test. Take notes, record the session if possible, and observe their behavior.
6. Analyze what happened
Review and analyze the test data, but resist pulling a few quotes and calling it good. Look for friction points, task success patterns, and sentiment themes across sessions. Then, validate findings against real user behavior in your analytics. If five test participants struggled to find your shipping policy and your live data shows the shipping policy page has near-zero traffic, you’ve got converging evidence.
7. Put your results to work
Translate each prioritized issue from step 6 into a specific design or copy change with an owner and a deadline. Then build re-testing into the same cycle: when the change goes live, run a smaller round of testing on the specific element you changed, to confirm the fix did what it was supposed to do.
In an episode of Shopify Masters, Susie Harrison, co-founder of Hearth Display, says, “Getting those Figma screens in front of individuals and not just asking them to give qualitative feedback on what that looks like, but actually asking them, ‘Would you be willing to pay for this solution?’ was really important for us to home in on what those go-to-market features were going to look like.”
Hearth Display used that market validation approach to raise $2.8 million in institutional seed funding for its family organization product.
Sample user testing questions
The right question pulls out information you couldn’t have anticipated. The wrong one confirms what you already thought you knew. Three principles separate the two, all grounded in Nielsen Norman Group’s research on open-ended questions:
- Open-ended over closed. NN/g recommends favouring questions that begin with “how” or “what” over “do” or “did,” because closed questions stop the conversation and miss the motivations, mental models, and concerns you didn’t know to look for.
- Specific over generic. “What did you expect to happen when you clicked there?” pulls out a concrete moment.
- Concise and non-leading. NN/g warns that leading questions interject the answer you want to hear into the question itself, making it awkward for participants to express a different view.
In Just Enough Research, Erika organizes research questions by what they’re trying to answer. She identifies four types: generative or exploratory, descriptive and explanatory, evaluative, and causal.
The examples below adapt that framework to the questions you’d ask shoppers at each stage.
Generative questions: What’s the problem worth solving?
These are for product discovery interviews and early-stage research, before a prototype exists.
Hall’s sample questions from a museum-website design project give the shape: “Tell me about your job. Walk me through a typical week in your life.”
Here’s how to adapt them for ecommerce:
- Walk me through the last time you [completed the task your product addresses].
- What’s the most frustrating part of how you currently do this?
- What have you tried before? What did and didn’t work?
- Tell me about the moment you decided you needed a better solution.
Descriptive questions: What’s happening, and how?
For surveys, interviews, and behavioral observation, when you need to understand the current state of how shoppers behave:
- How often do you shop for [category] online versus in-store?
- When you buy [product type], what do you usually compare before deciding?
- What does a successful purchase look like for you?
- What’s the last thing that made you abandon a cart?
Evaluative questions: Is this design working?
For prototype testing, usability sessions, and pre-launch concept validation:
- What do you think this page is for?
- What would you do first?
- Where would you click to [accomplish the task]?
- What did you expect to happen when you clicked there?
- If you were spending your own money on this, what would you want to know before you bought it?
Causal questions: Why did that happen?
For follow-up interviews, diary studies, and post-purchase research—when you’ve observed something and need to understand why:
- You hesitated at this step; can you tell me what was going through your mind?
- Walk me through how you used the product this week.
- Was there a moment when it didn’t do what you wanted? What happened next?
- What would you tell a friend who was thinking about buying this?
User testing examples for ecommerce
Below are four examples of how Shopify store owners and Shopify-built tools have applied user testing methods to real problems, with real results.
Each takes a different testing approach, which is the point—there’s no single right way to test a product or a storefront.
1. Iterative product testing (Suri)
Suri, the London-based brand that makes sustainable electric toothbrushes, treats testing as the entire development process rather than a final check before shipping.
Co-founder Gyve Safavi says, “We did quite a few iterations, probably like 20 iterations, before we launched. We conducted testing with dentists and sustainability experts early on.”
The methodology had two distinguishing features. First, the testing population: dentists for clinical credibility, sustainability experts for materials and life cycle questions. Each group surfaced problems the other wouldn’t have.
Second, the iteration count: 20 rounds is far above the industry default, which tends to treat product testing as a final pre-launch checkpoint rather than the operating mode of development.
“The main thing for us was that we needed to achieve minimal lovable product,” says Gyve in an episode of Shopify Masters.
Suri scaled to $30 million in sales within two years and secured retail partnerships with Boots in the UK and Erewhon in the US.
Read: Product Testing: A Complete Guide for Retailers (2026)
2. Quant-plus-qual product-fit testing (ThirdLove)
ThirdLove, the San Francisco lingerie brand co-founded by Heidi Zak in 2012, built its entire product line on the premise that 80% of women wear the wrong bra size. To validate that claim and design a sizing system that fixed it, the team needed something more rigorous than a focus group.
Hundreds of women came to ThirdLove’s two-story walkup, undressed, and tried on sample bras while the team gathered fit feedback in person. They paired the qualitative input—what fit, what pinched, what gapped, what felt wrong—with quantitative measurement of where existing sizing systems were leaving women between sizes.
The result was the half-cup. “Shoes have half sizes, now bras do too,” Zak says.
ThirdLove launched with twice as many sizing options as traditional brands, and the data validated it: the company’s virtual fitting room has now helped more than 18 million women find their size, and 65% to 70% of them receive recommendations for a different size than the one they were wearing.
3. Post-purchase surveys (SilkSilky)
SilkSilky, a Chinese brand selling ultra-premium 6A-grade mulberry silk pajamas, robes, and bedding, needed a way for tools to gather customer feedback at scale across multiple international markets. Email surveys were producing low response rates, not nearly enough data to act on.
So the team built a survey directly into the checkout flow instead. Shopify’s checkout page customization capabilities let SilkSilky add a user survey triggered immediately after a customer places an order, paired with a 15% discount to boost participation.
The response rate was more than three times that of email marketing. The survey gave SilkSilky real feedback on regional preferences that informed product decisions and helped the brand grow sales 680% in two years after upgrading to Shopify.
4. Accelerated checkout button testing (Everlane)
Everlane, the San Francisco apparel brand known for transparent sourcing and quality essentials, had a custom-built checkout that had become too complex to maintain.
But rather than replatform, Everlane tested Shop Pay as an accelerated checkout layer on top of its existing infrastructure. The hypothesis to test was: Would shoppers convert faster if they didn’t have to re-enter credit card and shipping details every time?
The result, based on Shopify’s internal measurement of Everlane’s performance, was checkout conversion rates up to 70%, with 15% of US transactions processed through Shop Pay within the first 30 days.
The most telling signal came when the team briefly removed Shop Pay checkout during testing. “The only feedback we got from customers was when it was gone,” said Anna M. Peterson, product lead.
Shopify’s accelerated checkout button testing tool lets you force every variation to display in your preview, so you can confirm each one works before any of those buttons reach a paying customer:
- Activate the payment methods you want to test in your payment settings; e.g., Shop Pay, Apple Pay, Amazon Pay, Google Pay, or PayPal.
- Go to your storefront from Online Store > Themes > View your store, and navigate to a product page or featured product section.
- Append a debug parameter to the URL to force a specific button to display. For example, “?shopify-debug=true&show=Amazon” forces the Amazon Pay button.
- Repeat for each payment method to confirm every configuration displays and functions as expected.
AI in user testing
AI is changing user testing in two ways worth pulling apart: who runs the sessions, and who participates in them.
The first is AI-assisted moderation—large language models (LLMs) conducting interviews, transcribing sessions, and surfacing themes across hundreds of responses faster than a human researcher can. But the risk is that AI summarization flattens the signals a skilled moderator would notice, like hesitations, contradictions, or body language proxies.
The second is AI-simulated users—synthetic shoppers that interact with a storefront before any real customer sees it. SimGym, launched in the Winter ’26 Edition, uses AI shoppers with human-like personas to test redesigns and new campaigns on a Shopify store. You can see how synthetic shoppers navigate, what they add to cart, and what they abandon before exposing the changes to real traffic.
The SimGym-Rollouts validation pipeline pairs synthetic signals with real traffic in two stages: first, real traffic, then synthetic signals.
According to Shopify’s Business Strategy and Planning Survey (Q4 2025), AI-using store owners test MVPs and prototypes with users at about 11%, compared to 3% for store owners who don’t use AI.
This makes AI-adopting store owners roughly 3.7 times more likely to run real user tests as part of product validation.
User testing FAQ
What is the difference between user testing and usability testing?
User testing is a broad process used to understand how users interact with a product, while usability testing focuses specifically on how easy the experience is to use.
For example, remote usability testing may involve watching participants complete tasks on a mobile or desktop device while researchers collect feedback through screen recording software and a live test feed.
What are the different types of user testing?
There are several types of user testing, including moderated testing, unmoderated testing, A/B testing, prototype testing, and remote usability testing. Companies may run one or two tests to validate a feature quickly or scale into more tests using user testing websites with a global network of participants.
Both remote and in-person experiences can help teams gain valuable insights that improve the overall design process.
What is user acceptance testing?
User acceptance testing (UAT) is the final testing stage before launch, where real users confirm that the product works as expected. During this phase, testers often provide feedback on workflows, functionality, and edge cases to ensure the product meets business requirements and supports real-world use cases for many customers.
How do you conduct user acceptance testing?
To conduct user acceptance testing, teams first define success criteria and realistic user scenarios. Participants may need to answer screening questions before completing tasks on a mobile or desktop device.
Teams then review feedback, analyze different data types, and use findings to improve the product before release. Running new tests after fixes can help validate that issues were resolved successfully.
What questions should someone ask in a user test?
Good user testing questions help uncover friction, confusion, and opportunities for improvement.
Common examples include:
- “What did you expect to happen here?”
- “Did anything feel difficult?”
- “How would you improve this experience?”
Even a quick practice test with a website tester can reveal insights about how users feel, offering a unique perspective that internal teams may miss.
