Validating Product Ideas Part 2: Testing Your Solutions

Prototype testing should be an ongoing process during the development phase.  The first prototype should rarely be the last.

Before we get started, let’s answer a stupid question:  Why test a prototype?

• To refine prototypes and the solutions they offer even if this means throwing out our first ideas and going back to the drawing board (or sketch pad).
• To learn more about our potential customers and how they will interact with our product.
• To get an attitude adjustment (also known as changing our point of view). Maybe we have failed at solving the solution or maybe we didn’t quite understand the problem.

Now, let’s lay out a framework for testing prototypes.

1) Choose your Test Users:

In Part 1 of this series we interviewed potential customers and built our customer profiles or avatars which were a generalized view of our potential customer types and the problems that our product can solve.  This is the perfect place to start when looking for test users.

There is no set amount of users that you should test with, but the more feedback you receive, the more ideas you will get on how to meet or beat your potential customers’ expectations.

2) List your business goals.

Before Prototype Testing, define and write down the specific business goals you wish to achieve through your products solution.  Have this definition handy while observing test users.  Does the product successfully solve the test user’s problem? Does solving the user’s problem lead to actions that achieve your business goals? 

To achieve these business goals you will need feedback about several qualities of your product and its potential users:

• Look and Feel
• Functionality (does it do what it is supposed to?)
• Usability (How will it be used?)
• Pricing

*Note on Pricing:  Asking test users “Would you buy this product for $X?” will never give reliable results.  Being creative on how you test the price point of your product can start very early in the development phase.  For example, try asking: “Will you buy into my beta program right now at a discount?  I will send you product updates for free as we make improvements.”  This is one easy way to generate capital for your product while building a real-world testing team.  Remember to reward and acknowledge these early adopters often and make them feel like they are a huge part of the success of your project.

3) Define User Tasks.

Define a minimum of 2 user tasks: 1 direct task and 1 scenario task.

A direct task will tell the test user exactly what to do (Example:  Open Door with Product X). A direct task will have a specific outcome.

A scenario task will create a situation where the test user can explore ways to use the product (Example:  You locked your dog in the car. How will you use Product X?)  A scenario task will help users decide for themselves how best to use your product and will help you develop new uses for it while also defining its limitations.  Basically, a scenario task will help define how your test users think.

These scenarios may depend on the physical space in which your product will be used.  Try and recreate these possible situations with your test users.

4) Moderate and Observe

When prototype testing, lead the test user through the tasks you have defined.  This is important because your prototype is not yet a finished product and many features may still not be working.

Show…don’t tell.  Put the prototype in the test user’s hands and give them the basics.  Don’t get into deep explanations of your concepts about how the product should work.

Have them talk through the experience.  Ask them to explain what they are feeling and seeing while using the prototype.

Ask open-ended follow up questions when they are done.  For example:  Don’t ask “Would you buy this product?” Maybe use: “What would be a scenario in which you would never buy this product?.  Follow every question with “Why?” or “Can you explain that a little more?”  Get them to talk about their experience.

Have someone observe the test without participating.  If this isn’t possible try to record video of the test.

5) Conclusions

After you are done with the test, use a separate sheet of paper to define your prototype testing conclusions.  Divide them into 5 categories:

What worked

What did NOT work

New questions

New ideas

General observations

I’ve wrapped all of the above steps in an easy to use prototype testing worksheet PDF which you can download here.  You can print it and fill it out by hand or send it to your tablet or phone for digital input.

The framework we have created in this article will work for testing any kind of product, even if it includes a physical space (like a brick and mortar store) or a sales scenario where the product will be used.  Many hardware developers create prototypes with electronics to record how the test users interact with the product.  The possibilities are limitless but the same questions will need to be answered to test successfully.

Maybe you don’t feel like a “people person” and would like to outsource these face-to-face tests.  In a worst case scenario, get someone else to moderate the interviews but at least be there as an observer.  Remember, creativity is about connections.  The more you connect with your potential audience the more creative solutions you will come up with to solve their problems.

Test and test again, this is the answer!  After going through this process, refine your prototype, make adjustments and test it again using a new group of test users.  The more you test, the more likely your final product will serve potential customers’ and be a hit.