Chapter 9: Testing and Experimenting

This chapter covers several topics related to an entrepreneur’s efforts to test their offering, to (in)validate assumptions about their business model, and to strengthen the product or service that they are developing.

Launching the Imperfect Business: Lean Startup[1]

New businesses need time to develop their identities, engage their target market, create and develop the right products, and sharpen their strategies. Startups don’t have fully developed and proven business models like established companies do. A business’s starting focus or idea may evolve into something else because the initial feedback from the prototype points in a different direction.

Take Dropbox, for instance. The company initially provided a short video description of the Dropbox product, without even having it coded to the actual program first.[2] In this way, they could test the business idea and receive customer feedback, without losing time or money on products that weren’t viable. Furthermore, if any changes were needed, they had the flexibility to make them, because it was just an early prototype and no big investment on the product had been made yet. Many of the early changes to Dropbox’s “minimum viable product” (MVP) were to the interface design that involved buttons and actions that could be taken, and storage capabilities that customers consistently gave feedback on.

Dropbox co-founders Andrew (Drew) Houston and Arash Ferdowsi knew that they did not have the perfect product. To gain insight into how they could improve upon their initial idea, they employed a system called the lean startup, a methodology that entrepreneurs use to help them innovate by continuously testing their offerings and getting feedback from customers in real time. This methodology calls for a product to be shared with early adopters in its early stages of development in order to gain immediate feedback. It ensures that people actually like a product and are willing to buy it. Tech companies have become enthusiastic users of this methodology, because they have learned that too many years working on a product without measuring desirability (i.e., the “field of dreams” philosophy previously introduced) is very risky – especially in a rapidly changing technological environment. The lean startup approach, though, is applicable to almost every business and industry. Let’s take a closer look at this approach and how it got its start.

Lean Startup Principles

Author and entrepreneur Eric Ries developed the lean startup methodology after sacrificing much time, effort, and money when his first tech company had to close. After he experienced his first business loss, Ries recognized the problem of companies devoting time and money on projects rejected by consumers, and he wanted to identify a solution. Fortunately, Ries’ frustration led him to study other successful companies and lean manufacturing programs, including Toyota’s highly successful lean manufacturing process. This taught him how to be flexible and quick while building products. He learned that he didn’t need to have a fully designed product resulting from years of development; instead, he could have something imperfect that worked without the bells and whistles, something that people could test and provide feedback on over time.

Ries then created a second company, called IMVU. IMVU is an avatar platform, or “metaverse,” for people who want to shop for clothing, furniture, and accessories in an online community and keep their identity safe. Essentially, IMVU is a virtual reality world with an e-commerce engine, user-generated content, and 3-D characters. While Ries was building IMVU, he and his co-founders worked endless hours for six months to create a prototype version of its 3-D avatars. The initial platform was designed to be a minimum viable product (MVP), which is a very early prototype of a product that is still functional (i.e., viable with the minimal time and cost).

According to Ries and the lean startup philosophy, an MVP should be the bare minimum to help people understand what the product or service is about. A product doesn’t even have to be built to be an MVP; instead, it can be represented in sketches, videos, and explanations of how it might look (more on this and other prototype examples a bit later in this chapter!). MVPs may also include a basic version of the product, just like IMVU’s website, which was not very good at first. The point is to have a basic product to show to potential users, with enough substance to elicit feedback on what people find useful and what attributes are important, without having to invest a lot of money and time. There is no right or wrong type of MVP – it is up to the owner to decide how to showcase the idea and test it to find out what people like and don’t like, and how to take the idea to the next level.[3]

Once he and his co-founders had a low-quality platform running, they decided to charge a fee for the service. However, they later learned that no one wanted this offering. During their first month, they made a total of $300. The next month they made $400, after begging friends and family to try it. When Ries saw IMVU’s faithful first customers disappear, he and his co-founders decided to talk to potential customers. They tested their product with teenagers and heavy users of technology, as well as with mainstream customers. The mainstream customers never knew what to do with the product, and they thought it was too strange. But those that were more tech savvy and younger were happy to try it.[4]

Ries enabled his teams to put the product through iterations – or small changes to the current version of a product to make it better fit consumer needs. At this phase, and using lessons from his first failure, he strove to create a version of a product that worked well enough to provide core value to the customer, garner customer feedback, and make small adaptations around what users considered most essential. This allowed IMVU to enhance the product in a way that brought the company, step by step, closer to providing the best value. In order words, customers feedback resulted in creating a better version of the avatar community.

For example, one iteration that proved enormously successful was adjusting how the avatar moved through its virtual reality world. The original avatars didn’t have the ability to walk around like they do in, say, popular multiplayer video games. Even though Ries knew this was a disadvantage, he sent the version to early adopters and asked for feedback. Customers responded that the lack of movement suggested the software was low quality. Reluctant to make the major tech investment for this movement, Ries and his team decided to try a very small iteration as an alternative: They made the avatar disappear from its starting point and reappear in its new location. Customers responded positively, seeing this as superior to what existed and as a high-quality variation. The quick and low-cost iteration turned out to be the best path to success. They sent dozens of changes to their loyal followers until they developed a product that would work for a larger customer base. These changes included adjustments such as adding new avatars, new avatar movements, choices of clothing, and new worlds for avatars to explore.

IMVU was a pioneer of what has since been referred to as the lean startup approach, which new ventures all over the world now use. By embracing customer feedback and an iterative process of building and learning, IMVU has grown to be a successful enterprise and the most adopted social metaverse with over 7 million monthly active users.[5]

The Build-Measure-Learn Loop

Based on what he learned through his entrepreneurial experiences, Ries developed a step-by-step approach for how to maneuver a startup and accelerate its expansion. This framework, called the Build-Measure-Learn loop, helps entrepreneurs develop their idea into a minimum viable product (MVP), measure its impact on people, and learn if there is a need to pivot (i.e., change course) or persevere (i.e., keep going). In other words, a prototype is first built (or at least the idea is written down). The offering is then pitched to (potential) customers, who provide feedback so that the company can learn what to keep and what to change. Based on what is learned, the company then makes changes to the prototype and starts all over again, repeating the process until the prototype is good enough to go to market. However, even after an offering hits the market, this loop can be used to continually enhance it. This iterative process is represented in Figure 9.1.

Figure 9.1          The Build-Measure-Learn Loop

Attribution: Copyright Rice University, OpenStax, under CC BY 4.0 license

The first stage of the lean startup approach (Build) involves developing an MVP that has only its core benefits and no extra features. For example, Dropbox founder Drew Houston and his developers created a basic platform for early adopters to test. Early adopters are people who like to try new products as soon as they come out. They don’t mind, for example, buggy software or cumbersome designs, because they are innovators and like to test new things. Dropbox did not create a perfect product initially, because the developers did not know what it would be until they got feedback from customers/users. After early signups and feedback on the first prototype, they were able to create the first version of the product. Later, they conducted iterations, or additions and changes to the initial version of the product, to incorporate their customers’ ideas and suggestions.

Zappos is another example of a successful business that launched with an MVP. Nick Swinmurn, creator of the online shoe company, launched the idea for Zappos in 1998 by creating a bare bones website to sell shoes and measuring traffic to the site. The idea started after he took a trip to the mall and could not find shoes in the styles or colors he wanted. He figured other people had equally frustrating experiences. His first hypothesis was that people were interested in buying shoes online. To test this hypothesis, Swinmurn created a plain webpage that started with basic photos of shoes and their prices. Swinmurn also ran ad campaigns, measured web traffic, and adjusted the webpage. Instead of spending a lot of time building a detailed plan about this business idea, Swinmurn used a lean startup approach and started with a simple, just like Dropbox did. This approach helped him to make quick adjustments to the product, the target market, and strategies using the build-measure-learn loop.[6]

By building the MVP, entrepreneurs can show potential consumers a sample of the product and see how they react. This low-quality product can help to Measure user reactions. Metrics (i.e., measurements) are developed to test the assumptions of your business model. In particular, the MVP helps an entrepreneur to test product-market fit by asking questions about the design and usability of the offering, the value proposition, and other factors that impact the customer experience. For example, different design attributes can be measured by asking customers if they like a feature or set of features of the product they are testing. The number of likes or dislikes can be tallied, and comments can be collected on what additional features should be added or deleted.

Finally, during the Learn stage, the entrepreneur uses the feedback obtained to assess the progress of the product in an objective manner. Feedback is critical in the lean startup process, as it supports the entrepreneur’s learning and supports the design of better products to better serve the venture’s target customers. Companies often work with early adopters for a while to learn as much as possible, before they expand their reach to a broader market with a better product that they believe they can entice more people to use.

Based on what an entrepreneur learns from customer feedback, they can make changes to Build a new MVP, and the starts the cycle over again. As an entrepreneur makes changes to their MVP based on customer feedback, they can also Measure whether the changes made to the offering actually help them to make progress toward developing a viable business. Ries advocates for using a minimal financial evaluation concept that is called innovation accounting, which aims to assess whether the changes made to an offering are creating the desired results. This means that entrepreneurs should not just look to the basic financial measurements traditionally used businesses (e.g., sales, profits, and return on investment) because these measurements aren’t necessarily the most relevant metrics for a startup at this stage. Instead, entrepreneurs should measure progress that directly test assumptions about the business, such as sign-ups, downloads, and retention rates. These data can be collected immediately to assess customer responses as product changes are made. This iterative lean startup process continues until the entrepreneur has reached a point where they either determine that they can accelerate and scale or that the venture needs to pivot.[7]

Pivoting

Pivoting is a crucial and often difficult change done to test a hypothesis regarding a startup’s offering, its growth potential, and business model. Once the product has been tested and retested for changes in customer behavior and/or an improvement on any metric that the company aims to capture, if it’s not attaining making progress, then the next step is to pivot instead of continuing down the current path.

This is a time for tough decisions: Should you continue working on the business, product, or project – or should you change? Is the company making progress toward its goals, or do the strategies need major adjustments? While measurable data are gathered during the measure stage, the lean startup process also has an aspect that is creative, intuitive, and visionary. In other words, knowing when to change cannot be determined solely through number crunching – it also requires human judgment. When coupled with the build-measure-learn formula, an entrepreneur’s vision can lift a company out of mediocrity.

If a business is not growing, it is either shrinking or stagnating.[8] A pivot, just like in basketball, requires keeping one foot down on the ground, and moving the other one. This means that you retain what you have learned from customers as your grounding foot but explore a different direction in which to move.

Here are several examples of pivots that famous companies have made:

• Twitter transformed into a social media giant from a company called Odeo, a broadcast platform that was initially created to transmit video, sound, and podcasts. However, Apple beat them by launching iTunes podcasting, and it didn’t make sense for the founders of the company to pursue that path. So they decided to create Twitter instead.
• PayPal (formerly Confinity) allowed people to send electronic payments or “beam” payments from their Palm Pilots (which were the first personal digital assistants created by Palm Computing) as well as from other devices. At that time, PayPal users registered a card number, downloaded the beam application, and were able to make transfer payments from one device to another. As technology changed, it become the preferred payment method on eBay, helping the company go mainstream.
• Avon was created by a traveling book salesman who would add perfumes to his sales as perks for his woman customers. Eventually, he realized that women were more interested in the perfumes than the books, so he started a new business from a small office in New York.
• Fab.com, a design e-commerce website, was initially called Fabulis, and it started as a social media site to connect gay men. After the social media site failed, the co-founders took it into a new direction by selling their design picks for the best home goods, accessories, and clothing for everyone.[9]

You may wonder, how many times can a company pivot? Ries likes to use a runway analogy: You must measure how long you have until your company can achieve its goals, and you either fail to lift (i.e., you burn through all your cash resources) or have a liftoff (i.e., generate sales, sign ups, add new customers). A startup’s runway can be calculated by taking the amount of cash that is in the bank and dividing that by how much is being spent or drained on that balance each month. So, if your company has $550,000 in the bank, and it drains about $50,000 per month (i.e., its burn rate), then you have a projected cash runway of 11 months (550,000/50,000 = 11). A way to slow down the cash drain (and to extend the runway) is to cut some costs or to ask investors for additional funding.[11]

There are also different types of pivots for different needs. Table 9.1 shows ten pivots an entrepreneur can make according to feedback.

Table  9.1           Pivot Strategies from The Lean Startup

Revisiting the Design Thinking Process

Prototyping[12]

A guiding principle of the Lean Startup methodology is the importance of getting early customer/user feedback and testing your assumptions with the use of a minimal viable product (MVP) or prototype. We were first introduced to the concept of prototyping earlier in the semester, when we learned about the design thinking process (see Figure 9.2). Prototyping, which begins after the ideation stage of the process, involves transforming your highest potential idea(s) into a simple version that can be used to get feedback from your target customers/users.

Figure 9.2 The Design Thinking Process

Prototyping requires getting ideas out of your head and into the physical world. Building prototypes means exploring, evolving, and communicating ideas in some shape or form, using people, paper, or pixels. IDEO, the renowned design thinking firm, suggest that during this part of the design thinking method, it is critical to embrace “a bias toward action” (in other words, rather than endless strategizing, planning, and debate). Your prototypes should be designed as rudimentary models that enable and inspire your target users/customers to engage with your proposed solutions, to discuss and ask questions about it, and offer their feedback. In the process of creating and sharing even early and very rough prototypes, you will have to think through your idea in practical and preliminary ways. Even a quick napkin sketch can be useful if it helps you to collect direct responses from potential end-users or inspires ideas to further improve and refine your solution.

Prototyping products, services, and/or customer experiences will quickly highlight possibilities and opportunities, as well as problems and mistaken assumptions that must be addressed sooner rather than later. As Google’s Jake Knapp writes in Fast Company Design: “Untested assumptions are like takeout containers in your fridge: If you leave them for very long, things get nasty.”[13]

A prototype can be anything that makes your idea tangible – this may be in the form of a wall full of post-it notes, a role-playing activity, an object, a web interface, a storyboard, etc. Fidelity, or the level of detail and functionality of your prototype, should be proportionate with your progress in your project. So in early explorations, you should aim to keep your prototypes rough and rapid, to allow yourself the opportunity to learn quickly and investigate a lot of different possibilities.

Here are a few key tips for developing a prototype of your product or service idea.

Tip #1: Make your ideas tangible.

Prototypes are most successful when people (the design team, the customer/user, others) can experience and interact with them. What you learn from those interactions can help drive deeper empathy with your user/customer, as well as provide meaningful insights that can help shape successful solutions.

You don’t know if an idea will work until you try to make it.  Creating mockups and models is the best way to make progress toward your goal, because the act of creating a prototype forces you to make decisions and choices, as you translate your idea into something tangible. In other words, it’s about building to think.

Tip #2: Start with low-fidelity prototyping.

Regardless of which format you select to express your idea, keep it simple. You should initially aim to make a version of your idea with the materials that you have at hand. Tim Brown, founder of IDEO, shared that based on his experience, the more “finished” a prototype is, the less likely the designer will be to welcome (and want to act on) constructive feedback. Similarly, customers may be reluctant to provide negative feedback on a prototype that looks very polished.

Low-fidelity prototyping encourages you to practice a highly flexible stance while developing solutions, enabling you to make changes on the fly, to learn along the way, and to incorporate those learnings into new ideas as you develop increasingly higher resolution models of your offering. Therefore, sharing raw representations of what you are building should start early in the life of a project.[14]

Tip #3: Make it – and break it – quickly.

Prototyping is not about getting it right the first time! The best prototypes change significantly over time. Another benefit of low-fidelity prototyping is that it enables you to detect and figure out ways to fix design problems early in your creative process. It is of course better to “fail fast” with a new idea before you get too far down the road, before you overthink, or waste resources on the wrong solution, or before you realize that you’re solving the wrong problem altogether. In addition, when you spend a long time designing a solution, you may grow more attached to it. As interaction designer Marc Rettig explains, “Spend enough time crafting something, and you are likely to fall in love with it.”[15] And as an entrepreneur, you want to fall in love with solving your customers’ problems, rather than being wedded to your own solution.

You’ll learn a lot by starting with simple expressions of your ideas, then later developing them to be closer to the final solution with each iteration. Keep a “parking lot” for questions that come up while you build future prototypes. You can revisit and answer them as you develop and refine your idea further. Remember, prototypes are only meant to convey your idea – they are not meant to be perfect! Embracing this principle helps you to move quickly through iterations of your offering by incorporating and building on what you’ve learned from your target customers, each time (hopefully!) getting a bit closer to a more desirable and viable version of your offering.

Tip #4: Plan ahead. 

Most concepts cannot be fully realized with just one prototype. Continuous iteration on your concept, requires various resources and capabilities, namely, money, time and people. As you work to make your idea a reality, you should consider what you will need to build out your concept. For example:

• What materials you will need to build your low-fidelity prototype? Do you have access to them?
• What people (customers, users, partners, etc.) will you need to get onboard to realize your solution?
• What special skills and capacities do you need to develop your offering?
• What networks of people will you need access to?
• How will you raise awareness of your prototype – and ultimately – your offering?
• As you progress toward higher fidelity prototypes, what funds might you need, and where will you get them?

Tip #5: Embrace prototyping as problem-solving.

Because prototypes are often somewhat unfinished, typically the initial model will fall well short of capturing either your complete vision or the perfect solution. Instead, prototypes are a tool to accelerate your learning, because they help to surface opportunities and limitations early on.  Your comprehension of both the problem and the solution will likely deepen, first as you engage in building your prototype and then again as you test it with your target customers and learn from their feedback. This makes prototyping a very efficient and effective method of problem solving.

Tip #6: Design your prototype to test your riskiest assumptions.

When developing your prototype, you should consider how it will help you to test your riskiest assumptions about your business model – in other words, where you have the most uncertainty. For example, you may have an assumption that your target customer experiences the particular problem that you have identified and is in fact motivated to do something to address it. To that end, your prototype should clearly illustrate the problem that you believe exists and how your offering solves it. You may have assumptions about the features that your product should include and the value proposition that you will deliver to your target customer. In this case, the prototype that you develop should clearly communicate that to the customer, so you can get feedback that will help you to refine your offering and messaging. You may have assumptions about who your target customer is, how you reach them, how they will want to pay for your offering, etc., etc. Think about what your major assumptions are and keep them in mind when considering the various types of prototypes that you could create.

Tip #7: Consider various types of prototypes.

There is rarely one way to best express your ideas to others. When you are ready to share your offering idea as a prototype, be creative and think through how you will build it. If you’re developing a physical product, perhaps you want to sketch it out as a drawing on paper, or create a digital rendering, or build it using paper/cardboard and/or other (cheap and easily available!) materials. If you are developing a service, maybe you could create a storyboard or simple video that illustrates the process through which your target customer experiences a problem and how your solution addresses it. Again, the type of prototype that you choose should effectively communicate your idea and engage your target customer in providing valuable feedback that will help you to test your assumptions and refine your offering.

Here is a (non-exhaustive) list of potential prototype options to consider. You can always find other options and great examples online for more inspiration!

• Share a website mockup
• Build a physical version of your product using available materials
• Perform a role-play
• Create a video or animation
• Present a storyboard
• Sketch out drawing of your product idea
• Sketch a wireframe of how an application works

Testing[16]

The general rule of thumb when developing a new or improved product, service, business model or experience is to test early and test often. That’s because everything that we think we know about our business model is inherently an assumption, and we always need to test our assumptions. It’s critical to find out if/how you are wrong, so that you can address the issue and develop a stronger, more viable product or service.

Gathering early-stage feedback helps to inform the next phase of decisions that you have to make as you iterate toward the final version of your offering.  To accomplish this, you need to “get out of the building”. You should aim to talk to your potential customers, listen to them, observe them, and understand how they feel. This reflects the empathy that is critical in human-centric design.

For that reason, it’s important to make testing an ongoing activity, so that you can develop an authentic understanding of how real people experience your offering and how they would recommend improving it. By conducting quick and continuous assessments, starting during the very first week of your project, you’ll increase the likelihood that you achieve the goals that you have for your startup.

Recruiting Testers

Testing involves identifying representative users/customers (i.e., from your target market) and learning more about them, so that you can build better products and services. However, depending on your timeframe or budget, and as part of the continuous cycle of testing your ideas frequently with as many people as possible, you may also find it useful to conduct some impromptu prototype testing with people you know. This approach is referred to as convenience sampling, and it can be an effective preliminary idea validation technique that requires minimal overhead. Fast, easy, and inexpensive research sessions involving people close to you can produce valuable informal feedback on your offering-in-progress.

To successfully recruit online testers who genuinely represent your future target customers, you will first need to decide who exactly you most need to speak to. To this end, preparing initial customer personas is recommended – they will help you to clearly define what kind of customers/users you most need to hear from, understand, and satisfy.

Recruiting Online. A relatively low-cost approach to recruiting prototype testers is to use online channels. One way to find relevant testers is to keep track of anyone who has expressed an interest in your offering and then create a go-to panel of “research” participants. When it comes time to run a test, you can send them a request via email or whatever channel you use to connect with them.

Another way to recruit online testers is through social media communications channels. First, select the specific platform(s) where you expect your target customers/users to be. When deciding between social networking platforms, consider the demographics of users on each channel. Some will skew young, or female, or professional－ so you need to know who you are looking for before using this approach. Once you’ve identified the most relevant platform, you can post a brief introduction for your project and provide a link to your test request (i.e., complete an interview, survey, etc.). When attempting to attract social media users to participate in a survey, it is recommended to focus on simple questions that can generate quick, clear responses. It can help to think of it in terms of a very specific problem or hypothesis that you want to address. For example, you might ask your participants, “Which messaging to you prefer?” or “What are your (least) favorite feature of this product?”

Using social media to run quick and simple prototype tests can deliver feedback on your idea within minutes. This quick, straightforward, brief, and (hopefully) honest feedback can help you move forward with developing a better, clearer offering.

In-Person Testing. In-person testing if often the preferred way to test a new prototype, because you have the opportunity to observe your target customer/user actively interacting with your product or service. This can generate particularly relevant and useful feedback for you to use as you refine your offering. To get the most out of an in-person test session, here are some general tips to consider.

Tip #1: Clarify your goals.

What do you want to learn? Identify your questions, concerns, areas of interest, and the purpose of the research. Also clarify the scope of the session. Be very specific about exactly what you are testing. Rather than attempting to test every assumption all at once, it can be helpful to plan out which elements of the product or service each round of testing will cover.

Related to this, you should also decide on clear metrics that you want to measure. This requires considering what a successful outcome will look like. Consider tracking completion rates, time on task, frequency of errors (critical and noncritical), total time spent confused, etc.- as well as subjective data such as user satisfaction, likes, dislikes, perceived ease of use and perceived success.

Tip #2: Come up with your script and questions ahead of time.

It’s important to know what you are going to say when you get your prototype in front of a potential customer. First, you should consider developing a “script” of sorts, by which you can introduce yourself, your product, and the testing activity. One sample product testing script goes as follows: “You’ve been invited to help us understand which parts of our offering work for you and which are confusing. The [product/service] that we’re testing is [insert 1-sentence description here]. Even though we’re calling this activity a test, you are not being tested – the product is being tested. There’s nothing you can do wrong. It’s not your fault if you can’t get something to work, and you won’t hurt anyone’s feelings if you say something bad about the product. It’s really important that you speak all of your thoughts aloud.”[17]

You should also decide on the focal tasks or questions that you have for your testers. During testing, you should ask participants to perform the key activities (i.e., tasks) that they would typically complete while using the product, service, or interface. For example: When testing a reservation app for a restaurant, you might ask the participant to “book lunch for tomorrow at noon for a party of six.” Additionally, here are some sample open-ended questions to get you thinking about what you might ask your testers:

• What would you expect to be able to do with this product/service?
• If you could change one thing about this offering, whether it is major or minor, what would be at the top of the list?
• Was there anything missing that you were expecting to see?
• Who do you think might want to use this service?

Tip #3: Consider the logistics of your testing situation (location, date, duration).

When selecting a location to conduct a prototype test, look for a setting with low or no distractions. Spaces such as the shopping mall food court, or a library common area might be too loud and hectic at certain times of the day. If at all possible, do your testing where people would normally interact with the product or service (e.g., home, work, etc.), so that you can observe them engaging with your offering in their natural environment.[18]

Decide on how many sessions you will conduct on given testing day and exactly how long each will last. Remember, testers may arrive late (especially if you do not send a reminder notification with directions to the test location), so leave a buffer between sessions.

Also consider any questions that you might need to conduct your test. For example, do you need a computer or internet connection? Do you plan to do video recording, and in that case do you need to think about lighting and a microphone? If you opt to conduct a test in the customer’s environment, make sure that your prototype will work in the field. For example, a physical version of your prototype has to be portable.

Tip #4: Focus on active listening and gratitude.

When planning in-person tests, it’s important to keep your questions simple and straightforward, and listen more than you speak. Don’t share too much information about your offering – instead, be patient and let testers find their own way. You want to speak as little as possible and let them do (all) the talking. By aiming to listen more than you speak, you’ll be less likely to resort to a confirmation bias, which is the tendency to hear and focus only on the feedback that confirms what you already think.

Prepare yourself to listen and be willing to pivot or otherwise adapt in response to the feedback you hear. The trick is to get very good at listening to your testers – try not to explain, get defensive, or feel defeated if they don’t “get it”. Instead, let the people you’re developing your offering for direct you to a preferred solution. You often can’t predict what users will say or do. Have the flexibility to change course if you receive new information or something goes completely wrong during your test sessions. Remember, the purpose of using prototypes to conduct testing is not to convince testers that you are right, but to learn from them!

Conclusion

This chapter covered why testing and experimenting is a critical part of developing a new product or service – that is why we see these concepts in both the Lean Startup and the Design Thinking approaches! It’s very difficult to fully understand another person’s problems, pain points, and perspectives until you put a prototype of your solution in front of them and observe their honest responses. And remember, you are mostly likely wrong about at least some of your business model assumptions – you just don’t know which one(s) yet!

When you build a prototype of what you think your solution should look and feel like, and share it with others, you are engaging in an active and iterative learning process. You are very likely to walk away with highly valuable insights into how you can improve, simplify, or pivot your solution. At the end of the day, the success of your business depends on developing a desirable offering for a clear target customer. This means that it is essential to put your ideas through systematic and varied testing. If you do this, you will better understand what your customers see, think, feel, and need and how to develop an offering that they will want and love to use.

Chapter 9 References