Hypothesis-Driven Development is based on the development of new ideas, products, and services with the aim of determining whether an expected outcome will be achieved or not. The cycle of formulating hypotheses, testing, and analyzing experiments is a continuous process that is repeated until a desirable outcome is obtained or the idea is deemed unfeasible. The main result of an experimental approach is measurable evidence and learning. (Barry O’Reilly)
The process of experimentation involves identifying problems or needs, creating hypotheses, and validating whether the solutions meet the desired expectation or not. With the experimental approach, it is possible to test solutions more rapidly and optimize the process of solving the right problems. Instead of creating solutions simply because they are innovative or because competitors have them, information, data, and testing are used to build the solution desired by consumers and relevant to the business.
The steps of the hypothesis-driven development process are:
Adapted and translated process from InvisionApp.
Identifying problems and opportunities
Start by identifying the problem or need. Use information based on real user research findings, product data, or insights. Identifying doubts and assumptions regarding the product can also help in the process of creating hypotheses. Be careful not to create extravagant solutions as it is very easy to engage in ideation exercises and come up with various solutions, but they may not be genuinely connected to the problem you want to solve.
The "How Might We" technique from IDEO is very useful for formulating questions and turning assumptions into questions. Applying this technique facilitates understanding the problem to be solved by the team and also helps initiate the discussion that will frame your possible solution. Start the "How Might We" technique by describing insights or problematic points, and then rephrase these insights into questions, beginning each sentence with "How might we...".
For example:
Problem: Users do not complete the order on the payment screen
From this problem, it is evident that users are interested in placing an order, but they end up abandoning the purchase flow during the final step. There are doubts about the reasons that may be causing this drop-off. Thinking about this, some examples of sentences using the "How Might We" model could be:
How might we make the payment process simpler? How might we make the payment process more reliable? There is no limit to the number of questions that should be produced. Try to think if there is any overlap in questions that address the same theme or even prioritize more strategic themes, as this can facilitate the definition of the correct problem for the team to focus on.
Understanding what hypotheses are
Hypotheses are statements that summarize the objective of the validation to be performed. The hypothesis is important to have an idea of the expected results of an experiment and helps to better understand its focus and vision. The hypothesis is a guess that a specific solution will produce an expected result. It is a statement made with limited knowledge about a certain situation that requires validation to be confirmed as true or false, enabling the team to continue its exploration or define the best solution for a problem. It is closely related to the process of experimentation, right?
Formulating hypotheses
It is important to identify a real user or business problem with supporting evidence to form hypothesis statements. Hypotheses can be formulated in various ways; however, it is important to include the following elements in the chosen format:
Solution and/or benefit User persona Expected result There are models that can be used to include these elements, such as:
a) 4-step Framework (Meclabs Institute)
If [Descriptive summary]
By [Removing/Adding/Changing]
Then [We will achieve a result]
Because [Consumer insight]
b) Framework (Invision)
We believe that [Creating this experience]
For [Persona]
We will achieve [Expected result]
Fictional example:
Problem: How might we encourage registered users to make their first purchase in the app? Hypothesis: We believe that by offering a discount coupon for the first purchase
To people who registered within the last month and have not made any purchases on the app
We will achieve a 10% increase in active users for the month
There are many tools and models that can support the creation of hypotheses; here are some:
Lean Product
Canvas User
Story Mapping
Service Blueprint
Empathy Map
Prioritizing hypotheses
After going through the process of problem and opportunity identification, formulating hypotheses, it is time to prioritize the hypotheses that will be validated. Some models can help stimulate discussions and decision-making regarding what should be done. Some factors to consider in decision-making are:
Perceived value and risk: Based on available information, determine whether the hypotheses can have a significant impact on the business and enhance results. Also, identify the risks associated with each hypothesis, such as negative user perception, technical application risks, potential financial losses, operational risks, etc.
Effort and implementation capacity: Number of sprints or time required to design and develop tests; this will depend on what the team determines as the necessary timeframe. Consider the testing volume capacity that the team can develop and manage. Depending on your team and company context, this can be a deciding factor in prioritization.
Strategy or OKRs: Reflect on whether the test helps optimize a critical flow and has the potential to impact a key indicator? In this case, prioritize tests that can directly impact your result metrics. The prioritization of hypotheses is crucial to invest the team's capacity and time in what really matters. However, even with a well-defined prioritization process and ideal criteria for your product, understand that there will still be inherent uncertainties in the experimentation process.
Creating experiments
By this stage, it is important to have completed the problem and opportunity identification, hypothesis formulation, and prioritization. To know all the necessary steps to implement an experiment, access this template here. In addition to the step-by-step guide, you also have access to a checklist of questions to create your own experiment.
Learning and building
The experimentation process helps teams gain rapid learning and build what really matters to the consumer and the business. The learning cycle: Build, Measure, Learn — is continuous and facilitates understanding what is important to the user and what still needs to be explored in the product.
Translated and adapted by the author from Research Gate
Tips
Not everything belongs in the backlog: Not every idea needs to enter the backlog; think strategically and invest energy in what will bring value to the business and the user.
Use data and evidence to decide what should be part of the backlog or not.
Don't rush to a solution: Focus on the problem, need, or opportunity identified to formulate the hypothesis; try to have in mind what you want to validate with the hypothesis, rather than the solution that needs to be defined.
Define success indicators: Although it is difficult to define expected results, try to use learnings you have already gained from the product and estimate what you expect to achieve with the hypothesis test.
Avoid putting tests live without understanding how you will validate the hypothesis, as this will make analyzing results and decision-making more challenging.
Create scenarios: Reflect and have at least an idea of what the next steps will be if the hypothesis is validated or not. Document and share learnings: Document the results for the team's learning and other teams. This can also prevent teams from conducting similar or identical experiments and promote more synergy.
References: