Our objective at MyBusinessLab is clear:
What it is (and why we need Business Experimentation)
Experimentation in business refers to the practice of conducting controlled trials to test hypotheses (related to new ideas, strategies, processes or products) before implementing them at scale.
It is a "forward-looking" data-driven approach, enabling organizations to make more informed decisions about what works, accelerate learning (by testing many, many ideas) and minimize the risks associated with adopting new initiatives.
It allows us to prioritize initiatives based on real evidence of the value they bring, test many more ideas, with limited risk, focus or scale those that work and discard those that add less value.
It therefore stimulates curiosity, creativity, passion for learning, the search for continuous improvement and possible change, and respect for evidence as fundamental values of your company's culture.
Experimentation is the way knowledge is built in the sciences, and it can now be imported, with low complexity, into the business context.
Experimentation in the organizational context can be applied in different areas, such as marketing, product development, operations, human resources and more.
An experiment only generates reliable and replicable results (i.e., it provokes true learning) if it is constructed and executed rigorously, and on representative samples of the population to be analyzed.
At My Business Lab we work with a 5-step experimentation methodology, with a loop format, where each experiment is designed, executed, the result is analyzed, learned and leads to concrete actionables and new experiment questions and opportunities.
In their most common form, experiments are AB TESTING (or "RCT", randomized controlled trial).
We construct a representative sample divided into a treatment group and a control group on which we will test a hypothesis regarding our value proposition.
We expose the treatment group to a modified version of the value proposition ("variation").
We measure the impact of variation on a target metric (and on complementary secondary metrics).
But experimentation is much more than AB Testing. According to the business problem and the organization, we work with more than 10 different experiment formats (multivariate, quasi-experiments, multi-armed-bandit, among others).
Myth
Experimentation kills intuition and good judgment.
Reality
Intuitive ideas, based on good judgment and common sense, are good sources of hypotheses with which to approach experiments, and guide decisions once results are obtained.
Myth
Experiments are only good for incremental changes and take "disruptive" changes out of focus.
Reality
Continuous iteration of cumulative incremental changes shows consistently superior results for companies than inaction waiting for disruptive changes as the only form of progress. However, experiments also test large changes.
Myth
We do not have sufficient hypotheses to test.
Reality
An executive may be intimidated by the idea of companies testing 25,000 experiments a year, such as Booking.com. But organizations start small, with few experiments and build up speed based on their capacity.
Myth
Traditional or non-digital companies cannot experiment.
Reality
Although experimenting is easier in digital environments, it is entirely feasible to run experiments in "100% physical" and B2B companies. Experiments with larger expected effects (requiring smaller samples) and assistance from relatively simple data techniques are required.
Myth
Experimentation does not bring results.
Reality
Studies at scale on thousands of companies show positive impact on different business metrics when adopting experimentation (Konig and Hasan, HBR, 2020).
Myth
We already work with Big Data, we don't need experiments.
Reality
Data complement and assist experimentation. They tell us about the interaction of our customers with the current value proposition (they lose predictive power in the face of innovation) and provide correlations on which we build hypotheses. Experiments seek to detect and measure causal relationships (and therefore predictive power).
Myth
Running business experiments is unethical.
Reality
As long as they follow protocols regarding confidentiality and no harm to participants, testing business variants does not present ethical objections. On the contrary, it allows to innovate and detect improvements for the user, with controlled and limited risks.
Complete our test and know in a few minutes your starting point. From there, we will work together on the road to becoming an experimental organization.
Buenos Aires, Argentina. 2024