Our objective at MyBusinessLab is clear:

LATIN AMERICA TO BE A HUB WHERE MORE AND MORE ORGANIZATIONS RUN EXPERIMENTS AND LEARN FROM THEM.

What it is (and why we need Business Experimentation)

Experimentation in business refers to the practice of conducting controlled trials to test hypotheses (related to new ideas, strategies, processes or products) before implementing them at scale.

 

It is a "forward-looking" data-driven approach, enabling organizations to make more informed decisions about what works, accelerate learning (by testing many, many ideas) and minimize the risks associated with adopting new initiatives.

 

It allows us to prioritize initiatives based on real evidence of the value they bring, test many more ideas, with limited risk, focus or scale those that work and discard those that add less value.

It therefore stimulates curiosity, creativity, passion for learning, the search for continuous improvement and possible change, and respect for evidence as fundamental values of your company's culture.

Experimentation is the way knowledge is built in the sciences, and it can now be imported, with low complexity, into the business context.

  • We test a relationship that we believe to be true: a hypothesis. We seek to learn with certainty, to distinguish the true from the false, with predictive capacity.
  • We seek to detect causal relationships: a relationship where a treatment on A (i.e., an independent variable) produces an effect on B (dependent variable).

Experimentation in the organizational context can be applied in different areas, such as marketing, product development, operations, human resources and more.

the methodology

An experiment only generates reliable and replicable results (i.e., it provokes true learning) if it is constructed and executed rigorously, and on representative samples of the population to be analyzed.

At My Business Lab we work with a 5-step experimentation methodology, with a loop format, where each experiment is designed, executed, the result is analyzed, learned and leads to concrete actionables and new experiment questions and opportunities.

AB TESTING AND OTHER TYPES OF EXPERIMENTS

  • In their most common form, experiments are AB TESTING (or "RCT", randomized controlled trial).
    • We construct a representative sample divided into a treatment group and a control group on which we will test a hypothesis regarding our value proposition.
    • We expose the treatment group to a modified version of the value proposition ("variation").
    • We measure the impact of variation on a target metric (and on complementary secondary metrics).

 

  • But experimentation is much more than AB Testing. According to the business problem and the organization, we work with more than 10 different experiment formats (multivariate, quasi-experiments, multi-armed-bandit, among others).

AB TESTING AND OTHER TYPES OF EXPERIMENTS

In their most common form, experiments are AB TESTING (or "RCT", randomized controlled trial).

We construct a representative sample divided into a treatment group and a control group on which we will test a hypothesis regarding our value proposition.

We expose the treatment group to a modified version of the value proposition ("variation").

We measure the impact of variation on a target metric (and on complementary secondary metrics).

But experimentation is much more than AB Testing. According to the business problem and the organization, we work with more than 10 different experiment formats (multivariate, quasi-experiments, multi-armed-bandit, among others).

7 myths about experimentation

Myth

Experimentation kills intuition and good judgment.

Reality

Intuitive ideas, based on good judgment and common sense, are good sources of hypotheses with which to approach experiments, and guide decisions once results are obtained.

Myth

Experiments are only good for incremental changes and take "disruptive" changes out of focus.

Reality

Continuous iteration of cumulative incremental changes shows consistently superior results for companies than inaction waiting for disruptive changes as the only form of progress. However, experiments also test large changes.

Myth

We do not have sufficient hypotheses to test.

Reality

An executive may be intimidated by the idea of companies testing 25,000 experiments a year, such as Booking.com. But organizations start small, with few experiments and build up speed based on their capacity.

Myth

Traditional or non-digital companies cannot experiment.

Reality

Although experimenting is easier in digital environments, it is entirely feasible to run experiments in "100% physical" and B2B companies. Experiments with larger expected effects (requiring smaller samples) and assistance from relatively simple data techniques are required.

Myth

Experimentation does not bring results.

Reality

Studies at scale on thousands of companies show positive impact on different business metrics when adopting experimentation (Konig and Hasan, HBR, 2020).

Myth

We already work with Big Data, we don't need experiments.

Reality

Data complement and assist experimentation. They tell us about the interaction of our customers with the current value proposition (they lose predictive power in the face of innovation) and provide correlations on which we build hypotheses. Experiments seek to detect and measure causal relationships (and therefore predictive power).

Myth

Running business experiments is unethical.

Reality

As long as they follow protocols regarding confidentiality and no harm to participants, testing business variants does not present ethical objections. On the contrary, it allows to innovate and detect improvements for the user, with controlled and limited risks.

WHERE IS YOUR ORGANIZATION ON THE ROAD?

Complete our test and know in a few minutes your starting point. From there, we will work together on the road to becoming an experimental organization.

The CINEFYN Model: deciding in contexts of uncertainty

  • Cause and effect relationships , in contexts of uncertainty, are only known ex post.
  • Experimentation allows us to identify these low-cost, low-riskrelationships .
  • We test ideas, quickly detect what works and scale it up.
  • We become learning learning machines: ideate, test, scale.