Source: Pinterest

Simpson’s paradox occurs when we observe a certain trend in the aggregate data but not in the underlying segments that comprise the data. In the A/B testing domain, Simpson’s Paradox can occur when the overall mean conversion rate and/ or average order value of the experiences tested point to a result different from the mean conversion rates and/ or average order value of the underlying segments.

Let me illustrate this with an example from Georgi Georgiev’s blog post, instructor at CXL. Suppose you run an A/B test between Page A and Page B and see the following results:

I am a big fan of the Experimentation Culture Awards, so imagine my excitement when I was invited to be on the jury! Here are my learnings.

Experimentation Culture Awards

For those who don’t know what Experimentation Culture Awards (ECA) is, the awards started in 2020 as an event to promote a culture of experimentation globally. The goal is to recognize people who are trying to grow an evidence-based decision making culture.

It is not about that one experiment, it is about improving the process, structure, trustworthiness, democratization and motivation for experimentation and data-driven decision making.

I wrote about my learnings from the…

What does driving social change and experimentation have in common? And what can we learn from this field to drive experimentation culture at organizations?

Photo by Pauline Bernfeld on Unsplash

Much has been written about building a culture of experimentation, including one of my previous posts on how to create experimentation as a habit. I was recently listening to one of my favorite podcasts, Hidden Brain by Shankar Vedantam, a social scientist. The episode was called “The Snowball Effect” and started off as follows:

“Why do some companies become household names, while others flame out? How do certain memes go viral? And why do some social movements take off and spread, while others fizzle?

I was immediately intrigued. Was there something I could learn that would help me understand why…

Source: Pinterest

I have been exploring how experimenters* work in and with product teams. Towards that end, my last post sought to demystify Scrum, especially its three pillars of transparency, inspection, and adaptation and covered how experimenters can align their processes with Scrum events and artifacts. This post will cover ways product owners can use experimentation to refine product backlog.

A product owner essentially creates a roadmap for a product with the aim of delivering value to customers. This roadmap known as the product backlog is a prioritized list of items that when developed is expected to deliver that value to customers…

This post covers some Scrum vocabulary and processes that product teams use and how they align with the process experimenters’ use.


Let’s start with a basic question — why do experimenters need to understand Scrum?

Experimenters need to understand Scrum because a large number of organizations are using Agile & Scrum practices and you are likely at one such organization. Typically, such organizations are focused on adding value to their customers through their product. Running experiments provides insights all throughout the product lifecycle from discovery to exploration to validation. …

Photo by OC Gonzalez on Unsplash

The Weekly Buzz has gone through a few iterations already in true experimentation spirit — not really backed by data yet, but we’ll get there. For now, in case you are here and want to catch up on some of the important posts in the last few weeks, here they are on LinkedIn.

The LinkedIn posts have takeaways from the articles/ posts below for easy reference.

TWB — Feb 11

👉 Which experimentation metrics should you be measuring? Katie Kelly from Speero (formerly CXL) provides a primer.
🤔 Your turn -> Which metrics do you track for your experimentation program?

👉 Need help deciding…

The Weekly Buzz — Experimentation/ CRO/ Conversion Optimization News

In case you are wondering what this is all about, read my previous post introducing The Weekly Buzz and what to expect. And now without further ado, here’s the latest installment of The Weekly Buzz. Scroll down to the video for key takeaways.

  1. Want to know which half of your advertising spend is wasted? Experimentation to the rescue! Read Tim Wilson’s latest blog post.
  2. Is breaking a UX convention worthwhile? Read NN/g’s latest post on maintaining consistency & adhering to standards.
  3. How to get SEO and CRO to dance together? Get AWA Digital’s free e-book.
  4. How founders…

Photo by Donovan Silva on Unsplash

While I took a break from writing in the last few weeks, I didn’t take a break from creating experimentation focused content. In December, I began collaborating with Experiment Nation. Spearheaded by Rommil Santiago, Experiment Nation connects experimenters from around the world and is a platform for sharing their experiences and learnings.

So, what is the The Weekly Buzz?

It is a way for busy professionals to stay in touch with the latest in the experimentation space. Every week, I pick 5 important stories covering experimentation news, A/B testing stats, UX research etc. and create a 2-minute video which has bite-sized takeaways from these stories for quick…

I got my Conversion Optimization certification from CXL Institute about two weeks back. And pursuing this course has been quite a journey for me with a lot happening alongside — dealing with the layoff, supporting my daughter’s online schooling, starting this blog and starting off as a freelance consultant. So I am really reveling in the sense of achievement at least for now! It’s also that time of the year. :)

As the conversion optimization (CRO) field matures, I am finding more folks interested in figuring out ways to get some formal education and potentially pursuing the mini-degree. I am…

Bithika Mehra

On the path to learning all things insights and optimization ( | Foodie | Environmentalist| Loves to travel | Player of a few riffs

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store