B2B go-to-market and the law of large numbers

April 11th 2023

Image credit: https://math.stackexchange.com/questions/2834535/visualizing-the-weak-law-of-large-numbers

In B2B marketing and sales we often have benchmarks for the probability of an outcome occurring - often this is in the form of a typical conversion rate. For example:

Number of impressions required per click
Number of sales opportunities required per win

In other words, we know how many times we normally need x to happen, for y to happen.

For example, over 3 years, you might have built up a body of data showing that the typical conversion rate of landing pages on your website is 2%. Meaning that when you create a new landing page, broadly, you expect a 2% conversion rate.

But if a new landing page performs worse than usual, at what point should you change it?

This is where it’s helpful to know about the ‘Law of large numbers’: the more times something happens, the more the results will be in line with what you expect.

Here is a chart showing what happens when a coin is tossed 200 times:

(Image credit: https://statisticsbyjim.com/basics/law-of-large-numbers/)

Early on, the results differ significantly from what you would expect (0.5). Then as the number of times the coin is flipped increases, the results settle close to 0.5.

The law of large numbers is something that we instinctively understand: we wouldn’t expect to flip a coin 4 times and be guaranteed that it would land on heads twice.

But it’s harder to intuitively answer a question like: “if when x happens, there's a 20% chance of y outcome happening as a result, what's the chance of y outcome happening if x happens 6 times?”

For example, if you know that typically 5 sales opportunities lead to 1 win, and you’ve now had 6 opportunities in a row without a win, is this bad luck or is there something wrong with your approach?

The formula to work this out looks like this:

To break it down, the 0.2 represents the 1 in 5 chance of the outcome happening, and the superscript 6 represents the number of tries. The 0.74 represents the likelihood of the outcome occurring after 6 tries (74%).

So in this situation, you can see that there is basically a 1 in 4 chance that the fact you haven’t had a win is bad luck. Which would probably suggest it’s not worth making drastic changes or going back to the drawing board yet.

As the number of opportunities increases, the likelihood that you’re suffering from bad luck decreases:

To the extent that if you expect 1 win from every 5 sales opportunities, and you’ve had 15 opportunities without a win, there is just a 4% chance that this is due to bad luck.

Here is another example at a different end of the spectrum, in terms of probability: the likelihood that someone will click on some kind of paid digital ad. If we imagine that for a certain ad format, you typically see a 2% click through rate, what would the relationship be between the number of impressions and the probability that someone would have clicked?

After the first 50 impressions, there’s only a 64% chance that someone would have clicked:

In the above sum, 0.02 represents the 2% probability of a click happening, the 50 represents the number of impressions, and 0.64 is the likelihood of the outcome (a click) happening.

Here's what happens as the number of impressions increases:

If you run an ad and you expect a 2% CTR based on past performance, and you don’t get the first click until after 150 impressions, the chances that this is bad luck and that over time the ad will perform as well as you expect is <5%. In other words, you should monitor subsequent performance closely, and probably start to think about making some changes to the creative and / or targeting.

What are the implications of the law of large numbers for B2B GTM?

The law of large numbers strengthens the argument for a measured long-termism. You can see that mathematically, even when you have made enough attempts that you might expect an outcome to happen, there’s still only about a two thirds chance that the outcome will actually happen. Ie:

33.33% chance of something happening, 3 tries = 70% chance of the thing happening
20% chance of something happening, 5 tries = 67% chance of the thing happening
10% chance of something happening, 10 tries = 65% chance of the thing happening
5% chance of something happening, 20 tries = 64% chance of the thing happening
2% chance of something happening, 50 tries = 64% chance of the thing happening
1% chance of something happening, 100 tries = 63% chance of the thing happening

But there should be limits to this long-termism, and a sensible rule of thumb might be that when you get to 3x the number of tries that you would theoretically need to happen to lead to an outcome, there’ll be a >95% chance of the outcome happening, so if it hasn’t happened, something is almost certainly wrong (or different). Ie:

33.33% chance of something happening, 9 tries = 97% chance of the thing happening
20% chance of something happening, 15 tries = 96% chance of the thing happening
10% chance of something happening, 30 tries = 96% chance of the thing happening
5% chance of something happening, 60 tries = 95% chance of the thing happening
2% chance of something happening, 150 tries = 95% chance of the thing happening
1% chance of something happening, 300 tries = 95% chance of the thing happening

B2B go-to-market and the law of large numbers

What are the implications of the law of large numbers for B2B GTM?

Further reading:

Casla

Sign up to our mailing list