The pitfalls of problem solving with averages

If you were a quarterback and threw the ball one foot too far ahead of your receiver half the time, and one foot too far behind your receiver the other half of the time, you wouldn’t see a lot of playing time. But if we were measuring the accuracy of your throwing, we might conclude that, on average, you were extremely accurate. And that is the basic problem with averages: they can hide what you need to know.

Averages hide variation. Averages are simple to calculate and are sometimes a lazy way of determining past performance. For example, over some period of time a performance level may have started at 50% and ended up at 70%. Simple math then determines the average to be 60%, so 60% is now used as a baseline to measure future performance against. If the next performance level measured is 65%, you might conclude that performance has improved, which it has over the baseline. But in fact it has degraded from true starting point of 70%. Your next decisions could lead you in the wrong direction.

Averages are easy to manipulate. The simplest way to manipulate averages is to change the base period or exclude the “outliers,” which are those data points that seem to be abnormally high or low when compared with the majority. One of our clients once told us that the key to understanding what drives performance is not to exclude outliers, but to study them. By looking at the best and worst months over the last year, he learned what most impacted his results. Another outlier study we learned about (from the automotive industry) is a concept called BOB WOW, which stands for best-of-the-best and worst-of-the-worst. This one we find effective for looking at variances in individual performance. Studying the outliers gives insight into what helps or hinders success.

In one retail project we did for a shoe manufacturer we observed the company’s most and least successful salespeople. We found the simple act of getting people to try on shoes had a marked impact on their likelihood to purchase. The sales reps couldn’t specifically articulate it, but their greeting, questions and even movement were designed to get you to invest time and energy in putting on shoes. Over time they intuitively knew that one action led to better odds of making a sale.