Category Archives: Data

The Data Will Never Be Good Enough. And That’s OK.


Courtesy lisasolonynko.

You have a bunch of sales data, or marketing data, or engineering data. And you have an open question. What customers are most likely to buy my new product? What market segment will grow the quickest? What material will be strong enough and light enough for my design? Alas, the data is just too murky to answer the question.

Life has given you lemons. You have two options:

1. Throw the lemons out. Ditch the data and do what people have always done (go with intuition, seek counsel from experts, make a pure guess, etc.)

2. Make lemonade. Adjust your expectations. Embrace the uncertainty. Find the scenarios where the data informs you about your question, even without a complete answer.

Both options are useful

In some cases, the first option is best. The pendulum has swung too far in support of data-driven decision-making, which is strange coming from someone (me) who has built a whole career in in this area. The data-driven approach is great. But it’s only a hammer, and we know all problems aren’t nails.

Continue reading

Monte Carlo Simulation 101: Embracing Uncertainty in Data

jkrimmel_calculator_20150308I explain below, using a simple example, how Monte Carlo simulation works. I have included a link to an Excel file at the bottom of this post that shows all the calculations involved. The Monte Carlo approach offers a way to gracefully manage the uncertainty in data-driven challenges. For our complex world, it’s a way to simplify your problem without sacrificing what makes the problem interesting.

Posing the question

Let’s say we want to estimate how much fuel all the vehicles in the United States will consume annually through the year 2025. Fuel consumption depends on a lot of different factors, each of which may change independently of the others. It’s a complex scenario. Let’s use Monte Carlo simulation to embrace the uncertainty.

Continue reading

Two Questions to Ask Before Jumping into the Data


Courtesy limecools. cc

What are you trying to learn about your business? What question is lingering in your head? What data set is sitting there, begging you to use it to increase your sales, or optimize your product portfolio, or identify your preferred customers?

All businesses inspire these kinds of questions. And we live in an age where using data to find answers is sexy. How do we extract the most value from these seemingly ubiquitous, under-utilized data sets?

The first step is to have the primary question in mind. Think of something like the following: “If I were to sell one instance of a new product or service, which existing customer would be most likely to buy it?” You might have existing sales data and maybe some general purpose marketing data to help you answer this kind of question.

Continue reading