5 Problems with Marketing Analysis

What do we often forget to mention?

I missed an important birthday last month; the banner ad turned 40.  We're certainly not going to see the 40% CTR that this ATT ad achieved back then.  In fact we're not likely to see much above a single digit or even decimal place.

At the time I don't think anyone had an idea what the click thru rate would be, but in today's data-driven world we should have some clues.  We have all kinds of systems and analytic tools that predict what a CTR might be. But each of those come with one or more constraints that we often forget to convey.

1. Time, data and resources will narrow the options we can use to analyze a problem; need an answer today? or next week?
2. Explanations always rest on assumptions as we reduce the problem to one that is manageable; did we remember to tell anyone?
3. Our mental models are biased to begin with, and we may not realize it.
4. Data is left behind due to data collection procedures or application code; design choices made before we started the analysis limit what we can and can not do/say.
5. Our use of data may be different than how it was intended; this is a common question when creating mashups particularly with public data.

So, say out loud "this is what we don't know"....