My favorite Dr. House episode is the one where they play high stakes poker – have I mentioned recently that I often binge Netflix when I am awake at night? If you have any favorite shows I’d love to hear about them, difficultly level: I consider Star Trek: The Next Generation to be the best TV show ever made. Good Luck.
Anyway – I recently had a discussion with a client about survey/panel type data vs collected data and how it was important to know the difference. TV always comes into conversations like that because of all the comparisons to Nielsen.He kept asking me why it was so important to get this concept and I finally said:
“Remember that Dr. House medical TV show with the diagnostician who was always trying to figure out what weird disease people had? One of his guiding principles was that all people lie. When it comes to survey data that same thing is true, that effect is real, that’s why it is important.”
If you ask your customers questions you get a significant quantity of lies. No disrespect to customers intended but this has been proven many times. A “how did you hear about us?” question box on a website always generates bad data. There are many causalities but let’s play Dr. House with this and discuss the two most common ones.
- They Really Don’t Remember
If a person habitually uses one search engine they have a greater than 1 in 4 chance of saying that was how they found you even if in fact it was from an alternate source.
- They consider how they “Should Answer”
When you ask people qualitative questions they will often consider the implied value judgment in their response, this is why so many surveys are anonymous. To put it another way – if you set up a survey table in front of a church on Sunday morning people will tell you they want to buy American made products and care about that. The same afternoon you’ll see those folks at the local big box store with carts full of imported plastic nonsense.
I’m not saying everyone should abandon surveys as a data collection methodology outright but it is important to remember what Dr. House says when using that data.
Here’s the bottom line for Internet Entrepreneurs: if you collect multiple kinds of data (survey data and ‘collected’ data like Google Analytics) make sure to remember which is which before you make any decisions.
In the meantime I think it’s time for some classic TV.