In a recent post by correspondent Steve King: Are Statistically Significant Research Methods Passe?. " ... A growing number of surveys are being based on informal groups of respondents rather than statistically significant population samples ... ". Speaking about the use of questionnaires. I comment at his post as well, where I agree it is a growing trend. It is also like the trend of design versus quantitative analyses. Like the 'Blink' hypothesis. A form of generate-and-test, but do not forget the test. Classic statistically sound methods are still important. It is OK to use roughly exploratory methods to understand the landscape, but not to make the big decisions. See related previous post, which suggests that chat is more important than carefully directed instruments.
Tuesday, September 16, 2008
A Move from Statistical Significance?
In a recent post by correspondent Steve King: Are Statistically Significant Research Methods Passe?. " ... A growing number of surveys are being based on informal groups of respondents rather than statistically significant population samples ... ". Speaking about the use of questionnaires. I comment at his post as well, where I agree it is a growing trend. It is also like the trend of design versus quantitative analyses. Like the 'Blink' hypothesis. A form of generate-and-test, but do not forget the test. Classic statistically sound methods are still important. It is OK to use roughly exploratory methods to understand the landscape, but not to make the big decisions. See related previous post, which suggests that chat is more important than carefully directed instruments.
Subscribe to:
Post Comments (Atom)
2 comments:
Franz: I totally agree. I'm of mixed minds about this trend. On one hand I like the speed, low cost and freedom informal methods provide. But the results may be meaningless or even misleading.
On the other hand, I think you learn a whole lot going through a statistically significant research process and the results tend to be more trustworthy.
I continue to think both approaches are useful.
On a somewhat related topic, the Ad Age article reminded just how long it takes for a new technology to move to the mainstream. Wasn't it 5 years ago when we presented at a meeting in Cincinati on this topic?
Steve
Good comment. We agree. In fact there is value in getting more people involved in the process of understanding what people think.
I repeat the generate-and-test idea, use overview sampling to help generate possible hopefully novel solutions, then test solutions with statistically valid experiments. Trouble is, the last part is often forgotten when people find an idea they like. Lots of confirmation bias out there.
Yes remember the talk we gave. Our talk is still getting traction. It does take quite a while for tech to get traction in the enterprise. In our case I was surprised the company moved as fast as it did.
Post a Comment