risks of quantitative studies | jakob nielsen’s alertbox

when it comes to discussing the contribution (and value) of design and design management to corporate performance people tend to demand for hard facts. hard facts in this case means: give me the numbers!

consequently a noticeable number of articles and design management disseration have been written about the value of design (management) investments ranging from product design to brand management. listening to expert reports at DMI conferences and reading dissertations about the financial value of design/branding always left me doubting about the correlations they draw in many cases.

in late 2003 I wrote two posts about usability guru jakob nielsen. nielsen publishes a biweekly column on his website ‘www.useit.com’. [as I already warned you last year: appreciate his content and ignore the site layout! ;-)]

in his current post nielsen writes about the risks of quantitative studies. as to summarize his findings:

“Number fetishism leads usability studies astray by focusing on statistical analyses that are often false, biased, misleading, or overly narrow. It’s better to emphasize insights and qualitative research.

… If you measure enough variables, you will inevitably discover that some seem to correlate. Run all your stats through the software and a few “significant” correlations will surely pop out. (Remember: one out of twenty analyses are “significant,” even if there is no underlying true phenomenon.)” >>>

even though nielsen comes from the area of web usability testing his article is worth a reading especially for design and design management students seeking to work on a reliable basis of their dissertation topic and related findings. [groeten naar rotterdam :-D]

one good example of research in the field of design management is the study (~ 130 kb; only available in german language) conducted last year under supervision of heithoff identity to get a clearer view on the attitude, application and willingness of future investment of north rhine-westphalian small and medium sized companies into design and design management procedures.

the study combines collecting quantitative data and drawing traceable conclusions while the research technique was a written questionaire sent out to some hundred companies. download the report here >>>



Leave a Reply

XHTML: You can use these tags: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <code> <em> <i> <strike> <strong>