My Opinions on the User Research Smoke & Mirrors Article

In this article, Christopher Fahey debunks the necessity of excessive and sometimes dramatized or artificially-engineered scientific research that tries to jump to conclusions about the usability in web design or be a credible basis for the re-design process. At the same time, he waxes lyrical about the virtues of “less-rigorous” qualitative user research such as simple persona creation and card sorting exercises that treat user-centric design as what it is – a subjective experience.

I have to agree with most of this article, particularly the examples of many designers conducting unnecessary “scientific” research that tries too hard to impress their bosses, although their results may be skewed too much or misinterpreted to derive wrong conclusions, thus inaccurately reflecting users’ actual concerns about the design. It seems that in this modern, technologically advanced day and age, much emphasis is being placed on empirical research churning facts and figures that try to conclusively and objectively prove right the assertions and claims made by advertisers, researchers and others. I realised that most of this research has an ultimate agenda – profit-making because they are using these statistics to win consumers and stakeholders over. One such example that I find amusing are detergent advertisements that try to compare the effectiveness of their brand with a no-name “brand X”, often by means of a graph or chart whose validity is questionable. The same is echoed in the design world as well, as in Fahey’s example of the automobile website being one example whose approach steers towards making profits rather than focussing on the real needs of the consumer. I believe that the highly scientific and “objectified” culture that emphasises the measurable instead of the intangible is partially to blame for such a widespread phenomenon. Hence, until design firms employ bosses who empathize with the actual users’ needs and see the value of the “softer” aspects of research, even the analysis of passing comments posted on users’ blogs, it seems that the profit-driven scientific research is here to stay. That could only mean bad news for consumers as their interests (who may not be profit-enhancing for many firms) may be replaced or overshadowed by the lure of profits that potentially misleading statistic-churning research may bring the firm.

For our project, I can imagine what would happen if “scientific methods” over qualitative research were employed. As it is an entertainment system for Fish & Co., this scientific approach would most likely arrive at a contrived conclusion as to how our customers may use our product. If heat-tracking technology is employed, we might have drawn false conclusions if say, most of our customers’ eyes are trained on the buttons. It could either mean that they either admire our buttons’ design or are finding difficulty in reading the words, two very different reasons. Hence, getting feedback from customers via survey forms or interviews would have been much more effective in zeroing in on the true reason for their prolonged staring at the buttons. This being said, eye-tracking technology could be used to provide a quick preliminary overview of what customers’ attention is drawn too, which could help us to identify potential problem areas and further our re-designing strategies and processes by following it up with qualitative analysis. Hence, “scientific methods” may not be as contrived or bad as what Fahey is hinting at, if we know how and to what degree to use it, ie. as a complementary, rather than core, analysis tool that can give us an indication on which aspects to focus our qualitative analysis on.

Next, I would like to comment on Fahey’s take on the Organic and IDEO examples, whose approaches towards design he takes a dim view of. I tend to agree with his assessment about Organic’s Persona Rooms being a waste of resources and not adding extra value to the design process that good UI designers would have already figured out using instincts and common sense. However, the idea at its core is not scientific, in fact, it is user-centric, an extension of the persona creation that we are used to doing. It is also ironic because the persona is characterized by the appearance of a confined space, ie. a room, when he/she could have been outdoors doing so many other things such as playing sports or clubbing, or hanging out with a large group of friends, things that cannot be captured effectively in the room’s appearance. Hence I think such rooms are just a show-off tactic on Organic’s part because I feel that it restricts, rather than expands, what the persona could be like, something which a paragraph of words could do better justice to.

However, for the IDEO case, Fahey may have overstepped his judgment about IDEO packaging its design process to be idealized and fit-for-TV, simply because we cannot tell for sure if that video, though condensed, represents the actual workings of the design process in that company, which it very well could. I have seen the shopping cart video in another class and I have to admit, it looks very believable and impressive, and contrary to what Fahey mentioned, there seems to be little fakery in it as the designers in the video seemed all fired up and willing to make a positive contribution to the design process. Even the layout of the office looks less rigid than other offices with toys, basketball hoops and even hanging bicycles signalling that the workers have indeed incorporated their hands-on and “playful” and experimental approach into their office spaces. Then again, even if such approaches to design are not faked, it may not be applicable to all types of design, like for example, a corporate website which needs a much more structured design process than that of a shopping cart.

In conclusion, we can draw some very interesting insights from Fahey’s article as it illustrates the sometimes ludicrous and downright laughable extent to which “user research” in the real world is used to validate design researchers’ decisions in the re-design process. Generally, as we can infer while going about conducting qualitative methods in our projects, such scientific methods may not be as effective as at the end of the day since users’ needs, preferences and evaluation of our product are subjective and cannot be “objectively measured” by conclusive charts, diagram and statistics. Having said that, such methods can be used as a guide to better inform us which aspects of our design we should seek to improve.