ntschutta.com

I consider myself a client side guy. I arrived there rather by accident – I was one of the least experienced (in terms of years worked) members of a team that was slated to build the company’s first Java based web application. When it came time to break the overall group into small teams, the three greenest (myself included) staffers were assigned to the client tier. I guess that sent a subtle message – the stuff the user would actually see wasn’t all that important so put the junior team members there. Ironically enough, the three of us also had the majority of the OO experience on the team (can you say procedural Java?)…but that’s a matter for another blog.

Oddly, we did spend a fair amount of effort on the user interface and we spent a remarkable amount of time doing usability testing. We had a fantastic UI architect that worked really hard to come up with a look and feel that was consistent between IE and Netscape (ah, the glory days of the late 90’s and serious browser incompatibilities – I love the smell of browser checks in the morning!) For reasons that still escape me, we actual brought in a rather pricey IBM usability expert to help us test the app and it was at this stage of my career where I discovered the joy of paper based prototypes (otherwise known as low-fidelity or low-fis).

The nice man from IBM taught a few of us how to moderate testing sessions. He told us to remind users that *they* weren’t being tested, that failures were actually our mistakes in the design. He taught us how to let users struggle, that telling them what to do meant, again, that we’d failed. We were coached on encouraging users into thinking aloud about their experience and tell us how they expected the app to work. But the most important thing he taught was that users lie. OK, he didn’t say it quite that way but it’s like Mom always said: “actions speak louder than words.” What a user says rarely matches what a user actually does. It’s why asking a user what they do day to day just doesn’t work – you need to actually observe them in their natural habitat (it’s kind of like being an anthropologist though truthfully it’s more like ethnography).

I’m not advocating that we completely ignore customer and I don’t mean to imply that they deliberately mislead us (well some do) but yes, listening to users is harmful. According to Nielsen, the First Rule of Usability? Don’t Listen to Users. Those of us that spend an inordinate amount of time dealing with ephemeral stuff like code are pretty good at working with abstractions (though we don’t hold a candle to those theoretical physicist types…) however, the average user has a hard time imagining what might be. Not to quote Kathy twice in one paragraph, but there is some quantum mechanics at work here. Again, actions and words don’t always match up – many of us respond to a kind of social calculus and consciously or not, we seek approval by giving people the answers that we think they want.

Of course what users want will be impacted by what we give them. It’s one of the main reasons heavyweight methodologies just don’t work – the feedback loop is too long and it gets short circuited by a desire to resist change. I used to think that my customers were just neurotic – we’d go build what they asked for, show it to them, then they’d say “it’s all wrong – change it.” It was considered rude to point out that we’d built exactly what they asked for…

I realize now that this is completely natural, my users were just reacting to what we were building. This is the fundamental strength of having frequent releases, customers get something to work with earlier and they can make adjustments to what they are asking for. Fundamentally, this is more efficient: if you’ve only spent a week or two developing a feature and discovered that it’s not useful, so what, you’ve only lost a couple of weeks. If you wait 18 months to discover that, well, you’ve just wasted a ton of money.

So, (mostly) don’t listen to what your users say. Watch what they do – get in front of real live users and watch them work. Asking what they do isn’t sufficient, a lot of times you’ll either get an incomplete answer (do you remember what you did yesterday morning?) or you’ll get the party line answer (well, this is what we are supposed to do). Don’t settle for the manager or the VP, even if they were the best at doing “foo” five years ago, things have probably changed (plus they probably wrote the proverbial book on doing the job and they misguidedly belief this is how things are actually done). So get out there and do some good old fashioned observation. If you’re lucky you might just build something people will actually use.