Home > Software, Usability > Users Lie

Users Lie

December 31st, 2005 Leave a comment Go to comments

I consider myself a client side guy. I arrived there rather by accident – I was one of the least experienced (in terms of years worked) members of a team that was slated to build the company’s first Java based web application. When it came time to break the overall group into small teams, the three greenest (myself included) staffers were assigned to the client tier. I guess that sent a subtle message – the stuff the user would actually see wasn’t all that important so put the junior team members there. Ironically enough, the three of us also had the majority of the OO experience on the team (can you say procedural Java?)…but that’s a matter for another blog.

Oddly, we did spend a fair amount of effort on the user interface and we spent a remarkable amount of time doing usability testing. We had a fantastic UI architect that worked really hard to come up with a look and feel that was consistent between IE and Netscape (ah, the glory days of the late 90′s and serious browser incompatibilities – I love the smell of browser checks in the morning!) For reasons that still escape me, we actual brought in a rather pricey IBM usability expert to help us test the app and it was at this stage of my career where I discovered the joy of paper based prototypes (otherwise known as low-fidelity or low-fis).

The nice man from IBM taught a few of us how to moderate testing sessions. He told us to remind users that *they* weren’t being tested, that failures were actually our mistakes in the design. He taught us how to let users struggle, that telling them what to do meant, again, that we’d failed. We were coached on encouraging users into thinking aloud about their experience and tell us how they expected the app to work. But the most important thing he taught was that users lie. OK, he didn’t say it quite that way but it’s like Mom always said: “actions speak louder than words.” What a user says rarely matches what a user actually does. It’s why asking a user what they do day to day just doesn’t work – you need to actually observe them in their natural habitat (it’s kind of like being an anthropologist though truthfully it’s more like ethnography).

I’m not advocating that we completely ignore customer and I don’t mean to imply that they deliberately mislead us (well some do) but yes, listening to users is harmful. According to Nielsen, the First Rule of Usability? Don’t Listen to Users. Those of us that spend an inordinate amount of time dealing with ephemeral stuff like code are pretty good at working with abstractions (though we don’t hold a candle to those theoretical physicist types…) however, the average user has a hard time imagining what might be. Not to quote Kathy twice in one paragraph, but there is some quantum mechanics at work here. Again, actions and words don’t always match up – many of us respond to a kind of social calculus and consciously or not, we seek approval by giving people the answers that we think they want.

Of course what users want will be impacted by what we give them. It’s one of the main reasons heavyweight methodologies just don’t work – the feedback loop is too long and it gets short circuited by a desire to resist change. I used to think that my customers were just neurotic – we’d go build what they asked for, show it to them, then they’d say “it’s all wrong – change it.” It was considered rude to point out that we’d built exactly what they asked for…

I realize now that this is completely natural, my users were just reacting to what we were building. This is the fundamental strength of having frequent releases, customers get something to work with earlier and they can make adjustments to what they are asking for. Fundamentally, this is more efficient: if you’ve only spent a week or two developing a feature and discovered that it’s not useful, so what, you’ve only lost a couple of weeks. If you wait 18 months to discover that, well, you’ve just wasted a ton of money.

So, (mostly) don’t listen to what your users say. Watch what they do – get in front of real live users and watch them work. Asking what they do isn’t sufficient, a lot of times you’ll either get an incomplete answer (do you remember what you did yesterday morning?) or you’ll get the party line answer (well, this is what we are supposed to do). Don’t settle for the manager or the VP, even if they were the best at doing “foo” five years ago, things have probably changed (plus they probably wrote the proverbial book on doing the job and they misguidedly belief this is how things are actually done). So get out there and do some good old fashioned observation. If you’re lucky you might just build something people will actually use.

Categories: Software, Usability Tags:
  1. Gear Daddie
    December 31st, 2005 at 22:12 | #1

    Excellent Post! Sometimes users like other times they are too grounded in their day-to-day work that they don’t have any idea what _can_ be done. I’ve recently worked on a system that was a web app port of a “green screen” application. Developers (note – there were no real designers as is very typical) did exactly what the users said they wanted and guess what the got: A complete copy of their green screen app with prettier fonts. What a waste…

  2. jsteil
    January 4th, 2006 at 08:25 | #2

    Another thing I don’t get is why projects don’t automatically plan for a second release after the initial goes in? Not that we should plan for things to be wrong, but it is inevitable that once a system goes live and the users start using it for real they’ll want to change some things to make their job easier. I have seen too many times recently where we dump a system into production and it just sits there without any enhancements after that.

    We have an end-customer app that went to production 2 years that hasn’t had a single update applied to it since even though our webmaster receives between 10-15 complaints about it per week. If you attempt using it over dial-up, you might as well not even try. Get an error in the middle of a multi-step process? Too bad, you have to start over. We’ve tried to get momentum behind enhancing the app, but the standard reply we get is that there isn’t any money in the budget to do it.

  3. January 16th, 2006 at 19:02 | #3

    Thanks guys, I really appreciate the comments (sorry I didn’t respond sooner – been a bit busy…) John, I’ve encountered EXACTLY what you’re talking about – well, we want this. Why? ‘Cause that’s what the old system did. It’s just amazing to me that people don’t grasp that rewrites allow you to reengineer the business process, a process that is all too often tailored around deficiencies in the systems! In fact, the last “real” project I was on at my former employer was no more than a direct port to the web – we couldn’t even modify the database to be, oh, I don’t know, relational!

    Jeff, great point about release 2.0. Too often I’ve seen projects derailed because users HAD to have all these features *right now!* Like waiting a couple of months would be the end of the world… Man, no updates in 2 years, what’s the point? Especially if there have been numerous complaints. Ahh, you wouldn’t want to listen to users on something like that – heck, they should just learn to do it right.

    Well, you two have inspired to write another post – hopefully soon!

  1. No trackbacks yet.