Day 2, Afternoon - Usability Week San Francisco

Dr. Nielsen began the afternoon session with an introduction to the Usability Toolbox -- the arsenal of tools at our disposal to rid the world of really bad design. They include:

• Server traffic log analysis
• Search log analysis
• Heuristic evaluation/expert review
• User testing
• Low-fidelity paper prototyping
• Surveys
• Field studies
• Participatory design
• Competitive studies
• Cardsorting
• Measurement studies

He then talks about a "Lifecycle Model" which includes Pre-Design, Design and Post-Design.

Pre-Design
In Pre-Design, field studies, cardsorting, testing the old design and testing competitive designs are your most valuable tools. Since some people may be resistant to testing the old design, Nielsen points out that, while you may know it's bad, you may not know WHY it's bad. If you just redesign, what you create will be different, but not necessarily better. Your old design is your best prototype for your new design. Your competitors are your second best prototypes.

Design
In the Design phase, iterative testing with low- to high-fidelity prototypes is most valuable, and final polishing can be done with a heuristic evaluation. Nielsen recommends developing low-fidelity paper prototypes early in the design process. During the training, we engaged in a team-based paper prototyping exercise. We used actual paper, markers, scissors, and sticky notes. At first, this seemed rather grade school, but as we got into it, I began to appreciate the team-based approach, the outpouring of creative ideas, and how easy it was to make changes to our design. We then tested the paper prototype, with one person acting as facilitator, another acting as the computer (changing pages or adding stickies as the user clicked), and another person taking notes. The user used the wrong end of a pen to indicate clicking, and the right end to write in forms.

In a heuristic evaluation, a small set (usually 2-3) of usability experts examine the interface to judge it's compliance with known usability principles (heuristics). Reviews can be done on draft designs, as well as on specifications and wireframes. Generally, one should take two passes through an interface when doing a heuristic evaluation -- the first to inspect the task flow and the second to inspect page details.

Post-Design
In Post-Design, Nielsen suggests tweaking the design as needed based on log file analysis and surveys. Regarding search log analysis, he said that people generally go for the easiest interaction (that they think will help them achieve their goal). If they use the search feature, they probably couldn't find it easily via other navigation options. Search logs can reveal what people are looking for, and the terms they use to refer to that information.

For surveys, Nielsen recommends a 1-7 rating scale, with 1 being really bad and 7 being really good. In 317 NNG studies, the mean rating is 4.9, leaning slightly toward the positive. Nielsen points out that it's really human nature that people tend to be more positive. The 1 and 2 ratings are hardly ever used, so it's almost like you're really using a 5 point scale. He also indicated that surveys weren't very reliable because what people say and what they do don't always match. One really great survey question, however, is "Why are you visiting our site today?"

Have Web Sites Improved in the Last 10 Years?
Based on the 317 surveys NNG has conducted, the results suggest not. Nielsen explains this by indicating that while web sites have certainly improved (we can do much more), user expectations have also increased. So satisfaction ratings are relative to user expectations. A site could get a good satisfaction rating one year, and even if nothing changes, get a poor satisfaction rating another year simply because user expectations have risen.

At end of Day 2, I felt the training content was really Usability 101, but I also recognized that I was picking up little nuggets of knowledge here and there that one could only get by spending time with someone like Nielsen and his team who have many, many years of practical usability experience.

More later about Day 3!