Day 3 - Usability Week in San Francisco

I know, this looks bad. Four months later, I'm finally getting around to summarizing Day 3. The impetus - I'm giving a presentation at work summarizing my conference experience. Since I'm preparing this for work, I figured I might as well share it with anyone else that might be interested. Who knows, maybe I'll get this blog thing going again :)

During Day 3, NNG covered a hodge-podge of objectives including:

• Field Studies
• Ethics of User Research
• Usability Report Analysis
• Variants of User Testing
• Special Areas of Research
• Financing Usability
• Cost Benefit Analysis
• Successful Usability Programs

Here are some of the points I found most interesting:

Field Studies

• Field studies can help define new features, tell you about tasks and work-arounds you may not have known about, and may identify new customers.
• Perform field studies early in the project while the information is still actionable.
• When recruiting for field studies, consider users of competitive or similar products (not just yours).
• Recognize that purchasers of your product may not be the users of your product.
• Make sure users know what their commitment entails (time, job shadowing, communication).
• Remember that a field study is about WATCHING people do their work; it is not to discuss their work. (Remember, what users say they do and what they actually do may be different.) Users should pretend you're not even there.
• 3-4 field study observers are recommended (e.g., product manager, developer, usability expert), each with an assigned role (e.g., facilitator, photographer, note taker).
• Consider creating a template with some guidelines, high level goals and plenty of room for notes and drawings.
• Count to 20 before you interrupt a user.
• Always let the user name the objects they're interacting with.
• While doing a field study, look for: processes, reasons, pain, tools, people, places, artifacts.
• Capture user quotes.
• Immediately debrief with observers to help you remember details and build consensus.
• Reserve a room to analyze findings ("war" room).
• Study outcomes may include: user profiles/personas, task lists/flows, prioritized issues, new feature ideas, dictionary of user terms, photos, videos, artifacts.

Variants of Usability Testing

• Co-discovery is when you two users attempt to do tasks together (e.g., house hunting). Consider if task is commonly done by two people.
• Remote testing is when faciltator and participant are in different locations; a good choice if you can't physically be there (e.g., international).
• Competitive studies are used when you want to test your own design as well as 1-3 competitors -- provides information on design elements that work and don't work, and allows you to avoid repeating others' mistakes.
• Longitudinal studies follow users over an extended period of time; users record their experiences and make comments.
• Eye tracking allows you to see where users are looking (first read, scan path, gaze time).

Special Areas of Research

• Users with disabilities
• Low-literacy users
• Senior Citizens
• Children
• International testing
• Hardware testing

I would add "domain-specific" testing here as well (e.g., automotive manufacturer).

Cost/Benefit Analysis

• Before/after metrics can include: sales, support calls, productivity (time to complete task), training time, customer satisfaction.
• The cost of training is the cost of bad usability.
• If you don't do a usability study, you could actually have negative improvement; however if you do a usability study, you'll always pick the design that proves improvement.

Successful Usability Programs

A good usability professional is experienced, balances diplomacy with assertiveness, is somewhat technical, and is driven by data; he/she is NOT timid, persecuted, or judging/finger-pointing.