Day 1, Morning - Usability Week San Francisco

Let the training begin! My day began at 8:30 a.m. with continental breakfast along with a few hundred other usability professionals. I sat down at a table with three women -- each coming from a different locale (Florida, Washington D.C., New Zealand) and each working in a different industry (Food, Government, Technology). In looking at the attendance roster, it was interesting to note that the people attending this event were coming from every corner of North America and beyond, as well as a wide range of industries. Dr. Nielsen has made quite a name for himself as a usability practitioner and writer; there are few areas where his expertise isn't valuable.

The program I enrolled in, Usability in Practice, started at 9 a.m. The presenter for the morning session was Amy Schade, a User Experience Specialist who works in NNG's New York office. Her pedigree includes a wide variety of hands-on experience in various industries, numerous training presentations, and she has co-authored many of NNG's reports. During her morning delivery, we covered User Testing Methodology (Planning User Tests, Conducting User Tests). Some key points I felt were especially relevant to our work at Wunderman included:

• The ability to observe user behavior is one of the key advantages of facilitated user testing. Sometimes what people say they do and what they actually do doesn't match up.
• Listening (not talking) during a user test is of utmost importance. The facilitator should stay quiet, observe and take notes. You want to test the design, not how good your instructions are; don't interrupt.
• When deciding what to test, identify the top 10 tasks to start. Don't try to cover everything at one time. Focus on large concepts or specific features -- ideally the most common tasks or those that have the most impact on the business.
• Metrics to collect can include Success Rate (pass/fail or partial credit with 0-4 rating scale), Task Time (use a stopwatch), Error Rate (including why error occurred), and Satisfaction Ratings (0-7 rating scale).
• Recruit the right participants; testing with the wrong users will (most likely) get you the wrong results.
• When developing screeners, ask open-ended questions (e.g., How much money have you spent online in the last year?), and be careful not to reveal the desired answer in the question.
• Ask your test recruiting vendor how they handle "no shows"; consider having "floaters" available -- people who are at the testing facility for an extended period in the event someone doesn't show up.
• Task writing guidelines include avoiding wording used in the design and micro-steps (too much detail).
• Conduct a pilot study of the test (at least 24 hours beforehand) to ensure tasks can be completed, tasks are clear, time allotted is reasonable, and number of tasks is appropriate.
• It is NOT necessary to return users to the home page at the beginning of a task (not realistic).
• Don't let your notetaking be distracting. (My personal experience is that it's better to have a facilitator focused on facilitating, and another person responsible for notetaking, ideally where the user can't see them.)
• Avoid mixing marketing questions (e.g., Would you use this feature?) with usability questions. If this is a requirement, save the marketing questions until the end.
• Some questions are just bad (e.g., Did you notice this link here? If you have to ask, they probably didn't.)
• You shouldn't have to describe every element on the page. Let the user discover on their own. If they don't mention, it's either not an issue, or it's not important to them.
• If you don't have a usability lab, it's not hard to mock one up. Consider a room where the user/facilitator sit at the front, with the screen projected on the wall, and observers sit behind the user/facilitator (out of sight).

During the Day 1, afternoon session, Dr. Nielsen made his appearance. More on this in the next post!