ORA ETech '06

Usability Testing
presented by Joel Spolsky and Betsy Weber

----------------------
collaborators: 
Gabe Hollombe - http://avantbard.com - gabe@avantbard.com
----------------------

users have about 6 mins to figure out your product before they give up

put out an ad in local paper (village voice) for people to come help with usability 
tests

goal of usability test is not to calculate statistics of how much works but rather 
to find places where you have usability probs

find the situations where the people have a different idea of how the product is 
supposed to work than your idea of how it's supposed to work

dont need a lot of users, about a dozen is enough.  more and you get bored and stop 
learning new things.  (after the first five or six you'll have found the major 
probs)

The Usability Test Uncertainty PRiciple
    - the act of observing ppl in a lab makes it impossible to test realistically
    - don't wait until too late or you wont want to make the big changes to fix the 
big problems found; UT a few weeks before shipping is Bad.

Paper Prototypes
    - draw the UI in a few states on some sheets of paper
    - show them the pages/screens, ask them to show you what they'd do
    - paper prototypes are nice because people react to the functionality exposed 
by the design, not the design itself

People often don't know the answers to what the software is asking them

Ask people if you can sit in their cubacle and video tape their own tasks with your 
product. pretty successful for larger products like MS Word because everyone uses 
it daily, etc.

if you focus too much on a UT, you can mess up the UI so everyone can pass the test 
but after your alterations it doesn't matter ("click here to start programs" over 
the start button in Windows98)

Learnability vs Usability
    - learnability = abil to learn how to use the prod for the first time
        * driving is hard to learn but easy to use once you learn how
    - usability = abil to use the product efficiently after you know how to use it
        * wizard dialogs are easy to learn but hard to use (if you needed to post 
8000 things on ebay, their wizards would get REALLY annoying)
    - keep both in mind

best reason to conduct a usability test is to get some sense into your programmer's 
heads about what human beings are _actually_ like
    - focus on make things extreemely easy to use, not just easy to use

hiring a company to do UT for you is like hiring someone to learn french for you.  
you dont learn anything.

Time & Money
    - excuses companies give to not do UT
    - UT can be a burden on your users

Project Aardvark - UX 1.5 (testing copilot)
    - paired up 14 people into victims and helpers
    - victim would call helper on phone and helper would direct to copilot.com and 
walk them through the target task by taking control of their screen
    - captured screens and users video
    - didn't take a long time to do the testing.  did the tests in a day
    - being able to observe _exactly_ what the users go through can be very 
insightful for everyone

UX 2.0 meets Web 2.0
    - Less Structure
    - Lose the opinions
    - Listen to customers
    - video captures of users using your product are great; you can't argue with a 
video

the experience is the application and the application is the experience

important to see not just whast users do but also what they're saying while they're 
doing it; their reactions

important to get customers involved and excited instead of burdening them
    - people are excited to be a part of the product dev experience 

web 2.0 apps aren't tied to a specific platform
    - any user
    - any time
    - any channel (cell phone, pda, pc, etc)
    - think about every way a customer can interact with the system
    
Jeff Bezos says amazon focused like a lazer on customer experience