paper prototyping

 
 

Sure, it's low-tech, but this usability testing method can help you sidestep problems before you write your code

Wouldn't it be great to find out what users (and marketing) want before you start coding? Paper prototyping lets you do just that. While it may seem counterintuitive to test an interface without using a computer, paper prototyping lets you get maximum feedback for minimum effort. After a few usability tests with a paper prototype, you'll have confidence that you're implementing the right thing.

What paper prototyping is

Paper prototyping is a method of usability testing that is useful for Web sites, Web applications, and conventional software. Here's how it works:

A facilitator (usually someone trained in usability) conducts the session while other members of the development team observe and take notes. The “computer” does not explain how the interface is supposed to work, but merely simulates what the interface would do. In this manner, you can identify which parts of the interface are self-explanatory and which parts are confusing. Because the prototype is all on paper, you can modify it very easily to fix the problems you find.

Figure 1.  A paper prototype of the File Setup dialog from Microsoft Word

paper prototype: an MS Word dialog

Figure 1 illustrates how various GUI widgets can be prototyped. This example shows the File Setup dialog from Microsoft Word. (In a usability test, this component would be placed on top of a prototype of the Word application rather than being shown by itself.)

Figure 2.  The shopping cart page from an e-commerce site

shopping cart page

Figure 2 shows how removable tape allows users to place any combination of items in the shopping cart. The “computer” can update the shipping cost and total accordingly by wiping them off the transparency with a damp paper towel and rewriting them. (It's not considered cheating for the “computer” to use a calculator!)

How good should the prototype be?

You can use screen shots of an existing design if you happen to have them, but it's also fine to hand-sketch them, especially when you're in the early stages of design. And as Figure 3 shows, sometimes hand-drawn elements are actually more readable than screen shots that use a dark background color.

Figure 3.  Hand-drawn versions of browser buttons compared to a grayscale screen shot

browser buttons: hand-drawn and screen-shot

You can also mix and match screen shots and hand-drawn components. The prototype only needs to be good enough for you to get answers to the questions you're most concerned about. Thus, most paper prototypes don't need:

It's fine if the prototype looks a bit messy. Very often, the first usability test will show you problems you'd never anticipated, and then you'll want to make changes. Don't spend time making the prototype look neat before you test it -- if it's legible, it's good enough. (For those of you who are thinking, “But I can easily mock up something that looks nice,” see the sidebar.)

What paper prototyping is (and isn't) good for

Paper prototyping is especially useful for gathering data about the following kinds of problems:

On the other hand, paper prototyping isn't ideal if your biggest questions pertain to:

Some development teams use paper prototypes in the early stages to smoke out the show-stoppers, and then do a couple of additional usability tests later with the real interface just to look for any additional surprises.

Key benefits for designers and developers

If you design and/or implement user interfaces, paper prototyping can provide several benefits:

  1. Test your design with users before you code
    Paper prototypes are ideal for finding out whether you're on the right track, before you write even a line of code. Sometimes the things you learn can have a significant impact on the interface or even the underlying architecture. For example, in testing a prototype of a travel Web site, we learned that users insisted on seeing some information that had to come from the airlines' databases. On one hand, this was bad news because it was technically difficult to get this information. On the other hand, the team learned about this need in time to address it in the first release, or else the site would have failed.
  2. Make fast changes
    With a paper prototype, you can revise the interface very quickly, even during a usability test. If a term is confusing, you can cross it out and try a different one. You can add an example next to a confusing edit field or a sentence of explanation. Simple changes like these can solve many usability problems. Even when you find the need for more substantial changes, you can often make them in a matter of hours. One team completely redesigned an e-commerce interface in one evening after learning from the first two tests that it had some fundamental flaws. The next day's tests confirmed that they'd fixed the problems.
  3. Find out what marketing really wants
    It's frustrating to implement something that wasn't quite what your marketing department had in mind. I call this the “Bud Light problem” after a funny series of Budweiser commercials in which someone says, “Bring me a light!” and is then presented with a lamp, flaming torch, fireworks -- anything but a light beer. It's not so funny when you've spent days or months developing the wrong thing. Interface specs -- even with screen shots -- don't solve the Bud Light problem because they don't show the behavior of the system, and they also don't show you what people (especially non-technical people) expect the interface to do. But when you put a paper prototype in front of someone and watch them use it, you'll quickly discover whether it meets their expectations. Invite marketing to your prototyping sessions, and make sure they attend the usability tests.
  4. Eliminate technology variables from the usability testing equation
    In eight years of usability testing, I've collected a number of “war stories” where technical glitches caused tests to be postponed or cancelled. This was disruptive to the development teams, which were relying on the tests to answer important questions about the interface. But I've never had to abort a usability test due to a problem with a paper prototype. I've even done paper prototyping during a power failure!

Paper prototypes give you complete control over how the interface behaves -- this is useful for short-notice demos as well as usability tests. Paper prototypes don't use databases, networks, servers, or any other form of technology, which eliminates most of the things that can go wrong. You don't have to worry about someone putting a change up on the development server that breaks the very thing you're trying to test (and this also means that development doesn't have to come to a complete stop every time you want to do a usability test).

You may be thinking, "But this is my application. I'm the one writing the code." Yes, but until you're well into the debugging stage, your code (or code written by others that yours depends on) may not be stable enough to allow users to do meaningful work without crashing. Bugs happen -- there's a big difference between the pleasant fantasy of &147;These ones and zeros obey my every whim” and the painful reality of “I'd be better off if I hadn't come in today.” I hope I'm not the only software developer who's ever said that!

Common concerns

Even people who recognize the benefits of paper prototyping may have some concerns about the technique. The questions I commonly hear fall into three categories: validity, professionalism, and resources.

  1. Validity ("Does it find real problems?")
    I'm often asked whether paper prototyping might have bad side effects, either in terms of missing important problems or finding false ones. The short answer is that paper prototyping does find most of the same problems that you'd find by testing the real thing.

    To my knowledge, there is only one published scientific study (see Resources) of the validity of paper prototyping. Three researchers at Verizon (formerly GTE Labs) compared the type and number of problems found with a low-fidelity (i.e., paper) prototype as compared to a working prototype. They found that there was a significant degree of overlap between the problems found using the two different methods. Although the working prototypes did uncover a few more problems, they also took significantly longer to develop than the low-fidelity ones -- weeks instead of days. These results indicate that there are diminishing returns from taking the additional time to develop a polished prototype.

    My own practical experience backs this up. I have conducted several hundred usability tests of software and Web sites, including more than 100 using paper prototypes. With only a few exceptions, I find the same kinds of problems with both.

  2. Professionalism ("What will others think of it ... and us?")
    The thought of showing an unfinished -- or even flawed -- design to outsiders makes some developers uncomfortable. Many of us are perfectionists by nature and/or training, and our jobs reward this trait. It's natural to be concerned that others will perceive our work as incomplete or sloppy.

    Fortunately, that's not what happens. Users (and coworkers) respond positively to a paper prototype, provided that they've been briefed appropriately. When participants arrive for a usability test, I explain that we know the design has some rough edges (sometimes literally!) but that we want to get their feedback before we've invested a lot of effort to build the wrong thing. I've found that users understand this, and I've never heard a user make a disparaging remark about a paper prototype. On the contrary -- users appreciate knowing that their input is being requested while there's still time to incorporate it.

  3. Development resources ("Do we really have time to do this? Is it just extra work?")
    Those who have never done paper prototyping (or even usability testing) often overestimate the amount of time it will take. If you're already doing usability testing and only the paper prototype part is new, estimate no more than 1-2 days for prototype creation if you're working mostly from screen shots, and up to a week if it's a new interface or a complete redesign. Any more than a week and you're spending too much time polishing the design before getting user feedback. (The other activities associated with planning, preparing, and conducting a handful of usability tests typically take on the order of a week of hands-on time, spread out over 2-4 weeks.)

    Sometimes paper prototyping will uncover planned functionality that doesn't need to be implemented. For example, I once tested a paper prototype of a distance learning Web application. The original design called for users to have avatars to navigate the virtual classroom, raise their virtual hands, etc. In prototype testing, we quickly learned that all this virtual 3D stuff got in the way of the real purpose of the application, which was to enable large companies to provide training to geographically dispersed employees. The designers realized that they were jeopardizing their whole product concept by trying to make the application feel like a video game. They dropped the 3D functionality, which actually shortened their development schedule. This was five years ago, and the product is still around, sans 3D.


Conclusion

It's normal for interfaces to undergo substantial changes between the initial concept and the actual release. No one enjoys writing code that ends up having to be overhauled or scrapped. Paper prototyping lets you find significant problems early in the design process, so you can experiment to find the best solution before investing the effort to implement it.

Resources

The article “Usability problem identification using both low- and high-fidelity prototypes” by Robert Virzi, Jeff Sokolov, and Demetrios Karis is available from the ACM Digital Library. (You must be an ACM member or pay a US$10 fee to download individual articles.)

Read the author's list of useful office supplies for paper prototyping, including hard-to-find items like removable tape. Includes links to specific items on two office supply Web sites.

The book A Practical Guide to Usability Testing by Joe Dumas and Ginny Redish is a good introductory reference for usability testing.

The Society for Technical Communication maintains a “Topics in Usability” site containing resources on usability testing.

From User Interface Engineering's site, see the articles “Five Paper Prototype Tips” by Matthew Klee, and “Using Paper Prototypes to Manage Risk“ by Carolyn Snyder.

Check out Carolyn Snyder's article “Seven Tricks that Web Users Don't Know” to find out what you may be taking for granted about your users.


Snyder Consulting * (603) 216-2255 * csnyder at snyderconsulting dot net