Testing Methodology

Flow Diagram | Testing Goals | Scenarios | Results Sheet | Consolidation | Solution Brainstorming

Process Flow

Our testing methodology is largely structured around the prototype iterations. Getting design information from our prototype, the sequence of steps in the user testing process generates new or revised ideas, which in turn provides feedback to the redesign iterations. Think-aloud sessions are the main usability testing method we used for our testing. Heuristic evaluation is also performed on some prototypes to eliminate any glaring usability problems before the think-aloud sessions.

Testing Goals

After designing each prototype, we identify specific areas of the design and formulate testing goals based on these features. These goals set a focus to each testing session, and help testers identify critical incidents specific to these goals.

Scenarios

The scenarios were constructed to reflect the graphing needs of our actual users in the city. With the testing goals in mind, we wrote the scenarios to give users motivation to use certain aspects of the interface during the testing session. For man-on-the-street user testing, we used instructive scenarios which required the users to accomplish certain tasks. When we user-tested with our actual users, we included scenarios which were more investigative and required users to explore the metric data for interesting points in a free-form manner.

Results Sheets

As we did the think-aloud user testing sessions, we realized that in addition to keeping track of whether certain tasks are accomplished, we also had to take note of a lot of critical incidents. As a result, we created a Results Recording sheet to record certain factual observations easily. The sheet consists of sets of requirements with checkboxes for each scenario. It also includes certain features like 'Search' and 'Expand All' functions that we were interested in seeing if the users used during the sessions. This Results Recording sheet has been very helpful for us to track the extent of tasks that were accomplished.

Consolidation

After each user testing, we wrote Usability Aspect Reports (UARs) to record specific critical incidents from the testing session. We also held consolidation meetings to categorize the problems and good incidents into the specific areas of the design. This allowed us to manage a huge amount of information from each testing, and also helped us focus on major design issues.

Solution Brainstorming

When we identified major redesign areas, we started brainstorming for various solutions for the problems. These redesigns will drive the testing goals of the next user testing round.

<< VB Prototype 6

Heuristic Evaluation I >>