User Test III

Description | Test Set-up | Findings | Links to Data

Description

General

After the last man-on-the-street user testing, we made a few major redesigns to the graphing option regions and made some minor visual tweaks to the graphing behavior. We wanted to see if people could create and understand the graphs more easily with this redesign.

In the previous user test, we learned that users had problems with multi-selection in the metric tree browser using control-click, as well as the calendar control. As a result, we decided to first train our users and get them familiarized with the tree and the calendar control, before they proceeded with the tasks. We wanted to assess the effectiveness of training in this user test.

Test Set-up

Goals

• General Graphing Options
• Do the users understand the Graph Data checkboxes (Values, % change, etc)?
• Can the users create graphs with different lines for divisions and years more easily?
• Given training, do they have problems with the calendar control?
• Metric Tree Browser
• Can they navigate to the required metric?
• Do they use the search function at all?
• Do they use the Expand / Collapse All function?
• Given training, do they know even to do multi-selection?
• Given training, can they do multi-selection with the tree structure?
• Graphing Behavior
• Are they comfortable with the graphing behavior (e.g. titles, axis labels, the time divisions)?

Training

Training Program

The goal was to familiarize the users with the calendar control and metric tree multi-selection interaction. We created a separate training program for demonstration as we did not want to instruct the users in how to use the actual prototype. So we created this program using the same controls, but with a different purpose. Our hope was that through the demonstration, the users could learn to use the controls.

Before the users read the scenarios, the tester would demonstrate a program which was used for purchasing a list of books to be delivered at a specified date. First, the tester showed the user how to navigate through the list of available books in the tree browser. He showed the user how to expand and collapse the items using the "+"/"-", as well as how to select multiple books with control-click. Next he used the calendar control to select the delivery date. The 'Today' function and the up / down arrow keys to change the year was quickly demonstrated.

Findings

• All the users were able to find the metric with the tree without using the search function.
• All the users managed to succeed in comparing by years and divisions
• Two users did not learn from the training. They still had problems with the multi-selection interaction in the metric tree browser, and the calendar control.
• We realized that the users had a work-around for selecting a date range for a year. The users compared by years, selected the year they wanted and then selected the months on the time axis control. This is not the intended, albeit valid way, to set the time range.
• The users are understanding the graphing area better.

Links to Data

Prototypes

<< Heuristic Evaluation II

CitiStat User Test II >>