User Test II

Description | Test Set-up | Findings | Links to Data

Description

General

At this point, we had a prototype with the most complete features so far, and we wanted to test as many features as possible. This was the first man-on-the-street user testing round with people who had no domain knowledge. We wanted to know whether general users could create different types of graphs with no training at all.

Test Set-up

Goals

• Can they navigate to the required metric?
• Do they even use the search function? Is it useful?
• Can they do multi-selection with the tree structure?
• Can they graph multiple metrics, years and divisions?
• Is the time axis control intuitive?

Scenarios

Since there was a huge combination of graphs that can be created using the prototype, we wanted to be able to user-test as many combinations as possible. As a result, we created a table showing all possible graphing combinations and designed three sets of scenarios to cover different subsets of all possible combinations. To ensure we had more reliable results, we made sure that each set of scenarios is tested with three users.

Findings

Metric Tree

• Users could navigate to the required metric easily without using the search function.
• Abbreviations in metric labels (#, %) were well-understood.
• Some users were not familiar with the control-click multi-selection interaction.

Time Axis Control

• Users found the calendar control very unfamiliar and tedious to use.
• Too much detail in the calendar were shown sometimes. When only months needed to be selected, users have to make sure they get the days right too.

Graph Options

• Users had difficulty creating graph to compare by divisions and by years.
• The graph option labels were not well-understood.

Graph Area

• Users did not understand the unit descriptions on the y-axis.
• The legend was redundant with just one line on the graph.

Links to Data

Prototypes

<< CitiStat User Test I

Heuristic Evaluation II >>