CitiStat User Test II

Description | Test Set-up | Findings | Links to Data

Description

General

This was the last user testing session with our real users in Office of Management and Budget. The prototype had come a long way, going through a series of iterations. We wanted to see whether the real users could even use the prototype without any training at all. Since they were users with actual domain knowledge, we took the opportunity to user-test with a more ambiguous scenarios which required users to investigate in a free-form manner. This mimicked the actual job of a budget analyst.

Certain features like exporting, saving graph options and displaying corresponding data table are newly incorporated in this prototype. We wanted to see if the real users could use these functions easily.

Test Set-up

Goals

• To test all the features in the prototype.
• Can the users use the prototype to explore the data for interesting points?
• Can the users save and load graph settings?
• Can the users export the data easily?

Scenarios

We set up two types of scenarios in this test, namely instructive ones and investigative ones. The former required the users to do specific tasks, and allowed us to direct users to different parts of the interface. The latter provided an opportunity for the user to use the interface to explore data for interesting trends, very much like what they do in their daily work. We would be able to observe if our prototype could support their current work effectively.

Findings

• None of the users used the "Expand All" function. Perhaps this function is not necessary.
• All of them could select multiple metrics easily with control-click in the metric tree browser.
• Search was very helpful in zooming down to specific metrics they had in mind.
• When the users tried to find a particular metric, all of them succeeded with either search function or using the metric tree browser.
• All the users could not save the graph settings. Instead of looking for the save option in the "Favorites", they looked for the option in the File menu.
• Copy-and-pasting of the graph and data table was intuitive.
• The users had a work-around method for setting the time range for one year.
• They wrongly thought that settings for comparing by years and divisions were persistent.
• They were generally comfortable with using the prototype to explore metric data.

Links to Data

Prototypes

<< User Test III

Our Solution >>