SUMMARY

As a volunteer for Open Oakland (a Code for America Brigade that meets weekly in Oakland to work on projects that support civic engagement), I worked on the user experience for the Open Disclosure project. This project aimed to make data on campaign finance contributions more visible to the city's voters. The Oakland Mayoral race was ramping up so it was important to share campaign finance data with the public in a way that was easy to understand.

CHALLENGE

We had been working diligently to get the responsive website up and running but had done no user research. Our team met Tuesday evenings at a room in Oakland's City Hall. Team members were volunteers with full time jobs, which made user testing a challenge. However, we understood the importance of user research on product development and decided to find a way to test our MVP website.

CIVIC USER TESTING

One of our fellow volunteers had heard about a group in Chicago, CUTgroup of Smart Chicago Collaborative, which had created a model for user testing civic digital products. We hoped to emulate their successful implementation of user testing in Oakland. Following their lead, we began creating a process for recruiting users who were representative of the city's population. The first project we would test was Open Disclosure.

One of the local libraries granted us access to a spare room on a Saturday so that we could set up a space for in-person user testing. Now we just needed to find users interested in giving us feedback!

RECRUITMENT

We wanted to ensure that we found a diverse group of testers who represented the diversity of Oakland. We created a web form to survey potential users and asked some basic questions to understand how users engage with technology and if they tend to vote.

Check out the  Cut Group Survey

SETTING UP THE STUDY

Now that we had date, setting & participants, it was time to get to define the details of the study. I worked with a seasoned user researcher to device a test plan based on the questions we had about the site. Here are some of the questions we hoped to answer through user testing:

• What campaign finance information do Oaklanders want to gain from the website?
• What aspects of campaign finance confuse or interest users?
• What features on the website are challenging for users?
• What do users like/dislike about the website?
• What additional features would users want to see on the next iteration of Open Disclosure?

Read the final  test plan.

USER TESTING

The in-person study took place at the West Oakland Public Library on Saturday, February 21, 2015 with 11 participants. Volunteers from Open Oakland came to facilite the tests. Everyone was trained on how to conduct the test and record their findings.

After the participants left, we had a quick debrief as a team to hear what each test facilitator heard. Each participants feedback was recorded with some screen recordings too. We also captured notes for each session to analyze.

ANALYSIS & REPORTING

Now that we had collected all this feedback, it was time to review the recordings and notes to put together a report summarizing the learnings. I provided some background information about why the study took place as well as methodology. Then I analyzed users' completion of tasks as well as any notes from their session. The study revealed a few areas where some updates might make the site more informative:

• Reconsider the information presented above the fold on the homepage to better illustrate the breadth of information found on the site.
• Additional filtering tools on the contributor lists would make it easier for users to search and understand the source contributions.
• Some of the interactive elements on the data visualizations were looked over and need further emphasis or labeling to aid discoverability.
• Users expressed interest in understanding further what affiliations individual contributors might have. In other words, names alone were often hard to interpret.
• For users who are less familiar with campaign finance, use simpler terminology or provide definitions for specific terms.

Read the final  report.

Our team took this valuable feedback and incorporated it into our design and later iterations. Additionally, user testing became a regular part of Open Oakland project work.