June 2004
Drive Product Development via User Research
Speaker: Sheryl Ehrlich, User Research Manager - Adobe Systems

Dr. Sheryl Ehrlich, User Research Manager at Adobe Systems, spoke at the June 2004 meeting of the SVPMA on Driving Product Development via User Research. Dr. Ehrlich explained how user research plays into development at Adobe and the varied methodologies her group employs. She illustrated many of her points through a series of case studies.

Dr. Ehrlich joined Adobe four years ago and was the first member of the user research team. She has since built the department to 18 employees. Prior to joining Adobe, she was a member of the research staff at Interval Research. She received her Ph.D. in Cognitive Psychology from the University of California at Berkeley.

The User Research group works collaboratively with Product Management, Design, and Engineering to conduct systematic research about Adobe's target users. Adobe has a wide range of products targeting a varied customer base made up of corporate users, consumers, video specialists, and creative professionals. This means there are more products than the User Research team can handle. Dr. Ehrlich is pragmatic in her approach to this challenge. She focuses on maximizing what they have and often bypasses formal research methods that are not always practical. One of her main focuses is to get beyond usability and move the research earlier in the design process where it can have the most impact.

The teams typical research centers on target customers, workflows, pain points, goals, and user profiles. They also assists in determining how to prioritize features. The group is becoming more involved in future products. To that end, they are involved in ethnographic research in the U.S. and abroad to observe how people live and interact and how future technologies might change this.

As a product moves from Concept to Release, there are three primary phases: 1. Formative Research 2. Idea Evaluation 3. Product Evaluation

During the formative research phase, the research team seeks to learn about users' goals, pain points, and their current usage. They employ a series of different methods that include site visits, interviews, observation, workflow analysis, diary studies, card sorts, and surveys.

Idea Evaluation is used to obtain user feedback on designs before engineering begins implementation. This might include using paper prototypes, interactive models, competitive studies, and participatory design. In participatory design, a user might cut out screen elements and design their own screen.

Product Evaluations are conducted pre and post release. These are conducted using task based scripts, and iterative studies. Dr. Ehrlich has found that the team gets much more out of iterating on the design and conducting more small studies than by performing one large study.

In one consumer market, adobe wanted to understand who the user was and what they were doing. They also wanted to identify the segments of users. The User Research team conducted a number of qualitative site visits at the users' homes. From this, the team was able to construct a series of case study cards that contained the person and profile, their goals, challenges, and key activities.

In another project, the team was focused on understanding cross-product workflows. For this project, the user research group pulled together an internal team to create their best guess at the primary workflows. Then the team confirmed these workflows with users and adapted them to real use cases. Doing this helped identify and prioritize features.

In a third project, the company knew the space and the target customer but wanted to identify an opportunity and workflow for a new product. The User Research group first assembled workflows. Then they used direct observation. Dr. Ehrlich commented that observation often yields better results than an interview since users will often omit steps in their explanation. The team then used card sorts to construct the underlying mental model. Users layed out the cards and grouped similar tasks into categories. This resulted in a break through for the researchers. Going into the project, Adobe thought there were only three buckets for features. The study concluded that the user views the tasks as mapping to eight categories. Dr. Ehrlich concluded with some words of advice:

• Advocate user research within your company
• Understand the market, but also the user
• Speak with a wide variety of users

  • This is inquiry, not selling
  • Suspend judgment
  • Probe deeply
  • Start early
• The data you collect will also help to build consensus

Lastly, she mentioned a few references that product managers might find helpful: