How to Measure Impact in B2B User Research

Matthew Morrison is a UX Research Manager at Braze, a customer engagement platform. Before Braze, he worked on the research teams at Etsy and WillowTree. It took him a while to figure out which research projects would have the most impact, and he's excited to share what he's learned along the way.

Impact is kind of a nebulous term. 🪐

• How do you determine what research is high-impact?
• Who decides what is high-impact?
• What does impact even look like or mean?

The list goes on — and the answers aren’t always very clear or easy to find. 🧐

Over the years, I’ve seen research both directly influence and have the highest impact on an organization’s strategy (like its product strategy). While measuring impact will naturally differ based on many factors — like industry, team size, priorities, etc. — through my own experience in User Research, I have some answers and guidelines for how to measure impact in B2B User Research.

First, how do you know if your research is high-impact?

Buy-in and visibility from leadership at the get-go will help set you off on the right foot for driving real business impact. Leadership can also help determine which projects would have the highest impact, and can help set clear priorities. And bringing leadership into your research will help you identify this impact.

Now, it’s important to note that not all projects will be high-impact. Depending on your team and org, lower-impact projects (e.g., usability testing) are usually done by the designer or product manager with guidance from a researcher. It’s important to mention that these are still important projects! That way you can focus and prioritize on high-impact research — and there will usually be plenty of high-impact projects to work on. For example, at Braze, the ratio of low- and high-impact research is 20:80.

How to measure impact

I measure impact through a process of iteration. Here are the steps I follow:

1. Determine how nebulous is the problem-space. If it's a complex problem space, then this is a good sign it will have a high impact.
2. Interview — casually or formally — stakeholders that would likely be affected/influenced by the research. In my experience, a lot of researchers forget this step. However, stakeholder interviews are helpful in getting buy-in, socializing your work, and refining your problem space.
3. Hypothesize what types of impact you think your research will have — product, cultural, and/or internal (based on Victoria Sosik's model).
4. Once the research is complete (or even if it isn't!) and you’ve socialized the work, see whether your hypotheses were correct.
5. Note if something didn't have the impact you thought it would or had another type of impact and use it to influence how you measure impact for future projects.

Utilizing Victoria Sosik’s model for measuring impact

Victoria Sosik’s model makes it quite easy to understand where your research has the most impact. In a talk given at a virtual UXR conference in 2021, Victoria shared 8 ways to identify impact in an organization. Research has impact when it:

• Influences product change
• Influences product strategy
• Increases stakeholder exposure to users
• Shares communication
• Prompts further research
• Prompts a new collaboration
• Elevating the role of user research
• Develops infrastructure

These can be grouped into larger buckets like product, cultural, and internal impact, which I mentioned above. For example, my research has the most impact if it influences a team's roadmap or strategy (product), if it elevates UXR's status and leadership relies more on research for decision-making (cultural), or if it helps determine a new process that gets picked up by teams (internal).

How do you communicate the impact your research has so that you can lobby for more UXR resources?

The two best ways to do this are:

• Show how your research unblocked a team so they could proceed with their roadmap. 🔓 Even smaller-impact evaluative studies, like concept testing or usability testing, can help save engineering effort by building the right thing the first time.
• Benchmarking is a great way to communicate impact. 🪜 Say you want to decrease the time it takes to perform a task, like adding to cart:
  

1. Establish a benchmark. For example, how long or how many steps it takes to add any item to your cart.
2. Do whatever research is needed to see how you can reduce the steps/time.
3. Complete the same study again to show how you improved this metric.

If UXR doesn't have full buy-in at the leadership level, one trick I've found in my experience is to share your research in an already existing meeting where leadership is present.

Tips for measuring impact

If you’re just starting to measure the impact of your research, here are 3 important things to do:

This last step is vital because it helps you know if your research is actually having the impact you want it to and allows you to track the various types of impact of your research. One way to document your impact is to keep a spreadsheet of studies you complete and record what impact you expect them to have and the impact they actually had.

Conclusion 

Ultimately, this is just one framework to help you start measuring your research impact. Feel free to modify it to fit your preferences, team, or organization. Also, take the time to learn what impact looks like for your organization; from there, you can hypothesize the types of impact your projects may have and begin prioritizing your projects accordingly. Finally, don't forget that your projects may not have the impact you thought they would. That's ok! Learn from that experience and iterate, just like you would in product development.

‍