Brad Orego (they/he), Research and Research Ops leader, joined the Rally team on June 20 for an AMA to discuss elevating the role of Research. Brad covered topics like building strategic partnerships, metrics for tracking the ROI of Research, the evolution of the role of Research, and why Research should be its own horizontal function. 

If you missed it, or want to revisit the highlights, read our recap below. If you’d like to watch a recording of the full AMA, follow this link.

Key Takeaways

🔑 Researchers need to make sure we’re doing the right work by understanding the business context and pushing back on some requests coming our way.

🔑 Ask the 3 Essential Questions: Why are we doing this? What are we hoping to learn? What will we do with the results?

🔑 Leverage Research Operations to identify where your efforts can have the greatest impact on driving process efficiency and enabling self-service research.

🔑 Self-service (“democratized”) Research is an important tool in growing and building research maturity across the entire organization.

🔑 Don’t let yourselves be pigeonholed into supporting Design or Product. Think broadly about the impact you can have and the teams you can collaborate with.

🔑 It’s time for Research to grow up. Start managing up and learn to speak the language of business. It’s the only way we’ll forge a path toward executive representation.

Who is Brad?

Brad is a Research and Research Ops leader with vast experience across agency, startup, and enterprise companies. Brad has established an impressive career in User Research, with notable experience in cultivating research practices that bring significant business value. At Auth0, their team played a critical role in setting strategic direction and influencing both product and marketing roadmaps, making them an invaluable resource.

How did Brad get into UX Research?

Like many in the UXR world, Brad’s path into User Research wasn’t your typical journey. As a teenager, Brad wanted to understand people and make video games — which led them to study both Psychology and Computer Science.

Fast forward a few years when Brad discovers human-computer interaction (HCI) and realizes “this is what I’m meant to do.” In the early days, startups would hire Brad to “do UX/UI stuff” — a scenario without guidance, mentorship, or management. And Brad had to just go figure it out for themself. Now it’s been 15 years and Brad has watched the entire UXR industry — and their path in it — evolve.

Why does Brad care about Research?

“Research is a way to be more efficient,” said Brad. “How many times has a product gone out that no one cares about or it doesn’t work right? Research ensures that we are doing the right thing and doing the thing right.”

Regarding Research Operations, Brad explained that it appeals to them for similar reasons. "Research Ops is about saving researchers time and making them more efficient. That's a very engineering-focused mindset." As a computer science graduate, Brad saw the birth of Research Operations as an opportunity to apply engineering concepts to research — finding places that are inefficient and making them more efficient.

Brad said they were able to utilize their coding background to increase their Research Ops impact through automation. “And guess what,” said Brad. “Automation takes a little bit of coding and technical knowledge.” Even if you’re not writing JavaScript, they continued, automation requires a process-oriented mindset. “My coding background has been pretty essential for me and gives me a different way of thinking about things,” said Brad.

{{quote-brad-1="/cms"}}

How to build strategic relationships with Design & Product

When Brad joined Auth0, they came in with a hypothesis: "What happens if we essentially ignore doing research for a year and focus on operations?" This approach can set the stage for a completely different relationship with these partners.

Brad explained that the conventional route often involves Product or Design teams working until they realize the need for a dedicated researcher. However, this method tends to slot researchers into a specific function or team. Brad’s approach challenged this dynamic. “We set up the context and nature of the relationships with Design and Product differently.”

At Auth0, both designers and researchers at Auth0 found that Product partners wanted to be challenged. “PMs know that they don’t know everything,” said Brad. “They want to gather as much information as they can to make the right decisions.” Because of this, Brad said, researchers need to push back and not be subservient to the product.

However, Brad cautioned that researchers should also understand the product manager's world. "I am always pushing my team to talk to their PMs and ask, ‘Why are we doing this? What’s the long-term vision? We’re building this thing right now, but what’s happening 12 months from now?’”

Understanding the bigger picture not only eases conversations but also improves one's skills as a researcher. "At the end of the day, they know they need research, but they don't necessarily know what research they need. Having more context will help make sure you’re doing the right research."

Additionally, Brad shared three important questions to ask to determine if you are doing the right research. Your answers to these questions will help you do more research that is tied to concrete objectives and outcomes. 

1. Why are we doing this? 
2. What are we hoping to learn? 
3. What will we do with the results?

How do you set up this different type of relationship?

Brad’s approach to setting up relationships was both top-down and bottom-up. The top-down approach requires you to ensure that everyone in leadership knows the role of the researcher: Not to just do what they are told, but to provide the necessary insights and intelligence to make decisions. Brad said to be clear that you want to work with leadership, just not only execute their requests. “Ask questions, point out blind spots in the roadmap,” said Brad. “Have those conversations.

The bottom-up approach involves understanding the team's needs and helping them understand where research fits in. It’s important to spend time educating, evangelizing, and helping the cross-functional partners and researchers understand how research is not just something you do at the beginning or end of a project, but all along the way.

At Auth0, they conducted monthly "research deep dives" – 45-minute lecture-demonstrations followed by Q&A on various research topics. This process not only helped answer recurring questions but also served as a rich library of resources for newcomers. “It was basically a college level course in how to do UX Research,” they joked.

Brad admitted they initially made a mistake by starting only top-down but soon realized the need to bring everyone along and spend more time and resources to guide people, not just introduce a new thing and move on.

What were some of the positive impacts of the research deep dives?

Brad explained that they initiated the deep dives because they found they were repeatedly answering the same questions. They wanted to document and automate as much of the onboarding process as possible. However, they also understood the importance of human interaction. So, when a new member joined, they'd ask them to go through the onboarding course first, which would ideally answer about 80% of their questions. The remaining 20%, usually the more specific questions, would then be addressed personally.

To determine some of the topics for these deep dives used in the onboarding process Brad asked questions like:

• What are the questions we keep getting asked?
• What are topics someone needs to know when they join the company?
• What’s the current onboarding process like?
• How much of that process can we document and automate?

This approach also helped with project-related questions. For example, if someone needed to conduct a card sort, they'd be asked to watch a relevant deep dive session first, establishing a base level of understanding. “Give people the information they need upfront so they can digest and go through it on their own time.”

Internal & external metrics for tracking the ROI of Research

Brad divided the tracking of Research ROI into two categories: internal and external metrics.

Internal metrics include cycle time, number of users you’re talking to, and the amount of research conducted. For instance, the number of studies completed could indicate an increase in research activities, but its value depends on the impact made by these studies. A single transformative study could be more valuable than several smaller ones.

External metrics focus on how many people within the company are utilizing research resources, such as the insights repository, or attending research presentations. It's also crucial to know where research requests are coming from — Product, Marketing, Customer Success, etc., Brad said.

These details can help to identify the areas where the Research function is making an impact and inform budget discussions. For example, if a significant portion of requests come from marketing, but no budget is allocated from there, it could be an area for negotiation.

Additionally, it's important to understand whether the business values more internal or external outcomes. If the focus is on driving the product forward, then research efforts might be more aligned with improving key performance indicators (KPIs) relevant to the product, said Brad. On the other hand, if the company is more interested in internal metrics, research might focus on how to achieve this, for example by decreasing the delivery time for engineering and design tickets, or decreasing the time it takes to onboard a new hire.

Brad highlighted that research could lead to significant labor savings, which can be a compelling argument during budget discussions. By showcasing potential savings in labor hours thanks to efficiency improvements brought about by research, you can justify the need for more research resources.

How to track Research impact on your org’s KPIs

Tracking Research impact on an organization's KPIs can be challenging, especially when there are no existing analytics in place, explained Brad. "The very first conversation I had with our chief product officer was about not having analytics in place,” they said. “That was probably the moment where I realized we likely won’t be able to tie our value as a Research function to increasing conversion or retention or anything like that, because we can't report on it or show any value.”

However, if you're at a small company or have some influence, you might be able to make a change. "Find the tool that works for you and your organization, then convince the engineering team to put it in place." If analytics are already in place, then work with the owner of the analytics to find a way to track Research's influence, said Brad.

When it comes to tracking impact, "it's always about contrast and comparing — here's a new feature that we launched that had research, here's one that didn't, now what's the difference in performance between these two?"

How do you expand the reach of Research outside of your organization?

"This is where your partnerships with marketing are really important along with partnerships with external analysts," they said. By connecting with external analysts from companies like Gartner or Forrester, they were able to share their research more broadly.

Blogging was another strategy Brad mentioned to help expand the reach of their research. "We did a lot of blogging at Auth0,” they said. “It was a channel that we control ourselves that is completely independent from anything else."

Some of the blogs Brad and their team wrote would end up featured on the official Auth0 blog, such as the blog they created when they launched their Research program. Once featured on the official company blog, it was also shared on the company’s social media channels.

Moreover, they stressed the significance of presenting research at various professional organizations and consortiums. For example, Brad presented passkey research to the FIDO Alliance. The presentation caught the attention of large tech companies who expressed interest in further discussion.

"I push my team a lot to present at local meetups, go to conferences, pitch your ideas,” said Brad. “Anyone can go write. Anyone can put things out on the internet.”

How has the role of Research evolved?

When Brad first started in User Research, they embarked on the usability engineering route. “That’s what UX and UX Research used to be,” they said. “Research used to focus on whether a button was in the right place or if we were saving a few milliseconds on every click.”

Brad claimed the industry began to move past that, but “we got stuck.” They further suggested that the industry hasn't grown as much as it should have, a factor which might be contributing to the current layoffs. "Research has not evolved as much as it should have,” they said.

Brad argues that researchers often find themselves caught in a rut. "Many folks just do the work they're told to do without thinking critically or understanding the business case."

"Think more broadly about how you can serve your organization," Brad continued. “What if Research worked more closely with the marketing team? What about customer success? Or sales?” they suggested. At Auth0, Brad would present and provide Research team updates as part of the training for sales team members. “Being integrated with teams like sales is something that I see very few Research functions doing.”

Why should Research have its own horizontal organization?

“I’ve been advocating for Research to move out of the Product org since my early days at Auth0,” said Brad. “When Research is under Product or Design, they control your budget.” This, they explained, leads to situations where there’s a need for more research but a lack of resources or proper prioritization.

Brad stressed the need for a culture change, one that includes positioning Research as an independent function. "Part of that is moving Research out of Product and Design, which will make it easier for us to work cross-functionally," they said.

Additionally, Brad said, there needs to be a new role: Chief Insights/Intelligence Officer. “It should be someone who understands what Research does and can effectively communicate our value.” Brad explained that this role would be over any disciplines that generate or analyze data about customers or the market.

Brad emphasized the necessity of reorganizing the structure of organizations as putting Research inside Design or Product doesn’t make sense. They explained, “We should create a Centralized Intelligence function that includes the Data Science team, the Pricing and Strategy team, the Growth Marketing team, and the Competitive Intelligence team.”

To implement this at Auth0, Brad formed the Customer Intelligence Alliance, a group that brought together members from various parts of the organization to increase open discussions and collaboration. “Building these relationships and having these conversations is the first step towards the change,” they said. “Over time, multiple voices saying the same thing to leadership can highlight the missed opportunities due to the current organizational structure.”

{{quote-brad-2="/cms"}}

Brad explained that this shift would provide research with the resources it needs and create room for meaningful work. "Such a shift would allow us to build the team we need, refuse projects that don't align with our focus, and dedicate our capacity to initiatives that are both interesting and important.”

What does a Chief Research Officer / Chief Insights Officer look like?

"Right now, in your organization, who is advocating for Research at the executive level?" Brad believes there's often no one with a nuanced understanding of Research and its potential value in those high-level conversations.

The creation of a Chief Research Officer or a Chief Insights Officer role would, according to Brad, facilitate a proper understanding and representation of Research at the decision-making level of an organization. “Until we get someone at that level, and until we really start speaking the language of business, organizations are not really going to understand what we do, and they're going to continue to suffer," they said.

{{quote-brad-3="/cms"}}

Brad also emphasized the need for researchers to educate those above them about the breadth and scope of their work, encouraging researchers to "manage up." They explained that if the Research leader can ensure their director or VP understands the full potential of Research, they are better equipped to make budget and strategy decisions that include Research perspectives.

However, Brad pointed out that being a Research manager is different from being a researcher, and many Research leaders haven't necessarily been trained in management. “Research management is fundamentally different from being a researcher,” they said. This is an area where many research leaders and managers could improve.

Brad’s advice for Research leaders

“Talk to peers at other companies,” they said. “See what they’re doing, what’s working, what’s not working.” Additionally, Brad recommended asking superiors for frequent feedback and guidance.

Additionally, Brad suggested participating in peer mentorship programs, hackathons, and informal coffee chats. These activities, according to them, not only help increase the visibility of the Research function but also provide different perspectives and learning opportunities. “Regardless of what level you are, you need to build relationships outside of your immediate circle,” they said.

⚡Lightning-round Q&A

We often don’t have time to cover every topic and answer every question live, so, as promised, here are a few additional responses not included in the recording:

How can a Team of One educate and evangelize in a low-maturity organization and get to do strategic research?

• Ruthless prioritization. The first step is understanding where they’re at and meeting them there. NN/g has a good model for UX maturity you can use. 
• Building out operations can then buy you the capacity & credibility to do more future-focused work.

How can we elevate the role of Research if our organization doesn’t see the value?

• Two concepts here: understand what their objections are so you can talk through them, and let them feel the pain themselves.
• You can lead the horse to water, but you can’t force it to drink. Sometimes people need to learn from their own mistakes before seeing the value.

How do you decide what’s appropriate for Self-service Research and what should be owned by the Research team?

• Effective democratization is about making clear-but-arbitrary distinctions. How and where you draw those distinctions depends on what’s appropriate for you and your team, and will shift over time.
• The more research maturity, the more people can self-serve. The more self-serve, the more you can focus on larger, more complex, and more impactful work.

How do you balance Research impacting strategy with PMs “owning” the Product/Service direction?

• One team, one score: we’re all trying to achieve the same goal, which is delivering an incredible experience. It doesn’t matter how we get there.
• It’s not our job to make decisions (because we don’t have to deal with the consequences). We’re there to inform and guide.

How do you prioritize where to focus energy with limited capacity and numerous opportunities?

• One part intuition, one part alignment. Priorities will be influenced by leadership and by product strategy, but you should also form your own opinions.
• A lot of the more fruitful research I’ve done has come from my own curiosity about a topic or a problem space.

How would you build a Research function at a marketing company?

• Consider your audience. What’s their current understanding of Research? What context are you engaging with stakeholders in (i.e. in-house or agency)?
• Make sure you’re learning and iterating on your process with every project. Knowledge sharing is essential in agency settings.

Thank you, Brad! 

We’re grateful to Brad for joining us and sharing their thoughts and experience with Research and Research Ops. If you’d like to watch the full webinar, follow this link. And if you love what you’re hearing and want more, Brad is available for 1:1 coaching. 

Elevate your Research with Rally

Rally’s User Research CRM helps you scale up research processes, empower teams to engage with their users responsible, and grow the impact of User Research across your company. With Rally, you can build templates, track research activity, set up SSO and user permissions, create governance rules, and more. Book a demo now.