Live from THiS 2024: A conversation with UserTesting’s SVP of Product

During UserTesting’s 2024 THiS conference in October, Rally’s CEO & co-founder, Oren Friedman, sat down live in Austin, Texas, with Andrew Jensen, SVP of Product at UserTesting, to discuss how UserTesting used Rally to build a scalable customer panel and streamline manual research processes.

Can you share your journey in building a research- and customer-centric culture, and how that’s evolved over time?

Andrew: At UserTesting, no surprise, we try to practice what we preach by being user-centric. We focus on getting customer perspectives into our product lifecycle and validating decisions at every stage, whether it’s product strategy, roadmap validation, design iteration, or release planning.

Our goal has always been to lead by example and demonstrate the benefits of being user-centric. This effort really evolved during the merger of UserZoom and UserTesting. Suddenly, we had more researchers, designers, and product team members, which allowed us to scale.

It was interesting because UserZoom had a traditional approach where researchers handled all the insights. Product teams and leadership relied on researchers for validation. On the other hand, UserTesting emphasized democratization – designers and product managers were empowered to directly connect with customers.

Both approaches had their strengths, and combining them allowed us to scale strategically. Researchers focus on strategic decisions like roadmap validation and foundational studies, while designers handle quick feedback loops by engaging directly with customers.

However, as our research scaled, we faced challenges, particularly in keeping up with the demand for customer engagement.

What challenges did you face as you scaled your research efforts?

Andrew: One major challenge was creating consistency across teams as we grew. We invested more in Product Operations and Research Operations to standardize processes.

Another challenge was engaging with customers at scale. While we used UserTesting’s network to cover about 75% of our needs, we also needed a structured way to connect with our passionate customers: UX professionals and researchers eager to give feedback. Unfortunately, we didn’t have a smooth onboarding process for our customer panel, and we lacked visibility into the breadth of the panel, leading to over-surveying the same participants.

This created poor customer experiences, with some opting out due to overuse. Managing incentives was another issue. Using tools like Amazon gift cards was cumbersome. These challenges prompted us to look for better solutions.

What solutions did you try before Rally came into the picture?

Andrew: Initially, like many of our customers, we relied on disparate systems – Salesforce, spreadsheets, web forms for opt-ins, and tools like Monday.com. Maintaining these systems was labor-intensive without improving efficiencies.

When we attended our Human Insights conference in Seattle last year, we started exploring Rally. After seeing how Rally helped our customers, we decided to implement it.

How has implementing Rally impacted your research workflows?

Oren: Fun fact – it’s actually been a year since UserTesting launched their User Experience Research Program powered by Rally at THiS 2023. Since then, we’ve worked closely together to refine and expand how Rally supports their Research Operations. Let me give you a quick overview of what the platform does.

As the first Research Operations Platform, Rally is, at its core, a participant CRM.

That means we provide a highly flexible, scalable participant database where companies like UserTesting can store hundreds, thousands, or even millions of participants – if you’re lucky enough to have that many willing participants. Rally helps organize these participants based on any data you have, funneling in information from tools like Snowflake and Salesforce, which are both Rally integrations that UserTesting leverages, to keep participant panels fresh and up to date.

On top of that, we add a layer of governance, security, and guardrails because we all know how valuable and sensitive this participant data is. We ensure that only the right people within an organization have access to personally identifiable information (PII) and other critical data.

And layered on top of that is the entire recruitment process, which Rally streamlines. It’s one thing to have a well-organized database; it’s another to be able to:

✔️ Easily contact participants with the right messaging

✔️ Screen and schedule them for studies

✔️ Obtain consent

✔️ Send them to the tools you need for research (like UserTesting)

✔️ Close the loop with incentive management in Rally

✔️ Integrate seamlessly with the rest of your research stack

Andrew: Exactly. Rally has improved our customer experience immensely. For example, we now use Rally for customers to join our experience panel. Customers can even opt in via QR codes at events like this, selecting the type of research they want to participate in.

Internally, our teams can easily find participants for specific studies, whether for quick feedback on a design or more in-depth research. Incentive management is also much smoother. Additionally, we can see when invitations are sent out through email via Rally. We can track which customers have seen the email, who has clicked on it, and who is actually engaging and taking the test.

The same goes for our moderated capabilities. Our workflow allows us to schedule calendars so everything is properly coordinated. When setting up and running a moderated session, Rally ensures that both the moderator and participants are aligned, making the process seamless.

The result has been remarkable efficiencies: reducing recruitment timelines from weeks to just days and improving customer satisfaction.

How do Rally and UserTesting work together to support research?

Oren: At the end of the day, companies have an equally important need to connect with both non-customers through a panel like UserTesting and their own verified customers when a study requires that type of feedback.

But when you’re recruiting your own customers, you don’t want to be locked into just one methodology. You need the flexibility to send them to UserTesting, conduct moderated sessions on Zoom, or use whatever tool best fits your research needs. That’s why Rally and UserTesting complement each other so well, enabling teams to conduct research with their own user base while also seamlessly tracking and managing participants.

At the same time, it’s critical to ensure that designers, PMs, and other stakeholders have the right guardrails, governance, and templates in place. Research with existing customers can be complex, sometimes even requiring approval from other teams just to reach out.

Using Rally alongside UserTesting gives our customers confidence that outreach to their users is done safely and with proper approvals. It ensures that messaging is aligned with marketing, that outreach is on-brand, and that customer engagement is approved by customer success teams when needed. Since all of this flows back into Salesforce, where customer teams operate, everything stays connected.

Ultimately, our goal is to elevate the research practice and make it easier for teams to connect with their users.

Can you share some success metrics from the past year?

Andrew: Over the past year, we’ve onboarded 550 customers. Recruitment timelines dropped from 1-2 weeks to just 2.4 days. This has been a total game- changer because our recruitment process before Rally was slowing down our product decisions and making it hard to keep up with the agile needs of our product teams.

This is something that all our teams, especially our Research Ops team, are super proud of.

We also achieved a 104% fill rate, improved email open rates, and significantly enhanced customer satisfaction by targeting research opportunities more effectively.

When we can better reach the right customers and conduct more targeted recruiting, we see a significant improvement in open rates, engagement, and overall participation. By connecting customers with research that’s relevant to them, they’re more likely to engage and provide meaningful feedback.

Oren: Recruiting the right customers can take weeks, and we don’t want recruitment timelines to be a bottleneck for research. You already have so many other priorities, and since recruitment is the first step in the process, delays at this stage can slow everything else down.

When recruitment drags on, it holds up the entire research process and that doesn’t help anyone. Cutting that timeline down to days instead of weeks, and sometimes even hours, allows teams to deliver insights faster and drive a more user-centric culture.

Watch the full conversation

This conversation just scratched the surface of how UserTesting built a scalable, efficient customer panel with Rally. Watch the full recording here.

Want to improve your Research Ops? Get the ReOps Maturity Matrix to start assessing your team’s strengths and opportunities for growth. Download your copy now. 

Want to see Rally in action? 

Explore our Interactive Demo Center to experience how Rally helps teams build and manage customer panels, automate recruitment, and integrate seamlessly with tools like UserTesting and Salesforce – all in a hands-on, self-guided experience.