STORIES

Defining research impact

UX RESEARCH
By Rebecca D.
7 min to read
November 18, 2020
A machine on the left made of pie charts and other research-related icons feeds energy through a pipe to a cityscape on the right.

SUMMARY

An internal study of Facebook researchers and partners has helped us clarify what impact means to us, what gets in its way, and some of the best ways to achieve it.

“You shouldn’t do research just for the sake of research,” goes the familiar refrain. “Your work should have an impact.” But what does that mean, exactly? What does impact look like? What gets in its way? Without a clear, shared understanding of what impact is or how we can drive it, it’s difficult to identify the research that matters most.



We on the Messenger research team realized we didn’t have such an understanding. After some brainstorming, we launched a study that included our cross-functional partners and other researchers around the Facebook family to gain a deeper understanding of how impact is perceived at Facebook.



This article summarizes the insights we gathered. We hope it provides a framework for thinking about impact, especially as it relates to evaluating current and future goals, and that it inspires you to seek a deeper understanding of impact in your own organization. We don’t intend the article to serve as a complete, static checklist for doing impactful research.



What we did

Ideation with Messenger research

At a two-day offsite, the Messenger research team came together to brainstorm what impact means, what it looks like, and how it gets blocked. We collected and synthesized the insights to create a baseline understanding of impact.




Interviews with leaders

To learn how our cross-functional partners perceive research impact, we conducted four interviews with cross-functional leadership around the Facebook family of apps, including leads in Messenger, Internationalization, Ads & Business Platform, and Instagram.




Survey of researchers

Lastly, we followed up with a survey about impact that was sent to all Facebook researchers, collecting a total of 52 responses.



What we found

Our study yielded four distinct (though sometimes overlapping) categories of research impact.



Informing product direction

One obvious way to make an impact with research is informing product direction by identifying areas where user needs are unmet. Ideally, research insights inspire a product team to deliver experiences that delight users and avoid harmful side effects.



Another type of product-related impact is helping products be more inclusive. Researchers often do this by generating deep understanding of populations dissimilar to themselves. Examples of informing product direction:



    • Uncovering new insights that help the team create something that improves launch outcomes.
    • Helping a product team quickly understand what we need to do to maximize the success of a project.
    • Stopping a product from launching because research indicates a bad user experience.

  • Influencing product or team strategy

    Influencing strategy was a repeated theme across all modes of data collection. This includes understanding what “good” looks like for the product team or org; what’s needed to get there; and how far the current state is from that north star. Research like this typically helps teams develop an expansive sense of the team’s mission, but can also lead to outcomes like blocking a product launch that would have been problematic. Examples of influencing strategy:



    • Helping a product team make crucial decisions about what to work on and what to build for.
    • Partnering with data science and product marketing management to deepen understanding of our target audience and use those insights to inform the product roadmap for the next half.
    • Identifying new audiences and problem spaces.


  • Contributing to team success

    This type of impact is all about helping the people around us, whether they’re researchers or not. On a day-to-day basis, it can help build a happier team, strengthen cross-org research connections, and help each other through peer reviews.



    On the cross-functional partners side, this kind of impact results in a common language around user needs and expectations around the globe. More tactically, it also means generating influence by smoothing cross-functional relationships.



    A few researchers also told us that for them, impact is helping their team get to a thorough roadmap that reflects user needs. Examples of contributing to team success:



    • Leading by example with your research roadmap to help the team understand how to leverage research and build people-first products.
    • Helping to onboard new team members (research or cross-functional) to your team.
    • Contributing to the team’s knowledge by sharing how to approach new methodologies.


  • Leveraging your expertise

    Expertise might be the most straightforward form of impact a researcher can provide. This involves building on each other's work and using creativity to unlock new insights that help our product teams get closer to true user needs and delight. However, this category is as much about rigor as it is about novelty and creativity. Examples of leveraging your expertise:



    • Sharing your research insights to help an org outside of yours get a deeper understanding of a similar problem space or audience.
    • Figuring out a gnarly technical problem and helping to scale understanding of the solution.
  • How do you know if you’re making an impact?

    Even if you have a clear definition of impact, it’s not always easy to tell whether your work is delivering it. Our study uncovered eight common hallmarks of impactful research:



    • Usage: Research gets used by the team to make meaningful product changes.
    • Education: Research methods are understood by stakeholders, and the pros and cons of different approaches are clear.
    • Resources: Research is given time and space early on in product development.
    • Memory: Findings are used beyond just the research cycle itself. cross-functional members recall previous research findings.
    • Trust: Researchers are seen as experts. They get invited to important conversations and asked to present at team meetings.
    • Partnership: Research is seen as a partner to the product team and has a seat at the table during strategy and product discussions.
    • Shifts: Research findings inform product direction, sometimes including significant pivots such as roadmap changes, products or features getting canceled, etc.
    • Implications: Research findings have org-wide utility, with relevance beyond your pillar.


  • Barriers

    Our study also identified factors that can limit impact. Here are the most commonly cited obstacles, along with suggested solutions:



    Time: Insufficient time to execute on a project.


    • Solution: Prioritization exercises with stakeholders, thinking of creative and flexible methods that can adapt to the time you have.


  • Understanding: Stakeholders are often working with research for the first time. This can limit their understanding of methods, hindering impact.


    • Solution: Prioritization exercises with stakeholders, thinking of creative and flexible methods that can adapt to the time you have.


  • False urgency: Working on the question that’s most immediate, not necessarily the most important.


    • Solution: Ask a lot of questions to understand where the urgency is coming from.


  • Meetings: At a large organization, communicating can take a lot of time.


    • Solution: Do a meeting audit. Look at your calendar for your week and prioritize the most important syncs. Block out time on your calendar for work time and thinking time.


  • Logistics: Some processes and tools can be inefficient, time-consuming, or duplicative.


    • Solution: Share tips and tricks with colleagues on efficiencies, ask your manager to help unblock you, set expectations with your team (e.g. they should book their own travel).


  • Ill-formed question: The question presented to the researcher by partners isn’t always appropriate.


    • Solution: See if the question is better answered by another discipline, or if there’s an underlying question that would be more valuable to investigate.


  • Out of sync with team: Team is working at a different pace or unclear how research can be a part of the process.


    • Solution: Roadmapping with the team can lead to understanding of key decision points and areas of focus, building stronger relationships with cross-functional partners.


  • Elaborate deliverables: Creating a compelling deck complete with highlight reels and pixel-perfect design can be daunting.


    • Solution: Work with your cross-functional partners to learn how to best meet their report-out needs. Sometimes a deck is warranted, but a note might be all your team needs to get moving.


  • Now what?

    It’s not enough to understand one’s impact. We also need ongoing conversation and community to support us in doing work that matters to our users, our org, our product teams, and ourselves. We recommend the following potential next steps:



    • Share this article within your teams and encourage discussions about expectations, experiences, and blockers.
    • If you’re a manager, identify opportunities for your team to share case studies with the broader research community about instances of high-impact research. These should describe how they identified the project, how they got buy-in from cross-functional partners, how they influenced teams to drive impact (ideally a mix of impact types — product, thinking, strategy, blocking a launch, etc.), and methodologies (try for a mix of short- and long-term work).
    • Read more about influencing product direction in Dave Hora’s article The researcher’s journey: leveling up as a user researcher.








  • Design at Meta is for everyone who touches user experience and design.

    Whether you’re a product designer, writer, creative strategist, researcher, project manager, team leader or all-around systems-thinker, there’s something here for you.

    Design at Meta is a global resource for all things design, with specialized focus on Meta technologies and programs. We believe everyone is a designer, in some shape or form.