A resource for newsrooms: Measuring the success of audience engagement efforts

Click here to download full report of The Engagement Metrics

Our multi-disciplinary group (see bios of some of the folks involved) focused our conversations around specific strategies for audience engagement, what their value is to the news organizations, and how the success of the efforts can be evaluated. We spent a day filling out a google spreadsheet together, and what you'll find here is an edited version of that document. It's not intended as a comprehensive guide to engagement, but instead as a sampling of practical ways to be strategic in our efforts.

Many people in the room have jobs that involve audience outreach and engagement, and they talked about their need to be able to show the concrete results of their work. They also felt the need to persuade their colleagues and their bosses to value engagement. Metrics are key. And by metrics, we don't just mean web analytics. The number of people who show up for a newsroom tour is a metric. The number of people who like your Facebook page is a metric. The number of story ideas or new sources resulting from a community conversation is metric. Research also has a lot to teach us about measurement beyond counting. A content analysis, for example, can help us track the civility of conversations or the diversity of sources.

And if it's information we seek, don't overlook what we already know how to do: report. If we want to know if our work is making a difference in the community, what if we assigned reporters to find out, whether we publish it or not?

We urge you to read this report with this in mind: What do you wish you knew about your relationship with your community?

SOME THINGS TO KEEP IN MIND

1. This is not a guide to engagement possibilities. Many ideas we discussed didn't make the cut, and many more didn't find their way into the conversation. This is a list of strategies we discussed that we thought we could measure.
2. There's no one-size-fits-all solution. What works in one newsroom might not work in another. Strategies, and therefore metrics, must be specific to your newsroom, and perhaps to individual projects.
3. What success looks like in engagement is continually evolving. Once you've figured out Twitter, the world will be on to something else. When you've made inroads with a specific group of people in your community, you'll be starting over with another. The industry is changing rapidly, as is our culture. Journalists need to learn to adapt more quickly, to be comfortable with shifting definitions of success and to invest in continued strategic planning.
4. Some of our measurement suggestions are easy to implement now. (Example: Use outbound links, not just incoming links, to track whether we're the first stop for someone online or the last). Others would take some sophisticated knowledge or software. (Example: What if our metadata had fields for things like story idea genesis, so we could track the benefits of listening to diverse voices?) Others are more like dreams than suggestions — things we wish we could measure but need help figuring out how. (Example: Making a network map of our newsroom's community connections and relationships.)
5. This is a working document. If you have ideas you'd like to see reflected, resources that should be shared or clarifications to make, please let us know. Email Joy Mayer at mayerj@missouri.edu.

The full downloadable PDF report starts with suggestions for getting started, and being strategic about engagement (also listed below). Then it moves on to specific recommendations, broken into three engagement categories. The categories are the result of dozens of interviews conducted throughout Joy Mayer's RJI fellowship. Joy asked journalists about audience engagement — what it meant theoretically, and also practically. She then took the strategies and tips and grouped them using more specific terminology. There is undoubtedly crossover among the three categories, but our hope is that they can serve as a starting point for talking about the purpose of engagement. Here is a summary of what each means:

• COMMUNITY OUTREACH -- Outreach includes efforts to share ourselves, our expertise and our content with our community. It involves: Taking the content to the audience, rather than hoping they'll find us. Identifying information needs, catering our products to meet them and distributing them in a way that makes sense. Being willing to participate in the community as individuals, building connections and personalizing our brand. Inviting the community to get to know our people and our processes. Enriching our community, sharing our own knowledge and supporting other community enrichment efforts.
• CONVERSATION -- Being in conversation with our community means listening as well as talking, and adjusting what we do and cover based on what we hear. It involves: Hosting discussions in person and online on topics that matter to the community. Participating in conversations we're not hosting, both in person and online. Valuing how a continuing dialogue can make us better journalists and improves the journalism. Using web analytics to better understand what people are showing us they value in what we do, and basing at least some of our decisions about content and staff resources based on what we're seeing. Recognizing that journalism is a process, not just a product, and involving more voices in the process means more diverse journalism.
• COLLABORATION -- Collaborating with our communities, the highest form of engagement, means we have a shared investment in and influence over our journalism. It involves: Soliciting and relying on user contributions. Soliciting and using user input about what we should cover and how we should allocate our resources. Valuing the role the users play in reacting to and sharing our content. Recognizing that we can accomplish things with the cooperation of the community that we could not do alone.

PLANNING FOR ENGAGEMENT

As you move forward with engagement efforts, it might help to consider these suggestions:

• Develop a written strategy for the organization's efforts.
• Start every project with a measurable goal. In 12 months, six months, one week or one day, what would success look like?
• Consider whether your goals align with any other community organization's goals, and consider partnerships.
• Build capacity within the organization (hire people who know engagement, assign specific duties, train staff, etc.)
• If engagement is a priority, it should be valued in the newsroom. Consider including engagement strategies in the performance evaluation of individual journalists, and of departments. Consider hiring people (perhaps even non-journalists) with different expertise, such as analytics, marketing or research. Embed them in the newsroom, and hold them accountable for newsroom training and culture changes. Consider allocating staff time for audience interaction and understanding, and for thoughtful strategizing, as Google does with its 20 percent innovation time. 
• Consider learning from disciplines like marketing, analytics, nonprofits, anthropology, social media, civic activism, etc. Whether you want to launch a community project, learn about your community, increase the community's investment in your work or change the perception of your brand, there are folks who know more about how to do it than you do.
• Identify specific communities in which to make inroads. Consider a long-term investment in a specific community (community of interest or geography or socio-economic level or education or any other unifying characteristic), with a goal of addressing that community's needs, or of transitioning individuals from non-participants to collaborators.
• Build community profiles of target communities, like the CIA world factbook reports on different countries

RESOURCES

• A few specific websites and articles came up as we were talking, as examples of how journalists can use metrics, or find help using them.
• How Postrank measures engagement: https://analytics.postrank.com/docs/engagement
• How Philly.com measures engagement (from Nieman): www.niemanlab.org/2010/10/getting-beyond-just-pageviews-philly-coms-seven-part-equation-for-measuring-online-engagement/
• The transcript of the #wjchat on social media metrics: wjchat.webjournalist.org/2011/03/chat-3-30-11-social-media-metrics/
• Gawker's approach to measuring "branded traffic" (from Nieman): www.niemanlab.org/2010/03/a-reader-affection-formula-gawker-creates-a-metric-for-branded-traffic/
• Confusion Online: Faulty Metrics and the Future of Digital Journalism: www.journalism.columbia.edu/page/633/437
• Information about Joy Mayer’s fellowship work at RJI: www.rjionline.org/blog/highlights-joy-mayers-community-engagement-fellowship-blog
• Links Joy has saved on metrics: www.delicious.com/mayerjoy/metrics

Please click link at top of this article to download the full report.

• Allowed HTML tags: <a> <em> <strong> <cite> <code> <ul> <ol> <li> <dl> <dt> <dd>
• Lines and paragraphs break automatically.
• Filtered words will be replaced with the filtered version of the word.
• Web page addresses and e-mail addresses turn into links automatically.