Measuring PIN in your newsroom: A how-to guide

Joellen Easton
Business Development Manager
Public Insight Network

Download the full Public Insight Network evaluation how-to guide here.

If something is important to you and you want it to improve, you measure it.  Want to get in better shape?  Metrics can help.  If you’re trying to become a better baker, lowering the ratio of burned-to-not-burned cookies is a good thing.  When journalists hear phrases like “impact evaluation” they start to hyperventilate. It’s like hearing “draft environmental impact statement” or “planned unit development.”

But that’s changing: by now we’re all pretty accustomed to traffic and click data on stories, and while viral success seems like lightning in a bottle (or not, as Eric Athas and Teresa Gorman found) it’s a point of pride when your story gets lots of shares and retweets.

At PIN, we believe passionately that engagement improves journalism.  We don’t want to trust our assumptions blindly, so from time to time we make a point of measuring PIN’s effectiveness, and of putting our heads together with smart people who think about these things too, like Joy Mayer and Reuben Stern at RJI.  We also believe that it’s OK to expect that your work has an impact.  As journalists it’s not our business to proscribe that impact, but it is certainly our business to understand it.

When I worked as a Public Insight Analyst, I often struggled to communicate with my colleagues the value of engaging with the audience around content — especially around ideas that weren’t yet published.  Producers were more inclined to engage with readers after a piece of content had aired than beforehand as a part of reporting the story, and I’m the first to admit that I failed many times to communicate why it’s important to engage people before the story is all tied up with a bow.

Since I failed then, let’s see if I can do a better job today: Engagement can improve your journalism.  You’ll find stories you didn’t know were there.  When you ask people what they know about a topic, and then you report on that topic, those people will see themselves reflected back.  An engaged audience is more loyal.  An engaged audience will forward your content.  I’ll stop there, but you can see where I’m going.

And it only makes sense that if what I just said is true, it is also measurable.

With this belief in mind, we have combined journalism and engagement evaluation best practices with what we have learned from recent PIN evaluation projects to bring you a guide to evaluating the utility of PIN for your organization (PDF).

This how-to guide doesn’t proscribe how to evaluate your work with PIN.  What it will do is discuss what you can measure about PIN and share guidelines for how to do it, including tips on how to get the most out of the PINfluence tracking tool and some easily available metrics from AIR. You’ll find recommendations for how to design your evaluation and how to communicate what you measure with your staff and sources. We also include two pre-designed evaluation templates courtesy of PIN’s evaluation partner, LFA Group.

  • One template will help you measure in-newsroom PIN dynamics and utility. This survey  asks questions about how PIN works in the newsroom, and how individual journalists use PIN, what they expect of it, and how they perceive PIN and PIN sources. The survey is online, and powered by Google Forms.
  • The second template is a set of tips and questions for anyone who wants guidance on creating a case study of the community impact of a PIN-informed story or project. Use this template if your newsroom wants to understand and deepen their community impact, and is using PIN to help facilitate that impact.

If you have feedback on PIN measurements that you’ve found useful in your newsroom or on how you’ve used the guide, please email me: jeaston@americanpublicmedia.org.

Joellen Easton Business Development Manager
Public Insight Network
Joellen Easton gets to chart the next mission for PIN: New ways to use the network for meeting information needs, with new partners and new technologies. All shiny objects cross her desk.

Her background is in anthropology, and she has produced for WGBH, PRI’s The World, and as a freelancer. She first worked with PIN at Marketplace as a PIN analyst for four years, and after that as partners manager for three years, leading the training and support team. She is degreed from Tufts and MIT.