Case Study: Reframe Pilot with InvestigateWest
Published October 2021
In 2021, Reframe set out to test its tools and resources with a working newsroom in a six-month-long pilot. Seattle-based investigative outlet InvestigateWest was the ideal partner, Reframe editor Aubrey Nagle writes.
At Reframe, our goal is to help journalists use more authentic and humanizing language in their reporting, especially that about communities frequently stigmatized, marginalized, or misrepresented in media. We accomplish this through workshops, online resources, source and content analysis, and a text analysis tool we’re developing to assist reporters during the writing process.
In 2021, we sought to pilot our tools and resources in a working newsroom to understand what worked and what could be improved upon. While journalists have taken our text tool for a spin, consulted our online news guides, or attended one of our workshops over the past two years, this pilot set out to be the first comprehensive use of all that Reframe has to offer. In a way we wanted to, as Senior Editorial Associate John Hernandez says, “suffocate the problem” by trying out a variety of avenues for behavior change synchronously.
To do so, we needed a strong and enthusiastic pilot partner. InvestigateWest, the nonprofit investigative newsroom based in Seattle and serving the Pacific Northwest with a focus on environmental and climate change reporting, was that ideal partner.
InvestigateWest’s then-Executive Director Allison Augustyn learned of Reframe while seeking resources for diversifying the newsroom and their coverage. Their team was already creating new hiring practices for their staff and board but were looking for additional support to ensure their coverage was inclusive. The goal in working together, then, was for Reframe’s tools and resources to bolster the work InvestigateWest’s leadership was doing in strengthening their journalism and community ties.
Our work with InvestigateWest came at a crucial time for the industry. When we first discussed working together in fall of 2020, the COVID-19 pandemic was in full swing, leveling financial blows to local media across the country at a critical juncture for public health information. Simultaneously, newsrooms across the country were responding to the uprising following George Floyd’s murder by examining the diversity of their coverage and staffs and reckoning with the industry’s role in perpetuating racism through harmful reporting. Both these events underlined the ongoing crisis of eroding trust between news media and the communities they cover and continued conversations on journalistic standards and objectivity.
It’s with this background that the InvestigateWest team decided to become a pilot partner, signing on to think deeply about the language used in their reporting, establish meaningful and equitable relationships with the communities they serve, and examine their sourcing practices.
Getting to work
We began our work together in earnest in January 2021, kicking off with our flagship workshop on language and framing. The workshop begins with instruction on why words “work” the way they do: how we use them as symbols, how different people interpret them in different ways thanks to their personal experiences and cultural connotations, and how this sometimes creates gaps between the intent and the impact of our words.
The second half of the workshop focuses on the framing of news stories: how we define the problem at hand, who is responsible for causing and solving it, and what the solutions might be. The core of the lesson here is also constructedness; there is no natural or inherent way to report the news, just a series of choices that journalists make that can shape the way an audience interprets and remembers events they can’t witness firsthand. Understanding how our framing choices can create similar intent and impact gaps prompts journalists to examine how their habits and newsroom standards come to be, and how those may reflect their personal experiences (or their managers’ personal experiences).
In a feedback survey sent to InvestigateWest’s team after the workshop, participants anonymously shared praise for its teachings. “Great balance of information and sophisticated subjects combined with fun and ability to interact,” one contributed. “The entire training was just reasonable and helpful and HUMANE. Thank you so much, everyone is chattering about how they are excited!!!”
Another participant shared, “Loved this workshop — a great approach to building off of information folks working in the journalism industry might already have. I appreciate the focused delivery and pointed examples.”
With this solid foundation under our belts, we next worked with InvestigateWest to develop a Community Standards Panel that would meet regularly with the newsroom’s leadership to discuss issues of representation in media, language and framing, and community trust. We had three main goals for the panel:
- To connect the newsroom directly with community members from their coverage area who are passionate about media’s impact on their lives and want to see their communities represented authentically in the news
- To amplify the voices of those panelists so that their experiences and perspectives would echo through INVW’s future work
- To initiate a new way of listening and engaging to the INVW team
Our goal for the makeup of the panel was to assemble a diverse group of 10-12 people with different professional and personal backgrounds. We reached out to a wide variety of community and service organizations around the Seattle area and Pacific Northwest to ask that they share our application with their networks. In response, we received three dozen applications from an incredible array of residents. It was truly difficult to narrow the panel down to a number we felt would be tenable for both breakout groups and single-group discussion, even if a few panelists couldn’t make a session last minute (which naturally happens with a large group). But in the end, we came away with a panel rich with cultural, regional, and generational differences that made for great conversation. We met once a month for four months with sessions running from 60 to 90 minutes each and each panelist was compensated by Reframe at $75 per session. Contributing guidance and perspective to the InvestigateWest team, as well as sharing personal and at times difficult stories, is valuable work. Thus it was important the panelists be compensated fairly for their time and their labor.
Our first session was focused solely on getting to know each other; if we were going to have some tough or deep conversations about news and representation, everyone needed to feel comfortable, safe, and supported. Throughout this session we took inspiration from a trauma-informed psychoeducational training that Resolve Philly staffers attended early in 2021. Facilitated by Rebecca J. Alvarez and Iris Bowen, the training taught us valuable lessons on working with those who’ve experienced trauma without causing further harm. For example, we began our sessions with some group agreements that explained how we would engage with each other respectfully, how we could resolve any issues or disputes that arose, and how to ensure everyone had the chance to take and make space. We took that lesson into our sessions with the Community Standards Panel.
The following three sessions were themed — language and word choice in news; narratives and story framing; and how to keep this engagement going — and, though the facilitation of each meeting was planned ahead of time, we let the flow of conversation take the lead when appropriate and adapted on the fly. This space was really for the panelists to share their voices and stories and to exchange learnings with the journalists on hand, so fruitful discussion took precedence over the schedule.
The panelists had many positive things to say about their experiences. As one participant offered in a follow-up survey, in the sessions they felt, “the sense of an immediate community that wanted to better all communities.” This sense of community was visible, too, in the warmth of the conversations and the panelists’ eagerness to share contact information and share each others’ work with their networks. That chemistry was integral to the work of aligning the newsroom staff and the panelists and building understanding that both parties were working towards the same goals of supporting their communities through accurate and authentic information.
The INVW staff felt the impact of the conversations as well. Former Engagement Director Ariel Shearer said, “Working with the Community Standards Panel was powerfully eye-opening. We were humbled by the willingness of our panel’s participants to share their insight and experiences with us, including experiences that reflect harm caused by the media. The meetings allowed our team to get familiar with a panel model as a space to listen to community members, witness their insight and be held accountable for our responsibilities as media professionals. It was a space that allowed us to start building lasting relationships with community members and start visualizing new alternatives to extractive reporting. We’re working to continue regular meetings with a number of panel participants even after our time with Reframe has ended.”
We’re so pleased to see this engagement work will live on!
Tracking & testing
The next stage in our pilot was to develop a system for tracking the diversity of the sources featured in InvestigateWest’s stories. Many retrospective source audits seek to establish a demographic baseline by surveying sources within, for example, the last six months of a newsroom’s coverage. Often a third party, like an editor or even a hired researcher or academic team, does this outreach and collects and analyzes the data on behalf of the reporters. This creates data to judge progress against while keeping the workload off of reporters’ desks. Then, this progress might be checked months later in a regular or annual update.
The Reframe theory of change and source tracking philosophy, on the other hand, requires that journalists be very involved in understanding the makeup of their source rolodexes during the reporting process. Our protocol requires participating reporters to survey their sources directly, either during their interviews or, when the situation requires, following up with an email (for instance, in the event of a press conference, following up with the press contact). This information is submitted via forms into a database that can be used to create beat- or newsroom-wide reports to monitor progress.
With this order of operations the reporter is conscious of the diversity of the sources they are leaning on for a story while it’s being reported, leaving time to course-correct if the story isn’t representing a wide range of voices. If the reporters’ main interaction with their source data only occurred during a yearly review for a newsroom-wide performance benchmark, they may not feel the same ownership over the results or understand how best to diversify their sourcing. By participating in sourcing analysis throughout the editorial cycle, the reporter is instead focused on diversifying their sourcing as a means of improving their coverage, ensuring it tells the full story. It may be a bit of extra work on the reporters’ part, but we feel the observation effect (the disruption of a habit or system because it’s being observed) is crucial to the behavior change that source tracking purports to seek.
This strategy does require a consistent point person within the newsroom (like a managing or desk editor) with knowledge of the editorial calendar to ensure reporters are submitting their sources. The InvestigateWest team is enthusiastic about the potential of source tracking, but due to personnel changes within the organization during our pilot period, establishing a consistent practice became a challenge. Of course, anyone who’s ever led a new initiative at their workplace knows that transitioning those responsibilities is difficult. A major learning from this experience is to proactively build contingency or transition plans into any workplace change efforts for when staff moves on. For the future, the InvestigateWest team now has a ready-to-use system that is resilient yet flexible.
“Old habits take time to change, and newsroom transformation requires patience and dedication. While we hope to continue growing and evolving after our Reframe experience, the external support is a great asset and motivator for internal change,” said then-Engagement Director Ariel Shearer.
Finally, two of InvestigateWest’s staff members have beta tested our text analysis tool to get a feel for how it works, learn its recommendations, and discover how it might fit into their workflow. The tool prototype is a browser extension for Chrome and Firefox which analyzes text within live Google docs. It comes pre-set with about 50 entries that flag terms within a text that should be reconsidered by the author, explains why a term was flagged, and offers options for replacement. The context and alternatives are based on best practices and community feedback we’ve collected.
The test entries feature topics that are too often reported on with stigmatizing language, including incarceration, substance use, immigration, disability, and age. The entries also feature terms that encourage the writer to share power with the source in question by using their descriptions of identifiers like gender and race. For instance, the tool would flag “illegal alien” in a text due to its dehumanizing nature, and suggest replacing the term with the subject’s specific immigration status or using a phrase like “undocumented immigrant” because these more accurately describe the subject without othering them.
The testing of the tool elicited positive responses from the InvestigateWest team members and underlined its importance in creating consistency across the newsroom. Shearer said, “I am so glad to have this as a resource for us to use moving forward, especially when a new executive editor joins the team.”
Katie Hayes, a Daily Herald reporter and former freelancer for InvestigateWest, has also tested the tool with positive results. “When it’s more widely-available, I would love to introduce it to my newsroom. I think it would be helpful for all the reporters in my newsroom to be on the same page about the language we use and why we use it,” she said.
The tool is currently limited in its application as we further develop its features and dictionary to incorporate community feedback into its guidance. We look forward to being able to share it with more reporters soon.
Learning & growing together
Our time together has greatly excited both the Reframe and InvestigateWest teams. The change to the InvestigateWest newsroom is clear from the takeaways of its leadership.
“Working with Reframe was incredibly informative to us as an organization. We learned how to rethink language use and gained access to powerful tools for source auditing and article language assessment, all of which continues to help us move toward our goal of more inclusive and representative reporting and newsroom practices. In short, it was a transformative experience that empowered us to start making vital changes to our practices, changes we need to see throughout the journalism world,” Shearer said. “Reframe sets a new standard for reporting conduct that should be taught in every journalism program, and proves that resources exist to help all publications make transformative changes for more inclusive and representative reporting practices and newsroom culture. No more excuses!”
InvestigateWest’s former Executive Director Allison Augustyn also offered her thoughts at the conclusion of our pilot:
“Reframe is an incredible journalism resource. The tools are extremely thoughtful and designed not just for a data-driven newsroom, but also the audiences we serve. As a nonprofit news organization driven by mission, not profit, InvestigateWest relies on well-run, well-managed programs like Reframe to support our work. Reframe’s trainings have led to in-depth conversations on our team and with our communities, as well as changes in practice, making our process and reporting even stronger. How we create the news is as important as the news we make, and Reframe knows this, approaching change and challenge with a smart, holistic approach. This is the future of news.”
The Reframe team learned so much from this pilot process, and we’re so grateful for the InvestigateWest staff’s participation and enthusiasm. In the end, we believe the pilot proved that Reframe’s “suffocating the problem” approach is beneficial to newsrooms looking to make big changes and put in the real work toward a more inclusive, effective journalism. We look forward to following InvestigateWest’s further progress as they implement the lessons learned from our tools and resources.