‘We’re a metal detector’: How local news is using AI to tell better stories and hold leaders accountable

Before we were Satchel, there was 'Maria' and 'Paul' -- experiments in AI that caught the attention of the media. Read more about how we got where we're at today below or click here to read the original article.‍‍

Mark Talkington runs a lean operation at the Palm Springs Post, a small, bootstrapped local news outlet he started two-and-a-half years ago to cover the nuts and bolts of local government. The region isn’t a news desert, per se, with the large regional daily The Desert Sun operating across the surrounding Coachella Valley community. But as the Sun, part of the ailing Gannett newspaper chain, has gone through rounds and rounds of cutbacks, it created a space for Talkington and his small staff to zero-in on the machinations of city hall. The daily newsletter boasts roughly 14,000 subscribers, and regularly sells out its advertising space.

It might seem odd, then, that Talkington has kept two new reporters, Paul and Maria, on the sidelines, chronicling meetings but never getting their work in print.

That’s because Maria and Paul are AI programs, which summarize meetings and write drafts of stories, respectively. Talkington and a colleague, Peter Loforte, created and fine-tuned them to experiment with the ways technology can help local newsrooms, and speed up and simplify time-intensive tasks.

The grassroots Palm Springs Post doesn’t have the benefit of an R&D budget or tech team, so Loforte and Talkington, who met as Microsoft employees in 2021, built and experimented with the AI systems—and their ability to observe, evaluate, and report out local meetings—on the side.

Talkington’s test places him at the center of an emerging confluence of technology and local news coverage. Paul and Maria have done a good job taking notes and sketching reports, generating output from live recordings of events using  numerous publicly available large-language models; Maria has even synthesized a meeting and sent participants a thank-you message with custom videos summarizing their main points. But they’re far from mastering the nuances and narrative skill needed to produce a compelling, accurate news story, and Talkington is very clear that he hasn’t published any stories created with any form of AI—he’s just experimenting. He doesn’t want to go that far into AI-produced journalism anyways.

“I’m just hoping to use it to assist me,” he sats. “I don’t want to turn AI loose.”

Journalists have long, and rightfully, feared the way big media businesses eye the potential of artificial intelligence to generate stories and revenue without having to pay a reporter’s salary. That’s especially true now, in a media landscape that has been brutal for local news outlets. As of May 31, more than 17,000 journalism jobs had been cut this year, a 315% jump from 2022 and the most such cuts on record. The 2022 Medill State of Local News Report found that a quarter of the nation’s local papers had closed since 2005, and a third will disappear by 2025, hitting smaller communities the hardest. A Reuters Institute Poll found the majority of editors and CEOs in news weren’t confident about the year ahead, and even the optimistic group saw layoffs and other cost-cutting measures coming by the end of the year.

The issue isn’t that AI is replacing journalists—bad business decisions and business models already have. Using tech in the right way, as a tool to enable and empower journalists, might just be a way to help local outlets prosper.

So far, AI authorship has amassed a poor track record. Examples such as overwrought high school sports copy, brusque management-led experiments that “aren’t ready for prime time,” and even a horrifying AI-penned obituary have gotten reader attention for all the wrong reasons. But viewing technology solely as a competitor misses an opportunity. AI can also bolster the necessary, and time-consuming, work of local journalists.

“While we, the technologists and the journalists, spend all of our time debating if AI will or won’t replace journalists, we’re missing an opportunity to actually start to make the tools we do need,” says Jeremy Gilbert, Knight Chair in Digital Media Strategy at Northwestern’s Medill School of Journalism. “What I think is the most likely scenario is that AI is a complementary tool that helps us do some of the more specific things that we need to do.”

Talkington, who previously worked as a daily reporter for The Desert Sun, helped launch ESPN.com and worked as a homepage editor for MSN before being replaced by algorithms. He has observed that every time a publication “flipped the switch” and let human editors go, the end result may have saved salary costs, but it broke down the audience’s trust in the news product. As larger, corporate media outlets experiment with and lean into AI-aided content creation, it leaves more room for smaller, independent outlets to provide more responsive, relevant reporting.

“Journalism is alive and well,” says Talkington. His publication is part of LION, a group of independent publishers that grew its membership 13% last year, with 454 independently owned publications in Canada and the U.S. “There’s so much good journalism being practiced right now. It’s just not being practiced by major newspaper chains.”

“Everyone is crying that the news is dying, and journalism is dead, and they’re just absolutely full of bullshit,” he adds.

It’s exactly those types of papers and websites that many media technologists want to help. Talkington’s AI reporters, which he’s said could potentially become the backbone of a publicly available tool for reporters, are two examples of a growing list of tools seeking to help journalists better evaluate public meetings and hearings, and do it in less time.

As local journalism and smaller publications have withered under economic pressures, a sharp divide between digital winners and losers has emerged. Gilbert saw it happen when new means of digital storytelling emerged, and publications like the New York Times could invest in whole teams to experiment, incorporate new ideas, and create tools that were often inaccessible to local newsrooms.

The push among digital journalism advocates now is to create open source, more widely available tools leveraging generative AI for the public good. The Knight Lab, Associated Press, and Michigan Public Radio have collaboratively built a tool called Minutes that allows a reporter to enter in a YouTube feed of a government proceeding and obtain a transcript.

Two Yale Law School classmates, Matt Post and Mat Hernandez, who also have experience serving on local school boards, were a bit taken aback by just how much happened in these forums, where important decisions were made out of the purview of most of those being impacted. Their response was to cofound LocalLens, an AI tool that summarizes local government meetings by scrubbing through audio or video recordings, even meeting minutes.

When asked about the program, which went live earlier this year serving select areas of New Jersey, Post quickly made clear that it isn’t meant to be journalism, or a replacement for it. When a Politico reporter picked up LocalLens coverage of a school board meeting, the resulting piece described the program as a “reporter” and described its output as an “article,” which raised serious questions about whether the tool was, in fact, moving beyond summations into editorializing, and if readers could tell the difference.

Post underscored that LocalLens is simply a starting point to help short-staffed publications wrap their arms around the actions of the roughly 90,000 local government entities in operation across the U.S.

“We’re a metal detector for noteworthy stories,” he says. “We see ourselves as a tool to restore transparency.”

Allowing journalists to spend more time reporting and investigating, and less time in four-hour meetings, seems like a good step. Talkington has says that during Palm Springs Post’s run, he’s come to understand that readers value a few core issues: homelessness, housing affordability, short-term vacation rentals, and the larger issue of inequity. These involve lots of players at City Hall, which means lots of time in meetings.

Still, expecting smaller publications to get on board with AI as a tool and core part of their work will require a cultural shift and more education.

“Most of the newsrooms that I talked to are only beginning to have that understanding of what artificial intelligence, specifically generative AI, can do, and why they should either be excited or worried about it,” Gilbert says.

There are also limitations with this type of technology that need to be sorted out and iterated upon. LocalLens has says they’re working to include timestamps with transcripts, to improve trust and fact-checking, and are experimenting with the formatting and how best to present the most important issues to readers.

There are also the same issues with hallucinations (making up false answers and presenting them as fact) and transcription errors that other AI programs have had; Post claims that the most common mistakes with LocalLens have been misspellings of names, and that “we haven’t seen strings of factual errors in our testing.” The large language models that underlie these programs are better at summaries, he says; answering Google-like queries is often where the mistakes come in. There’s also room here for local governments to help, by assisting with transcript annotations, appending relevant data and documents, and improving transcript and data availability.

Talkington agrees that there are still outstanding issues and tweaks to be made; when Paul first reported on meetings, it wrote a chronological summary, instead of being able to take the next, more useful, step of writing a summary with the important points at the top. It’s currently being fed all the existing Palm Springs Post coverage, to help glean more context and make additional connections.

But meetings are just the beginning. Gilbert believes AI can offer smaller newsrooms more ways to save time, and even generate story ideas. The Knight Lab is working with a newsroom in India to create an AI chatbot trained on 80 years of local election data, allowing reporters to ask conversational queries (i.e., “has the opposition party ever controlled a majority of state governments?”) that can spark trend stories and deeper analysis. Other examples include tools that can ferret out financial irregularities, or parse through databases of political contributions. QuickTrace seeks to help investigative journalists make sense of large collections of documents and discover the connections that create big stories.

Another program, SourceScout, aims to help amplify marginalized experts and diversify the experts cited in stories, what its developer Alicia Stewart, a former CNN producer, calls “marginalized genius.” Created during a media conference and hackathon this summer in Chicago, where it won the top prize, the program is a ChatGPT plugin that connects journalists with a vetted, curated list of experts.

“This is a co-creator,” says Stewart. “This is not replacing who that journalist is.”

There’s certainly funding out there for projects like these. OpenAI just donated $5 million to a cadre of 41 news organizations to help them find innovative ways to use the firm’s underlying technology, like sorting through complex databases or freedom of information act requests. The MacArthur Foundation also announced a $500 million investment in strengthening local news that, while not specifically focused on AI technology, philosophically aligns with many of the goals that these tools’ creators suggest is their mission, empowering authentic community coverage.

“I think there’s a version of AI journalism that is funded by private equity people and is intended to gut newspapers further,” says LocalLens’s Post. “That’s not what we want. We want to be a nonprofit tool for journalists.”

And if those tools are developed for and owned by smaller publishers, says Talkington, then that can be a formula for success.

“I want to be in a direct conversation with my audience, and when you start to involve technology, that’s when you lose the human aspect,” he says. “I want to do as much as I can to keep that personal relationship.”