Since the public launch of ChatGPT nearly three years ago, the media industry has cycled through many phases of AI adoption and experimentation. Andrew Deck has witnessed it all. As an AI staff reporter at Nieman Lab, Andrew is one of a small group of reporters specifically dedicated to tracking AI developments in journalism and media. For Andrew, reporting responsibly on AI means cutting through the “ambient anxiety” and speculation of most AI coverage and providing real-time updates on how AI is working, and not working, in newsrooms around the country.
We asked Andrew about what audiences actually want to know about AI use, AI “bias detectors,” how AI can help and hurt under-resourced newsrooms, and how he works to produce accountability coverage without undue alarmism. Read excerpts below and the entire interview here.
Ryan Howzell, Ethics and Journalism Initiative Assistant Director: You’ve been covering AI’s impact on journalism and media for Neiman Labs since March 2024, but have reported more generally on generative AI for other outlets. As someone who’s been following this closely, would you be able to explain to our readers how things have maybe changed since you started tracking AI in the media industry? Is it fair to say the industry is moving in one direction, or is it splintering across outlets?
Andrew: There’s one pretty notable change in the discourse that I’ve noticed in the last several months. Upfront, a lot of the energy has been focused on AI adoption in newsrooms and how that might lead to productivity gains. A lot of conversations about off-the-shelf tools like Chat GPT and how to integrate those into editorial workflows or workflows of other teams in newsrooms.
The shift that I have been seeing, and I welcome it, is more recognition of how generative AI tools are actually shifting our audience landscape. And the ways that readers are and might in the future be discovering our original reporting or interacting with our reporting in a way that does not fit the conventional mold of an article page view.
ChatGPT is always there as the easiest example. We all know how that tool is synthesizing and summarizing the reporting of outlets without necessarily needing a user to click through to a news website. In the last month, in particular, there’s been this conversation about a “traffic apocalypse.” A lot of that’s oriented around AI overviews alongside tools like ChatGPT, and the fact that AI tools are eroding our search traffic.
I’m not sure that conversation has been borne out in the numbers quite yet. If you really take a closer look at some of this reporting, a lot of it is anecdotal. But I do think the concern is really valid, and I believe there should be a sense of urgency around this issue.
In the past, when I’ve talked to newsroom leaders, they’ve been most concerned about whether or not they should have a ChatGPT enterprise account and whether that would shave hours off an investigation in their newsroom. I think now we’re having a more serious conversation about whether their generative search products would actually kill, say, some of the traffic and also subscription conversions that might be paying for those investigations traditionally. So we’re having a more serious conversation, not just about adoption from journalists, but adoption among readers and some of the downstream effects of that. And I think that’s a really important conversation.
Ryan: Over the past year or two, there’s been this push within newsrooms to be very transparent and to explain in a lot of detail how journalists in a newsroom might be using AI. I feel like the narrative has been that audiences are confused. Have you seen evidence that news consumer AI literacy is changing?
Andrew: In the US, we’ve seen a lot of surveys. I can think of one from the University of Minnesota and Poynter, where a lot of American readers just say that they do not have interest in using AI tools to get information from their news organizations. Early on, we saw a lot of news organizations experimenting with chatbots. [For example,] The Washington Post came out with a climate-focused chatbot. There were a couple of chatbots from Hearst-owned newspapers, including one from the San Francisco Chronicle about food reviews in the city. And the readers are telling us sometimes pretty directly, we don’t want to use these tools, we don’t have an interest in them. But there continues to be this impulse from organizations to build them anyway. And I think that’s something that we need to have more honest conversations about. Is there actually demand for generative AI tools among readers? That’s one aspect of that conversation.
Then there’s also on the other side of it: transparency around generative AI tools that might be used in our own workflow. And how do we talk about them with audiences? And again, the survey research has been showing that people have a really negative perception in the U.S. towards knowing that AI was used in the production of journalism that they’re consuming.
I think a lot of newsrooms and rooms I’ve been in, people have been concerned about too much emphasis on disclosure, if that alienates readers from the important journalism that they’re producing. And I think that’s a really ethical gray area that has not really been resolved yet. If, say, a generative AI tool was used in the research phase of reporting out this story, do we have an obligation to disclose that to our audience? If there were several layers of human editorial intervention beyond the research phase, including the drafting of the story that was completely human-done, do we still need to disclose to our audience that we used AI, particularly if that will turn them off to the substance of the story that we reported immediately?
That’s a really challenging situation that a lot of newsrooms are in. As these generated AI tools first hit the market, there were a lot of publications like Wired that put a lot of emphasis on transparency. But things still fall through the cracks. Like we saw [in August], Wired hired a freelance reporter who ended up producing an entirely fabricated story using AI tools, and they published it on their site and made it through their fact-checking process and multiple rounds of edits before being retracted.
And it was just exposed [recently] that that had also happened at Business Insider. There are no easy answers in the transparency conversation. I think we’re beginning to have some more solid understandings of how readers might feel about the use of these tools and getting more granular with that understanding to know that different parts of the editorial operation readers might feel differently about AI being used.
Ryan: That’s really interesting and perhaps adds some much-needed nuance to the original transparency argument. As someone who’s been covering a variety of outlets in the industry at large, what recurring ethics issues are you seeing, and are they, like this transparency argument, changing?
Andrew: I think one really interesting trend that I’m noticing is whether or not content produced by generative AI tools and published on news sites should be held to the same editorial standards as content that is produced by human journalists. And the flashpoint for this debate has really been at POLITICO with its recent union arbitration hearing, which I did some reporting on [in August]. I’d point out one tool in particular, Politico Pro, which is a kind of subscription policy intelligence platform where readers are allowed to generate policy reports using a large language model. It pulls information from POLITICO’s article archive, but the actual output does not pass through any human editors. There’s disclosure language that says, “This information might not be accurate.”
But what POLITICO’s union has found is that, sometimes, it is egregiously inaccurate and is issuing content that would by no measure be in line with POLITICO’s editorial standards or even necessarily follow its style guide in certain cases. The union has filed a formal grievance saying that its contract’s language states any AI tool that qualifies as journalism effectively has to follow POLITICO’s editorial standards. The newsroom has come back and said, ‘It’s not journalism. This is a generative AI tool built by our product team that has no human editors or reporters directly involved in its copy, and users should be able to see that disclosure language and make the effort to fact-check the information themselves.’ But what is the place of a news publication like POLITICO if not fact-checking its editorial content for accuracy? We’re seeing this kind of loophole almost being used by some newsroom leadership where they’re saying, this is unlike the journalism that we traditionally produce, and therefore we don’t need to hold it to the same ethical and journalistic standards that we normally would.
I think we’re gonna see that conversation continue and touch down in other newsrooms, particularly as newsrooms try to build products that might be able to compete with some more tailored large language models on the market. Newsrooms, news publishers, they want to have a direct line to those audiences. They don’t want their content to always be mediated through ChatGPT or Gemini. But in trying to play in that sandbox, there is the danger of abandoning certain journalistic norms that I think we all should be paying really close attention to.
Ryan: Another AI issue we’ve been tracking at the Ethics and Journalism Initiative is AI bias checkers, which you reported on in July. The big story was that Law360 rolled out this bias checker, and their union also pushed back against that. Have you seen any additional newsrooms or additional movement towards using AI itself to effectuate these very standards and guidelines?
Andrew: When I got the tip about the Law360 story, I was shocked because I had never seen a tool like this deployed in a newsroom. I’ll say from my conversations with the News Guild, which represents Law360’s union, that was challenging the use of this tool, they had not seen it in any of the newsrooms that they represent at the time, either. So there was a novelty factor to this that is part of why I really wanted to report out that story.
Something similar but slightly different that I have seen deployed already is using large language models for fact checking, so similarly using it in the later stages of the editing process to tweak and fine-tune the language in the published piece to make it more accurate. AI fact-checking has been folded into quite a few startups that are trying to embed in large newsrooms. I’m seeing it mostly in Europe, actually. There’s a startup called Noah Wire Services that I’ve covered, and another startup called Open Mind that I’m aware of that are basically trying to automate parts of the aggregation workflow.
The idea is that reporters often in our digital media landscape are not necessarily talking to original sources all day and reporting original journalism. Many times staff reporters are rewriting the original journalism of other publications, and these startups are trying to automate significant parts of that process including identifying what stories might be viable for aggregation, actually drafting the copy of the aggregated articles, and then in a final stage checking quotes against quotes that have been published in other news outlets to make sure that the quote is accurate. It’s effectively spot-checking the article using a large language model. This is really concerning to me because of the propensity of large language models to hallucinate. And it’s at the center of any conversation about AI adoption when we’re displacing human fact-checking power to have automated fact-checking through large language model tools that are, by design, often inaccurate.
Ryan: It seems this conversation has been full of cautionary tales of contested rollouts or moments when things have gone wrong in some sort of way. Are you seeing any examples, or if not real-world examples, starting to think about what would be a more ethical and thoughtful way to roll out these types of tools?
Andrew: There are a few things that I would point to. Many local newsrooms, or at least several that I’ve reported on, are adopting AI transcription tools that are able to effectively sit in on local public meetings, including school board meetings or city hall meetings that journalists are not able to attend in person, generate transcripts, and create effectively a searchable database that journalists are then able to interact with through conversational AI.
Chalkbeat has adopted one of these tools in its newsroom. They have reporters, including one based in Michigan, who have to cover, say, a dozen school board districts throughout Michigan, and they’re just not physically able to be in all those places at once. So it’s really a way that they’re using generative AI to cover their blind spots in terms of editorial coverage. I know one of the stories they published, one of the characters in that story, they found by looking through school board meeting transcripts for testimony from LGBTQ students who were responding to Trump’s executive orders around trans students in schools. They were able to find this real student in a real town in Michigan who was able to speak on the record about their experience over the phone after the fact But they wouldn’t have known about them without this transcription tool. I think that’s a great example of using generative AI to not replace the work of journalists, but to augment and supplement their work, particularly in under-resourced lean newsrooms.
I’d also like to remind folks, and it sometimes feels quite technical to talk about this, but not all AI is generative AI. There are some kinds of machine learning techniques that have long been used by journalists, especially data journalists and investigative journalists.
I’ve been doing a story every year for the last couple of years where I interviewed the Pulitzer Prize winners who disclosed using AI in their submissions. I did one this year, and by and large, the four finalists that I spoke to were not using generative AI. They were using these more traditional machine learning techniques, things like complex data visualizations that lean on AI, building custom object detection models that allow them to analyze satellite imagery. These are things that are under the umbrella of AI, but they’re not what we often think about when we think about the AI boom. We’re often thinking more about AI-generated text or AI-generated imagery.
There are tools that used to have a really high barrier for entry because you needed to know coding languages like Python or R to use them. And what we’re seeing in the last couple of years is that more journalists without engineering backgrounds are actually able to get their hands on these tools because they’re able to use conversational AI to interact with them. They don’t need to learn a whole other language to interact with them. And I think that’s really exciting to me, and also is an important reminder that AI is more than one thing. And there are plenty of use cases that have been used by journalists even before this moment in time.
Ryan: As an AI staff reporter, how do you select your stories and curate your beat? And how might you advise AI watchers in journalism and in media and news consumers generally sort through this kind of general ambient anxiety and get more helpful information?
Andrew: Something that I’m really conscious of and something that I was very intentional about when I started this role was that there is a lot of shallow crystal ball forecasting in AI coverage. There is a lot of unsubstantiated opinion and speculation that seeps into coverage, and my goal was to really document the reality of AI adoptions in newsrooms in real time as it unfolds, and spend less energy than a lot of voices out there on speculative coverage and coverage that is trying to predict the future of AI. I think what I’m most interested in is ‘What is the reality of AI in journalism right now?’
I think, as a beat, if we spent more of our energy focused on that, we’d actually have some more interesting insights. There’s the hype and the fear cycle; they feed each other. My most optimistic self thinks we can break out of the cycle. I don’t know if we can, but we can certainly try. Often, some of these high cycles are also being cultivated by large tech companies that are building these tools. It is a form of marketing for their products. I have a background in tech journalism beyond this job, and I have tried to build up some inoculation to that because it’s really easy to get caught up in the possibilities and miss the real harms or real helps that are happening right now.