Using genAI to understand impact of journalism: The Agência Pública story
Here is how a Brazilian non-profit investigative journalism organisation is using generative AI to measure the impact of its work beyond tracking republications
Agência Pública, a non-profit investigative journalism organisation in Brazil, faced a growing challenge in measuring the impact of its work.
This news agency, founded in 2011 by three women journalists, has developed a strong presence in Brazilian and Latin American media due to its innovative approach to investigative reporting, its coverage of human rights and social issues, and its evidence-based approach to journalism.
The problem
As their work gained increasing relevance at both local and international levels, they began measuring their impact. However, assessing the impact of a journalistic organisation is not straightforward. While media outlets traditionally gauge their impact by the number of times their content is republished by other organisations, Agência Pública’s team realised that tracking republications alone was insufficient.
Marina Dias, Director of Communications at Agência Pública, has been part of the organisation for over 10 years. Initially responsible for managing republications and the organisation's presence across platforms, her role expanded to encompass monitoring the overall impact of the agency’s work.
The team recognised the need to evaluate the broader impact of their stories, such as mentions and citations across various media outlets. “Back then, it was a lot of manual work for us, mostly done with spreadsheets. We even had a form that our reporters would fill in if they encountered any relevant cases. Then, last year, with everyone talking about artificial intelligence (AI) in journalism, we realised we needed to explore how AI could be used in our newsroom — including defining what can and can’t be done with AI at Pública,” says Marina.
The genesis of the solution
In 2020, Agência Pública developed an internal system to monitor media mentions using a WordPress site. However, the system proved inefficient, slow, and did not significantly reduce the manual workload.
A breakthrough came when Agência Pública participated in the Open Society AI Challenge for newsrooms in 2023. “We already had one project in mind: using AI to read our reports aloud. We tested it for a while but abandoned it because we realised it didn’t resonate with our audience — they weren’t consuming content in that format. Then I said, ‘If we’re going to solve a long-standing problem, let’s use AI to track the repercussions and impact of our work,’” Marina explains.
The team identified the potential of AI to automate parts of the monitoring process — specifically routine tasks that, with the increasing volume of mentions, were becoming burdensome for staff.
They decided to apply AI, particularly generative AI tools like ChatGPT, to analyse mentions of Agência Pública in the media and categorise the impact of their work more effectively for future analysis.
The new workflow
For the AI solution to succeed, Agência Pública first needed to establish clear categories for media mentions and impacts. The "Pública IQ" project emerged from this process, combining simple automations with ChatGPT’s natural language processing capabilities.
“What was really interesting for me, after seven years working on this, was that I thought our impact monitoring process was thoroughly documented, but it turned out there were gaps in how we categorised things. One of the tasks ChatGPT performs is classifying each result into categories we thought we had already defined.”
The solution involves six key steps, from extracting text from media mentions to using AI for categorisation and analysis. Integrating these processes into a WordPress-based system allows for easy management and prompt modification, enhancing flexibility. ChatGPT is used at two key stages: first, to determine whether the extracted content from URLs is about Agência Pública, and second, to analyse and summarise the collected information.
Human oversight remains essential: staff still review the AI’s analysis to ensure accuracy. “Theoretically, we could allow the system to complete all the steps independently, but we prefer to review it. We quickly check everything is correct before finalising the registration. We don’t encounter many errors with ChatGPT. Occasionally, there are misclassifications, but they’re rare. The analyses can vary slightly in format — sometimes in bullet points, sometimes as continuous text — but overall, it works well.”
What we’ve learnt
If your organisation already works with databases and performs routine categorisation tasks, generative AI technologies can support your work.
However, beyond simply using a specific tool, Agência Pública’s experience offers valuable lessons:
Start with a well-thought-out process to clearly understand the challenges and how AI can complement human effort.
Analyse existing workflows to determine which processes are easiest to automate.
Don’t be afraid to revise categories if necessary.
Use widely available tools, such as ChatGPT, within a custom-built workflow.
Keep human oversight central to ensure quality control.
Develop scalable systems that other newsrooms can adopt, creating sustainable models for organisations with limited resources but significant data-monitoring needs.
Responsible AI usage
Adopting AI also requires a strategy to guide decision-making. “During the Open Society Challenge, we realised we needed guidelines for AI usage. By October last year, we had already established those rules,” explains Marina.
“The main rule is that Agência Pública doesn’t use AI to write reports. Our reporters still conduct investigations and write their stories,” she continues. According to the organisation’s principles, AI can assist reporters with tasks such as data analysis or transcription, but always under human supervision and review. “Let the machines handle what machines can,” recommends Marina.
Final advice
Marina emphasises the importance of focusing on existing challenges: “The first thing is to solve existing problems, not create new ones. Assess the challenges or demands the organisation already faces. It doesn’t necessarily have to be a problem — it could be a demand, like wanting to launch a product that is currently too costly or complex to develop. Start with what you know well.”
She also warns against unrealistic expectations: “AI is great, but it’s not magic. It requires study, dedication, and a lot of patience. Not every problem can be solved with AI, so it’s crucial to understand what you can and can’t achieve with it.”
Note: Whisper and Claude were used to transcribe the interview, and make the final edits to the text, respectively.