Salco: Semi-Automated Local Content

Published: 16 December 2019

Aims

How can we use automation to deliver local news at scale?

The project

The BBC is known for its broad regional news offering, through local radio, TV and online news. There are, however, many untold stories that may be derived in part from public data, whether election results, crime figures or NHS performance statistics.

Such stories lend themselves to semi-automation. Journalists can prepare templates that describe the possible scenarios, which are combined with data to produce potentially thousands of granular local news stories that can reach hyperlocal audiences.

Crucially, this is done without tying up the time of regional journalists who would otherwise have to produce each permutation. This increased capacity can allow us to bring high quality journalism to many segments of the audience for whom it would otherwise be impractical.

Salco (Semi-Automated Local COntent) is the BBC's first attempt to bring machine-written journalism to our audiences. Beginning in late 2018, we have been developing the infrastructure to generate, review, and deliver stories.

From A&E to planting trees

We decided to first test our idea by telling stories about local A&E performance.

For some time, BBC News has maintained an NHS tracker, which meant we were able to draw on the experience our colleagues have with processing and analysing the performance datasets periodically released by the NHS.

Working with journalists in Birmingham, we produced a template of a story that would summarise the performance of a given NHS trust, handling the different outcomes that could come from the data. We then developed a web-based dashboard that allowed an editor to trigger the generation of a set of stories for a particular English region. They could review the text of each story, and it would automatically publish to the relevant local news stream via Vivo - the BBC's platform for short-form storytelling.

The dashboard used to review and publish Salco stories — The web dashboard used by editors to review and publish the stories Salco generates.

We successfully published dozens of stories for each NHS hospital trust in the East of England, and you can learn more about how that project went on this blog.

Last Summer, we built on this by using Salco to generate local text stories and radio scripts for BBC Local Radio about tree planting, to accompany a long-form story on the BBC News website and packages for TV News. This let us tell a story in a range of styles to suit different audiences across several outlets. You can read more about how we built this story here.

To the polls

The local elections in May 2019 were another opportunity to deploy Salco, but we knew we'd need a new workflow to support the editing and publication of stories that are drip-fed over the course of hours or days.

A screenshot of a local election results story — An example of a story generated during the local elections.

Instead of the interface for batch processing monthly datasets, we built a system that subscribed to a notification service announcing each result, then generated a story as each declaration came in. Journalists checked and subedited the stories that were generated, with results graphics inserted into the stories.

You can learn more about how the local elections went on a blog written by Tamsin.

The BBC's exit poll projected on the outside of Broadcasting House — The BBC reveals the result of the exit poll on election night.

By the middle of 2019, it felt like a general election was looming so we started work to adapt our local elections system for what would be Salco's most comprehensive outing yet: 690 stories, with one for each of the UK's 650 constituencies in English, and another 40 in Welsh.

To best tell the range of stories about the winners and losers of the night, we iterated on an increasingly sophisticated template over several weeks, taking feedback from journalists and editors.

We assembled and trained a team of journalists who worked across the UK subbing and publishing articles throughout election night.

The sheer scale of the job, and having journalists working across four sites and in two languages, demanded a better notification system. Our web dashboard made way for a "Slackbot" that announced when a result story was generated. This allowed a journalist to pick up the story, edit and publish without duplicating work.

A slack bot announcing when each story was ready to be reviewed. — A Slackbot announced the arrival of each story, tracking its status as it was reviewed and published by a journalist.

Weeks of iterating and testing with the support of colleagues across the BBC meant the night went off without a hitch. We offered everyone in the country a high quality story about what happened in their area, within minutes of the result being declared. You can get a sense of how the night went on this blog written the morning after.

We're looking at new ways of pushing the technology further, and at opportunities to deploy Salco in other innovative scenarios.

Salco 2023: Making news hyperlocal

Results

We have successfully delivered automated local news for four stories: including A&E performance and the 2019 general election.
The general election was the biggest test of the system to date: with 690 stories generated over one night in two languages, constituting over 100,000 words of copy.