07 August 2019

Salco part 2: Automating our local elections coverage

In May, residents in England, Wales and Northern Ireland went to the polls in the local elections. We at News Labs were asking how we could make sense of the results on a local basis. Enter Salco!

Local elections on BBC News Online are covered at both a national level, to aggregate results and highlight trends, as well as locally by journalists working out of regional hubs. With 248 councils up for election in England alone, that means a huge amount of journalism in a short period of time.

We observed huge variation in election coverage across the country: Some councils were not covered. Some would simply take tweets from the @bbcelection Twitter feed. Others were there at the count, posting photographs and detailed results.

It became clear that it may not be possible to cover each result to the same level of detail, so we set out to make it as easy as possible for a journalist to publish stories for their local councils.

Seeing the results, wherever you are

The aim was for readers to log in to see results on their local news feed, whether that be in Canterbury or Calderdale and always be able to see who the winners and losers were. Salco election result for Calderdale constituency

From Hospital reports to Town Hall counts

With Salco’s A&E project, we had created a pipeline which took the source data, passed it through our Natural Language Generation tool, and pushed the resultant stories to a dashboard where a journalist could review and publish them.This did most of the work we needed for elections, too – to generate the story and send it to our in-house publishing platform, Vivo. Diagram of Salco architecture for NHS stories The main difference between the local elections and the NHS data was how the data would be made available. The A&E figures were published monthly as a spreadsheet on a website, so could be batch processed. The election data would be delivered on a near-live basis, council by council, and we wanted to benefit from that real time data by pushing stories through to journalists as soon as we had the results.

Technically, this is achieved by an AWS SNS results topic run by BBC News’ Data Systems team. We created an SQS which subscribed to the topic, allowing us to process each result as it was received.

Templating story types

To write our story template we worked with Daniel Dunford, a journalist from our Visual Journalism team. After the A&E project, we were keen to push the use of the natural language generation tool further. Dan drafted stories for different results scenarios and I wrote the templates, making extensive use of the conditional logic functions to vary the output depending on the type of result, who the winners were and whether it was a whole or partial council election. Daniel then selected a group of councils whose results were predicted to be announced at staggered intervals so they could give each story the attention needed to check accuracy and legibility – this was still a test project, after all!

What happened on the night?

As the results came through, we checked the stories, as did the journalists. A few needed some tweaking and were edited directly by the journalists though I also edited the story template, ensuring we wouldn’t keep seeing these errors. We’d also hoped to automate the inclusion of graphics, but ran out of time to do this, so these were manually added too.

As we worked on this project, many people were feeling that the chances of a sudden general election were high. Our thoughts are still on Salco’s potential for coverage of a general election – perhaps for every constituency in the UK. We’ve started a new project cycle for Salco in collaboration with the English regions news team, so look out for further experiments soon.

Careers

Love data and code?

We're looking for talented developers to join our team.