14 February 2019

Editorial innovation in news

It is easy to assume that the future of news will be inevitably technical, writes News Labs' executive product manager David Caswell. That assumption is dangerous.

Assumptions are like algorithms in our brains. They’re clearly useful for speeding up how we process information, but they also come with huge risks because they can exclude important knowledge that is different, rare, subtle or new. We all use assumptions, and journalism as a community of practice also uses assumptions. Some of those assumptions are about how we can innovate with journalism.

A common assumption is that journalists can’t fully participate in technical innovation in journalism. The revolution in communications that has disrupted news so thoroughly has been triggered and powered by technology, and its greatest beneficiaries have been people and companies whose skills are primarily technical. ‘Digital’ and ‘technical’ are seen as somewhat synonymous, and we hear a lot more about journalists learning to code than we do about engineers learning to write stories. It is easy to assume that the future of news will be inevitably technical.

That assumption is dangerous. Technical innovations like news bots and automation offer tangible ‘newness’, and their novelty naturally generates surprise and excitement. But meaningful innovation in journalism must produce news products that are genuinely useful to real people, month after month and year after year. To do that they must be rooted in something familiar and human.

Our work on news bots, natural language generation, voice interfaces, structured journalism and a host of other technologies must be as editorial as it is technical.

That ‘human element’ is hard to articulate, harder to define and impossible to capture as data or code, but it is critically important nonetheless. A primary challenge in innovating for journalism is finding ways to bring this essential characteristic into the digitally-native news products that we will need to sustain newsrooms in the digital communication environment.

Here at News Labs we are putting more emphasis on exploring editorial innovation for news products. This doesn’t mean that we are turning away from technical innovation, or that we’ve lost sight of the competitive realities of a communications environment in which everyone walks around with a broadcast studio and a printing press in their pocket. Instead it means that we recognise that our work on news bots, natural language generation, voice interfaces, structured journalism and a host of other technologies must be as editorial as it is technical. It requires radically new ways of reporting, analysing, writing, editing and producing — editorial innovations that will feel as deeply challenging to journalists as cutting-edge technical research feels to technologists.

Let’s look at an example. News bots use ‘dialog trees’ or similar structures to facilitate interaction with their audience. These are assembled from elements created with natural language — blocks of text that are very unlike the articles or scripts that most journalists produce. For a start, they are isolated from each other in ways that sentences or paragraphs in an article are not. They usually have a single purpose, such as providing a description, explanation, answer or a quote. That purpose must be explicit not only in the focus of the writing, but also in the metadata labels, or ‘tags’, that describe the element. The elements must be written in such a way that they are useful in different contexts — without necessarily knowing which other elements have been seen by the reader prior to, or following, their use. They must also be useful over time — and so they can’t assume that news consumers just ‘know’ what’s going on. They must also have provenance, or evidence, associated with them so that their journalistic integrity can be tracked and perhaps even communicated to consumers.

The larger structures into which these bot elements are assembled are also editorial — dialog trees or bot experiences need to cover certain subject matter, they need to be impartial, and they need to be able to expand as events unfold and stories develop. They need to be edited, verified and crafted to fit editorial guidelines and objectives, and maintained and extended as feedback or analytics data from news consumers becomes available. They are also likely to be authored by many journalists, working over an extended period of time, rather than by a single author working alone for a few days.

All of these challenges are editorial, but none of them resemble the practices, workflows and processes of traditional news production. They require exploration of radically new forms of writing and are deeply innovative by any standard, but that innovation is not technical.

Hilda Matheson interviewing an MP

Hilda Matheson (left), the BBC’s first ‘Director of Talks’, helped invent many of the editorial forms of radio news and current affairs.

Let’s look at another example. Newsrooms are increasingly seeking to improve productivity by using automation to produce large quantities of relatively formulaic but highly personalised stories from data. This practice is known as ‘automated journalism’ and it is enabled by a technology called ‘natural language generation’. Natural language generation is based on the creation of templates — complex structures of logic and language that generate customised text from an arrangement of data. Those templates are written by journalists but, like the dialog trees of news bots, they are utterly different from traditional articles or scripts. For a start they are about patterns across many stories, not about the particulars of a single story. They are analytical in nature, because they require a clear understanding of the data and where the story lies within the data. They are not authored using programming languages or word processors, but are instead created with user interfaces that combine language and logic in a single display. Perhaps most importantly, these templates are not one-off artefacts that are produced once, posted on a website and then dumped into an archive. Instead, these templates are true editorial infrastructure — capital assets that can be improved and perfected over time and that can produce personalised news content indefinitely if provided with the required data.

Exploring new forms is hard, mind-stretching, risky work — and the ambiguity inherent in editorial judgement and practice makes that more challenging, not less.

All of this is alien to traditional journalists, but nonetheless it is writing, not technology. It therefore requires exploration, experimentation and risk-taking that is editorial, not technical.

Other examples abound. Editorial innovation has been occurring in newsrooms for years, with new forms of writing showing up in live blogs, alerts, cards, etc. Some digitally-native news startups have pioneered new editorial forms focused on efficient consumption — Axios is a particularly successful example. New modular content management systems such as the Washington Post’s ARC or the BBC’s Optimo are specifically designed to accommodate news content created as nested blocks, with multiple variants of each unit of news — forms that are no longer constrained by the boundaries of a traditional article. Voice interfaces — currently a target of significant investment by newsrooms — are likewise dependant on new forms of writing if they are to develop beyond mere reading services for articles. Even metadata — the ‘tagging’ systems often disdained or ignored by journalists — is becoming increasingly editorial, pioneered by the examples of content platforms like Netflix.

Editorial innovation of this kind is not merely about adjusting the tone or style of articles or videos, but about much more fundamental changes. Exploring these new forms is hard, mind-stretching, risky work and the ambiguity inherent in editorial judgement and practice makes that more challenging, not less. Nonetheless this exploration is essential. Viable, sustainable new forms of news storytelling will likely be impossible without it.

We have plenty of stereotypes of innovative engineers in their hoodies, churning out code until the wee hours of the morning — but what does the editorial equivalent look like? Well, editorial innovators probably aren’t wedded to the traditional formats of article and programme script, and are genuinely open to alternatives. They are probably a bit more abstract than most journalists — willing to stand back and consider the patterns as well as the specifics. They may even be a little theoretical at times. Like most successful journalists they are probably fascinated by language, but perhaps that fascination is rooted in curiosity rather than just in application. Like technical innovators they will be willing to fail — a lot — but also willing to learn from failure and keep looking. They are probably also true explorers — looking for ways of communicating that don’t yet exist but that might be just over the horizon. They might even code.

In 2019 we may be standing on the cusp of new forms of writing. Just as the invention of the printing press enabled the novel and the newspaper article, and the invention of radio and television enabled the newscast and the documentary, so too will the internet enable its own information artefacts with their own unique writing practices. To become sustainable in an environment in which human attention is scarce and text and video are commodities, journalism must experiment with new ways of telling stories. That won’t be easy or comfortable, but it will be interesting and it will certainly be editorial.

If you like experimenting with new ways of creating, using and organising writing to communicate news, and if you’re interested in working at BBC News Labs, please get in touch.


Love data and code?

We'd like to hear from you.