Audience questions: Could the Explainer Builder have the answers?

Even for a story that impacts everyone, like coronavirus or climate change, journalists often fret about the level of knowledge they should assume on the part of their audience. Too little and they will lose the reader with a boring or even patronising article. Worse is the opposite, leaving out vital explanations, making the story impossible for the majority to understand.

Enter the Explainer Builder: a News Labs prototype that creates a searchable embed of frequently asked questions and answers within the story. Could this aid audience understanding by answering each reader’s questions and overcome the challenge of assumed knowledge? All without leaving the story.

We’ve been experimenting with both the audience-facing embed and the content management side of the prototype for journalists this year.

We’ve had great fun, enabling journalists to maintain a single source of truth on different topics at the BBC whilst creating a new audience experience. It’s also had its challenges. Here’s what we’ve learned so far.

No daily updates

News teams built initial collections of questions and answers on climate change, the coronavirus and US politics.

Our specialist correspondents’ in-depth understanding of their beats meant deciding on the initial questions to answer wasn’t too hard. Most started with the background explanations they have to reproduce again and again and also considered the most asked questions from the audience.

Journalists were careful not to include daily news in their content. In the case of the coronavirus this meant the rapidly changing statistics on the pandemic were not included. Instead the trends in the weekly evolution of the story and the background science on the virus and epidemics in general were included to help our audience keep up with the fundamentals.

Early, informal testing with journalists, editors and even readers showed that short answers with only a couple of sentences worked best in this format.

The selection of the three questions displayed by default in the embed proved vital to driving audience engagement. When the coronavirus first emerged as a story, basic questions about the disease - “What are the symptoms?” - were displayed. As the epidemic took on significance for audiences in the UK, more complex details - “Who is worst affected by the virus?” - were being expanded by over 60% of the readers that saw them in the story.

In the same way, questions that offered analysis and explanation from BBC subject matter experts - “Why is Joe Biden struggling?” - when the now presumptive Democratic party nominee got off to a slow start in the US primaries, also drove audiences to read answers.

By keeping an eye on the questions audiences are expanding, journalists can keep a finger on the pulse of overall understanding on a long-running story. Such data makes it possible to build out standalone explainer content with confidence that we are answering questions that will be useful to many people.

Design is substance

Journalists updating the tool with questions and answers gave us feedback on the interface and user experience. The initial minimum viable product was built as a hasty prototype to test the concept of journalists using a single source of truth for work they often duplicate across stories.

Whilst adding efficiencies to the typical journalist’s workflow was a strong selling point in early demos to news teams, we quickly realised that the usability of the prototype was also vital to getting buy-in from the newsroom and testing the hypothesis fully. Even pilot users now expect early prototypes to be simple and sleek.

Our designers reviewed the basic interface which crammed most of the options for editing and customising an explainer into a single screen and proposed a new user experience which makes the prototype more intuitive for journalists and editors.

Playing matchmaker

Matching free-text user questions to answers that meet the editorial standards of the BBC was something News Labs had experimented with for some time.

With this prototype we made a leap forward in the audience experience by extracting entities and hints from the text of the questions and answers and fed them into a matching engine. We then built a test feature, allowing journalists to manually test the matching by asking questions themselves. With journalists adding their own hints if needed after testing, all within the editing interface of the prototype, we created the best outcome yet.

The end result is a new audience experience that we see providing BBC-quality answers to each reader’s questions, all without leaving the article.

So many questions

With such a huge audience seeing the embeds, for example our coronavirus explainer had millions of user impressions on the news website in the last week of February, we worked on capturing the questions readers were asking.

By expanding on previous experiments in clustering user questions - we named our prototype "Clustertron" last year - we can now create buckets of user queries within any topic and show journalists the most asked questions on their beat.

The inputs aren’t just from the embed. We’re now also able to feed questions asked across the website, using submission forms on the bottom of stories, back to teams in the newsroom in a digestible format. These questions can even be automatically added to an existing explainer allowing journalists to further narrow the gaps in their coverage for the audience.

We’re excited to see how teams will use this new feature in the coming months and if we can achieve a sustained conversational loop with audiences while adding to their understanding.

All together now

Many questions remain about whether or not a news organisation as diverse as the BBC can gather around one source of truth. Whilst it would drive efficiency by having expert correspondents curate one set of facts that could be used across various outlets, from online to smart speakers and even television reports, it also requires in-depth collaboration.

Early testing with potential internal users shows that various teams want to have a level of control on formatting the content for their output channel, for example a conversational style of answer on voice platforms. Building this functionality into the prototype whilst maintaining a single source of truth at its core is something we could explore in the future.

For the moment we’re excited to see how journalists use it, and how audiences react.

You can see more on the project and take a look at stories the embeds appear in here.