Experiments with AMP's new 'Stories' format for news

When the AMP Project announced their new Stories format in February, publishers took notice.

AMP (Accelerated Mobile Pages) is an open-source initiative created by Google; with a lightened page load and Google-boosted search placement and caching, it has arguably been a huge success since it launched in 2016. The BBC added an AMP article renderer to its News article publishing pipeline, so for little cost and no extra editorial effort all of our News output is now available in AMP. This is vital at a time when we are trying to increase our reach around the world and deliver news that is optimised for mobile.

Because of this, senior BBC News managers were keen for us to experiment with the new Stories format. Similar to Instagram and Snapchat stories, an AMP Story is told through a number of cards that combine images, text and video, which can be navigated between using a tap-through interface.

Working with Oleg Mikhaylov, the Multimedia Desk Editor for the BBC Russian Service, myself, and developers Lei He and Pietro Passarelli had six weeks to test whether the story style appealed to World Service audiences — and also, to gain a better understanding of how difficult it would be to produce stories in the new format. In this post I'll be talking through the technical part of how we built a simple CMS and what the future of the format is at the BBC.

Building a CMS for Stories

What makes AMP different to regular HTML pages is its extensive use of Custom Elements, which allow Javascript behaviour to be attached to HTML elements — and in AMP's case to be written for better performance on mobile. So the <img> tag becomes <amp-img>, and it handles the loading of that image.

Stories introduce a new set of AMP-HTML tags, including <amp-story> and <amp-story-page>. While the mark-up is easy enough to hard code, it would be inefficient to do so for several different stories, so we set out to create a lightweight CMS and publishing system.

Fortunately, we weren't starting from scratch: Mic News had already created its open-source projects Story JSON and Story JSON to AMP. Story JSON is a structured document format for card-based stories, and Story JSON to AMP converts its output to valid AMP Story HTML.

However, as an emerging open source project we found it had some limitations — some of which stem from the fact that it's designed as a flexible format that can be used to create various outputs (video as well as AMP Stories). For example, it uses Flexbox to handle the layout, which results in some image distortion that the AMP Story layout styles are better at handling.

To create Stories that were visually appealing we asked the design agency Magnetic North to create a style guide. After briefing them on our work and providing them with some branding assets, they rapidly created a set of eight templates which we could use for covers, pull quotes, labelled images and text-heavy cards.

With our design templates in hand, our next task was to express them as a number of Story JSON templates. Using their specification, we wrote a Javascript object for each template style, adding in variables to represent the background images, logos, text and video files.

In order to populate the variables with editorial content, we had to create a system that would be straightforward for journalists to use and simple enough for us to build in a couple of weeks. We experimented with some web forms generated through JSON schemas, but found for our (short-term) purposes that a spreadsheet allowed us to easily access the data and let journalists to see an overview of each story at a glance.

Using a scraper to retrieve the contents as JSON, we had our user input ready to turn into HTML. With each sheet representing a story, and each row representing a card, we could specify which template each card should use from a drop-down list, and pass that data to the Story JSON templates. These could then be converted to AMP-HTML, and finally copied to a build directory.

Because we were creating Stories for both the English and Russian language services, there were a number of language variants we needed to account for. By adding a language code (e.g. 'ru') to each story in the spreadsheet, we added conditionals to ensure that the correct branding assets were used, the correct HTML 'lang' attribute was set, and the HTML output was placed in the correct build directory for that language service.

Serving Stories

While Google cache the story HTML, they don't cache any of the media, so we had to find our own solution for video and images that would work at scale. For video, we made use of the existing BBC tool Castaway, which journalists use to upload video (including vertical video as it would be in this case), and Media Selector, an API used to serve video across all BBC products, fronted by specialised media CDNs. For images, we used another in-house tool with a CLI to upload files programmatically to a S3 bucket, also fronted by a CDN.

Finally, we had to host the content on the BBC website. We asked for a number of new routes (e.g. bbc.co.uk/news/ampstories/*) to be created at the VTM (Virtual Traffic Management) level and point to a couple of Amazon S3 buckets where we would upload the stories. The VTMs also offered a caching layer for any traffic coming directly to the BBC site.

With each component in place, we needed to automate publication of each story. We wrote a script to trigger the Jenkins endpoint (our continuous deployment server) which starts a build and runs the Node application inside a Docker container on Jenkins, then copies the build assets to S3.

Writing Stories

While we were busy building the CMS, Oleg had been working on a number of stories for both the Russian and UK services. You can find the Russian stories here and the UK stories here. We asked Oleg for his thoughts on writing AMP Stories:

"These stories are quite easy to write — if we compare to regular stories on our website. The only thing you have to consider — you have to fit everything in a few sentences. It can be challenging when you have a long, complicated story.

In terms of editorial workflow — it was easy, especially when I was aware of the Russian Services' plans, so you always have an option to plan ahead and create regular stories and AMP in parallel. Some of the stories were re-versioned from CPS/Shorthand (regular and long-form article formats) equivalents and some of them were created specifically for this format."

We also learned that editing videos to meet AMP Stories specifications could be quite labour-intensive for journalists; Oleg said that multiple programmes were needed to edit files into the correct format. And with multiple videos per story, the process could be time-consuming.

Measuring performance

With our stories published and added to the sitemap, we waited for the traffic to roll in. Unfortunately, it wasn't that simple — without promotion on the BBC site, very few viewers were discovering the stories through search alone.

Following a blog on the Russian site and some promotion across social media, our numbers did increase a bit. Our data shows that the Russian-language AMP Stories actually received close to six times as many total views as the English-language Stories. We also added tracking to measure how far users were progressing through stories, engagement times and clickthrough rates for related articles, but we suspect there was an issue with the tagging we used to collect this data based on the figures that came back.

The bottom line is that we could only promote the stories so much. Many publishers taking part in the original trial were featured in a special carousel for Stories in Google search. Google weren't able to add us to Search during our trial, which limited our reach; most of the traffic to our English-language AMP Stories came from links we shared on Twitter. And an unusual limitation for those part of the carousel trial was that users would have to search for the publisher (e.g. CNN) rather than the story, which is not typical user behaviour.

Producing the headlines

In parallel with our bespoke Stories, we'd also been developing a format created by a team in BBC R&D who have spent the year prototyping New News formats. The team had already built upon their Expander prototype using AMP HTML and the BBC News index as a story feed, so it seemed a natural step to try it as an AMP Story.

In its original form, a page consisted of several short story summaries, each with a headline, images and one line summary. You could tap the story to expand it to several sentences, and tap again to read the full story — all while remaining on the same page.

The AMP Story version, which we called Headlines, presented each story as a card, with just the headline, image and one-line summary. To read more, you could tap a button to read the full article, or continue tapping through to browse all the headlines.

For the build, Mathieu Triay from the New News team had already built a RSS feed scraper. We wrote a Node app with an HTML template that would scrape the feed and insert the headline, image URL and text. We then set up a Jenkins job to run every 30 minutes, and copy the resulting HTML files to the S3 buckets using the existing routing setup.

The New News team were able to run a small (five) user test of our AMP Stories during their other work. Some of the feedback included:

Four out of five liked that you could click through to the full article. One participant said she would prefer to stay on the same page.

Some users liked the idea of a personalised story selection:

Interestingly, all tried to swipe at first but discovered the tap interaction without prompting. Overall, four out of five liked it and said they would use it now.

The future of AMP Stories

Since our short trial finished, we're pleased to say that AMP Stories will continue at the BBC... for now. Our six Indian-language services have each published AMP Stories about the people 'Making More of Their World' (here are the Punjabi, Hindi, Tamil, Gujarati, Marathi and Telugu versions) as an experiment. And in September, the BBC Visual Journalism team will start publishing three stories a week, taking the format further and we hope securing the all-important search carousel.

Since the launch, several publishers who were part of the initial trial have stopped publishing new content, so is the format running out of steam? Producing AMP Stories has some upfront costs for creators, in terms of designing and implementing a tool for journalists, as well as committing to regularly producing stories for this format. Indeed what we created is only a temporary solution that requires further consideration. But until Google roll out the product across their platform, publishers may be unwilling to make this commitment.

The format shows a lot of promise, with a good response in our tests for users who prefer more visual content. For now, Stories continue somewhat tentatively, and we'll watch closely for further developments.