GoURMET: A step towards bringing down language barriers across the BBC

Creating content 24/7 in more than 40 languages, the World Service is unique among global organisations, all the more so among media corporations; a mosaic of global, diverse, distinct teams.

In recent years, technological advancements in natural language processing, particularly in machine translation (MT) beckons the day when the translucent language barriers across World Service teams and outlets can disappear for good.

Means to weave in diverse perspectives from various countries on shared interests, better contextualisation, and robust autonomous fact-checking no longer seem too distant possibilities.

We are closer to seeing audiences converge for truly global conversations irrespective of their languages - where nations shall speak peace unto each other.

Fast forward 20 years from the day I first walked under the World Service’s doric columns at Bush House, it has been a joy and privilege to be part of the team that drove the BBC’s most ambitious attempt at exploring the role of MT to facilitate that.

Bush House was the home of World Service between 1940 and 2012

This is the story of three multilingual tools, five international partners, 16 languages, 34 open source models, 42 months, 400+ human evaluations, a potential audience of more than 100 million people, heaps of learning and how it all came about.

An international effort

In 2019, BBC News Labs partnered with Deutsche Welle’s (DW) innovation team and some of Europe’s foremost academic researchers in computational linguistics and MT from Alicante (Spain), Amsterdam (Netherlands) and Edinburgh (UK) universities for an unprecedented project. Combining our skills, expertise and resources, we aimed to expand research on MT development, focusing on communities that could particularly benefit from improved information and news flows.

Funded by the European Union’s groundbreaking Horizon 2020 innovation portfolio, the project was called Global Under-Resourced MEdia Translation - or GoURMET (Grant no: 825299) and had a dual motivation:

• Firstly, spurring on research to ensure no languages were left behind in the global race to expand MT capabilities, dominated over the past decades by global technology companies.

We focused on languages that had strategic importance for global media but did not have sufficiently high quality MT models, or the volume of data readily available to train these. Several BBC World Service languages, despite audiences totalling hundreds of millions, fell under this “under-resourced” category. (It must be said that the project’s timeline saw mainstream providers begin to develop or serve these previously left-behind languages by the time our work was completed.)

• Secondly, exploring and developing fit-for-purpose solutions utilising these capabilities in real-life newsroom workflows.

The BBC was leading the work on this latter aspect, focusing on getting machine translated output assessed by journalists and seeking to integrate models into workable newsroom solutions. GoURMET provided the time and resources to investigate the current landscape of MT, realistically evaluate the quality of various models in different domains, identify opportunity spaces, and experiment with ideas.

By the time we completed our work, the Global Under-Resourced MEdia Translation project generated MT models in 16 languages into/from English, all trained on genuine news content on our websites.

These are Amharic, Bulgarian, Burmese, Gujarati, Hausa, Igbo, Kyrgyz, Macedonian, Pashto, Serbian, Swahili, Tamil, Tigrinya, Turkish, Urdu, and Yoruba.

We sought to identify genuine use cases and built tools and features to demonstrate and test how global media could best utilise MT and other language technology capabilities. We experimented with continuously enhancing these capabilities.

BBC News Labs’ Multilingual Journalism Suite, born out of this project, was shortlisted for the BBC’s News Innovation Award in 2021.

News Labs was also “highly commended” as Technology Innovator of the Year in the Computing Technology Product Awards thanks to this work.

A remote but close collaboration

It involved a lot of team-work, multidisciplinary experimentation and cross-border collaboration (in the shape of umpteen zoom calls due to Covid-19 travel restrictions).

The project involved machine translation experts from Alicante, Amsterdam, Edinburgh Universities and the innovation teams of BBC and Deutsche Welle.

Work began with a wide-ranging conversation on the use cases and the range of languages which could benefit most from MT capabilities.

We tried to identify languages which have:

• sizeable audiences, where the BBC had a robust portfolio and growth potential as well as data
• limited (if any) commercial alternative MT models for our use cases
• interesting linguistic research questions and challenges for algorithms
• overlaps with our media partner DW’s language pool where applicable
• language teams expressing a willingness to try out the outputs.

We worked in cycles, selecting four languages for each, applying more of our insights on data collection and augmentation for the next iteration. This has enabled us to respond to changes in strategic priorities better and amend our practices and evaluation methods.

The BBC World Service provides content in 14 of the 16 languages the project covered:

Amharic, Burmese, Gujarati, Hausa, Igbo, Kyrgyz, Pashto, Serbian, Swahili, Tamil, Tigrinya, Turkish, Urdu and Yoruba.

Every week, these languages jointly reach over 100 million people.

That’s according to the Global Audience Measure, which records the total weekly number of adults accessing the BBC around the world, published in October 2022.

What could MT do for this vast audience and their content creators?

The value proposition

The first task was drawing up the framework on the role of translation in journalism. This was tricky: translation is traditionally seen as inferior to original content (be it in media or literature) and a common retort from our colleagues was "we are journalists, not translators”.

Which was true: Unlike its post-war radio days, where the World Service did employ some “translators”; these days content generators at the World Service are multimedia journalists, who often specialise in their respective fields.

However, coverage of international stories particularly in a global broadcaster like the BBC often involves perusal of some non-native sources as a minimum; and the theory of “news translation” is increasingly defined in functional terms by scholars: as a new offer of information specifically redesigned for intended target recipients.

Could translation support a level playing field to discover content in any language?

Journalists told us they were open to using MT capabilities to monitor and discover stories, but were less keen when it came to serving output involving MT to their audiences.

MT output into non-English languages has in certain cases been described as “mechanical and dry” whereas English was generally regarded to be “more adequate” by non-native journalists.

More than 200 BBC and DW journalists contributed to the project as data validators and evaluators.

Hearing out the concerns and preferences we decided to play to the strengths of the technology. Our answer was to try to bring as much of the BBC’s content to a level playing field, boost transparency and visibility. This would enhance opportunities for content exchange and compliance while also allowing other languages to benefit from natural language processing capabilities that are a lot more varied and advanced in English.

What we achieved

We built a suite of three prototypes exploring different angles of workflow for the BBC. These included a monitoring platform to follow Live Page news updates in any language which proved particularly potent during key global events like the Russia-Ukraine War; a discovery tool to identify best and most interesting original pieces from language outputs, and a semi-autonomous graphics generator. As well as ensuring our language models were open sourced, we built a publicly available website to enable everyone to try out the models themselves. To experiment with improving quality for specialised domains, we built a custom medical glossary ensuring the range of vocabulary is suited for news consumers rather than medical professionals.

During the three-and-a-half years of work (2019-2022), we were able to demonstrate that it is possible to compete with - and surpass - the translation quality of mainstream providers, even on a single training cycle.

Serbian, Pashto, Turkish, and Urdu were among languages with promising results.

There is lots to explore in terms of developing workflows whereby models are improved continuously, as well as building an efficient pipeline that can work with the huge range of production tools employed across the BBC. The research-oriented, experimental models were sometimes not compatible to be seamlessly plugged into systems, or scaleable or fast enough in their initial iterations.

The GoURMET project has also highlighted the importance of finding the right use cases that play to the strengths of developing technological capabilities while mitigating reputational risks, being transparent about the use of such solutions and managing expectations of “acceptable quality”.

News Labs continues to explore multilingual solutions in a bid to remove language barriers across BBC Newsrooms.