Gates Foundation launches coalition to build more representative language data sets for AI

NEW YORK (AP) — The Gates Foundation is convening Anthropic, Google, OpenAI Foundation and dozens of other organizations to help make artificial intelligence more accessible in underrepresented languages.

The partners — 60 in total, according to Monday’s announcement — include frontier AI labs, corporations, philanthropies and others already working to expand the number of languages available in AI tools that the Gates Foundation believes could combat global inequality. The coalition aims to reach more than 3 billion people over five years by better coordinating existing efforts.

It’s work that must continue “full speed ahead,” according to Gates Foundation CEO Mark Suzman, even as some of the largest AI companies urge their industry to slow down development of advanced models. Governments must regulate the impacts on cybersecurity or children’s development, he said, while they extend AI’s humanitarian applications to poor communities currently “shut out” from the technology.

“Even if AI was frozen right now — which I don’t expect and I am not calling for — we would want to be building these language sets and making them usable with the tools that we have available right now,” Suzman told The Associated Press.

The collaboration follows last week’s Goalkeepers report, where the foundation committed $1 billion toward AI-focused efforts to improve health outcomes, upgrade educational tools and inform small farmers’ practices around the world. Unrepresentative language data could lead a model to mistranslate a pregnant Malawi woman saying her “water has broken” into the direct English that she’d “thrown away water,” the report warned.

The “original sin” is that many AI tools were trained on language data scraped from the internet, according to Mozilla Data Collective CEO E.M. Lewis-Jong, whose company was incubated by the Mozilla Foundation and is now a subsidiary of the not-for-profit Mozilla.org. The data-sharing platform seeks to help communities upload cultural and linguistic data sets on their own terms rather than have that data taken from the web without their explicit consent.

“The internet is not a representative space,” she said. “Why would you think that you were going to get a culturally diverse and representative system out of something that was predominantly trained on Reddit?”

Details such as the coalition’s governance are still getting hammered out, according to Suzman. A secretariat will track each signatory’s commitments and the Gates Foundation may nudge partners to fill larger gaps when necessary.

Google, a member of the coalition, has been funding an effort to collect more than 150,000 hours of audio across every district in India. Known as Project Vaani, the push underscores the need to gather speech data on dialects within languages, according to Google senior vice president James Manyika. To get there, he said, Google is working with local partners to record speech in the field.

Anthropic, whose CEO published an essay calling for industrywide cooperation on decelerating advancements, was already working with the foundation to speed up vaccine developments and bolster its chatbot’s data set of local crops. Elizabeth Kelly, the head of beneficial deployments at Anthropic, acknowledged the company’s products “lag in many African languages in particular.”

“We’re acutely aware that we can’t achieve any of the benefits we want to see in terms of improving patient outcomes or improving literacy and numeracy for students across the globe unless we actually get this language piece right,” she said.

___

Associated Press coverage of philanthropy and nonprofits receives support through the AP’s collaboration with The Conversation US, with funding from Lilly Endowment Inc. The AP is solely responsible for this content. For all of AP’s philanthropy coverage, visit https://apnews.com/hub/philanthropy.