Singapore builds ChatGPT-alike to better represent Southeast Asian languages, cultures

“Do we want to force every person in Southeast Asia to adapt to the machine, or do we want to make it more accessible so people in the region can make full use of the technology without having to be an English speaker?” he said.

“We are not trying to compete with the big LLMs; we are trying to complement them, so there can be better representation of us,” said Teo, senior director for AI products.

Japan author’s AI revelation sparks debate: ‘some readers may feel cheated’

There are over 7,000 languages spoken worldwide. Yet LLMs including Open AI’s GPT-4 and Meta’s Llama 2 that are used to build AI systems such as chatbots and other tools, have largely been developed for, and are trained on, the English language.

These models can help local populations participate more equitably in the global AI economy that is largely dominated by big tech firms, said Nuurrianti Jalli, an assistant professor at Oklahoma State University’s school of communications.

“Regional LLMs are also needed because they support technology self-reliance,” she said. “Less reliance on Western LLMs could provide better privacy for local populations, and also align better with national or regional interest.”

‘We need to verify and filter’

Multilingual language models, which are trained on text from several languages at once, can infer semantic and grammatical connections between high-resource languages that have more data, and low-resource languages, researchers say.

These models can be used in a variety of applications from translation to customer-service chatbots, to content moderation on social media platforms that have struggled to identify hate speech in low-resource languages such as Burmese or Amharic.

About 13 per cent of SEA-LION’s data is sourced from Southeast Asian languages – more than any other major LLM, Teo said. More than 9 per cent of its data is from Chinese text, and about 63 per cent from English.

Multilingual language models often train on translated text and other poor quality data that may have errors, so AI Singapore is “careful” about the data used in training SEA-LION, Teo said in his office at the National University of Singapore.

“The age of pristine data has passed – a lot of the stuff on the internet now is material that is generated by LLMs, so we need to verify and filter,” he said.

“We cannot be perfect, but we also cannot take out everything we consider to be bad,” he added.

More governments are contributing data, and businesses are testing SEA-LION, which due to its smaller size can be deployed faster and is cheaper to fine-tune and adopt, Teo said.

At Indonesian e-commerce company Tokopedia, a majority of customer interactions is in Bahasa Indonesia, so models “with that local fluency will enhance our ability to connect with customers and improve their experiences,” said Paul Condylis, Tokopedia’s associate vice-president of data science.

Bias in the data

As more countries and regions build their own LLMs, digital and human rights experts fret that they will reproduce only the dominant views expressed online, which can be particularly problematic in nations with authoritarian governments or strict media censorship, or those lacking a strong civil society.

Chinese social media platforms, for example, censor references to the Tiananmen Square uprising and criticism of the government, while several Southeast Asian nations have enacted laws to curb content that authorities deem as misleading.

“Training models on such data risks perpetuating biased, prejudiced, incomplete and even misleading narratives,” Jalli said.

“The models may fail to surface important sociopolitical issues like human rights abuse, corruption, or valid criticism of political powers,” she said.

In response to a query on Indonesia’s former president Suharto, for example, Llama 2 and GPT-4 mentioned his spotty human rights record, while SEA-LION’s response focused largely on his achievements.

If a model is only trained on favourable articles about a government, then the model is “likely to adopt a world view where the government is wholly positive and leave behind dissenting viewpoints,” said Aliya Bhatia, a policy analyst at the Centre for Democracy & Technology, a US non-profit organisation.

“Regional LLMs may better reflect the linguistic and cultural nuances of local language speakers, but they may also have less information about the world in general,” she added.

“There is a real risk of government-backed models instilling a revisionist view of history and undermining democratic values.”

The dark side of unchecked AI use: laziness and learning loss

But the alternative – relying entirely on Western LLMs with “disproportionately large influences” from wealthy, liberal, Western democracies – means perpetuating different biases related to cultural values, political beliefs and social norms, according to AI Singapore.

“These LLMs have a very particular West Coast American bias – they are very woke. They do not represent us,” Teo said.

“We are not saying ours is the only perspective – we are just trying to rebalance it.”