Latent racism in AI models — mostly unintended — has been a concern since the emergence of AI in its current form several years ago. Despite increased awareness and efforts to expunge racial bias (not to mention sexist and cultural bias), the scourge continues to pollute AI output.
That’s the conclusion of researchers at Stanford University, who note that “despite advancements in AI, new research reveals that large language models continue to perpetuate harmful racial biases, particularly against speakers of African American English” as dialects are translated into text.
Such persistent underlying racism affects African Americans’ opportunities where AI is now being applied — in housing, education, and employment, as well as adversely affecting criminal sentencing. Covert racism against African American English persists in the major LLMs — including OpenAI’s GPT2, GPT3.5, and GPT4, as well as Facebook AI’s RoBERTa, and Google AI’s T5, the study finds.
While it’s perceived that AI models are getting better with every iteration, and presumably weeding out racism and bias, it unfortunately has not been the case. Often, instances of AI bias are only fixed as they surface, without addressing the underlying problem, the researchers state. Essentially, overt racism gets papered over, “by superficially obscuring the racism that language models maintain on a deeper level,” the study shows.
LLM developers already “spend significant effort fine-tuning their models to limit racist, sexist, and other problematic stereotypes,” the study’s authors — Valentin Hofmann (Allen Institute for AI), Dan Jurafsky (Stanford University), Pratyusha Ria Kalluri (Stanford University), and Sharese King (University of Chicago) — point out. Despite years of efforts, these models “still surface extreme racist stereotypes” dating back to the 1950s and earlier.
The researchers blame training data — often scraped from the Web — for racism seeping in at a covert level. “Developers of LLMs have worked hard in recent years to tamp down their models’ propensity to make overtly racist statements,” the researchers related. “Popular approaches in recent years have included filtering the training data or using post-hoc human feedback to better align language models with our values. But the team’s research shows that these strategies have not worked to address the deeper problem of covert racism.”
There is a need for greater awareness of covert racism, the researchers urge. Greater and deeper evaluation by AI proponents is needed. They even recommend that policymakers consider “banning the use of LLMs for academic assessment, hiring, or legal decision making.”
What should AI proponents and developers do to address this potential racism that could surface within AI output? AIForward provides some actionable strategies to help expunge racism from LLMs:
- Identify bias in datasets: “Ensure that the data faithfully represents the various traits and attributes of the population it intends to cater to,” the AI Forward authors recommend. “Deliberately identify underrepresented racial and ethnic groups in your dataset and actively seek out data sources that reflect their diversity. Collaborate with community organizations or experts to acquire comprehensive data.” In addition, they recommend proactively collecting diverse and representative data.
- Focus on algorithmic fairness: This is “the practice of modifying machine learning algorithms to ensure that they do not discriminate against specific racial or ethnic groups, gender, or any other sensitive attributes,” the AI Forward authors state. Measures that can be taken include defining fairness metrics, which may include “disparate impact, equal opportunity, and demographic parity.” In addition, they urge model modification in which a model may be “penalized” for “making predictions that disproportionately favor one group over another.”
- Assemble a diverse AI development team: “A crucial step in ensuring that your AI solutions are developed with a wide range of perspectives and experiences.” This can be supported with appropriate training, and collaboration with outside experts and organizations with diversity experience.
- Establish an ethical AI review board, with feedback loops: This involves “assembling groups of experts with diverse backgrounds, including ethicists, sociologists, and domain specialists.” This board would oversee periodic reviews of AI models, algorithms, and policies, as well as responsive feedback channels.
- Monitor and evaluate on a continuous basis: Continuously monitor the performance of AI systems in real-time, as well as through regular reports. This includes implementing “automated systems that continuously monitor AI predictions, interactions, and outcomes for fairness and bias.”
- Promote critical thinking and awareness: Encourage users “to think critically about the content they consume and raising awareness about potential biases and stereotypes is essential,” according to AI Forward. “This education empowers users to recognize and challenge biased content.”
