Photo by Beyza Yılmaz on Unsplash
Researchers Reveal AI Models Show Racial And Socioeconomic Bias In Medical Advice
- Written by Andrea Miliani Former Tech News Expert
- Fact-Checked by Sarah Frazier Former Content Manager
A new study published in Nature Medicine this Monday reveals AI models show racial and socioeconomic bias in medical recommendations when different socio-demographic labels about the patient are provided.
In a rush? Here are the quick facts:
- A new study reveals multiple AI models show racial and socioeconomic bias in medical recommendations.
- Researchers considered 9 LLMs and 1,000 cases for the study, including racial and socioeconomic tags.
- The results showed AI models make unjustified clinical care recommendations when including tags such as “black” or “LGBTQIA+”
The research, Sociodemographic biases in medical decision making by large language models , was conducted by multiple experts from different institutions and led by the Department of Genetics and Genomic Sciences at the Icahn School of Medicine at Mount Sinai in New York.
The researchers considered 9 Large Language Models (LLMs)—proprietary and open-source—and analyzed more than 1.7 million outputs from 1,000 emergency department cases—half of these real and the other half fictitious—including 32 variations.
The abstract of the study states:
LLMs show promise in healthcare, but concerns remain that they may produce medically unjustified clinical care recommendations reflecting the influence of patients’ sociodemographic characteristics.
In the variations, the researchers included sociodemographic and racial identifiers, revealing that the outcomes had a strong influence in these. For example, cases with the LGBTQIA+ subgroup tag or identified as black patients were suggested to receive more mental health analysis, get more invasive treatment, and were recommended more often to visit urgent care.
The researchers wrote:
Cases labeled as having high-income status received significantly more recommendations (P < 0.001) for advanced imaging tests such as computed tomography and magnetic resonance imaging, while low- and middle-income-labeled cases were often limited to basic or no further testing.
The researchers claimed that the behavior was not supported by clinical guidelines or reasoning and warned that the bias could lead to health disparities. The experts note that more strategies to mitigate the bias are needed and that LLMs should focus on patients and remain equitable.
Multiple institutions and organizations have raised concerns over AI use and data protection in the medical field in the past few days. A few days ago, openSNP announced its shutdown due to data privacy concerns , and another study highlighted a lack of AI education among medical professionals .
Photo by Ilyass SEDDOUG on Unsplash
UN Says AI Will Impact 40% Of Jobs Worldwide
- Written by Andrea Miliani Former Tech News Expert
- Fact-Checked by Sarah Frazier Former Content Manager
The United Nations Trade and Development (UNCTAD) shared a new report this Monday, Technology and Innovation Report 2025 , revealing that AI could affect 40% of jobs across the globe and urging nations to take action. The agency also estimates the AI market will reach $4.8 trillion by 2033.
In a rush? Here are the quick facts:
- UNCTAD published the Technology and Innovation Report 2025 , revealing that AI could impact 40% of the jobs globally.
- The agency estimates that the AI market will reach $4.8 trillion by 2033.
- Experts urge governments and policymakers to study the impact of AI, prioritizing human workers and including developing countries.
According to the official document shared by the agency, the use of AI can bring multiple benefits to workers, but also risks and job losses. The UNCTAD also acknowledges that AI shows different patterns and behaviours compared to other historic technological waves.
“AI can perform cognitive tasks and impact a far wider range of activities, conceivably affecting 40 per cent of global employment, transforming production processes and business operations,” states the document. “AI can bring productivity gains and increase the income of some workers, but also cause others to lose their jobs, reshaping workplace dynamics and labour demand.”
The organization explains that AI has already been impacting in multiple ways. It can enhance jobs, particularly in developing countries, by boosting productivity and creating new roles. At the same time, AI models—and combinations of emerging technologies—are expected to replace some human jobs. For instance, the report notes that AI can monitor financial transactions in the banking sector to detect fraud or anomalies more efficiently. In healthcare, AI can assist doctors in diagnosing cancer by analyzing radiographs and electrocardiograms.
Considering the global economy and development landscape, UNCTAD warns about the location and the businesses in power over frontier technologies. According to the study , most developing countries lag behind in research and development—except for China.
The United States and China hold 60% of AI patents and 33% of AI publications. In infrastructure, the U.S. takes the lead with around 50% of the world’s computing power and a third of the top supercomputers.
“There is a significant AI-related divide between developed and developing countries,” states the report. “This could widen existing inequalities and hinder efforts by developing countries to catch up.”
UNCTAD urges governments and policymakers to understand the complex dynamics of AI to ensure an equitable impact of AI, support job transitions, and create AI solutions for developing countries—always prioritizing keeping human workers.
The European Union has already implemented the first AI Act and recently published guidelines to prevent misuse of the technology . However, other nations still have a long way to go.
