The Hidden Dangers: Bias and Ethical Risks in Medical AI

Dr. R. Amarnath Trivedi

Research Fellow, George Institute for Global Health

Keywords: Artificial Intelligence, Ethics, Bias, Medicine, Risk

The influences of Artificial Intelligence (AI) technologies in today’s healthcare are notable, from more precise diagnostics to predictive analytics concerning patient outcomes. As innovative as it is, there is always a risk that comes with the embracing of medical AI technologies, especially the unobserved dangers of bias and ethical predicaments.

If these issues are not tended to in a prudent manner, great AI advancements will become a paradox by amplifying negative effects pertaining to discrimination on healthcare services.

Medical AI pertains to the application of machine learning, deep learning, natural language processing (NLP) as well as other technologies in the analysis of intricate medical datasets. Its uses encompass

• Diagnostic Equipment Cancer detecting systems from radiology images.
• Clinical support decision systems
• Risk prediction modeling for various diseases.
• Patient data and health monitoring.
• Health assistant bots.

These technologies can deliver more efficiency in service, personalized treatment, and better results to patients. Still, their accuracy depends on the quality, relevance, and ethics of the algorithms and data.

AI bias may not be a new concept, but it certainly comes with shaky implications when combined with health care. A good number of AI are trained on data sets that do not adequately capture the population demographics. An algorithm developed within a white patient’s data corpus will not perform well on any racially diverse patient population. Some researches out there indicate that some dermatological AI tools are less efficient in identifying skin cancer in people with darker skin types.

Health care systems are always a reflection of the discrimination system’s social hierarchy. Any AI systems trained on data from these caregivers who are sent to support sick people in propagate deep seated discrimination norms will likely end up developing bias out of discrimination norms.

Medical AI bias consequences can be life-altering.  A conservative AI algorithm will fail to diagnose a condition, especially when there’s an underrepresented demographic, resulting in a misdiagnosis. Ranging access to poorer populations can further worsen the problem of unequal treatment access to medical services.  

The lack of reliable medical tools for specific demographics can provoke the erosion of trust amongst the general public and medical professionals.  

Health inequalities that AI automated tools are intended to help level out can ultimately worsen due to persistent bias built within AI algorithms.  

An infamous case was reported in a healthcare algorithm: black patients in the united states were classed into an inferior group where their health spending “needs” were systematically underestimated compared to white patients who were put in a privileged group. The algorithm bypassed entire spending on healthcare proxy facilitated systems revolving around spending proxies such as dial-on devices whose spending on the “free” economic filter homogeneous equal enhanced consumption – outcomes never desired but imagined “overs” actually meant high patient need-surplus health.”

There are ethical risks regarding the applying of AI in the healthcare sector.Providing interpretability precedes Justification as There has to be every reason justifying that output has been reasonable and logical.

Epidemiological information is very critical. AI needs a lot of data which makes concerns regarding information such as patient consent, anonymization, and possible exposure surpassing confidentiality promises to rise.

Ethical concern with regards to obligation is whose duty is it to take when predictive algorithm miscalculates. Is it the engineer who builds, the health provider who runs it or the health management system determining matters? Suitable authority allocation must be set out.AI overshadowing the practitioner may result in prescriptions made without practitioners critically analyzing patient information.

References:

1. Topol EJ. High-performance medicine: the convergence of human and artificial intelligence. Nat Med. 2019;25(1):44–56.
2. Esteva A, Kuprel B, Novoa RA, et al. Dermatologist-level classification of skin cancer with deep neural networks. Nature. 2017;542(7639):115–8.
3. Blease C, Kaptchuk TJ, Bernstein MH, Mandl KD, Halamka JD, DesRoches CM. Artificial Intelligence and the Future of Primary Care: Exploratory Qualitative Study of UK General Practitioners’ Views. J Med Internet Res. 2019;21(3):e12802.
4. Adamson AS, Smith A. Machine learning and health care disparities in dermatology. JAMA Dermatol. 2018;154(11):1247–8.
5. Obermeyer Z, Powers B, Vogeli C, Mullainathan S. Dissecting racial bias in an algorithm used to manage the health of populations. Science. 2019;366(6464):447–53.
6. Mehrabi N, Morstatter F, Saxena N, Lerman K, Galstyan A. A survey on bias and fairness in machine learning. ACM Comput Surv. 2021;54(6):1–35.
7. Goddard K, Roudsari A, Wyatt JC. Automation bias: a systematic review of frequency, effect mediators, and mitigators. J Am Med Inform Assoc. 2012;19(1):121–7.
8. Rudin C. Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead. Nat Mach Intell. 2019;1(5):206–15.
9. Price WN, Cohen IG. Privacy in the age of medical big data. Nat Med. 2019;25(1):37–43.
10. Gerke S, Minssen T, Cohen G. Ethical and legal challenges of artificial intelligence-driven healthcare. In: Artificial Intelligence in Healthcare. Academic Press; 2020. p. 295–336.
11. Vyas DA, Eisenstein LG, Jones DS. Hidden in plain sight—reconsidering the use of race correction in clinical algorithms. N Engl J Med. 2020;383(9):874–82.
12. Wahl B, Cossy-Gantner A, Germann S, Schwalbe NR. Artificial intelligence (AI) and global health: how can AI contribute to health in resource-poor settings? BMJ Glob Health. 2018;3(4):e000798.
13. Chen IY, Szolovits P, Ghassemi M. Can AI help reduce disparities in general medical and mental health care? AMA J Ethics. 2019;21(2):167–79.
14. Doshi-Velez F, Kim B. Towards a rigorous science of interpretable machine learning. arXiv preprint. 2017; arXiv:1702.08608.
15. Floridi L, Cowls J, Beltrametti M, et al. AI4People—An ethical framework for a good AI society: Opportunities, risks, principles, and recommendations. Minds Mach. 2018;28(4):689–707.
16. U.S. Food and Drug Administration (FDA). Artificial Intelligence and Machine Learning in Software as a Medical Device. [Internet]. 2021 [cited 2025 May 21]. Available from: https://www.fda.gov/
17. Mesko B, Győrffy Z. The rise of the empowered physician in the digital health era: viewpoint. J Med Internet Res. 2019;21(3):e12490.