Clinical AI Requires Living Oversight
Legal and Ethical Grounds for a New Accountability Framework
DOI:
https://doi.org/10.34625/issn.2183-2705(39.2)2026.ic-5Keywords:
Accountability, Living oversight, Stewardship, Informed consent, Human dignity, Clinical IA, Redress mechanismsAbstract
This article contends that the European frameworks governing clinical AI fail to sustain accountability once systems are in use, and that certification and documentation are not apt to capture distributed, evolving decision-making. Drawing on comparative doctrine, the paper diagnoses the gap between formal compliance and lived responsibility. It advances living oversight as a governance practice that is continuous and substantively human, reframes stewardship as a shared responsibility across developers, clinicians, institutions, and regulators, and sets out practical implications: dynamic consent, post-deployment auditing, and enforceable redress. Accountability has to be designed into operation – not verified after the event – if clinical AI is to remain compatible with the moral foundations of healthcare.
References
Normative Instruments
CHARTER of Fundamental Rights of the European Union [online]. Official Journal of the European Union, C 326, 26.10.2012. Available from: https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=celex%3A12012P%2FTXT [viewed 4 February 2026].
COUNCIL OF EUROPE. Convention for the Protection of Human Rights and Dignity of the Human Being with Regard to the Application of Biology and Medicine (Oviedo Convention) [online]. Strasbourg: Council of Europe, 1997. Available from: https://rm.coe.int/168007cf98 [viewed 4 February 2026].
COUNCIL OF EUROPE. Committee of Ministers. Recommendation CM/Rec(2020)1 of the Committee of Ministers to member States on the human rights impacts of algorithmic systems [online]. Strasbourg: Council of Europe, 2020. Available from: https://rm.coe.int/09000016809e1154 [viewed 4 February 2026].
EUROPEAN PARLIAMENT AND COUNCIL OF THE EUROPEAN UNION. Directive (EU) 2022/2555 of the European Parliament and of the Council of 14 December 2022 on measures for a high common level of cybersecurity across the Union (NIS 2 Directive) [online]. Official Journal of the European Union, L 333, 27.12.2022. Available from: https://eur-lex.europa.eu/eli/dir/2022/2555/oj [viewed 4 February 2026].
EUROPEAN PARLIAMENT AND COUNCIL OF THE EUROPEAN UNION. Regulation (EU) 2017/745 of the European Parliament and of the Council of 5 April 2017 on medical devices [online]. Official Journal of the European Union, L 117, 5.5.2017. Available from: https://eur-lex.europa.eu/eli/reg/2017/745/oj [viewed 4 February 2026].
EUROPEAN PARLIAMENT AND COUNCIL OF THE EUROPEAN UNION. Regulation (EU) 2017/746 of the European Parliament and of the Council of 5 April 2017 on in vitro diagnostic medical devices [online]. Official Journal of the European Union, L 117, 5.5.2017. Available from: https://eur-lex.europa.eu/eli/reg/2017/746/oj [viewed 4 February 2026].
EUROPEAN PARLIAMENT AND COUNCIL OF THE EUROPEAN UNION. Regulation (EU) 2024/1689 of the European Parliament and of the Council of 13 June 2024 laying down harmonised rules on artificial intelligence (Artificial Intelligence Act) [online]. Official Journal of the European Union, OJ L, 2024/1689, 12.7.2024. Available from: https://eur-lex.europa.eu/eli/reg/2024/1689/oj [viewed 4 February 2026].
EUROPEAN PARLIAMENT AND COUNCIL OF THE EUROPEAN UNION. Regulation (EU) 2025/327 of the European Parliament and of the Council of 11 February 2025 on the European Health Data Space and amending Directive 2011/24/EU and Regulation (EU) 2024/2847 [online]. Official Journal of the European Union, OJ L, 2025/327, 5.3.2025. Available from: https://eur-lex.europa.eu/eli/reg/2025/327/oj/eng [viewed 4 February 2026].
ORGANISATION FOR ECONOMIC CO-OPERATION AND DEVELOPMENT. Recommendation of the Council on Artificial Intelligence [online]. Paris: OECD, 2019 (amended 2024). OECD/LEGAL/0449. Available from: https://legalinstruments.oecd.org/en/instruments/OECD-LEGAL-0449 [viewed 4 February 2026].
UNITED NATIONS. Universal Declaration of Human Rights [online]. New York: United Nations, 1948. Available from: https://www.un.org/en/about-us/universal-declaration-of-human-rights [viewed 4 February 2026].
UNITED NATIONS GENERAL ASSEMBLY. Resolution A/RES/78/265: Seizing the opportunities of safe, secure and trustworthy artificial intelligence systems for sustainable development [online]. New York: United Nations, 2024. Available from: https://docs.un.org/en/A/res/78/265 [viewed 4 February 2026].
UNITED NATIONS HUMAN RIGHTS COUNCIL. Resolution A/HRC/RES/47/23: New and emerging digital technologies and human rights [online]. Geneva: United Nations, 2021. Available from: https://undocs.org/A/HRC/RES/47/23 [viewed 4 February 2026].
UNITED NATIONS EDUCATIONAL, SCIENTIFIC AND CULTURAL ORGANIZATION. Universal Declaration on Bioethics and Human Rights [online]. Paris: UNESCO, 2005. Available from: https://unesdoc.unesco.org/ark:/48223/pf0000146180 [viewed 4 February 2026].
UNITED NATIONS EDUCATIONAL, SCIENTIFIC AND CULTURAL ORGANIZATION. Recommendation on the Ethics of Artificial Intelligence [online]. Paris: UNESCO, 2021. Available from: https://unesdoc.unesco.org/ark:/48223/pf0000381137 [viewed 4 February 2026].
UNITED STATES. Food and Drug Administration. Quality System Regulation [online]. Code of Federal Regulations, Title 21, Part 820. Washington, DC: US Government Publishing Office, 2024. Available from: https://www.ecfr.gov/current/title-21/chapter-I/subchapter-H/part-820 [viewed 4 February 2026].
WORLD HEALTH ORGANIZATION. WHO guidance on ethics and governance of artificial intelligence for health [online]. Geneva: WHO, 2021. Available from: https://www.who.int/publications/i/item/9789240029200 [viewed 4 February 2026].
WORLD HEALTH ORGANIZATION. Regulatory considerations on artificial intelligence for health [online]. Geneva: WHO, 2023. Available from: https://www.who.int/news/item/19-10-2023-who-outlines-considerations-for-regulation-of-artificial-intelligence-for-health [viewed 4 February 2026].
Jurisprudence
Daubert v. Merrell Dow Pharmaceuticals, Inc., 509 U.S. 579 (1993).
Google Spain SL and Google Inc. v Agencia Española de Protección de Datos (AEPD) and Mario Costeja González, C-131/12, ECLI:EU:C:2014:317, Court of Justice of the European Union [online]. 2014. Available from: https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX%3A62012CJ0131 [viewed 4 February 2026].
Bibliographic References
ALLEN, Matthew R.; WEBB, Sophie; MANDVI, Ammar; FRIEDEN, Marshall; TAI-SEALE, Ming; KALLENBERG, Gene. 'Navigating the doctor-patient-AI relationship – a mixed-methods study of physician attitudes toward artificial intelligence in primary care'. BMC Primary Care [online]. 2024, 25(1), 42. https://doi.org/10.1186/s12875-024-02282-y.
AMERICAN LAW INSTITUTE. Restatement (Third) of Torts: Liability for Physical and Emotional Harm. St Paul, MN: American Law Institute, 2010.
BLEHER, Hannah; BRAUN, Matthias. 'Diffused responsibility: attributions of responsibility in the use of AI-driven clinical decision support systems'. AI and Ethics [online]. 2022, 2(4), pp. 747–761. https://doi.org/10.1007/s43681-022-00135-x.
BOUDERHEM, Rabaï. 'Shaping the future of AI in healthcare through ethics and governance'. Humanities and Social Sciences Communications [online]. 2024, 11(1), 416. https://doi.org/10.1057/s41599-024-02894-w.
BURRELL, Jenna. 'How the machine "thinks": understanding opacity in machine learning algorithms'. Big Data & Society [online]. 2016, 3(1), 2053951715622512. https://doi.org/10.1177/2053951715622512.
CESTONARO, Clara; DELICATI, Arianna; MARCANTE, Beatrice; CAENAZZO, Luciana; TOZZO, Pamela. 'Defining medical liability when artificial intelligence is applied on diagnostic algorithms: a systematic review'. Frontiers in Medicine [online]. 2023, 10, 1305756. https://doi.org/10.3389/fmed.2023.1305756.
CORREIA, Mónica; RÊGO, Guilhermina; NUNES, Rui. 'Gender transition: is there a right to be forgotten?' Health Care Analysis [online]. 2021, 29(4), pp. 283–300. https://doi.org/10.1007/s10728-021-00433-1.
CORREIA, Mónica; RÊGO, Guilhermina; NUNES, Rui. 'The right to be forgotten and COVID-19: privacy versus public interest'. Acta Bioethica [online]. 2021, 27(1), pp. 59–67. Available from: https://actabioethica.uchile.cl/index.php/AB/article/view/63954 [viewed 4 February 2026].
CORREIA, Mónica; RÊGO, Guilhermina; NUNES, Rui. 'The right to be forgotten versus the right to disclosure of gamete donors' ID: ethical and legal considerations'. Acta Bioethica [online]. 2021, 27(1), pp. 69–78. Available from: https://actabioethica.uchile.cl/index.php/AB/article/view/63955 [viewed 4 February 2026].
CORREIA, Mónica; RÊGO, Guilhermina; NUNES, Rui. 'The right to be forgotten regarding genetic data: a legal and ethical analysis'. Acta Bioethica [online]. 2024, 30(2), pp. 231–243. Available from: https://actabioethica.uchile.cl/index.php/AB/article/view/76137 [viewed 4 February 2026].
DUFFOURC, Mindy; GERKE, Sara. 'Decoding U.S. tort liability in healthcare's black-box AI era: lessons from the European Union'. Stanford Technology Law Review [online]. 2024, 27(1), pp. 1–58. Available from: https://law.stanford.edu/wp-content/uploads/2024/02/Publish_27-STLR-1-2024_Decoding-U.S.-Tort-Liability-in-Healthcares-Black-Box-AI-Era.pdf [viewed 4 February 2026].
FEDERAL JUDICIAL CENTER; NATIONAL RESEARCH COUNCIL. Reference Manual on Scientific Evidence [online]. 3rd ed. Washington, DC: National Academies Press, 2011. Available from: https://nap.nationalacademies.org/catalog/13163/reference-manual-on-scientific-evidence-third-edition [viewed 4 February 2026].
FLORIDI, Luciano. The Fourth Revolution: How the Infosphere Is Reshaping Human Reality. Oxford: Oxford University Press, 2014.
FLORIDI, Luciano. The Ethics of Artificial Intelligence: Principles, Challenges, and Opportunities. Oxford: Oxford University Press, 2023.
GOZUM, Ivan Efreaim A.; FLAKE, Chastene Christopher D. 'Human dignity and artificial intelligence in healthcare: a basis for a Catholic ethics on AI'. Journal of Religion and Health [online]. 2024. Available from: https://doi.org/10.1007/s10943-024-02206-1 [viewed 4 February 2026].
GROTE, Thomas; BERENS, Philipp. 'On the ethics of algorithmic decision-making in healthcare'. Journal of Medical Ethics [online]. 2020, 46(3), pp. 205–211. https://doi.org/10.1136/medethics-2019-105586.
HERZOG, Christian; BRANFORD, Jason. ‘Relational ethics and structural epistemic injustice of AI in medicine’. Philosophy & Technology [online]. 2025, vol. 38, no. 4, 160 [viewed 4 February 2026]. Available from: https://doi.org/10.1007/s13347-025-00987-1
KEETON, W. Page; DOBBS, Dan B.; KEETON, Robert E.; OWEN, David G. Prosser and Keeton on the Law of Torts. 5th ed. St Paul, MN: West Publishing Co., 1984.
KISELEVA, Anastasiya; KOTZINOS, Dimitris; DE HERT, Paul. 'Transparency of AI in healthcare as a multilayered system of accountabilities: between legal requirements and technical limitations'. Frontiers in Artificial Intelligence [online]. 2022, 5, 879603. https://doi.org/10.3389/frai.2022.879603.
KOLFSCHOOTEN, Hannah B. van. 'A health-conformant reading of the GDPR's right not to be subject to automated decision-making'. Medical Law Review [online]. 2024, 32(3), pp. 373–391. https://doi.org/10.1093/medlaw/fwae029.
LAY, Winnie; GASPARINI, Loretta; SIERO, William; HUGHES, Elizabeth K. 'A rapid review of the benefits and challenges of dynamic consent'. Research Ethics [online]. 2025, 21(1), pp. 180–202. https://doi.org/10.1177/17470161241278064.
LE MOLI, Ginevra. 'Intelligence artificielle vs dignité humaine: quand la sous-performance humaine est légalement requise'. Revue européenne du droit [online]. 2022, 4(1), pp. 122–127. https://doi.org/10.3917/red.004.0122.
LEE, Ah Ra; KOO, Dongjun; KIM, Il Kon; LEE, Eunjoo; YOO, Sooyoung; LEE, Ho-Young. 'Opportunities and challenges of a dynamic consent-based application: personalised options for personal health data sharing and utilisation'. BMC Medical Ethics [online]. 2024, 25(1), 92. https://doi.org/10.1186/s12910-024-01091-3.
LORENZINI, Giorgia; ARBELAEZ OSSA, Laura; SHAW, David Martin; ELGER, Bernice Simone. 'Artificial intelligence and the doctor–patient relationship expanding the paradigm of shared decision making'. Bioethics [online]. 2023, 37(5), pp. 424–429. https://doi.org/10.1111/bioe.13158.
MALIHA, George; GERKE, Sara; COHEN, I. Glenn; PARIKH, Ravi B. 'Artificial intelligence and liability in medicine: balancing safety and innovation'. Milbank Quarterly [online]. 2021, 99(3), pp. 629–647. https://doi.org/10.1111/1468-0009.12504.
MEDTECH EUROPE. Liability challenges in AI medical technologies: key considerations for policymakers and industry [online]. MedTech Europe White Paper. Brussels: MedTech Europe, 2022. Available from: https://www.medtecheurope.org/resource-library/liability-challenges-in-ai-medical-technologies/ [viewed 4 February 2026].
MOLBÆK-STEENSIG, Helga; SCHEININ, Martin. 'Human rights and artificial intelligence in healthcare-related settings: a grammar of human rights approach'. European Journal of Health Law [online]. 2025, 32(2), pp. 139–164. https://doi.org/10.1163/15718093-bja10146.
O'FLAHERTY, Michael. Human rights oversight of artificial intelligence [online]. Strasbourg: Council of Europe, Commissioner for Human Rights, 2025. Available from: https://www.coe.int/en/web/commissioner/-/human-rights-oversight-of-artificial-intelligence [viewed 4 February 2026].
ORWAT, Carsten. 'Algorithmic discrimination from the perspective of human dignity'. Social Inclusion [online]. 2024, 12, 7160. https://doi.org/10.17645/si.7160.
PALANIAPPAN, Kavitha; LIN, Elaine Yan Ting; VOGEL, Silke; LIM, John C. W. 'Gaps in the global regulatory frameworks for the use of artificial intelligence (AI) in the healthcare services sector and key recommendations'. Healthcare [online]. 2024, 12(17), 1730. https://doi.org/10.3390/healthcare12171730.
PIERCE, Robin L.; VAN BIESEN, Wim; VAN CAUWENBERGE, Daan; DECRUYENAERE, Johan; STERCKX, Sigrid. 'Explainability in medicine in an era of AI-based clinical decision support systems'. Frontiers in Genetics [online]. 2022, 13, 903600. https://doi.org/10.3389/fgene.2022.903600.
PRICE, W. Nicholson II. 'Medical malpractice and black-box medicine'. In: COHEN, I. Glenn; FERNANDEZ LYNCH, Holly; VAYENA, Effy; GASSER, Urs (eds.). Big Data, Health Law, and Bioethics. Cambridge: Cambridge University Press, 2018. https://doi.org/10.1017/9781108147972.027.
RUSTAMZADE, Aykhan; BAGIROV, Anar Ramiz; HUSEYNOV, Sahil Zahir. 'Justice and artificial intelligence'. Revista Jurídica Portucalense [online]. 2024, 36, pp. 163–184. https://doi.org/10.34625/ISSN.2183-2705(36)2024.IC-7.
SANTOS, Elisabete. 'Workers under the aegis of artificial intelligence: new risks and challenges in the light of data protection'. Revista Jurídica Portucalense [online]. 2025, 38, pp. 427–449. https://doi.org/10.34625/ISSN.2183-2705(38)2025.IC-22.
SCHMIDT, Jelena; SCHUTTE, Nienke M.; BUTTIGIEG, Stefan; et al. 'Mapping the regulatory landscape for artificial intelligence in health within the European Union'. npj Digital Medicine [online]. 2024, 7(1), 229. https://doi.org/10.1038/s41746-024-01221-6.
SUBRAMANIAN, Harishankar V.; CANFIELD, Casey; SHANK, Daniel B. 'Designing explainable AI to improve human-AI team performance: a medical stakeholder-driven scoping review'. Artificial Intelligence in Medicine [online]. 2024, 149, 102780. https://doi.org/10.1016/j.artmed.2024.102780.
TANG, Lu; LI, Jinxu; FANTUS, Sophia. 'Medical artificial intelligence ethics: a systematic review of empirical studies'. Digital Health [online]. 2023, 9, 20552076231186064. https://doi.org/10.1177/20552076231186064.
THE PRECISE4Q CONSORTIUM; AMANN, Julia; BLASIMME, Alessandro; VAYENA, Effy; FREY, Dietmar; MADAI, Vince I. 'Explainability for artificial intelligence in healthcare: a multidisciplinary perspective'. BMC Medical Informatics and Decision Making [online]. 2020, 20(1), 310. https://doi.org/10.1186/s12911-020-01332-6.
TONEKABONI, Sana; JOSHI, Shalmali; McCRADDEN, Melissa D.; GOLDENBERG, Anna. 'What clinicians want: contextualising explainable machine learning for clinical end use' [online]. Preprint, arXiv, 2019. https://doi.org/10.48550/ARXIV.1905.05134.
VILAS BOAS PINTO, João. '"I don't know where I'm going, I know I'm (not) going that way": some notes about the opportunities, challenges, and limits of artificial intelligence in the scope of administrative law'. Revista Jurídica Portucalense [online]. 2025, 38, pp. 286–307. https://doi.org/10.34625/ISSN.2183-2705(38)2025.IC-15.
VILLEGAS-GALAVIZ, Carolina; MARTIN, Kirsten. 'Moral distance, AI, and the ethics of care'. AI & SOCIETY [online]. 2024, 39(4), pp. 1695–1706. https://doi.org/10.1007/s00146-023-01642-z.
YEUNG, Karen; LODGE, Martin (eds.). Algorithmic Regulation. Oxford: Oxford University Press, 2019.
ZHANG, Jie; ZHANG, Zong-ming. 'Ethics and governance of trustworthy medical artificial intelligence'. BMC Medical Informatics and Decision Making [online]. 2023, 23(1), 7. https://doi.org/10.1186/s12911-023-02103-9.
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2026 Eva Dias COSTA, Mónica CORREIA, Rui NUNES
This work is licensed under a Creative Commons Attribution 4.0 International License.
Authors who published in the journal agree to the following terms:
- The Authors grant the Journal the right of first publication, and other non-exclusive publishing rights, licensed under the Creative Commons Attribution License which allows the sharing of work with recognition of its initial publication in this journal.
- Authors are able to take on additional contracts separately, non-exclusive distribution of the version of the paper published in this journal (ex .: publish in an institutional repository or as a chapter in a book), with an acknowledgement of its initial publication in this journal.
- Authors are permitted and encouraged to post and distribute their work online (eg .: in institutional repositories or on their website) at any point before or during the submission process, as it can lead to productive exchanges, as well as increase the impact and the citation of published work (See The Effect of Open Access).
RJP does not apply submission, publication or any other fees of any nature. Its articles are open access, with the goal of disseminating scientific knowledge and the debate of legal topics in the area of Legal Sciences.
