On the issue of ethical aspects of the artificial intelligence systems implementation in healthcare

Full Text

INTRODUCTION

According to GOST R 59277-2020, artificial intelligence (AI) is defined as a set of technological solutions that simulates human cognitive functions, including self-learning, identifying solutions without a predetermined algorithm, and achieving insight, thereby obtaining results that are comparable at least to human intellectual activity. Thus, systems based on AI technologies have certain characteristics, such as the ability to “evolve” and apply “new knowledge.” This imposes on society the responsibility to develop new ethical standards for the application of such systems, such as those that help physicians diagnose diseases and make decisions to provide substantial help in medicine. AI systems must be effective (due to their variety of options), safe, and assured by regulatory control, including ethical standards (development of a code of ethics and national and international ethical standards for using AI technologies in medicine) and technical regulation that are subjected to reforms and updates. Despite their similarity to a black box, AI systems should not cause distrust as their use the benefits are much greater, such as handling staff shortages and obtaining independent second opinion due to the emerging well-established norms and rules.

The development of AI systems brings forth ethical issues regarding the impact on decision-making processes and interactions in medicine and the overall healthcare system [1]. Even with the use of AI systems, human rights should be respected, which requires systematic ethical discussions; certain ethical standards must be established for their global application [2].

The ethical part is a characteristic of all stages of the AI lifecycle, a set of processes from the development (scientific and clinical research, design, and creation) and operation (market launch, financing, maintenance, monitoring, and performance evaluation) to the withdrawal from an operation. Bearing in mind the ethical issues during the interaction of objects with AI systems is of particular relevance. In this case, objects involved with AI systems are defined as any party that is involved in any of the stages of the AI lifecycle, such as legal and physical persons, including researchers, scientists, physicians, data processing specialists, engineers, information technology (IT) specialists, commercial and public companies, universities, and research centers.

Particularly, the ethical issues of AI application in global and national healthcare systems are worth mentioning [3, 4]. AI systems that involve medical products are increasing. Therefore, minimizing the potential for errors associated with medical and personal patient data that are used to develop and test AI technologies is necessary. To perform these tasks, access to sufficient verified, high-quality medical data of patients is required, which may be used for disease prediction, diagnosis, treatment optimization, as well as external and internal testing algorithms [5–7]. Thus, a set of solutions is needed, including state regulation, involvement of highly qualified experts (engineers, physicians, and IT specialists); raising patient awareness to solve the ethical problems associated with the development and implementation of medical AI systems is also warranted.

Therefore, this study aimed to present the currently available documents that regulate the ethical aspects and norms of the global AI application, particularly in the healthcare industry. Moreover, the normative legal and technical regulatory documents for AI systems worldwide were studied, along with the presentation of the Russian experience of the first national standards that will control the development and application of AI technologies in healthcare.

INTERNATIONAL AND NATIONAL EXPERIENCE TO SOLVE THE ETHICAL ISSUES OF AI APPLICATION IN HEALTHCARE: GENERAL PROVISIONS

Data privacy concerns are raised with the widespread use of patient data for AI training. Improving healthcare by maintaining a balance between the secondary use of data from other patients and the privacy of personal information presents several challenges. For example, the challenge at the individual level is to understand the extent to which personal data may be used at the secondary level and what elements are involved, who can access these data, how effective and complete their anonymity is, the potential use of data to harm the patients, the financial use of data, and the effect of the data privacy policy on the care received [6]. In addition, the issue of “liability awareness” for the ownership of personal data may arise at the level of medical institutions that share medical data, and such sharing may be problematic.

However, in case of a misdiagnosis made by a physician when using an AI system, the extent to which the involved party (physician, institution, healthcare system, or AI manufacturer) is ethically responsible is unclear. According to the existing normative legal acts (Federal Law of November 21, 2011, No. 323-FZ, Art. 98 [as amended on July 2, 2021] on Fundamentals of Healthcare in the Russian Federation), the responsibility of violations of various legal norms in healthcare lies on the medical organization and the physician.

The responsibility for medical errors becomes more ambiguous at the state level. Thus, AI systems may not meet all the established requirements, and risks are present when using medical devices, including AI systems, for diagnoses. The United State Food and Drug Administration states that these medical devices may be risky due to the following five reasons: increased false positives leading to unnecessary additional procedures, increased false negatives, use in unsuitable settings, misuse by people, and malfunction due to improper product (AI system) introduction in the market [8]. Thus, a transparent and well-established state regulatory system is needed to minimize the risks, thereby partially removing or controlling the ethical issues associated with using AI systems at the state level.

By definition, a regulatory system is a set of rules. For products manufactured by an industry, such as drugs, vaccines, medical devices, and AI systems, the role of regulations is to limit the risks associated with the product, such as an unsafe product, the product not meeting the purpose for which the product was intended for (inefficient product), or the risk of not meeting up with quality standards based on the global report of the World Health Organization (WHO) on Ethics and Governance of Artificial Intelligence for Health [9]. The national legislation is responsible for developing such rules and creating the conditions to ensure compliance with these laws. Among those who are required to follow these rules are manufacturers and users of AI systems (medical organizations, physicians, and other medical staff). For example, the AI manufacturer is recommended to use the accepted classification of pathologies according to approved thesauruses and the International Classification of Diseases or name the phenomena as recommended by professional associations.

GENERAL APPROACHES TO SOLVE ETHICAL ISSUES

WHO experts have formulated principles to address ethical issues, which apply to AI systems in healthcare [9].

1. Protecting autonomy

With AI development, physicians began to raise concerns about being replaced by these technologies. However, the proposed principle of protecting autonomy implies that AI systems should be designed to help in decision-making and be fully controllable when operated by medical staff. In addition, this principle requires data protection and privacy on the part of AI manufacturers, dataset creators, and national governments, which should be enshrined in regulations. In Russia, an experimental regime of personal data processing was introduced in Moscow (Federal Law No. 152-FZ of July 27, 2006, on Personal Data), which allowed changes in the informed consent of patients. Moreover, changes were initiated in the regulatory legal documents related to medical devices in 2020. These amendments outlined the role of the AI system as an aid to physician decision-making, thereby removing the issue of using AI without physician interventions.

2. Ensuring patient safety and well-being

AI systems should be designed not to cause physical or mental harm to the patient and must meet the approved requirements for safety, efficiency, and quality throughout the life cycle. Therefore, quality control and monitoring mechanisms will be put in place to ensure that AI systems function as intended. The recommendations of the International Medical Device Regulators Forum to perform a clinical evaluation of software were approved and are being adopted in Russia [10]. In addition, standards for protocols and reports on clinical trial results are being developed. For example, randomized controlled studies are considered the gold standard of experimental design to provide evidence on the safety and efficiency of AI systems. In this regard, the extension of Consolidated Standards of Reporting Trials—AI (CONSORT-AI) to CONSORT 2010 and Standard Protocol Items: Recommendations for Interventional Trials—AI (SPIRIT-AI) was developed [11].

3. Ensuring transparency, explainability, and intelligibility of AI systems

The intelligibility of AI systems includes ensured transparency and explainability. AI manufacturers must demonstrate transparency by publishing and presenting data to the professional community on how the AI model was developed, its application scenario, the parameters of data sets that were used to develop and validate the AI system, and clinical studies that are conducted to demonstrate the efficiency of this system [10].

AI algorithms and the principle of data processing and decision-making should be explained as much as possible to inform the individuals and patients.

In addition, the specified principle of AI intelligibility is related to the principle of safety as it requires disclosure of information about the performed tests and their completeness and quality.

4. Enhancing responsibility and accountability

Engineers and manufacturers of AI systems along with medical experts must develop and visualize scenarios for applying this technology, including the efficiency and performance results obtained. In this case, concerned parties are responsible to use the AI system for its intended purpose and within the stated conditions, and all its incorrect operations must be documented and further monitored.

5. Ensuring equity

AI technologies should be available to all segments of society, irrespective of age, gender, race, financial status, and social status. AI manufacturers must ensure and provide information that the data sets for AI training and testing are free from systematic sampling errors (biases) and are therefore accurate, complete, and diverse, which give equal parameters for the AI efficiency within the entire population.

6. Promoting AI that is responsive and sustainable

Responsiveness requires the manufacturer and user of the AI system to continuously collect and analyze information on its use to monitor the compliance of its performance with the stated requirements. The parameters of resources and external systems for the AI functioning, as well as its integration (such as medical information systems), should be ensured. Responsiveness is relevant in assessing the place of a particular AI system in healthcare, the opinions of physicians, and the execution and efficiency of anticipated scenarios for AI applications.

The principles presented for solving ethical problems are general and should be adapted to national contexts. Many world powers presented their national strategies with consolidation of basic principles of AI development, which includes those previously mentioned.

INTERNATIONAL STRATEGIES FOR AI DEVELOPMENT IN HEALTHCARE

Main documents from the world-leading powers were reviewed to study the international strategies for AI use and development in healthcare.

Most world powers, such as the United States, China, Japan, South Korea, the European countries, and Russia, recently developed and published national strategies or policy documents defining the goals, objectives, and plans for AI development, all of which emphasized the ethical component in healthcare (Fig. 1) [12, 13].

 

Fig. 1. Timeline to develop and approve artificial intelligence strategies worldwide

 

Japan, being one of the first countries to develop a national AI strategy and a report on the Artificial Intelligence Technology Strategy in 2017, identified healthcare as a major industry for further development [14]. In the same year, Singapore launched an AI Singapore National Program, which focused on the national health system [15]. In addition, South Korea published its AI Development National Strategy in 2019, which also prioritized the healthcare system [16].

The European Commission adopted the AI for Europe strategy in 2018, which presented an initiative aimed at ensuring appropriate ethical and legal approaches by creating a European AI Alliance and developing guidelines for AI ethics [17].

Moreover, the United States published the National AI R&D Strategic Plan (2019 Update), which identified eight strategic priorities, namely, long-term investments, effective methods of human-AI interaction, regulation of legal and ethical AI implications, AI safety and security, publicly available data sets, AI standardization and testing, advancing the AI R&D workforce, and public-private partnerships [12]. In the same year, the report on Current State and Near-Term Priorities for AI-Enabled Diagnostic Support Software in Health Care was published. Therein, experts substantiated the need for changes in government policy and regulation to ensure safer and more effective AI applications in healthcare [18]. This paper provides an overview of the current regulatory framework that governs the development and use of clinical decision support systems, as well as diagnostic support. The experts concluded that regulatory changes for AI systems in healthcare based on the following tenets are needed:

First, providing evidence that AI systems improve patient outcomes, increase the quality of life, and reduce the cost of care, as well as provide physicians with relevant information that is “useful and trustworthy.”

Second, assessing the potential risks of using AI products in clinical settings. “The degree to which a software product comes with information that explains how it works and the types of populations used to train the software will have significant impact on regulators’ and clinicians’ assessment of the risk to patients when clinicians use this software,” the researchers note. “Product labeling may need to be reconsidered and the risks and benefits of continuous learning versus locked models must be discussed.”

A White Paper on Ethical and Legal Issues Related to Artificial Intelligence in Radiology was published in Canada in 2019 [6]. This official document of the Canadian Association of Radiologists provides a framework for exploring legal and ethical AI issues in medical imaging. Particular attention was paid to patient data privacy, AI systems (levels of autonomy, liability, and legal aspects), Canadian healthcare practices (best practices and current legal framework), and the projected opportunities that AI may provide for healthcare.

Ethics Guidelines for Trustworthy AI published by a group of experts of the European Commission in 2019 contain a set of key requirements, which elevates AI systems to the level of trustworthy technologies, such as human agency and oversight, technical robustness and safety, privacy and data governance, transparency, diversity, non-discrimination and fairness, societal and environmental well-being, and accountability [19].

In 2020, the European Parliament postulated the creation of a new legal framework that outlines ethical principles and legal obligations for the development of AI and robotics, and their deployment and use were all described in a White Paper on Artificial Intelligence Regulation [3].

The Council of Europe (the Ad hoc Committee on Artificial Intelligence, CAHAI), including the representatives of Russia, actively developed approaches to European regulation [20].

Moreover, UNESCO developed draft guidelines on AI ethics to be adopted by its member countries at the end of 2021 [21]. Thus, Europe is becoming a global player in AI ethics and actively developing documents in this sphere, including those for the intended use of AI systems in healthcare [20].

Notably, the United States has a freer market approach to AI compared with Europe, and the United States market for medical devices using AI-based software is one of the largest around the world. AI is predicted to contribute up to 13.33 trillion euros to the global economy in 2030, and the regions estimated to benefit most are China and North America, followed by Southern Europe [22].

China’s strategy for AI development correlates with that of the scientific and technological development of the Russian Federation that focused on the transition to “digital and intelligent production technologies, robotic systems, new materials, and methods of construction, as well as creating systems for processing large data volumes, machine learning, and artificial intelligence” [23]. The State Council of China published a New Generation Artificial Intelligence Development Plan in 2017, which outlined step-by-step goals for AI development; subsequently, China AI Industry White Paper was developed in 2018 [24]. Both documents listed healthcare and ethics among their priorities.

Separate studies raise the issue of ethical development and implementation of AI systems in the medical community. For example, a joint statement by the European and North American communities addressed basic ethical principles similar to those proposed by the WHO, whereas the radiology community stated its intention to develop a unified code of ethics to create a sound ethical foundation with rapid technological advancements [25, 26].

In Russia, the current priority is AI development and implementation in healthcare, which could help in achieving strategic goals and objectives of the Healthcare National Project, including decreased morbidity and mortality and increased life expectancy. This strategy will be discussed in more detail in the following section.

NATIONAL STRATEGY FOR THE AI DEVELOPMENT IN HEALTHCARE

National processes of AI development in Russia, including AI in healthcare, are built on the principles of openness with the involvement of researchers, AI manufacturers, and government agencies. The work on the Code of AI ethics was initiated in 2020 by the order of the President of the Russian Federation [27].

Russia attaches great importance to international cooperation in AI development [28]. Concurrently, proper management of AI potential and the creation of necessary foundations and standardized approaches for the safe and ethical development of AI systems, which should include creative force and contribute to the development of human potential and abilities, is important.

Russia is an active participant in AI development and follows all global trends. Thus, the Russian government developed and adopted a comprehensive policy that allowed the national strategy to be further implemented [29]. In October 2019, the Russian President signed a Decree on the Development of Artificial Intelligence in the Russian Federation, which introduced the National Strategy for the Development of Artificial Intelligence until 2030 and approved the plans for initiating the AI Federal Project (introduced in 2020 as part of the national project for Digital Economy of the Russian Federation) [30]. The Federal project includes six main blocks, namely, support for scientific research, creation of a comprehensive legal regulation system, software design and development, improvement in data accessibility and quality, increased hardware accessibility, staff training, and increased public awareness on AI technologies.

The uniqueness of the Russian national strategy, which differs from the strategies of other countries, is their priority in the development of AI in healthcare, including the need to move from general principles to a more practical level. One of such routes is the creation of normative legal and technical regulations (standardization). In Russia, software, created using AI technologies and intended by the manufacturer to provide medical care refers to medical devices (software as a medical device).

The Technical Artificial Intelligence Committee for Standardization (TC164) was created in July 2019 to develop technical regulations and improve the AI standardization efficiency [31]. In addition, Artificial Intelligence in Healthcare PC01 Subcommittee was created within the TC164 Committee to elaborate the national and international standards that cover the requirements for the development, testing, application, and operation of medical AI software and coordinate the work on unification and standardization of requirements used in the development, testing, and operation of AI systems in healthcare, as well as set certification requirements for medical products based on AI technologies [32].

Russian experts recently developed and prepared a corresponding standard for approval to regulate the process of AI clinical trials (as part of clinical assessment for independent testing of AI systems), where their safety and efficiency are evaluated, and AI systems are registered as medical devices. Ethical review is recommended before conducting clinical trials of unregistered medical devices (AI systems). This review includes issues related to the ethical components and analysis of possible adverse events arising from the AI application. Moreover, the Ethical Committee should review the clinical trial program (in the trial design approval and formation of data sets monitoring), and evaluate the compliance of the qualifications of investigators with the proposed trial.

Remarkably, the challenge of recent years, namely, the coronavirus disease-2019 (COVID-19) pandemic, has increased the understanding of the ethical AI application aspects in healthcare, including in the diagnosis of this disease using chest computed tomography [33]. The paper shows that AI technologies may adjust the patient flow by tracking radiological sign categories of pulmonary changes in COVID-19, which influences the tactics for managing such patients, because AI systems automatically process patient images and, accordingly, assess the volume of changes in the lungs more accurately; physicians perform such an assessment visually. Another ethical aspect of AI application during the COVID-19 pandemic was the issues of privacy, data protection, and their use for virus tracking [21].

CONCLUSIONS

Currently, standards for AI regulation in healthcare are actively developed at the national and international levels. However, these regulatory agencies did not form a full-fledged legal framework and control systems and have insufficient experience in using AI technologies. Therefore, ethical aspects of this process remain unresolved or controversial.

Russia actively develops legal and technical AI regulation, as well as issues of assessment, measurement, and standardization of AI technologies to improve and finalize normative and legal documents. The most active work is performed in the national industry standardization of healthcare. The unification of best practices will structure the development, evaluation, regulation, and security control in the application and development of AI technologies, thereby increasing public confidence.

A system based on ethical performance must be created to achieve the desired public goal for AI systems to benefit the society. Therefore, the existing legal framework must be reviewed and updated for new technological advancements. Regulations and industry standards in clinical medicine require that AI systems be developed and evaluated according to ethical standards, especially those intended for physicians and other members of the medical community. AI systems must be “transparent” in understanding by healthcare providers and protect the interests of physicians and patients, especially in the area of ethical regulation. Only by this can a trustworthy and ethically safe relationship be built between the physician, patient, and AI manufacturer.

The international community and private companies should develop ethical monitoring and verification mechanisms to identify, prevent, and minimize the impact of AI systems in human rights, good scientific practice, legality, and protection, and evaluate the efficiency of such mechanisms. States are encouraged to consider forms of governance, such as AI certification mechanisms and mutual recognition of standardization while considering the healthcare industry and the use of AI systems and their potential impact on human lives, as well as other ethical aspects.

The AI safety and security may be improved by developing reliable systems against unauthorized access to personal information complex solutions that will provide more effective training of AI models based on quality data.

Importantly, public and political discussions should focus on the ethics of healthcare. AI has enormous potential to improve healthcare. However, its realization will be possible if ethical problems facing society are addressed as soon as possible.

Whether the use of AI systems is ethically acceptable depends on the extent to which AI technologies contribute positively to individual patients and the entire society, including medical organizations and physicians (users). All levels of interaction should be guided by a relative level of equity between the principles of fairness and expected positive outcomes.

ADDITIONAL INFORMATION

Funding source. This article was prepared with the support of the Moscow Healthcare Department as a part of Program “Scientific Support of the Capital’s Healthcare” for 2020–2022 (Unified State Information System for Accounting of Research, Development, and Technological Works No: АААА-А21-121012290079-2).

Competing interests. The authors declare the absence of obvious and potential conflicts of interest related to the publication of this article.

Authors’ contribution. Concept of the article — D. Sharova, V. Zinchenko, E. Akhmad; collection and processing of the materials, writing the paper — D. Sharova, V. Zinchenko, E. Akhmad; compilation of the reference list — D. Sharova, E. Akhmad; editing — D. Sharova, V. Zinchenko, E. Akhmad, O. Mokienko, A. Vladzymyrskyy, S. Morozov. The final version of the article was approved by all co-authors, who equally share a responsibility for the integrity of all parts of the article. All authors confirm their authorship in accordance with the international ICMJE criteria. All authors made a substantial contribution to the concept development, research and article preparation, read and approved the final version before the publication.

About the authors

Daria E. Sharova

Moscow Center for Diagnostics and Telemedicine

Email: d.sharova@npcmr.ru
ORCID iD: 0000-0001-5792-3912
SPIN-code: 1811-7595
Russian Federation, 28-1 Srednyaya Kalitnikovskaya str.,109029 Moscow

Viktoria V. Zinchenko

Moscow Center for Diagnostics and Telemedicine

Email: v.Zinchenko@npcmr.ru
ORCID iD: 0000-0002-2307-725X
SPIN-code: 4188-0635
Russian Federation, 28-1 Srednyaya Kalitnikovskaya str.,109029 Moscow

Ekaterina S. Akhmad

Moscow Center for Diagnostics and Telemedicine

Email: e.Akhmad@npcmr.ru
ORCID iD: 0000-0002-8235-9361
SPIN-code: 5891-4384
Russian Federation, 28-1 Srednyaya Kalitnikovskaya str.,109029 Moscow

Olesia A. Mokienko

Moscow Center for Diagnostics and Telemedicine

Email: o.Mokienko@npcmr.ru
ORCID iD: 0000-0002-7826-5135
SPIN-code: 8088-9921

MD, Dr. Sci. (Med.)

Russian Federation, 28-1 Srednyaya Kalitnikovskaya str.,109029 Moscow

Anton V. Vladzymyrskyy

Moscow Center for Diagnostics and Telemedicine

Email: a.Vladzymyrskyy@npcmr.ru
ORCID iD: 0000-0002-2990-7736
SPIN-code: 3602-7120

MD, Dr. Sci. (Med.)

Russian Federation, 28-1 Srednyaya Kalitnikovskaya str.,109029 Moscow

Sergey P. Morozov

Moscow Center for Diagnostics and Telemedicine

Author for correspondence.
Email: Morozov@npcmr.ru
ORCID iD: 0000-0001-6545-6170
SPIN-code: 8542-1720

MD, Dr. Sci. (Med.), Professor

Russian Federation, 28-1 Srednyaya Kalitnikovskaya str.,109029 Moscow

References

Supplementary files