Exploring the possibilities of an artificial intelligence program in the diagnosis of macular diseases

Full Text

BACKGROUND

According to the World Health Organization, approximately 2.2 billion people in the world suffer from visual impairment, and the largest group include patients aged >50 years [1]. Macular diseases, particularly age-related macular degeneration (AMD) and diabetic macular edema (DME), are significant causes of irreversible blindness and low vision quality [2].

As of 2020, 196 million patients have AMD worldwide, and this figure is expected to reach 288 million by 2040 [3]. Clinically, AMD may present with drusenoid deposits, retinal pigment epithelial changes, macular neovascularization, exudation, and hemorrhage. At advanced stages, geographic atrophy of the retinal pigment epithelium occurs [4]. These manifestations lead to severe visual impairment.

The number of patients with diabetes mellitus is increasing steadily, including its ocular manifestations. By 2045, the number of cases of diabetic retinopathy (DR) is predicted to reach 160.50 million, and the number of patients with sight-threatening DR will reach 44.82 million [5]. DME, which occurs at various stages of DR, is the most common cause of vision loss in patients with diabetes, and nearly 75,000 patients are newly diagnosed in the United States each year [6]. A direct correlation was found between the prevalence of DME and DR severity, and 70% of the cases are at the proliferative stage. Among patients with type 1 diabetes mellitus, DME develops in 27% within 9 years of disease onset [7]. In Russia, >630,000 patients have various stages of DR, and its prevalence in patients with type 1 diabetes mellitus is twice as high [8].

Central serous chorioretinopathy is the fourth macular disease that causes decreased visual acuity, characterized by serous detachment of the neuroepithelium above the area of leakage from the choriocapillaris. Central serous chorioretinopathy affects 9.9 individuals per 100,000 of population. The process is often unilateral; however, bilateral involvement was observed in 40% of cases [9]. The process has a chronic nature in 5% of cases, and central serous chorioretinopathy relapses within 12 months in 30%–50% of the patients [10].

Abnormalities of the vitreomacular interface include full-thickness and lamellar macular holes, vitreomacular traction, and epiretinal membrane. In some cases, vitreomacular interface changes may not cause significant functional impairment but have a significant effect on the macular area; the symptoms are accompanied by significant vision loss and negatively affect the patient’s quality of life. Macular holes affect 3.3 individuals per 1,000 patients aged >55 years [11]. Annual incidence ranges from 4.71 to 8.5 individuals per 100,000 populations [12].

During the 20-year follow up in the Beaver Dam Eye Study (BDES), optical coherence tomography (OCT) detected an epiretinal membrane in 34.1%. Despite the different pathogenesis mechanisms, the clinical manifestations may be identical. An epiretinal membrane may develop as an idiopathic disease or as a concomitant pathology in previous eye diseases, such as DR, retinal vein occlusion, or previous cataract surgery [13]. Epiretinal membrane without posterior vitreous detachment may be a prerequisite for macular edema and lamellar macular holes caused by tangential traction syndrome [14].

At present, with the increasing life expectancy of the population, early detection of age-related diseases is relevant. Timely disease detection plays a major role in its treatment and is one of the keys to reducing the incidence rate. Because DME and AMD remain the leading causes of low visual acuity, regular ophthalmological screening for these pathologies is necessary to identify patients in need of specialized ophthalmological treatment. This strategy will prevent progression to blindness at the early stages [15]. Timely treatment initiation is often central to a favorable outcome with positive morphological and functional results (primarily visual acuity).

The use of vascular endothelial growth factor inhibitors (anti-VEGFs) is the gold standard treatment of neovascular AMD and DME [16]. To date, different protocols for the management of patients with these diseases have been developed. All proposed protocols involve regular treatment assessment based on clinical studies and OCT findings. Currently, OCT is the most informative and widely used method for diagnosing retinal pathology because it enables accurate monitoring during therapy. Based on OCT findings, the current state of the macular area is assessed, and treatment techniques for an individual case are selected.

The increasing number of patients with macular diseases who require regular ophthalmological examinations puts a significant strain on the healthcare system. The need to diagnose a large number of patients in a limited period, evaluate the data obtained, and determine the treatment techniques requires significant labor and time resources, which may cause their shortage. A possible solution to this problem is to develop and implement innovative methods for analyzing ocular images in clinical practice and the use of artificial intelligence (AI)-based technologies [17].

With advancements in AI technology, a whole new area has emerged, which aims at the development and implementation of intelligent systems in routine clinical practice. In Russia, the National AI Development Strategy has been approved until 2030.¹

In Russia and all over the world, research on creating and validating AI-based software for diagnosing retinal pathology in clinical practice is ongoing. The authors of the Google DeepMind project (UK, USA) examined images obtained in 32 ophthalmology clinics that covered different population groups. The findings of the image analysis may be used to quantify retinal morphology and obtain measurement results for specific pathologies. The company’s software is highly efficient in sorting images, nearly the same as specialists or superior. The authors examined 53 pathologies consistent with the national areas of development, presented a novel system capable of high-quality, efficient analysis of clinical OCT images, and offered recommendations. In the future, they plan to focus on a greater number of clinical disorders and expand the scope of their work.

The RetInSight software developed by researchers at the University of Vienna (Austria) under the guidance of Prof. U. Schmidt-Erfurth was designed to monitor the treatment effectiveness of neovascular AMD using AI algorithms. The program is based on the principle of segmenting intraretinal and subretinal fluid and retinal pigment epithelium detachment and dynamically assessing the volumes of these structures [18].

Chicago-based Altris Inc. (USA) launched the Altris AI software designed for automatic analysis of OCT scans. According to the developers, the cloud platform provides rapid analysis and visualization of 100 pathologies and pathological signs, including rare ones. The embedded algorithms can diagnose glaucoma, AMD, DR, and other retinal diseases. Users have access to modules for screening, segmentation/classification, and reporting [19].

B.E. Malyugin et al. [21] focused on creating an algorithm for the automated detection of biomarkers of anti-VEGF therapy effectiveness in patients with AMD. Seven biomarkers were identified, namely, pigment epithelial detachment, pigment epithelium, subretinal fluid, intraretinal fluid, ellipsoid zone, retinal nerve fiber layer, and subretinal hyper-reflective material. The Dice score was 0.8 for retinal pigment epithelial detachment, 0.4 for the pigment epithelium and subretinal fluid, and 0.3–0.15 for other biomarkers. In the future, the authors planned to expand the set of training data and increase the accuracy [20].

AIM

To assess the AI capabilities in diagnosing macular pathology based on structural OCT scans.

MATERIALS AND METHODS

Study design

Structural OCT scans were obtained using an RTVue XR 110-2 OCT scanner (Optovue, USA). To analyze OCT scans, Retina.AI software (Digital Vision Solutions LLC, Russia) was used for processing digital medical images. The single-center, retrospective, observational, sampling-based, single-arm study had a non-inferiority design.

Eligibility criteria

Inclusion criteria:

• Confirmed diagnosis of DME, AMD (dry and exudative), central serous choroidopathy, and vitreomacular traction syndrome (vitreomacular traction, epiretinal membrane, and full-thickness macular hole and lamellar macular hole)
• Central visual acuity loss and suspected macular pathology.

Exclusion criteria:

• OCT scanning is impossible because of the severe lack of clarity of the optical media (corneal opacity, mature cataract, hyphema, hemophthalmos, etc.)
• Patient’s inability of visual fixation (nystagmus, Parkinsonism, etc.).

Subgroup analysis

The study included a total of 129 patients, with a mean age of 65.9 years. Women made up 55.8% (n =72), whereas 44.2% were men (n =57). The number of examined eyes was 200, with the following nosological entities:

• DME
• Cystic macular edema
• ARMD (macular neovascularization and retinal drusen)
• Central serous choroidopathy
• Vitreomacular traction syndrome
• Full-thickness macular hole
• Lamellar macular hole
• Epiretinal membrane.

Moreover, eyes without macular pathologies were examined. The distribution by nosological entity is presented in Table 1.

 

Table 1. Distribution of the study eyes by nosological entity

Diagnosis	No. of eyes
Diabetic macular edema/cystic macular edema	52
Age-related macular degeneration: macular neovascularization	41
Age-related macular degeneration: retinal drusen	40
Central serous choroidopathy	25
Vitreomacular traction syndrome	25
Full-thickness macular hole	23
Lamellar macular hole	25
Epiretinal membrane	72
No pathology	34

 

Study conditions

This study conducted a comparative clinical analysis of structural OCT scans of patients undergoing examination and treatment at the Federal Research and Clinical Center for Specialized Medical Care and Medical Technologies of the Federal Medical-Biological Agency of Russia and the Moscow Regional Research Clinical Institute named after M.F. Vladimirsky. The study duration was 12 months.

Medical intervention and primary outcome

Structural OCT scans were analyzed by the AI program in two stages. First, the program segmented the biomarkers. Second, it calculated the probable pathology using a differential diagnostic search algorithm. The conclusion resulting from the analysis of OCT scans by the AI program was compared with the clinical opinion of two ophthalmologists.

Ethics review

The study protocol was reviewed by the Ethics Committee of the Federal Research and Clinical Center for Specialized Medical Care and Medical Technologies of the Federal Medical-Biological Agency of Russia, Extract 2 from Protocol No. 11_2022 dated November 29, 2022. The conclusion read as follows: “To approve the retrospective analysis of eye fundus photographs and OCT scans of patients selected from medical databases using Retina.AI software for processing digital images in the diagnosis of eye pathologies by analyzing fundus photographs and structural OCT scans according to TU 58.29.32-001-60003594-2022 on the basis of the Research and Clinical Center for Specialized Medical Care and Medical Technologies of the Federal Medical and Biological Agency of Russia.”

Outcome measures and statistical analysis

To evaluate software accuracy, the following parameters were calculated:

• Number of true positives (TP)
• Number of false positives (FP)
• Number of false negatives (FN)
• Number of true negatives (TN).

Based on the presented parameters, sensitivity, specificity, and accuracy were calculated for each nosological entity using the following formulas:

Precision: the proportion of true-positive cases out of the predicted positive cases:

Precision= $\frac{TP}{\left(TP+FP\right)}$

Recall/Sensitivity: the proportion of true-positive cases out of all positive cases:

Recall= $\frac{TP}{\left(TP+FN\right)}$

Specificity: the proportion of true-negative cases out of all negative cases:

Specificity= $\frac{TN}{\left(TN+FP\right)}$ [17].

RESULTS

Study objects (subjects) and primary outcome results

The study used Retina.AI software for processing digital medical images to diagnose pathologies by analyzing structural OCT scans. The study was conducted without patient participation; it only involved the assessment and analysis of clinical data.

During the analysis of OCT scans using the program, the following pathological structures were identified:

• Intraretinal cysts, subretinal fluid
• Retinal pigment epithelium detachment
• Subretinal hyper-reflective material
• Retinal drusen
• Epiretinal membrane
• Vitreomacular traction
• Full-thickness macular hole
• Lamellar macular hole.

The program automatically generated a conclusion based on the identified pathological signs. The general parameters of the method were as follows: sensitivity, 95.16%; specificity, 97.76%; accuracy, 97.38%. Detailed results of the comparative analysis of the conclusions by the AI and ophthalmologists are presented in Table 2.

 

Table 2. Sensitivity, specificity, and precision of Retina.AI software in diagnosing macular area diseases

Diagnosis	Sensitivity, %	Specificity, %	Precision, %
Diabetic macular edema/cystic macular edema	96.08	97.48	97.14
Macular neovascularization	97.50	97.65	97.62
Age-related macular degeneration: retinal drusen	97.37	96.51	96.67
Central serous choroidopathy	92.59	98.36	97.62
Vitreomacular traction	86.96	97.33	96.19
Full-thickness macular hole	95.65	98.40	98.10
Epiretinal membrane	100.00	97.12	98.10
Lamellar macular hole	95.83	97.85	97.62
No pathology	94.72	97.69	97.25

 

Fig. 1 shows a structural OCT scan of the macular zone of patient B (54 years old) with type 2 diabetes mellitus. The ophthalmologist’s report recorded DME. In the analysis of the structural OCT scan by Retina.AI, the algorithms segmented the following pathological signs: intraretinal cysts, subretinal fluid. In conclusion, the patient was diagnosed with DME.

 

Fig. 1. An example of the optical coherence tomography scan analysis of a patient with diabetic macular edema by the artificial intelligence algorithm: a — structural optical coherence tomography scan; b — optical coherence tomography scan after segmentation of the pathological features (subretinal fluid — green mask, intraretinal cysts — blue masks); c — scan analysis report (the reporting table of the differential diagnostic search, probable pathology is highlighted in red — macular edema).

 

Fig. 2 shows a structural OCT scan of the macular zone of patient B (68 years old) with exudative AMD. The diagnosis was established by an ophthalmologist. In the analysis of the structural OCT scan by Retina.AI, the algorithm segmented the following pathological signs: intraretinal cysts, subretinal fluid, retinal pigment epithelium detachment, and subretinal hyper-reflective material. The program report registered macular neovascularization.

 

Fig. 2. An example of the optical coherence tomography scan analysis of a patient with exudative age-related macular degeneration by the artificial intelligence algorithm: a — structural optical coherence tomography scan; b — optical coherence tomography scan after segmentation of the pathological features (subretinal fluid — green mask, intraretinal cysts — blue masks, retinal pigment epithelium detachment — orange mask, subretinal hyperreflective material — yellow mask); c — scan analysis report (the reporting table of the differential diagnostic search, probable pathology is highlighted in red — macular neovascularization).

 

Fig. 3 shows a structural OCT scan of the macular zone of patient K (69 years old) with a full-thickness macular hole diagnosed by an ophthalmologist. In the analysis of the structural OCT scan by Retina.AI, the algorithm segmented the following pathological signs: full-thickness macular hole, intraretinal cysts, and epiretinal membrane. The program report registered a full-thickness macular hole and epiretinal membrane.

 

Fig. 3. An example of the optical coherence tomography scan analysis of a patient with macular hole, epiretinal membrane by the artificial intelligence algorithm: a — structural optical coherence tomography scan; b — optical coherence tomography scan after segmentation of the pathological features (macular hole — violet mask, intraretinal cysts — blue masks, epiretinal membrane — red masks); c — scan analysis report (the reporting table of the differential diagnostic search, probable pathology is highlighted in red — macular hole, epiretinal membrane).

 

Adverse events

No adverse events occurred during the study.

DISCUSSION

Primary outcome summary

During the clinical study, the AI algorithm-based Retina. AI software demonstrated high sensitivity, specificity, and accuracy in diagnosing macular pathology: these parameters exceeded 90% for all diseases, except for vitreomacular traction syndrome, with a sensitivity of 86.96%. The advantage of the software lies in its convenient user interface that highlights the identified pathological structures in an OCT scan, which allows the doctor to additionally double-check the performance of the program and fosters the doctor’s trust in the AI technology.

Primary outcome discussion

Practically, one of the most promising areas for introducing AI technologies into clinical practice is screening examinations that require examining a large number of patients and identifying those in need of specialized ophthalmological treatment. AI technologies open up the possibility of delegating some functions to nursing staff and organizing pre-doctor screening to free up the doctor’s time for more complex tasks and increase the throughput of the ophthalmologist’s office. However, AI technology cannot be viewed as a replacement for an ophthalmologist but as a tool to boost the efficiency of diagnosis and treatment. Implementing AI software may prove troublesome because it requires a stable Internet connection as the platform is cloud-based. For the convenience of continuous operation, developing a desktop application that does not depend on the Internet connection is promising.

Another encouraging area for using AI technologies is on monitoring the pathological process during treatment. In patients with DME and exudative AMD during anti-VEGF therapy, segmentation of intraretinal cysts, subretinal fluid, pigment epithelial detachment, and subretinal hyper-reflective material allows for a dynamic quantitative assessment of the volumes of these structures. At present, the primary quantitative parameters that doctors focus on during treatment include the best-corrected visual acuity, central retinal thickness, and area of the neovascular membrane in patients with AMD as visualized in OCT angiography. New quantitative criteria for assessing the effectiveness of anti-VEGF therapy by ophthalmologists in collaboration with developers of AI algorithms are urgently needed.

However, the technology is limited by the evaluation of a single OCT scan uploaded into the program. Thus, the software must be improved to allow for the analysis of the entire series of images of the same patient in the future.

CONCLUSION

During clinical validation, Retina.AI software based on AI algorithms demonstrated high sensitivity, specificity, and accuracy in the diagnosis of DME, dry and exudative AMD, central serous choroidopathy, and vitreomacular interface disorders based on analysis of structural OCT scans. The Retina.AI platform is a Russian development and is currently available for testing at www.screenretina.ru. However, the conclusion of the program is not a diagnosis and must be clarified by a specialist.

ADDITIONAL INFORMATION

Funding source. This study was not supported by any external sources of funding.

Competing interests. E.A. Katalevskaya and A.Yu. Sizov are part of the software developers. Other authors declare that they have no competing interests.

Authors’ contribution. All authors made a substantial contribution to the conception of the work, acquisition, analysis, interpretation of data for the work, drafting and revising the work, final approval of the version to be published and agree to be accountable for all aspects of the work. M.R. Khabazova — literature search and analysis, data analysis, manuscript writing, manuscript editing; E.N. Ponomareva — conducting research, data analysis, manuscript editing; I.A. Loskutov — discussion of the research results, editing; E.A. Katalevskaya — development of the artificial intelligence algorithms, data markup, manuscript editing, literature search and analysis; A.Yu. Sizov — development of the artificial intelligence algorithms, data markup, graphic materials preparation; G.M. Gabaraev — literature search, data analysis.

 

¹ Decree of the President of the Russian Federation No. 490 dated October 10, 2019, “On the development of artificial intelligence in the Russian Federation.” Access: http://www.kremlin.ru/acts/bank/44731

About the authors

Margarita R. Khabazova

Federal Research and Clinical Center of Specialized Medical Care and Medical Technologies

Email: rita.khabazova@mail.ru
ORCID iD: 0000-0002-7770-575X
SPIN-code: 2736-9089
Russian Federation, Moscow

Elena N. Ponomareva

Federal Research and Clinical Center of Specialized Medical Care and Medical Technologies

Email: ponomareva.en@fnkc-fmba.ru
ORCID iD: 0009-0001-0828-9844
SPIN-code: 7868-4425
Russian Federation, Moscow

Igor A. Loskutov

Moscow Regional Research and Clinical Institute

Email: loskoutigor@mail.ru
ORCID iD: 0000-0003-0057-3338
SPIN-code: 5845-6058

MD, Dr. Sci. (Medicine)

Russian Federation, Moscow

Evgenia А. Katalevskaya

Digital Vision Solutions LLC

Email: ekatalevskaya@mail.ru
ORCID iD: 0000-0002-5710-9205
SPIN-code: 7849-8890

MD, Cand. Sci. (Medicine)

Russian Federation, Moscow

Alexander Yu. Sizov

Digital Vision Solutions LLC; Nizhny Novgorod State Technical University n.a. R.E. Alekseev

Email: sizov_ost_vk@mail.ru
ORCID iD: 0000-0003-3338-4015
SPIN-code: 4468-1730
Russian Federation, Moscow; Nizhny Novgorod

Georgiy М. Gabaraev

Federal Research and Clinical Center of Specialized Medical Care and Medical Technologies

Author for correspondence.
Email: geor_gabaraev1@mail.ru
ORCID iD: 0000-0002-0759-3107
SPIN-code: 1802-3224
Russian Federation, Moscow

References

Supplementary files