Identification of indicators used to assess needs for telemedicine consultations in various profiles of medical care

Texto integral

BACKGROUND

Telemedicine is a highly effective and innovative healthcare service in different treatment areas [1]. Its widespread use reduces the time patients spend in healthcare organizations, increases the availability of healthcare—especially in underpopulated countries—by increasing the coverage of diagnostic tests and outpatient follow-up, and increases patient satisfaction with the overall quality of healthcare [2].

Telemedicine has been shown to increase the accessibility and clinical effectiveness of care and improve patient satisfaction across treatment areas [3, 4]. Additionally, evidence suggests that it helped contain the spread of COVID-19 during the pandemic [5]. Telemedicine has also proven effective in remotely monitoring the health status of patients with various diseases [6–9].

In the Russian Federation, a significant proportion of telemedicine consultations (TMCs) are performed by National Medical Research Centers (NMRCs). One of the primary objectives of the Federal Project “Development of the NMRC Network and Implementation of Innovative Medical Technologies” is the implementation of TMCs for regional, republican, territorial, and district healthcare organizations across the constituent entities of the Russian Federation. Positive experiences with TMCs have been described in some treatment areas:

• General Practice [10],
• Psychiatry and Psychiatry/Narcology [11–13],
• Ophthalmology [14],
• Oncology [15],
• Surgery (Transplantology) [16],
• Anesthesiology and Intensive Care in Pregnancy [17],
• Obstetrics/Gynecology and Neonatology [18].

Positive experiences of using TMCs are confirmed by “anchor” healthcare organizations of the constituent entities of the Russian Federation [19]. However, there is still no unified system for assessing the results and real contribution of TMCs toward improving the quality of care in the Russian healthcare system. Additionally, no factors have been identified to enhance the efficient use of financial resources allocated to the implementation of TMCs.

STUDY AIM

The aim of this study was to develop a system of parameters to differentiate the need for TMC in different treatment areas.

MATERIALS AND METHODS

The study included two phases:

1. Creating a list of parameters that determine the need for TMC in the delivery of healthcare services in various treatment areas,
2. Conducting a quantitative assessment of the significance of each parameter using a weighting coefficient.

The list of parameters was based on studying the established practice of TMC and its evaluation, including the following:

• Analysis of analytical reports on NMRC field events in the constituent entities of the Russian Federation and annual public reports on the activities of NMRCs for 2020–2022 in order to identify parameters influencing the need for TMC,
• Open interviews with representatives of the NMRCs to clarify and validate the identified parameters.

The expert survey was used to assess the quantitative significance of each parameter. Based on the results of the expert survey, weighting coefficients were calculated to assess the significance of parameters that determine the need for TMC in different treatment areas.

A structured form was developed to analyze the NMRC reports, including the following:

• Parameters used by the NMRC to determine the need for TMC,
• Specific and general suggestions for improving the effectiveness and efficiency of TMC,
• Parameters that determine the technical feasibility of TMC implementation in the constituent entities of the Russian Federation,
• Results of the assessment of the implementation of the recommendations made by the NMRC professionals during the TMC (if any).

Similar parameters determining the need for TMC were grouped for subsequent validation through open interviews with NMRC experts.

To clarify and validate the key parameters determining the need for TMC, face-to-face open interviews were conducted with 14 experts from five NMRCs. The experts were asked to determine the relevance of the parameters selected in the field report analysis phase, determine the scope of TMC for each treatment area, and formulate additional conditions that should be considered in determining the need for TMC for different treatment areas.

Based on the results of the first phase, a list of potential parameters was created to determine the need for TMC in different treatment areas.

In the second phase, experts assessed the significance of the listed parameters using a standardized form. To participate in the interview, experts had to meet the following criteria:

• At least 1 year of experience participating in the Federal Project “Development of the NMRC Network and Implementation of Innovative Medical Technologies,”
• At least 5 years of experience in management and methodology development at the level of constituent entities of the Russian Federation,
• Experience in planning and implementing TMC, calculating the cost of implementing TMC, and/or analyzing the effectiveness of implementing TMC at various levels,
• A self-assessed expertise level of ≥0.5 points¹ [20].

In total, 18 experts participated in the survey: heads of structural departments responsible either for management and methodology development in the constituent entities of the Russian Federation or for TMCs, as well as specialists of NMRCs and the Federal State Budgetary Institution “The Center of Expertise and Quality Control of Medical Care” of the Ministry of Health of the Russian Federation. The mean coefficient of expertise was 0.58.

Each expert was asked to rate the significance of considering each parameter on a scale of 1 to 5, with 1 being the least significant and 5 being the most significant. It was suggested that the significance of the parameters influencing the planned scope of TMC be evaluated separately for each treatment area. To assess the significance of parameters of effectiveness and efficacy of the TMC implementation and the validity of corresponding requests, it was decided not to use reference to specific treatment areas since, in an open interview, the majority of experts agreed that the significance of these parameters does not vary depending on the treatment area.

For each parameter, an average score (based on all expert responses) was calculated in points. Subsequently, the concordance of expert opinions was assessed using Kendall’s W [21], which is a statistic ranging from 0 to 1 that characterizes the concordance of expert opinions (ranks) for a set of criteria. The degree of expert opinion concordance was considered unsatisfactory at W < 0.3, intermediate at 0.3 < W < 0.7, and high at W > 0.7. To ensure the significance of the differences in the identified parameter correlation, the coefficient of concordance was evaluated using the Friedman test. Statistical analysis was conducted using Statistica 10 (StatSoft, Tulsa, OK, USA).

The final weighting coefficients for all parameter profiles were calculated by multiplying the mean assessment value by Kendall’s coefficient of concordance. Weighting coefficients for parameters influencing the planned scope of TMC were calculated for each profile separately. This was done for parameters of TMC effectiveness and efficacy and the validity of corresponding requests without reference to specific treatment areas.

RESULTS

A review of the NMRC reports and open interviews with NMRC experts identified three groups of parameters that determine the necessity of TMC for a range of treatment areas:

1. Parameters influencing the planned scope of TMC,
2. Parameters characterizing the effectiveness and efficacy of TMC implementation,
3. Parameters characterizing the validity of requests for TMC.

Parameters Influencing the Planned Scope of Telemedicine Consultations

The planned scope of TMC is defined by specific parameters that reflect the health status of patients who are receiving healthcare services in the specific treatment area, as well as certain characteristics of such services. The parameters are divided into five subgroups:

1. Mortality rates from diseases covered by the treatment area,
2. Disability rates,
3. In-hospital mortality rates,
4. Rates of emergency/urgent consultations (among all TMCs),
5. Rates of intensive care consultations (among all TMCs).

Table 1 shows the most used parameters in the various subgroups.

 

Table 1. Parameters influencing the planned scope of telemedicine consultations (most frequently used by National Medical Research Centers)

Subgroup	Parameters
Mortality rates from diseases covered by the treatment area	Mortality rates per 100,000 for individual diseases or groups of diseases by treatment area Overall mortality rate 100,000 for all diseases by treatment area
Disability rates	Disability rates per 10,000 for individual diseases or groups of diseases by treatment area Overall mortality rate 10,000 for all diseases by treatment area
In-hospital mortality rates	In-hospital mortality in different surgeries by treatment area
Rates of emergency/urgent consultations (among all TMCs)	Percentage of patients with various acute disease complications by treatment area
Rates of intensive care consultations (among all TMCs)	Rates of major emergency and urgent conditions (ICD-10) Prescription rate for a range of healthcare services (dialysis, ECMO, etc.)

Note. ECMO, extracorporeal membrane oxygenation; ICD-10, International Classification of Diseases, 10th revision; TMC, telemedicine consultations

 

Parameters Characterizing the Effectiveness and Efficacy of Telemedicine Consultations

The effectiveness and efficacy of the TMC are evaluated using subjective and objective parameters.

Subjective parameters include the satisfaction level of healthcare professionals in the constituent entities of the Russian Federation with the TMC outcomes, which can be assessed through surveys using a structured feedback form. At some NMRCs, experts are already using forms that ask questions, such as:

• Are you satisfied with the outcomes of the consultation?
• Does the outcome of the consultation meet the goals?
• Are you satisfied with the conclusion based on TMC results?
• Were the recommendations made during the consultation followed?
• Was the diagnosis clarified/changed as a result of the consultation?
• Was the treatment plan changed as a result of the consultation?
• Did you have any difficulties creating a TMС request?
• Please rate whether the expectation of the request matches the priority of the consultation (scheduled, urgent, emergency).
• Please rate your level of satisfaction with the work of the TMC NMRC Group.
• Based on the TMC results, the patient was referred to a federal healthcare organization for treatment/further examination (“yes” or “no”).

Each question is suggested to be answered in points, usually ranging from 1 to 5, with the overall score being the final result.

To objectively assess the effectiveness and efficacy of TMCs conducted by a constituent entity of the Russian Federation, experts most often consider it necessary to evaluate outcomes of diseases and hospitalizations for which the TMC was performed. Based on open interviews with TMC department heads, two parameters were recommended for an objective assessment:

1. The percentage (or absolute number) of positive TMC outcomes: recovery/improvement/remission + discharged/referred to another healthcare organization, federal healthcare organization, or NMRC,
2. The percentage (or absolute number) of negative TMC outcomes: deterioration/progression/complications/death + discharged/referred to another healthcare facility, federal healthcare facility, or NMRC.

Parameters Characterizing the Validity of Requests for Telemedicine Consultations

According to NMRC field reports and interviews with NMRC specialists, the validity of TMC referral by healthcare organizations of constituent entities of the Russian Federation is influenced by the following:

• Scope of examination prior to the consultation,
• The accuracy of the diagnosis,
• Expert assessment of the possibility for independent decision-making at the level of region or healthcare organization.

A complete examination of the patient before the consultation indicates that all necessary tests, including laboratory tests and imaging modalities, are conducted in accordance with clinical guidelines.

Accuracy of the established diagnosis means that the patient’s diagnosis is fully consistent with the clinical picture of the disease and that the wording and coding of the diagnosis are consistent with the current International Classification of Diseases and other generally accepted classifications.

Experts evaluate the possibility of making independent decisions at the regional or healthcare organization level, considering the availability of human and material resources for accurate diagnosis, the ability to perform appropriate examinations and tests, prescribe the necessary treatment, and the need for timely referral of a patient to another healthcare organization, without conducting TMC with the NMRC.

To determine the need for TMC, most experts have suggested evaluating the share of justified TMC requests from the total number of requests from a constituent entity of the Russian Federation.

Assessment of Parameter Significance and Calculation of the Weighting Coefficients

Table 2 shows the weighting coefficients based on expert assessment for Group 1 (separately for different treatment areas).

 

Table 2. Weighting coefficients for the significance of parameters influencing the planned scope of telemedicine consultations

Treatment area	Coefficient of concordance	P-value	Weighting coefficient of parameter significance
			Mortality rate	Disability rate	In-hospital mortality rate	Percentage of emergency TMCs	Percentage of intensive care patients
Obstetrics/Gynecology	0.2171	0.0036	0.74	0.66	0.87	0.77	0.92
Anesthesiology and Intensive Care (Pregnancy)	0.3188	0.0001	1.28	0.81	1.36	1.20	1.36
Anesthesiology and Intensive Care (Adults)	0.3723	0.0000	1.45	0.95	1.61	1.41	1.59
Anesthesiology and Intensive Care (Children)	0.3486	0.0001	1.39	0.91	1.53	1.32	1.49
Hematology	0.1151	0.0817	0.43	0.32	0.44	0.36	0.43
Geriatrics	0.0240	0.7863	0.06	0.05	0.06	0.05	0.06
Oncology and Hematology (Children)	0.1761	0.0130	0.76	0.58	0.69	0.63	0.67
Traumatology and Orthopedics (Children)	0.1734	0.0141	0.54	0.60	0.67	0.61	0.69
Infectious Diseases	0.3406	0.0001	1.02	0.78	1.31	1.06	1.23
Cardiology	0.1541	0.0255	0.59	0.49	0.66	0.52	0.62
Coloproctology	0.0201	0.8355	0.06	0.05	0.06	0.06	0.07
Medical Rehabilitation	0.0547	0.4146	0.10	0.13	0.11	0.11	0.14
Neurology	0.0425	0.5477	0.15	0.13	0.16	0.14	0.16
Neurosurgery	0.2201	0.0032	0.93	0.71	0.97	0.89	0.95
Neonatology	0.1307	0.0516	0.52	0.45	0.57	0.51	0.56
Oncology	0.1342	0.0466	0.57	0.45	0.49	0.45	0.48
Otorhinolaryngology	0.0973	0.1355	0.22	0.22	0.28	0.27	0.30
Ophthalmology	0.0859	0.1859	0.20	0.24	0.24	0.20	0.26
Pediatrics	0.1611	0.0206	0.59	0.47	0.66	0.55	0.64
Psychiatry and Psychiatry/Narcology	0.0760	0.2424	0.18	0.19	0.19	0.21	0.22
Pulmonology	0.0509	0.4536	0.16	0.37	0.47	0.40	0.47
Health Resort Treatment	0.0316	0.6852	0.05	0.05	0.06	0.05	0.05
Cardiovascular Surgery	0.2363	0.0019	0.95	0.75	1.05	0.93	1.02
Dentistry	0.0973	0.1811	0.22	0.22	0.28	0.27	0.30
General Practice	0.0582	0.3806	0.16	0.15	0.19	0.15	0.19
Traumatology and Orthopedics	0.1118	0.0898	0.35	0.37	0.44	0.37	0.42
Urology	0.0928	0.1539	0.26	0.23	0.29	0.27	0.31
Phthisiology	0.1376	0.0421	0.47	0.36	0.45	0.34	0.44
Surgery	0.1844	0.0100	0.64	0.53	0.72	0.63	0.73
Surgery (Organ and/or Tissue Transplantation)	0.0836	0.1979	0.32	0.26	0.33	0.29	0.33
Maxillofacial Surgery	0.0578	0.3850	0.15	0.15	0.18	0.16	0.19
Endocrinology	0.0458	0.5099	0.15	0.13	0.17	0.14	0.17

Note. TMC, telemedicine consultations

 

In Group 1, the significance of each parameter varied by treatment area. For example, in Obstetrics/Gynecology, the percentage of intensive care patients was the most significant (coefficient of 0.92), and the disability rate was the least significant (0.66). In Cardiology, the in-hospital mortality rate was the most significant parameter (0.66), with the disability rate being the least significant parameter (0.49). In Oncology, the mortality rate was the most significant parameter (0.57), whereas the disability rates and the percentage of emergency/urgent TMCs were the least significant parameters (0.45). In Medical Rehabilitation, the percentage of intensive care patients and the disability rates were the most significant parameters (0.14 and 0.13, respectively), and the mortality rate was the least significant parameter (0.10). However, all parameters had lower coefficients compared to other treatment areas (except for Coloproctology and Geriatrics).

The highest weighting coefficients for all five subgroups were found in four treatment areas such as Anesthesiology/Intensive Care (Adults), Anesthesiology/Intensive Care (Children), Anesthesiology/Intensive Care (Pregnancy), and Infectious Diseases (Fig. 1). In these treatment areas, the in-hospital mortality rate and the percentage of intensive care consultations were the most significant parameters. Less significant parameters for Intensive Care included the mortality rate and the percentage of emergency TMCs, and vice versa for Infectious Diseases. The disability parameters were the least significant.

 

Fig. 1. Four treatment areas with the highest weighting coefficients of the parameters influencing the planned scope of telemedicine сonsultations. TMC, telemedicine consultations.

 

Table 3 shows the weighting coefficients of the parameter significance for Groups 2 and 3 (not related to the treatment areas). The percentage of justified TMC requests from the total number of requests from a constituent entity of the Russian Federation was the most significant parameter (0.0743). Subjective quality of TMC (0.0679), the number of positive TMC outcomes (0.0658), and the absolute number of negative TMC outcomes (0.0638) followed.

 

Table 3. Weighting coefficients of parameters characterizing the effectiveness and efficacy of telemedicine consultations

Parameter	Coefficient of concordance	p-value	Weighting coefficient
Significance of the subjective quality of telemedicine consultations based on a feedback form obtained from a constituent entity of the Russian Federation	0.01852	0.71653	0.0679
Significance of percentage (or absolute number) of positive TMC outcomes			0.0658
Significance of percentage (or absolute number) of negative TMC outcomes			0.0638
Significance of percentage of justified TMC requests from the total number of requests from a constituent entity of the Russian Federation			0.0743

Note. TMC, telemedicine consultations

 

DISCUSSION

Based on the analysis of established expert TMC practices, our study generated a list of parameters to assess the need for TMC in different treatment areas. The significance of each parameter was evaluated using a weighting coefficient.

Most of Russian studies included a qualitative assessment of the effectiveness of TMC. For example, in several treatment areas, the active use of TMC in constituent entities of the Russian Federation has the following:

• Reduced length of hospital stays by optimizing drug therapy (Psychiatry and Psychiatry/Narcology [11–13]),
• Improved the effectiveness of minimally invasive surgery through adaptation of surgical techniques (Ophthalmology [14]),
• Supported criteria for initiation of anticancer drug therapy (Oncology [15]),
• Reduced postoperative morbidity and in-hospital mortality rates in some healthcare organizations (Anesthesiology/Intensive Care (Pregnancy) [17], Obstetrics/Gynecology and Neonatology [18]).

Our parameters for determining the need for TMC are partially consistent with those from other studies. For example, a systematic review by Khanal S et al. [22] evaluated 46 papers on 36 TMC programs in different countries and identified some predictors for the most appropriate and effective telemedicine use. For example, such criteria included treatment of urgent conditions (if real-time TMC is possible); this parameter was also highlighted in our study. However, Khanal S et al. did not quantify the significance of the identified factors.

In an open interview, we clarified the parameters that influence the need for TMC. A similar approach was used in a Norwegian study, which also conducted open interviews with key stakeholders at the national, regional, and local levels to identify the main factors determining the use of telemedicine by health authorities and healthcare organizations [23]. However, according to the study, organizational factors were critical, including the integration of telemedicine into daily healthcare services, a single body for telemedicine management, and training of healthcare professionals in the proper use of telemedicine.

Both studies emphasized the need for a quantitative assessment of the parameters and factors that determine the use of telemedicine and its necessity. The aim of our study was not only to identify the most significant parameters and factors that determine the need for TMC in different treatment areas but also to develop a methodology for quantitative assessment of parameter significance using weighting coefficients.

A quantitative assessment of the parameter significance and differentiated approach to treatment areas are the main differences between our study and other similar studies, which, in most cases, only conducted a qualitative assessment of the factors influencing the use of TMC. Similar studies that quantitatively assessed the significance of parameters and factors were not found.

However, our study had some limitations. It may be difficult or inaccurate to obtain primary data to calculate some parameters. The weighting coefficients reflecting the significance of the factors were determined by a small panel of experts, and a larger study may be needed to refine them in the future.

CONCLUSION

A system of parameters (with their significance considered) has been developed that can be used for a differentiated assessment of the need for TMC in different treatment areas.

ADDITIONAL INFORMATION

Funding source. This study was not supported by any external sources of funding.

Competing interests. All authors declare that they have no competing interests.

Authors’ contribution. All authors made a substantial contribution to the conception of the work, acquisition, analysis, interpretation of data for the work, drafting and revising the work, final approval of the version to be published and agree to be accountable for all aspects of the work. E.S. Samsonova — concept and design of the study, collection of material, writing the article, acquisition, analysis and interpretation of data; I.A. Mikhailov — concept and design of the study, data collection, analysis and interpretation, manuscript writing, approval of the final version; V.V. Omelyanovsky, M.V. Avksentieva, I.A. Zheleznyakova — concept and design of the study, editing, approval of the final version; G.G. Lebedenko — concept and design of the study, editing.

 

¹ Each expert was asked to evaluate their level of expertise in three domains: k1 for the level of theoretical knowledge of a subject matter, k2 for the level of practical knowledge (experience) of a subject matter, and k3 for the level of ability to predict future development of a subject matter. Each of the three domains was rated by an expert using the following scale: 1 point for the high level, 0.5 points for the intermediate level, 0 points for the low level. The coefficient of expertise (k) was calculated by averaging k1, k2, and k3.

Sobre autores

Elena Samsonova

The Center for Healthcare Quality Assessment and Control; N.A. Semashko National Research Institute of Public Health; Russian Medical Academy of Continuous Professional Education

Email: samsonova@rosmedex.ru
ORCID ID: 0000-0002-6085-1320
Código SPIN: 8372-5195
Rússia, Moscow; Moscow; Moscow

Ilya Mikhailov

The Center for Healthcare Quality Assessment and Control; N.A. Semashko National Research Institute of Public Health; Russian Medical Academy of Continuous Professional Education

Autor responsável pela correspondência
Email: mikhailov@rosmedex.ru
ORCID ID: 0000-0001-8020-369X
Código SPIN: 5798-0749
Scopus Author ID: 57203900904
Researcher ID: I-9035-2017

MD, Cand. Sci. (Medicine)

Rússia, Moscow; Moscow; Moscow

Vitaly Omelyanovsky

The Center for Healthcare Quality Assessment and Control; N.A. Semashko National Research Institute of Public Health; Russian Medical Academy of Continuous Professional Education; Scientific and research financial institute

Email: vvo@rosmedex.ru
ORCID ID: 0000-0003-1581-0703
Código SPIN: 1776-4270

MD, Dr. Sci. (Medicine), Professor

Rússia, Moscow; Moscow; Moscow; Moscow

Maria Avksentieva

The Center for Healthcare Quality Assessment and Control; Russian Medical Academy of Continuous Professional Education

Email: avksenteva@rosmedex.ru
ORCID ID: 0000-0001-6660-0402
Código SPIN: 3333-4237

MD, Dr. Sci. (Medicine), Professor

Rússia, Moscow; Moscow

Inna Zheleznyakova

The Center for Healthcare Quality Assessment and Control; Russian Medical Academy of Continuous Professional Education

Email: zheleznyakova@rosmedex.ru
ORCID ID: 0000-0003-2943-6564
Código SPIN: 2413-9265
Rússia, Moscow; Moscow

Georgy Lebedenko

The Center for Healthcare Quality Assessment and Control; Russian Medical Academy of Continuous Professional Education

Email: lebedenko@rosmedex.ru
Código SPIN: 5504-7272
Rússia, Moscow; Moscow

Bibliografia

Arquivos suplementares