Diagnostic value of lung ultrasound in COVID-19: systematic review and meta-analysis

Full Text

ABBREVIATIONS

CI — confidence interval

SMD — standard mean difference

CT — computed tomography

US — ultrasound

RT-PCR — reverse transcription polymerase chain reaction

ICD — International Classification of Diseases

INTRODUCTION

As of September 16, 2020, there are 29,155,581 confirmed cases globally, with 926,544 deaths [1] from the COVID-19 pandemic. The impact of the end of the summer vacation period and schools re-opening on the epidemic is uncertain. However, there is a possibility of a second wave of the disease [2], if it will follow a high transmission scenario. Amid rising number of new cases, Israel was the first developed country to announce a second nationwide lockdown [3]. Presently, since June 30, 2020, more than 700 cases of SARS-CoV-2 infection have been detected in Moscow. Effective and safe patient triage tools could aid decrease the COVID-19-associated pressure on the healthcare system. Several laboratory parameters help assess the disease severity, such as calculation of the viral load [4], platelet count [5], D-dimer concentrations [6], and others [7]. COVID-19 often leads to respiratory manifestations, and therefore medical imaging is one of the main techniques to assess its severity in patients [8]. Among the imaging modalities, including radiography, computed tomography (CT), and ultrasound (US), CT offers great sensitivity in detecting COVID-19-related findings [9]. Because of this, some experts suggest making it a diagnostic standard. CT imaging was one of the main diagnostic and triage tools in Moscow, Russia, during the lockdown period [10]. Unfortunately, CT it is not widely available and is associated with potential harm from exposure to ionizing radiation. Lung US could compensate for that, being a widespread and safe method. The technique is appealing, especially for pregnant women, children, and critically ill patients. Recent systematic reviews explore the potential utility of lung US [11, 12]. However, there are not enough scientific data to establish the functionality of this approach in making clinical decisions depending on the severity of the disease [13].

We reviewed currently available studies addressing cohorts of COVID-19 patients for the disease severity using US compared to CT, RT-PCR, and laboratory data, in order to assess the sensitivity and specificity of lung US for different degrees of pulmonary involvement.

METHODS

This manuscript follows the PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions [14].

Eligibility criteria

Types of studies. Inclusion criteria: (i) any study evaluating the performance of lung US in diagnosing COVID-19; (ii) studies reporting US sensitivity and specificity values or providing enough information to construct a 2 × 2 confusion matrix; and (iii) we placed no restrictions regarding country, patient age, sex, and race. Exclusion criteria were as follows: (i) studies with unavailable full texts; (ii) studies on non-human subjects; (iii) case reports, case series, and systematic review studies; and (iv) studies published before January 1, 2020.

Types of participants. Hospital patients of any age with signs and symptoms of COVID-19-associated pneumonia confirmed by CT, RT-PCR, or serological tests (ICD codes U07.1, U07.2).

Types of intervention. Studies comparing the diagnostic value of lung US, including point-of-care US (POTUS) with chest CT, chest radiography, and clinical follow-up data.

Types of outcome measures. Primary outcome measures: numerical values of sensitivity and specificity of lung US in COVID-19 patients of different severity grades. Secondary outcome measures: numerical values of sensitivity and specificity of lung US and POTUS for COVID-19 patients regardless of the disease severity.

Information sources. Studies were identified by searching the electronic databases PubMed and Google Scholar. The last search was run on September 1, 2020.

Search. We performed two types of searches in the PubMed database, using MeSH terms and text keywords since it takes about a month for PubMed to assign a MeSH term for a published study:

1) (“Coronavirus infections/diagnosis”[MeSH] OR “Coronavirus infections/diagnostic imaging”[MeSH]) AND “Ultrasonography”[MeSH]

2) (“lung ultrasound” OR “chest ultrasound” OR “thoracic ultrasound” OR “ultrasonography”) AND (COVID-19 OR “SARS-CoV-2” OR “coronavirus”) AND diagnosis

We used the query string “lung ultrasound diagnostic value specificity sensitivity COVID-19” to search the Google Scholar database.

Study selection. Two reviewers (RVR and DVL) assessed for eligibility in a standardized manner by an automatic search for words “sensitivity” and “specificity” in full texts. Three other researchers (NNV, NSK, and OAM) evaluated the selected manuscripts according to the study protocol to resolve discrepancies.

Data collection process and data items. We developed a data extraction sheet using the Google Spreadsheet service to ensure that all the reviewers have simultaneous and unrestricted access to the document. The data extraction sheet was pilot-tested on three randomly selected included studies and refined accordingly. Two reviewers (RVR and DVL) extracted the following data from the included studies: Authors, Affiliation, Title, Journal (or preprint service), Acceptance date, DOI, Population (number, age, % female, inclusion & exclusion criteria, medical centers location, start and end dates of the study), US protocol, US scoring, comparison protocol, comparison scoring, US outcome, and comparison outcome. The three other researchers (NNV, NSK, and OAM) verified the extracted data. Disagreements were resolved through a discussion among the authors. After the review started, we added the data from systematic reviews on specificity and sensitivity of reference standard methods if the values were not estimated in the included studies.

Risk of bias in individual studies. To assess the methodological issues associated with diagnostic accuracy studies, we followed the QUADAS-2 (Quality Assessment of Diagnostic Accuracy Studies) framework [15] recommended for systematic reviews by the Agency for Healthcare Research and Quality, Cochrane Collaboration. Four domains were used to organize each included study: patient selection, index test, reference test, and patient flow. A detailed description of each domain and judgment criteria are described in the Cochrane Handbook [16].

Statistical analysis. We used the random-effect model to analyze and pool lung US sensitivity and specificity across the included studies. To measure between-studies heterogeneity, we used estimates of τ², the percentage of variability I², and Cochran’s Q-statistic. As a threshold we used I² values of 25% (low heterogeneity), 50% (moderate heterogeneity), and 75% (substantial heterogeneity) and p-values < 0.05. The meta-analysis was performed using the dmetar [17] package for R 3.6.3 [18].

RESULTS

Study selection. We included 16 studies in this review. The search in PubMed and Google Scholar databases provided 245 studies imported into a Mendeley library. Of these, six studies were discarded because they were conducted on non-human subjects. After adjusting for duplicates, 236 studies remained. Of these, 220 studies did not meet the criteria and were discarded after abstract or full-text reviewing (Figure 1). We examined the full texts of the remaining 16 studies [19–34], and only six of these analyzed the diagnostic accuracy of US in the context of the disease severity [19, 20, 27–30]. However, only three studies enrolled patients of all clinical grades: mild, moderate, and severe stages of the disease [19, 20, 28]. The other three studies included only critically ill patients [27, 29] or evaluated the prognostic value of lung US in predicting the need for non-invasive respiratory support [30]. A study by Veronese et al. stood out because they analyzed the data of bedridden nursing home patients, aged 84.1 ± 9.8 years [24]. For these patients, mortality was associated with a lung US score of 4 (maximum value 36), primarily due to this cohort’s general health.

 

Figure 1. Flow diagram of the study selection.

 

With the exception of the study by Hatamabadi et al. that provided only the seven-day results [34], the average follow-up period in the included studies was 34 ± 15 days. The included studies involved 1696 participants, of which 1121 had confirmed COVID-19. There were 13 single-center and three multicentric studies, two of which were conducted in France and one in China. In total, four studies were conducted in France, three in China, two studies each in the USA, Turkey, and Spain, and the remaining three came from Iran, Italy, and Israel (Figure 2). The mean or median age of participants ranged from 27 to 69 years (with the exclusion of the study of Veronese et al. [24]).

 

Figure 2. The map of studies included in the review.

Note: the map template had been purchased from Shutterstock [35].

 

All studies had a test group (patients with confirmed COVID-19), while only five studies included a control group of SARS-CoV-2-negative participants [22, 25, 26, 31, 33]. Patients in the test group were diagnosed using the RT-PCR test. The specificity and sensitivity of lung US were estimated using RT-PCR in six studies [22, 24–26, 31, 33], clinical and laboratory data in two studies [28, 29], and chest CT in seven studies [19, 20, 23, 27, 30, 32, 34] as a reference standard.

Risk of bias. The main sources of bias came from the patient selection domain (Figure 3). The majority of studies (75%) included previously diagnosed patients. However, in all the studies, the participants met the criteria of the review protocol. The specialists performing lung US and analyzing the results were not blinded to the diagnosis, which could also be a potential source of bias.

 

Figure 3. Bar chart of risk of bias for the 16 included studies.

 

Seven studies properly reported the details of both index and reference standard tests. The interobserver variability was estimated only in three studies [21, 25, 32]. Three studies only (19%) reported the interval between the two tests, but the majority (87%) correctly indicated whether all patients used the same reference standard.

Scoring systems. The included studies used different scoring systems to assess the presence and severity of the disease. Dividing the imaging zone into separate regions, and providing a score reflecting the degree of pulmonary involvement to each region was common to most systems (87%). The total lung US score was calculated as the sum of individual scores. The most popular scoring system divided each hemithorax into six regions, with each region scored on a scale from 0 to 3, and a total score ranging from 0 to 36 [19, 20, 23, 24, 28, 30]. Three studies collected the lung US results from eight zones [27, 32, 33] but used a different scoring approach. While two groups scored each zone on a scale from 0 to 3 (total value 0–24) [27, 32], Favot et al. analyzed the lung US images for the presence of different patterns [33]. Two studies divided the chest wall into ten zones but used different severity scales with a maximum value of 40 [29] or 10 [34]. Yassa et al. collected the scores in a range from 0 to 3 from 14 zones (total value 0–42) [25, 26]. Finally, two groups performed a qualitative assessment of the lung involvement based on the US findings [21, 22].

Diagnostic accuracy of lung US. All included studies reported the lung US sensitivity and specificity values, with sensitivity ranging from 15.6% to 100% and specificity ranging from 51.9% to 100%. However, only three studies estimated the diagnostic performance of a reference standard test [23, 27, 32]. For the estimating of values in the review, we used the meta-analysis data on the sensitivity and specificity of RT-PCR [35] and chest CT [36]. For the studies using clinical and laboratory data as a reference standard test [28, 29], the control specificity and sensitivity values were set at 100% (Figure 3).

 

Figure 4. Forest plots of pooled specificity (A) and sensitivity (B). The symbols * and ** denote studies by Yassa et al. on interobserver agreement [25] and the role of lung US in COVID-19 screening [26], correspondingly.

 

According to the meta-analysis results, lung US has a specificity 81.6% ± 13.3% and sensitivity 79.4% ± 21.4% in diagnosing COVID-19. However, the Cochran’s test revealed a significant heterogeneity of the data: Q = 2244.8, p < 0.001, and Q = 1127.7, p < 0.001, for sensitivity and specificity, correspondingly.

The observed heterogeneity could be associated with the fact that the included studies assessed the diagnostic value of lung US for different purposes and cohorts of participants. For further analysis, we excluded the study by Veronese et al.[24]. We divided the remaining studies into two groups: in the first group, the researchers used US to screen for COVID-19 [19, 21–23, 25, 26, 31, 32], in the second, they used US to evaluate and follow-up critically ill patients [19, 27, 30, 32, 33]. We also did not include the studies by Lichter et al. [28] and Zhao et al. [29] in the second group, because the authors estimated the prognostic value of lung US to predict mortality and refractory situation, correspondingly. Lichter et al. reported a 62% sensitivity and 74% specificity in the ROC analysis of 30-day mortality, the cut-off value for lung US score was 18 (maximum value 32) [28]. According to Zhao et al., using the lung US score cut-off value of 32 points (maximum value 40) predicted a refractory situation with a 57% sensitivity and 89% specificity [29].

The index test characteristics remained heterogeneous, with the lowest Q-statistic and variability percentage obtained for lung US sensitivity in critically ill patients (Table 1).

 

Table 1. Lung US efficiency for patients with COVID-19

Group	Sensitivity		Q	I2, %	Specificity		Q	I2, %
	Mean, %	SD, %			Mean, %	SD, %
Screening	79,6	21,6	694,2	99,0	79,5	16,1	345,0	98,0
Severe	87,6	12,3	158,9	97,5	80,5	7,1	379,6	98,9
Moderate	72,8	7,1	11,24	91,1	74,3	2,7	0,26	0,0
Mild	80,4	16,5	59,5	98,3	66,6	27,0	33,3	97,0

 

We also pooled the sensitivity and specificity values for patients with different degrees of pulmonary involvement. From the data provided in the study by Lichter et al.[28], it was not possible to extract the numerical data to estimate the characteristics. Therefore, we did not include this study into the meta-analysis. In the study by Zieleskewicz et al., we obtained the sensitivity and specificity values with the maximum Youden index from the three zones on the ROC curve according to the lung US score thresholds [20].

The data was heterogeneous, except for the lung US specificity in moderately ill patients (Table 1). Note that we used the results for moderately and mildly ill participants from only two studies in this meta-analysis, and both of them did not include a control group of patients.

DISCUSSION

The variety of scoring systems in the included studies makes it impossible to directly compare the lung US score cut-off values used to estimate the outcomes. However, regardless of the scoring system, almost all authors agree that patients with severe disease had higher lung US score values than patients with moderate and mild disease. The first exception to this was the study by Veronese et al., where the authors did not find a significant difference in mortality risk between nursing home patients with a lung US score ≥ 4 and < 4 (maximum value 32) [24]. The authors did not interpret this observation, but we believe it is related to the general health of the nursing home residents, which were older adults, suffering from dementia, and bedridden. The other exception was the study by Benchoufi et al., which showed that the performance of the lung US scoring system used by the authors was lower to predict the disease classified as severe by chest CT compared with normal vs. pathologic and normal or mild vs. moderate or severe [32].

Overall, in confirmed symptomatic COVID-19 patients with severe disease, the lung US and CT scores positively correlated. According to our meta-analysis, lung US has a sensitivity of 88% and 80% specificity in this group (see Table 1). That is a specific cohort of patients, but for them, lung US has significant advantages compared with chest CT in terms of health risks and logistical limitations.

Low lung US scores were also valuable to exclude severe COVID-19-associated pneumonia. According to Zieleskewicz et al., chest CT would not be required if the initial US examination had a score <13 (out of 36) [20]. Lichter et al. reported that lung US could predict good clinical outcomes for symptomatic patients without any pleural thickening or subpleural consolidations [28]. Despite the relatively low efficiency of lung US in assessing mild lung lesions [19], this feature could have practical value for symptomatic patients in making triage decisions.

The highest discrepancy between the lung US and chest CT scores was observed for moderately ill patients. For this group of patients, lung US was least sensitive (see Table 1). Zieleskewicz et al., in their study, called the zone on the ROC curve from which we obtained the data, “a grey zone with inconclusive values” [20]. Therefore, despite the relatively modest statistical heterogeneity, the diagnostic value of lung US for moderate lung lesions is relatively low.

Screening for COVID-19 using lung US findings has several advantages in pregnant women. In the study of Yassa et al., 17% of the pregnant women, who had undergone a lung US exam and were RT-PCR-positive, initially had negative RT-PCR results. The RT-PCR test was repeated after a week due to their abnormal US findings [26]. Note that the specificity of lung US according to our meta-analysis, was significantly higher than the specificity of chest CT, a “gold standard” for medical imaging: 79% vs. 31%, correspondingly. It might be associated with the fact that most included studies were conducted in conditions of high pre-test probability. There was an evident patient selection and index test risk of bias that could affect the observed specificity value (see Figure 3).

Chest CT is superior to lung US in differential diagnostics of lung pathologies since it is sensitive for alternative diagnoses [37, 38]. Contrary to that, lung US cannot distinguish between pulmonary alterations: pneumonia, lung cancer, or atelectasis, which may show the same echographic pattern [11, 39]. Moreover, the accuracy of the lung US exam is highly dependent on the operator’s expertise level and could be affected by a pre-test probability of the disease. For example, in the study by Tung-Chen et al., three patients had lung US findings compatible with COVID-19; two patients were eventually diagnosed with viral bronchiolitis, and the other patient had metastatic pulmonary disease. The inter-rater agreement in the included studies, when reported, could be as low as 68%, which significantly reduces the applicability of the technique. However, a quick bedside lung US exam proved useful for real-time evaluation and monitoring of patients with rapidly progressing disease [19, 28].

Our study has limitations. Conventionally, at least five studies should be used for a meta-analysis. Although our final library contained 16 studies, the data was incomplete. For some analyses, we used the highly heterogeneous values obtained from only two studies. Significant data heterogeneity is also associated with the differing patient population, index test and reference standard protocols, and the outcome definitions across the included studies.

CONCLUSIONS

In 2020, several meta-analyses on lung US applicability for COVID-19 patients were published. All agree that the presence of lung US findings, although nonspecific, could be used for diagnosis, triage, and follow-up of the subjects with SARS-CoV-2 infection. Unfortunately, none of them focused on distinguishing between patients with different clinical status and prognosis. Chest CT is the gold standard in assessing the severity of the disease. However, depending on the patient cohort and the disease stage, other techniques could be advantageous. Lung US has adequate sensitivity and specificity for confirmed COVID-19 patients with severe lung involvement that have a risk of adverse events associated with transfer and exposure to ionizing radiation. Lung US is preferable for critically ill patients, pregnant women, children, and bedridden aged population. The technique is applicable for triage of patients with mild symptoms to rule out lung tissue damage. In patients with moderate disease, the diagnostic value of lung US is the lowest.

The high heterogeneity of the sensitivity and specificity values should be addressed in further studies. We believe that these studies need to be performed on large randomized cohorts of patients following a systematic protocol with clear and standardized definitions of the disease stages and including a control group of participants. Another issue that requires future research is the sensitivity and specificity of different scoring systems used to assess the severity of the disease.

ADDITIONAL INFO

Funding. The study had no sponsorship.

Conflict of interest. The authors declare no conflict of interest regarding the publication.

Authors contribution: Vetsheva N.N. — wrote the paper, performed the analysis; Reshetnikov R.V. — collected the data, performed the analysis, wrote the paper; Leonov D.V. — collected the data, performed the analysis; Kulberg N.S. — performed the data analysis and the proofreading of the paper; Mokienko O.A. — conceived and designed the analysis.

All authors made a significant contribution to the search and analysis work and preparation of the article, read and approved the final version before publication.

About the authors

Natalia N. Vetsheva

Research and Practical Clinical Center for Diagnostics and Telemedicine Technologies of the Moscow Health Care Department; Moscows regional research clinical institute n.a. M.F. Vladimirskiy

Author for correspondence.
Email: vetsheva@npcmr.ru
ORCID iD: 0000-0002-9017-9432
SPIN-code: 9201-6146

MD, PhD

Russian Federation, Moscow

Roman V. Reshetnikov

Research and Practical Clinical Center for Diagnostics and Telemedicine Technologies of the Moscow Health Care Department; Sechenov First Moscow State Medical University (Sechenov University)

Email: reshetnikov@fbb.msu.ru
ORCID iD: 0000-0002-9661-0254
SPIN-code: 8592-0558

PhD

Russian Federation, Moscow

Denis V. Leonov

Research and Practical Clinical Center for Diagnostics and Telemedicine Technologies of the Moscow Health Care Department

Email: d.leonov@npcmr.ru
ORCID iD: 0000-0003-0916-6552
SPIN-code: 5510-4075

PhD

Russian Federation, Moscow

Nikolas S. Kulberg

Research and Practical Clinical Center for Diagnostics and Telemedicine Technologies of the Moscow Health Care Department

Email: kulberg@npcmr.ru
ORCID iD: 0000-0001-7046-7157
SPIN-code: 2135-9543

PhD

Russian Federation, Moscow

Olesya A. Mokienko

Research and Practical Clinical Center for Diagnostics and Telemedicine Technologies of the Moscow Health Care Department

Email: o.mokienko@npcmr.ru
ORCID iD: 0000-0002-7826-5135
SPIN-code: 8088-9921

MD, PhD

Russian Federation, Moscow

References

Supplementary files