Streaming technology: from games to tele-ultrasound

Full Text

BACKGROUND

Tele-ultrasound is a new technique for ultrasound examination employing telemedicine. Digital ultrasound images from this examination are forwarded to an expert for remote evaluation. Applications for ultrasound systems with built-in remote consultation functionality are expanding constantly. However, updating the ultrasound diagnosis equipment remains an urgent challenge [1, 2].

A standard smartphone can be used to get a telemedicine consultation based on ultrasound findings, by sending an image or a cine-loop captured on the built-in camera [3–5]. However, because the message would contain personal data, this method is not always practical and quite risky.

The majority of ultrasound scanners older than 10–20 years have a video output for an external display and/ or video printer. Gaming solutions (specific devices for in-game audio and video capture [6]) and live game streaming software may increase the functionality of outdated diagnostic equipment.

The aim of the study is to investigate the possibility of applying the gaming industry’s information technology in telemedicine, like tele-ultrasound.

MATERIALS AND METHODS

The study subject was one of the authors who underwent ultrasound of the neck vessels in 2019. Greyscale B-mode, color duplex, pulse Doppler, and combinations thereof (greyscale B-mode + color duplex; greyscale B-mode + pulse Doppler) were the imaging modes that were used. Sequoia 512 Acuson ultrasound scanner was used. Images from the ultrasound scanner were recorded using Ezcap 295 HD video capture system (http://www.ezcap.com).

OBS Studio 24.0.3 by Open Broadcaster Software (http://www.obsproject.com) was used for remote communication of the study findings. OBS Studio is a free open-source software for video recording and streaming. Twitch (http://twitch.tv) was used as a streaming service. A personal computer with the following characteristics was used as a video capture workstation: AMD Ryzen 5 3400G processor, 16 GB RAM, NVIDIA GeForce RTX 2070 graphics card, Windows 10 Pro 64-bit OS, LAN 100 Mbs, BenQ 21.5” IPS monitor, 1920 × 1080. An expert used a laptop with the following characteristics for remote evaluation: AMD E-450 APU, 8 GB RAM, Radeon HD7470 graphics card, Windows 7 64-bit OS, WiFi 72 Mbs, 15.6” monitor, 1366 × 768. Redmi Note 4 smartphone with the following characteristics was used as a mobile device: 3 GB RAM, 5.5” IPS monitor, 1920 × 1080, 401 ppi, 4G Internet. Prestige 338 video recorder, 1920 × 1080, 25 fps, webcam mode, was used for recording the findings of ultrasound examination.

The image quality was evaluated by three functional (ultrasound) diagnosis professionals with a combined experience of more than 10 yr. The quality of the study was considered sufficient if an expert could assess the segmentation of the intima-media complex correctly.

Microsoft Excel was used for statistical processing of the results. The measured parameters were presented as mean ± SD. The correlation of the parameters was assessed by the Spearman’s rank correlation coefficient, since the data were nonparametric.

Figure 1 shows the flow chart of tele-ultrasound using streaming technology.

Fig. 1. Tele-ultrasound flow chart. The ultrasound system, to which Ezcap 295 HD (video capture device) is connected via S-Video (video output standard), which in turn is connected via USB to the workstation. The webcam is also connected to the workstation via USB. Video/audio streaming is performed via the video streaming service. The clients are a laptop connected to the Internet via Wi-Fi and a 4G-smartphone.

The expert opinion was based on a binary scale: If all three experts agreed that the server-side and client-side video quality were comparable, the evaluation was positive. If at least one expert concluded that the video quality did not match the original, the evaluation was negative. For the quality evaluation, each expert viewed the server-side and client-side image at the same time. The experts performed the quality evaluation independently.

RESULTS

The connection pattern was as follows: the ultrasound scanner’s video output was connected to Ezcap 295 HD video capture board’s input, which was then connected via USB to the video capture workstation with OBS Studio installed. An S-Video VCR output was used as the video output of the ultrasound scanner. On the video capture device, the corresponding video input was utilized. The maximum settings were set up by hardware: FULL-HD (1920 × 1080), 30 fps (the frame rate could not be changed and corresponded to the factory settings). Ezcap 295 HD was automatically detected on the video capture workstation once the video capture device driver was installed, and it was added as the video source in the OBS Studio. The image quality on the video capture workstation display was compared with the original quality on the ultrasound scanner display. Three experts evaluated the image, and they all came to the conclusion that the image quality on the video capture workstation display corresponds to that of the ultrasound scanner display.

Proper operation of tele-ultrasound requires two video streams (from the ultrasound machine and from the webcam pointed at the anatomical area under examination). These two video streams are merged in the video capture workstation and transmitted to a remote server. To test this feature, a webcam with 1920 × 1080 source image size was connected. To avoid interference, the image from the webcam was reduced to 640 × 360 by bicubic interpolation and superimposed on top of the image from the ultrasound scanner in the corner, outside the workspace.

The video capture device had a fixed resolution where the analog signal was digitized. The resolution of 1080p is unnecessarily large for the ultrasound scanner video output. As a result, bicubic interpolation the image size was used to reduced image size to 1280 × 720 by to save traffic during video streaming. The experts found no visual differences between the images on the video capture workstation display before and after interpolation.

The following step included configuring OBS Studio to stream video on Twitch and the streaming. The image quality was evaluated on remote devices (laptop and smartphone) connected to the video streaming web service. Nvidia NVENC hardware-based multithreaded encoder was used for video encoding in the video capture workstation. The video quality was set to maximum. These settings ensured CPU load not more than 15% and GPU load not more than 30%. The only parameter that required adjustment was the video bitrate. A constant bitrate was selected to ensure uniform video quality.

In this study, we searched for the lowest possible video bitrate for reliable remote evaluation by an expert. We decided to begin the video stream at 3,000 Kbps, with a subsequent decrease of the bitrate to the minimum level, where, in the view of the expert, the ultrasound image quality would differ significantly from the original image. Each bitrate value was subjected to three measures, with a mandatory stop and restart of streaming in between each measurement. The stream contained cine-loops in greyscale B-mode, cine-loops in duplex mode (greyscale B-mode + color duplex or greyscale B-mode + pulse Doppler), and freeze frames, including those with ongoing measurements.

The expert opinion was based on a binary scale: The evaluation was positive if all three experts agreed that the server-side and client-side video quality was similar. The evaluation was negative if at least one expert concluded that the video quality did not match the original. For the quality evaluation, each expert viewed the server-side and client-side image at the same time. The experts performed the quality evaluation independently. Table 1 shows the results of tele-ultrasound system testing depending on bitrate settings.

Table 1. Results of remote testing of the tele-ultrasound system by experts

The minimum bitrate of 200 Kbps resulted in a blurred image. The experts concurred that the image was not suitable for proper evaluation of ultrasound structures. Only freeze frames exhibited decent image quality at the 300 Kbps bitrate. According to the study, the ultrasound image was less clear when viewed on a laptop and more clear when viewed on a smartphone. For all devices and modes, the ultrasound image quality was satisfactory at 350 Kbps bitrate and above.

We also evaluated the time lag between server-side and client-side videos. Depending on the bitrate, the time lag was 3.94–4.92 s. As the bitrate decreased, so did the time lag (correlation coefficient: 0.82).

DISCUSSION

Currently, telemedicine research and consultations can be employed in areas where there is severe lack of medical professionals, particularly specialized ones, as well as in facilities with outdated equipment, as in the case of this study. There should also be consideration for the accessibility of mobile communications in our country. Therefore, using a smartphone is appropriate and enables speedy connection to a teleconference, particularly for remote counseling in case of emergency.

We were able to do an ultrasound with remote analysis by expert doctors using the Ezcap 295 HD, which the manufacturer markets as a video capture device for video games. This device provided high quality of video images required for efficient work of ultrasound diagnosis professionals.

We have chosen OBS Studio by Open Broadcaster Software as one of the well-liked free open-source streaming software tools for gaming. This software instantly detected and connected to Ezcap 295 HD. Twitch, which is marketed as a video game streaming service, was selected to test the streaming. In this study, we only used it to test the operability of the proposed system. Thus, the findings of this study only support the possibility of using the OBS Studio software for the purposes described above. It is possible to stream video to any video streaming platform with OBS Studio (according to the information on the site).

For the future implementation of tele-ultrasound, we advise using video streaming platforms that offer private streaming protected from unauthorized connection and viewing. The study did not assess such platforms. However, when it comes to remote research in general and tele-ultrasound in particular, information security is essential. Tele-ultrasound is used to stream two forms of confidential data. These are medical and personal data, the security of which must be guaranteed in accordance with the current rules and requirements of the Russian law. Private health information, such as medical data exchanged back and forth between the client and the server, should not be disclosed to third parties in any way. We advise that anonymized medical data (images from an ultrasound scanner) and associated medical documents (including personal data) should be transferred as two different streams using different security and encryption algorithms in order to increase the security of research. In the future, the transmission of anonymized ultrasound frames with a user identifier (UID) may be considered. The UID can be matched with the patient’s name and other personal data in the health information system database or using hardware/software security gateways (e.g., Vipnet Coordinator).

In this study, we assumed that the ideal bitrate for H.264 video streaming at 1280 × 720 would be 3000 Kbps [7]. However, the tests revealed that the image quality did not change significantly when the bitrate was decreased down to 350 Kbps. This can be explained by the fact that most of the image streamed from the ultrasound scanner is static. According to a detailed analysis of the ultrasound scanner’s image, the area of interest on the original image does not exceed 880 × 822. After bicubic interpolation, the area of interest does not exceed 586 × 548, which requires a three times lower bitrate. Furthermore, there are fewer bits needed to encode the ultrasound image because it is not in color. The color duplex mode has a limited color area/chart, which also ensures good performance at low bitrates.

We believe that we have selected the best possible video streaming settings for the study: 1920 × 1080 and 25 fps for the input image from the ultrasound scanner; 1920 × 1080 and 24 fps for the input image from the webcam; video output 1280 × 720, 24 fps; H.264 encoder (Nvidia NVENC); and minimum bitrate of 350 Kbps.

In conclusion, it can be claimed that the suggested settings enable stable transmission of a high-quality video image from an ultrasound scanner to any client device, including a smartphone, with a cellular Internet connection. This enables the technology to be used for the streaming of ultrasound findings via VSAT space communication technologies. Tele-ultrasound cannot be negatively impacted by a video broadcasting delay of less than 5 s.

In the literature, there are reports on the use of platforms for mass communication (e.g., Voip), which are highly efficient when employed in tele-ultrasound [8]. For example, A.S. Liteplo et al. [3] compared the most widely used Voip platforms, Skype, and iChat. The authors concluded that iChat was superior to Skype. Later, it was demonstrated that Skype could be used effectively as well [4, 9]. Additionally, it has been demonstrated that using Apple FaceTime for tele-ultrasound can be a good option [10].

The aforementioned technologies require the installation of specialist software and make connecting clients much more challenging if there are several clients. Our findings show that ultrasound video streaming technology enables viewing the video on any Internet-connected device without the need for client-side installation of specialist software. The proposed technology allows an unlimited number of users to view the streamed data. This may be useful for multidisciplinary team meetings and consultations involving specialists and experts from various health facilities, including those in remote areas, as well as distance education of medical personnel. We can also assume that improved image quality on an ultrasound scanner will improve the image quality on a laptop or smartphone. Thus, we used video capture at 1080p, the highest resolution possible for a video capture device. Technically, an ultrasound machine with a digital video interface (HDMI, DVI-D, and DP) and high resolution (Full HD and higher) can be used together with the video capture device under consideration.

Furthermore, this study differs from similar ones in that it used software and technical solutions developed for the gaming industry, which are outside the scope of other researchers. Nonetheless, we were able to show that these solutions could be successfully applied to address medical issues.

Modern ultrasound scanners can transfer the research findings in digital format, and some models can perform tele-ultrasound without the use of additional devices; however, many devices still lack these capabilities and have lower ultrasound image quality. The methodology we describe may be appropriate for health facilities with low-cost or outdated equipment. The display resolution for ultrasound examination recommended by the American Association of Physicists in Medicine and the Society for Imaging Informatics in Medicine is 3 Mp (2048 × 1536) [11]. The described technique can be used with ultrasound scanners of a higher class when using video capture devices that support this resolution.

Study limitations

We can assume that this technical solution is also applicable to other ultrasound scanners with similar or different outputs supported by a video capture device although this study only evaluated one ultrasound scanner.

There has only been one video capture device tested. We believe that a video capture device with performance specifications non-inferior to those of the Ezcap 295 HD can produce a comparable result.

In this study, the quality of ultrasound images was evaluated solely based on subjective criteria. We accepted this limitation because all ultrasounds are a priori subjective and involve a human factor present at every stage, from ultrasound image display to assessment of ultrasound findings.

Finally, this study did not provide for testing of other video streaming platforms.

To date, numerous codecs (devices or software for data/signal conversion) have been developed, but in this work, we used H.264. Noise suppression, which is common in the H.264+, H.265, and H.265+ codecs, can negatively affect the ultrasound image quality, necessitating additional research to evaluate the changes made to the quality of transmitted ultrasound signal.

CONCLUSION

Video game streaming technologies can be used in telemedicine, for example, for on-site tele-ultrasound or mobile hospitals that have a portable ultrasound machine with a video output is available. The advantages of these technologies are their availability and high quality of the broadcast video image while using a minimum bandwidth of communication channels. In addition, this ultrasound technology can be used for distance learning or remote counseling.

Given that it is intended to transfer medical data for full-fledged work in a clinical setting, the issue of communication channel security from unauthorized access to transmitted medical information must also be addressed.

ADDITIONAL INFORMATION

Funding source. This study was not supported by any external sources of funding.

Competing interests. The authors declare that they have no competing interests.

Authors’ contribution. All authors made a substantial contribution to the conception of the work, acquisition, analysis, interpretation of data for the work, drafting and revising the work, final approval of the version to be published and agree to be accountable for all aspects of the work.

К.М. Arzamasov ― research design development, volunteer during research; К.М. Arzamasov, Т.М. Bobrovskaya ― data analysis; К.М. Arzamasov, Т.М. Bobrovskaya, V.А. Drogovoz ― data interpretation; К.М. Arzamasov, V.А. Drogovoz ― writing a manuscript.

Acknowledgments. The authors express their gratitude to the Head of the Department of Functional Diagnostics of the SCC of JSC “Russian Railways” S.V. Ivanov for his assistance in organizing and conducting the study on the basis of the SCC of JSC “Russian Railways”, and also to the doctor of the Department of functional diagnostics of the SCC of JSC “Russian Railways” E.V. Andreeva for assistance in conducting the study. The authors express their gratitude to graphic designer T.A. Savosina for creating an illustration for the article, as well as M.V. Vlasova for the translation.

About the authors

Kirill M. Arzamasov

Author for correspondence.
Email: k.arzamasov@npcmr.ru
ORCID iD: 0000-0001-7786-0349
SPIN-code: 3160-8062

MD, Cand. Sci. (Med.)

Tatiana M. Bobrovskaya

Email: t.bobrovskaya@npcmr.ru
ORCID iD: 0000-0002-2746-7554
SPIN-code: 3400-8575
Russian Federation, Moscow

Viktor A. Drogovoz

Email: Vdrog@mail.ru
ORCID iD: 0000-0001-9582-7147
SPIN-code: 1804-2636

Cand. Sci. (Technical)

References

Supplementary files

Supplementary Files

Action

1. JATS XML

Download

2. Fig. 1. Tele-ultrasound flow chart. The ultrasound system, to which Ezcap 295 HD (video capture device) is connected via S-Video (video output standard), which in turn is connected via USB to the workstation. The webcam is also connected to the workstation via USB. Video/audio streaming is performed via the video streaming service. The clients are a laptop connected to the Internet via Wi-Fi and a 4G-smartphone.

Download (166KB)

Indexing metadata