Tele-ultrasound imaging using smartphones and single-board PCs

Full Text

BACKGROUND

At present, mobile devices show higher performance with lower costs. These factors expand the scope of mobile device applications, including medicine and telemedicine. [1–3]

Teleultrasonography (or teleultrasound) is a unique example of telemedicine. [4] In this diagnostic technique, a functional diagnostics specialist or an ultrasound specialist receives and analyzes data from a remote ultrasonogram and sends back a medical report or recommendations. This process requires a special software and hardware complex.

Some cases of teleultrasound using a smartphone have been reported, [5–8] i.e., when an image from the screen of an ultrasound scanner was captured by a smartphone camera and transmitted using communication programs. No additional equipment is required, and this is the great advantage of this technique. However, some technical difficulties were reported, such as the need for additional staff or special devices for holding a smartphone when recording an examination and the lower quality of ultrasound images delivered to an expert compared with the original.

At present, there are mobile video capture devices that can be connected not only to a personal computer but also to mobile devices such as single-board computers (SBCs) and smartphones.

This study aimed to evaluate technical opportunities for implementing teleultrasound techniques using SBCs and smartphones.

MATERIALS AND METHODS

Software

The study used Sequoya 512 Acuson and SonoAce-8000 ultrasound scanners. One of the study authors volunteered to undergo a standard ultrasound examination of brachiocephalic arteries performed by a functional diagnostics specialist. Based on an expert review (three specialists with >10 years of experience), the image quality and possibility of interpretation were subjectively assessed. Thus, fragments of the examination provided were stored in an ultrasound scanner in the form of 5-s cine loops, which were then cyclically reproduced on the ultrasound scanner. A total of nine cine loops were recorded, three for each mode: B-mode only, B-mode + color Doppler mapping, and B-mode + spectral Doppler. The expert could evaluate the image on the monitor screen of the ultrasound scanner and screen of the client device in real time. Moreover, the above technique allowed experts to evaluate results obtained with different software and hardware systems during the same examination (on the same cine loop).

We used a Defender C-090 USB camera and low-cost video capture systems from an ultrasound scanner. Since lox-cost systems use two types of microcircuits (chipsets) as an analog-to-digital converter, i.e., UTV007 and AMT630A, both options were tested:

1) Gembird UVG-002, a video capture device based on the UTV007 chipset with a resolution of 720 × 576 at 25 frames per second,

2) Espada USB 2.0 –RCA/S-video EUsbRca63 (hereinafter referred to as EUsbRca63) is a video capture device based on the AMT630A chipset with a resolution of 720 × 576 at 25 frames per second.

The following software was used for the on-server study:

• VLC version 3.0.8 is a free video transmitting and playing software with the option to stream video from external devices. It runs under different operating systems, including Linux and Android. The study used this software simultaneously on the server and client of the telemedicine system.
• Motion Release 4.3.0, free software designed for CCTV cameras, focused on motion detection in a frame. This software runs under the Linux operating system and was installed on SBCs. It has the option of running as a background process. However, this software can only broadcast images without sound. Access required user identification and a password.
• USB Camera version 9.7.9. is freely distributed software designed to transfer images from USB cameras, and a non-commercial version was used to operate as a server on Android devices.

The main criterion for software selection was the ability to use various operating platforms. Then, these software and hardware solutions were assessed for suitability for use as a telemedicine system. Data transfer speed was evaluated (expressed as mean ± standard deviation). Microsoft Excel software was used for statistical processing of the results.

Telemedicine System

A telemedicine system consists of a server and a client. In this paper, a server is a software and hardware complex that transmits a video image from an ultrasound scanner. A client is a device that receives and plays video. The client and the server were connected through a local network using a Wi-Fi router with a bandwidth of 72 Mbps. During testing, the highest video signal quality (maximum resolution and maximum frame rate) was selected. The client was connected to the server using protocols supported by the installed software, such as HTTP and RTSP. The following client programs were required for video playback: VLC for RTSP, Google Chrome on Windows 7 and Android, and Chromium on Raspberry Pi for HTTP. Each software installed on the server supported its codec: JPEG (Motion), VideoH.264 (VLC), and H.264 (USB Camera).

Two connection options are available (Fig. 1).

1. The server is a smartphone (Android 7.0, 2 GHz 8-core processor, 3 GB RAM, or Onyx Max3 Android 9.0, 2 GHz 8-core processor, 4 GB RAM) connected to a video capture device via an OTG-USB cable. The video capture device is connected to the video output of the ultrasound scanner. The client is a laptop based on AMD E-450 APU, 8 GB RAM, 64-bit OS Windows 7, or a smartphone.
2. The server is a single-board microcomputer (Raspberry Pi 1 Model B, CPU 700 MHz, 512 MB or Raspberry Pi 4, CPU Quad Core Cortex 1.5 GHz, 4 GB with Linux version 4.19.118-v7+ installed) connected to the video capture device via a USB cable with a USB webcam. Raspberry Pi 1 did not have a Wi-Fi module and was connected to the router via a LAN cable. The client is a laptop based on AMD E-450 APU, 8 GB RAM, 64-bit OS Windows 7, or a smartphone.

 

Figure 1. Connection diagram.

 

RESULTS

Ultrasound scanners. Two video outputs were tested on the Sequoya 512 ultrasound scanner, including a coaxial output for connecting a video printer and an S-Video output for connecting a VCR. When the image was analyzed at the output of the video printer, a black-and-white image was obtained, partially going beyond the fields of video capture devices. Conversely, when connected to the S-Video output, a color image was obtained, corresponding to the original on the monitor screen of the ultrasound scanner. Sequoya 512 was used with Raspberry Pi-based systems. SonoAce-8000 was used to implement teleultrasound based on Android devices. As in the previous case, S-Video was selected for the video output.

UVG-002. This device was successfully detected on all mobile devices and ran normally with all tested programs. It provided the declared maximum resolution on all devices. The maximum frame rate was reached on all devices, except for Raspberry Pi 1.

EUsbRca63. This device was successfully identified on all mobile devices and operated normally with all tested programs, except for VLC on Raspberry. The declared maximum resolution was not achieved on any of the devices. The maximum resolution was 640 × 480.

USB camera. The USB camera was successfully detected on all mobile devices and ran normally with all tested programs. The maximum resolution was 640 × 480, and the frame rate was 30 per second.

Motion software. It has been configured to run as a server: continuous recording and transmission of images from video devices with maximum resolution. Owing to the low performance, Raspberry Pi 1 was able to produce 1–1.5 frames per second. Raspberry Pi 4 had none of these problems; thus, it was possible to run several processes simultaneously with the same quality of the broadcast video.

VLC software. The successful installation on Raspberry Pi 4 allowed streaming from external video capture devices. We also successfully implemented two VLC applications simultaneously with video capture from a video capture device and a webcam, whereas the image quality and frame rate corresponded to the maximum. In addition, we tested the ability to run VLC and Motion simultaneously on a Raspberry Pi 4. VLC was installed on Android devices; however, in this version, it did not allow the operation of a server but only functioned in client mode. In the client mode, it was also used on a Windows laptop.

USB-Cam software. The implementation of a telemedicine server based on this software was extremely sensitive to the quality of the communication channel. Therefore, at a signal level below 80 dB, it was disconnected, and there was no difference in which side the low quality of the connection was from, the server or the client side.

DISCUSSION

Based on the study results, we prepared a list of devices and software solutions for the implementation of mobile teleultrasound systems. The test results are presented in Table 1: a video capture device based on the UTV007 chipset allows us to obtain a subjectively better image than a device based on the AMT630A chipset. The UVG-002 device can operate on all platforms and with all the used software. This demonstrated a high-quality picture.

 

Table 1. Test results for software and video signal devices

Parameters		Motion	VLC	USB-Cam
Codec		JPEG	VideoH.264	H.264
Connection protocol		HTTP	HTTP, RTSP	HTTP
Transfer rate, Mbps	UVG-002	18,7±2,8	0,64±0,17	5,2±0,3
	EUsbRca63	8,6±1,4	-	4,4±0,2
	Webcam	15,3±5	0,49±0,19	2,0±0,1

 

The EUsbRca63 video capture module showed satisfactory performance on mobile devices. We were unable to set this module for operating with VLC, even though Motion EUsbRca63 smoothly ran with the same driver. The quality of the resulting ultrasound image was extremely low on all platforms: the inscriptions on the image were illegible, and adequate assessment of the ultrasound image of such quality was impossible.

For optimal software selection, the platform planned to be used with the teleultrasound server was determined. Therefore, in the case of an Android-based smartphone, USB-Cam is the only software considered; however, in this variant, teleultrasound will require a special bandwidth of the communication channel. In our case, the channel capacity for transmitting ultrasound data was 4.4–5.2 Mbit/s, which is practically difficult to achieve using cellular networks. Conversely, when connecting to Wi-Fi, this requirement for the bandwidth of the communication channel becomes less important. During the testing, this software and hardware complex was highly sensitive to the quality of the Wi-Fi signal (i.e., distance from the access point). Given these facts, opportunities for the implementation of teleultrasound in mobile networks are seriously limited.

At present, SBCs are increasingly and actively used in telemedicine. [9, 10] We consider recent SBCs to be the optimal platform for the implementation of teleultrasound. The tested Raspberry Pi models showed a significant increase in computing power, with the first generation of Pi 1 being only able to transmit video at a speed of ≥1.5 frames per second (in our study, the minimum frame rate on the ultrasound scanner was 16 frames per second), which is thought to be insufficient to implement a teleultrasound server. However, this device may act as a client in this system. Raspberry Pi 4 demonstrated high performance by simultaneously broadcasting two video streams.

The increased computing power of SBCs allows the implementation of machine-learning technologies, [11, 12] including in healthcare. [13] The available computing power of an SBC may be sufficient for use with a medical decision support system in parallel with the teleultrasound server for evaluating ultrasound images for abnormalities.

The choice between Motion or VLC must be guided by the bandwidth of the communication channel. Therefore, our data show that one video stream for VLC required 0.5–0.6 Mbps, and for Motion, the bandwidth should be at least 10 times greater. The literature review demonstrates that a connection speed from 0.6 [14] to 1.5 Mbps [15] is the minimum allowable for comfortable work of a remote expert, provided that the frame rate is 15 per second and the original video resolution is preserved. If it is necessary to transfer only the image from the screen of the ultrasound scanner and there is no limitation on the bandwidth of the communication channel, the Motion software is the best solution because it allows autonomously starting and deploying the teleultrasound server without human intervention. In other cases, VLC may be selected. Our study also revealed the possibility of simultaneously using two different systems.

In a previous paper [16], we showed the possibility of using streaming technologies in a PC-based teleultrasound system. However, the use of smartphones and SBCs makes teleultrasound an even more mobile technique. A device for a teleultrasound server with the necessary cables fits easily into a doctor’s pocket, allowing him/her to keep the connection kit ready and, if necessary, organize broadcasting the examination within seconds.

Study Limitations

In this study, only two models of ultrasound scanners were analyzed. However, this technical solution can be also suitable for other ultrasound scanners that have similar or different supported video outputs. This study included only a small number of modern mobile devices, video capture devices, and software for working with video capture devices and webcams; however, we created several functioning low-cost systems for mobile teleultrasound. We focused on searching for the most affordable way to perform teleultrasound. During the study, the cost of a mobile device (Raspberry Pi4) and a video capture module did not exceed 6,000 rubles. At present, a similar package costs from 12,000 rubles. The software used (Motion, VLC, and USB-Cam) had a free license or was distributed with an open-source code. It is assumed that the operating system (Windows, Linux, and Android) is installed on the devices as part of the telemedicine system.

For such testing, the bandwidth [17] and security of communication channels used for healthcare data transmission must be considered. [3] The organization of VPN tunnels is one of the possible ways to ensure the security of transmitted data. However, the study of information security during teleultrasound is not the goal of this paper. This study is unique in that it proposes a new technical solution for teleultrasound.

Portable ultrasound systems are now available, including those with the function of image transmission (teleultrasound), such as Butterfly iQ + Butterfly Network Inc., Kosmos EchoNous, Vscan Air General Electric, and Lumif Philips Healthcare. However, these systems are expensive (>500,000 rubles) and do not have all the options necessary for a complete examination using a stationary ultrasound scanner. [18] The advantage of our approach is the use of ultrasound equipment, which is part of the standard ultrasound equipment.

CONCLUSION

The use of devices based on SBCs and smartphones allows the creation of a low-cost teleultrasound system, which potentially contributes to improving the quality of examinations performed for distance learning and consulting HCPs. These solutions can be also used in remote regions as a part of the “field” of medicine and other possible areas of mobile healthcare.

ADDITIONAL INFORMATION

Funding source. This study was not supported by any external sources of funding.

Competing interests. The authors declare that they have no competing interests.

Authors’ contribution. All authors made a substantial contribution to the conception of the work, acquisition, analysis, interpretation of data for the work, drafting and revising the work, final approval of the version to be published and agree to be accountable for all aspects of the work. K.M. Arzamasov — research design development, research volunteer; K.M. Arzamasov, T.M. Bobrovskaya — data analysis; K.M. Arzamasov, T.M. Bobrovskaya, V.A. Drogovoz — data interpretation; K.M. Arzamasov, V.A. Drogovoz — writing the text of the manuscript.

Acknowledgments. The authors are grateful to collaborators of Moscow Center for Diagnostics and Telemedicine: doctor of functional diagnostics, MD PhD D.V.Shutov and the leading researcher, MD PhD A.E. Demkina for their help in this study.

About the authors

Kirill M. Arzamasov

Moscow Center for Diagnostics and Telemedicine

Author for correspondence.
Email: ArzamasovKM@zdrav.mos.ru
ORCID iD: 0000-0001-7786-0349
SPIN-code: 3160-8062

MD, Cand. Sci. (Med.)

Russian Federation, Moscow

Viktor A. Drogovoz

Scientific and Production Association “Russian Basic Information Technologies”

Email: Vdrog@mail.ru
ORCID iD: 0000-0001-9582-7147
SPIN-code: 1804-2636

Cand. Sci. (Tech.)

Russian Federation, Moscow

Tatiana M. Bobrovskaya

Moscow Center for Diagnostics and Telemedicine

Email: BobrovskayaTM@zdrav.mos.ru
ORCID iD: 0000-0002-2746-7554
SPIN-code: 3400-8575

MD

Russian Federation, Moscow

Anton V. Vladzymyrskyy

Moscow Center for Diagnostics and Telemedicine; The First Sechenov Moscow State Medical University

Email: VladzimirskijAV@zdrav.mos.ru
ORCID iD: 0000-0002-2990-7736
SPIN-code: 3602-7120

MD, Dr. Sci. (Med.)

Russian Federation, Moscow; Moscow

References

Supplementary files