Design and Implementation of a Digital Teleultrasound System for Real-Time Remote Diagnosis John W. Sublett Bert J. Dempsey Alfred C. Weaver { sublett,bert,weaver} @Virginia.EDU Computer Science Department University of Virginia Charlottesville, VA 22903, U.S.A. Abstract We present the design and implementation of a digital image capture and distribution system that supports remote ultrasound ezaminations and, in particular, real-time diagnosis for these ezaminations. The system was designed in conjunction with radiologists and staff in the Department of Radiology at the University of Virginia hospital. Based on readily available microcomputer components, our teleultmsound system handles the acquisition, digitizing, and reliable transmission of still and moving images generated by an ultrasound machine. The digital images have a resolution of 640A80 with an 8-bit color plane, can be captured at rates up to 30 frames/sec, and are compressed and decompressed in real-time using specialized hardware. While scalable to communications networks of any transmission speed, initial deployment is envisioned for 1.5 Mbit/s T-1 leased lines. To achieve realtime still image distribution and to reduce the bandwidth necessary for motion video, the teleultrasound design employs lossy image compression based on the JPEG standard 141. The effects of JPEG compression on diagnostic quality are being studied in a sepamte signal detection study with the Department of Radiology at the University of Virginia. 1: Introduction Health care reform is a critical social issue today. While much emphasis is given to the need for quality health care for all citizens, the high costs of medical care under the current system make universal health care a difficult challenge. The most advanced equipment and highly trained professionals are generally located in large urban hospitals in order to ensure cost-effective use of these expensive resources. However, uniform health coverage requires that high quality care be readily available to all citizens, including those people living in smaller cities and rural areas. Under the current system, providing access to medical specialists for patients in sparsely populated areas often results in inconvenience and/or inefficient use of resources. Most commonly, a patient seeking care will travel to see a medical specialist at the urban hospital. These trips can require hours of travel time for a relatively short examination, and thus are neither convenient nor an efficient use of the patient s time. A different approach is to have the physician travel to remote clinics as a circuit rider. This solution wastes the physician s time with travel. A third conventional approach has been to provide medical imaging capabilities at outlying health centers, but then physically transport these images 292 1063-7125/95 $4.00 0 1995 IEEE
SAlA - DesigdMethodology 293 via a courier service to a central site for reading. The main drawback here is the long turn-around time, generally on the order of a few days to a week, before the patient and his local care providers learn of the results of the examination. Advances in computer networking offer an attractive alternative to these conventional scenarios: digital transport of medical imagery from outlying clinics to central reading sites. Transmitting images not only leads to less travel time for the physician and patient, but also enables faster, more efficient diagnostic procedures. With digital image transport, the specialist who interprets the images has access to the images very quickly. In some medical application domains, e.g. ultrasound, diagnosis immediately following image acquisition is feasible since the imagery has relatively low resolution and thus can be sent across the network rapidly. In this paper we describe the design and implementation of a PC-based digital teleultrasound system for real-time remote diagnosis. Because ultrasound uses relatively low resolution images, the rapid distribution of ultrasound images is feasible over network links with modest bandwidth. Since images are digitally encoded, the system can take advantage of the powerful flexibility afforded by digital representation, e.g., image enhancement techniques, multi-resolution encoding, and so on. By providing images to the remote radiologist as they are captured in an examination, our teleultrasound system enables the radiologist to provide interactive feedback to the technician during an examination. Such a dialogue between technician and radiologist will, we believe, reduce the likelihood of requiring a repeat visit by the patient due to the lack of a conclusive diagnosis from the first image set. Repeat examinations may be required when images are read offline. Our project to design and build a prototype for teleultrasound was driven by the needs of the Department of Radiology at the University of Virginia. In an effort to be a health care provider to a wider surrounding area, the University of Virginia hospital, located in Charlottesville, is moving towards supporting several outlying clinics. Three of these clinics are nearby, i.e., within 5 km of the hospital, while other sites are as much as 80 km away. Due to the low workloads at individual outlying centers, staffing each of them with a fulltime radiologist is not feasible. Our project was designed to explore the technical feasibility of providing a low-cost teleultrasound system for real-time examination and diagnosis to a clinic staffed with an ultrasound technician, an ultrasound machine, and a leased-line network connection, e.g., a 1.5 Mbit/s T-1 line. The completed prototype for this project confirms the feasibility of such a system today while incorporating the flexibility to accommodate faster networks and higher resolution imagery as new technologies become available in the future. 2: Teleultrasound System Since user acceptance is essential for a successful project, our design phase centered around a careful study of the current ultrasound examination paradigm used within the hospital at the University of Virginia. In this section we briefly discuss our findings and then outline the teleultrasound examination routine. In a typical in-hospital ultrasound examination, a trained ultrasound technician sits with the patient in an examination room and manipulates a transducer connected to an ultrasound machine. The transducer is passed over the areas of interest on the patient, and the images generated appear on a small analog display. When the technician observes an image that is of interest, he selects individual freeze frames and records them on radiographic film, and during an examination several such films are collected. At the conclusion of the session, the films are developed and placed on a lightboard where a radiologist views them
294 Eighth IEEE Symposium on Computer-Based Medical Systems Ultrasound Machine Patient U Technician Radiologist Flgure 1. Teleultrasound Examlnatlon. for diagnosis. According to our consulting radiologists, many diagnoses can be made from this initial set of films. However, in some cases, the first examination does not provide sufficient information for conclusive diagnosis, and a second set of images must be taken. The decision to perform a second examination can take many minutes. The film for each image study takes a few minutes to develop, and the radiologist is typically dealing with multiple patients at the same time. After the initial examination has been performed, the patient must remain in the ultrasound examining room in case a second image set is needed. In addition to the still images, it is general practice to record the entire examination onto a videotape. This is a simple process since the ultrasound machine has an NTSC output (standard TV signal). If the still images from the second examination do not yield a clear diagnosis by themselves, the radiologist may use the videotape to aid in diagnosis. In our experience, the value of the motion video sequences varies with the individual radiologist and with the nature of the examination. In any case the complete set of information gathered in an in-hospital ultrasound examination is a set of still images along with a videotape of the examination, and clearly any new system should provide a similar set of information. Figure 1 illustrates the examination scenario under our digital teleultrasound system. The patient is located at a remote site, e.g., a clinic or ambulatory care center. On location with the patient is an ultrasound technician, an ultrasound machine, and a transmitting workstation. A radiologist located at a central hospital is linked with the remote site by a data network and ordinary telephone service. When the examination begins, a network connection is opened between the transmitting workstation at the remote clinic and the receiving workstation at the hospital. In addition, at some point, the radiologist and the technician initiate an audio link using the telephone. Just as with in-hospital procedures, the technician scans the patient and captures still images with a simple button press. However, instead of being stored on film,each image is immediately digitized, compressed, and transmitted across the network link. The image is received for viewing by the radiologist almost instantaneously, with the exact speed dependent on the bandwidth of the communication link and the amount of compression.
SAlA - Desigdhlethodology 295 Based on the still images received, the radiologist can, if necessary, instruct the technician on what additional imagery to capture. Besides the still images, the ultrasound technician captures a small set of motion video sequences for transmission to the radiologist. As with the still images, the video is digitized and compressed as it is acquired. Unlike the still images, however, the video sequences are stored locally on the transmitting workstation and transmitted to the radiologist only at the end of the examination period. Delaying transmission of the video sequences ensures that the transmission of still images is very fast and reflects the secondary importance of video in certain diagnostic scenarios. As mentioned earlier, during in-hospital procedures, a videotape is made of the entire examination. However, digitizing and transmitting the motion video for the entire duration of every examination is generally not feasible over wide-area communication links. By limiting the video sequences in most cases to only those scenes believed by the technician to be of diagnostic value (generally from 30 seconds to 5 minutes), compressed video can be transported to the radiologist relatively rapidly. For example, in the prototype system, assuming a T-1 line as the transmission link, each second of compressed video requires approximately 2-6 seconds of network transmission time, depending on the amount of compression. Thus, a few minutes of transmission time is generally required for the video, and the radiologist can be viewing the still images from this examination (or some other one) on his workstation while the video sequences are arriving over the network. Allowing the technician to select portions of the video sequence is a workable design trade-off for several reasons. First, the technician generally has a very good feel for what motion video is useful, just as with the selection of still images. Second, in the envisioned interactive paradigm, the technician is in contact with the radiologist during the examination and thus can get additional guidance on which video sequences are important. For these reasons, and as corroborated in conversations with our consulting ultrasound technicians and radiologists, we believe that recording the entire examination digitally would be wasteful of both digital storage space and transmission bandwidth. The teleultrasound system we designed includes a rudimentary interface for viewing still and motion images at the radiologist s workstation. The still images can be displayed in any order by simple key presses, and the video sequences can be viewed with VCR-like functionality. As noted, the system is designed to allow the radiologist to examine image studies while files containing video sequences are arriving from the network. 3: Advantages of the Design Familiar paradigm. An important goal of this project was to create a system that would be readily accepted by ultrasound technicians and radiologists. Our final design fulfills that goal by ensuring that the networked examination process closely resembles the current in-hospital examination paradigm. Digital representation. The advantages of digital versus analog representation of images are well-documented (31. For this project, the most important advantages of digital representation include the fidelity of digital representation after network transmission, the high quality of freeze frames taken from a motion video clip, and the value of digital image processing techniques to diagnostic interpretation. By transferring the images over a digital data network, the image quality becomes insensitive to distance, in contrast with analog signals whose signal-tenoise ratio decreases over distance. Reliable transport protocols allow the (possibly compressed)
296 Eighth IEEE Symposium on Computer-Based Medical Systems image at the sending site to be reconstructed exactly at the receiving site. Another advantage of the digital representation is evident in the review of the video clips. When a radiologist views an analog videotape from an examination, a freeze frame on the tape is of lower quality than the corresponding still image on the radiographic film. With digitdy encoded images, freeze frames from a video clip are identical in quality to still images. Finally, digital image processing for image enhancement and feature analysis is widely regarded as very useful for radiographic images. Such capabilities appear in the latest commercial packages for teleradiology [5, 61. Real-time feedback. Allowing the radiologist to view the images concurrent with the examination results in several potential benefits. We believe this feature of teleultrasound can significantly improve the effectiveness of first-time examinations in producing a conclusive diagnosis. Eliminating a second examination benefits the patient by providing more immediate feedback on the diagnosis and removing the burden of scheduling another visit to the examination site. Flexibility. Real-time interaction between the technician and radiologist is envisioned as the primary mode of operation for our teleultrasound set-up. The speed at which images can be transmitted is a function of the network bandwidth available and the amount of compression used. Compression is necessary for the motion video since the uncompressed rate far exceeds the network bandwidth available from cost-effective wide-area links today. We provide the ability for users to adjust the compression ratio via a user-selected parameter for the JPEG algorithm, the @factor. In this way the responsiveness of the system can be traded against image quality, where feasible. Since the exact effects of image compression on the diagnostic quality of ultrasound imagery is an open research question, we are currently working with our consulting radiologists for this project on a signal detection study involving a set of JPEG-compressed ultrasound images. The system also supports non-interactive paradigms. A radiologist need not be present at the receiving station during the examination. Instead, the receiving station may accumulate the images from several exams to be viewed at a later time by the radiologist. While this method does not exploit the full capabilities of the system, it is nonetheless valuable since it permits the examination of remote patients without travel by either party. Scalability. Our teleultrasound design assumes initial deployment using T-1 leased lines. However, the networking software is easily tuned to take full advantage of whatever bandwidth the network offers. As discussed in the next section, the prototype system is capable of processing and transmitting images at very high rates if the network will support it. 4: Implementation of the Prototype We implemented a prototype of the teleultrasound system using PC-based hardware and communication software developed within our computer networks group. The hardware platform is a pair of 66 MHz Intel 486-based PCs with EISA busses. As shown in Figure 1, one machine (the transmitter) will be located at the examination site while the other (the receiver) will be at the radiologist s site. Here we describe the hardware and software components of the prototype implementation and discuss key performance issues. Each machine is equipped with two video processing circuit boards: the TrueVision Bravado video board and the Rapid Technologies Visionary compression board. The two
SAlA - DesigdMethodology 297 boards are connected with a dedicated bus, reducing contention and bandwidth on the EISA bus during network transmissions. During image acquisition at the transmitter, the Bravado video board digitizes the analog NTSC video signal output from the ultrasound machine. Each frame is digitally encoded at a spatial resolution of 640x480 pixels with a color plane of 8 bits per pixel and then sent over the dedicated bus to the Visionary board for compression. (The 640x480 spatial resolution exceeds the resolution provided by typical ultrasound machines.) At the receiver, buffers containing compressed frames arrive from the network and pass first through the Visionary card for decompression and then through the Bravado card for display on the VGA screen. The compression hardware in the prototype implementation is not altogether satisfactory in that it can not consistently process complex frames with very low compression ratios. Our experience has shown this limiting behavior begins to appear when using a Qfactor of about 20 for the JPEG algorithm. Observations by our consulting radiologists suggest that JPEG compression with a Qfactor of 40 is sufficient for preliminary diagnosis in most ultrasound applications. The problem is thus not a severe one in the current system, and in any case can be eliminated with more powerful video hardware, which is already commercially available. The network software for the system handles reliable transmission of each frame from the transmitter to the receiver. For this purpose we use a novel transport protocol, the Xpress Transport Protocol [l]. XTP has a number of salient features that provide flexibility in adapting the system to different network environments and future requirements. For example, the XTP flow and rate control facilities are the parameters that enable the network transmission code to be easily tuned to the bandwidth of the underlying physical network link. Our XTP software also provides a high performance networking subsystem. Even running on 486-based machines, benchmarks over a 100 Mbit/s FDDI local area network demonstrate that reliable transmission can be sustained at around 15 Mbit/s when overlapped with video acquisition and at over 40 Mbit/s without concurrent image acquisition. While this far exceeds the near-term requirements of the system, it suggests that high-speed links, if available at some future time, could be well utilized. 5: Summary As noted in a recent survey study of telemedicine systems [2], there is now considerable evidence of the effectiveness of teleradiology, though its cost-effectiveness is not wellestablished. Our project demonstrates by example that powerful teleultrasound systems can be constructed today using low-cost microcomputer technology. Our experience also suggests that teleultrasound will lead to new paradigms for ultrasound examinations. In our design the immediate delivery of still images to the radiologist provides the capability for real-time remote diagnosis and feedback from the radiologist to the ultrasound technician during image acquisition. The secondary nature of motion video for diagnostic purposes in some ultrasound studies permits relaxation of the real-time paradigm when transmitting the video sequences. In our prototype system, each second of compressed motion video requires from 2-6 seconds of transmission time over T-1 lines. The exact rate depends on the degree of compression and the frame rate. New paradigms for remote ultrasound examinations are likely to emerge as the experience base with these systems grows. One intriguing capability in our prototype system is the unique reliable multicast mode of the networking software (XTP). If the network links are available, the XTP multicast feature provides for the efficient distribution of images to
298 Eighth IEEE Symposium on Computer-Bused Medical Systems more than one site at the same time. That is, radiologists at two or more sites could receive and view the ultrasound radiographs within the same time needed to transfer the images to a single site. While multicast is not utilized in the current teleultrasound prototype, we believe its availability in the software opens up many interesting possible diagnostic scenarios for future generations of our system. Near-term enhancements to the teleultrasound system will focus on refinement of the image processing and viewing capabilities at the radiologist s workstation, integration of new video technologies, and better accommodation for ultrasound applications that inherently rely on motion video, e.g., echocardiology. Acknowledgments We gratefully acknowledge the contributions of our three partners in the University of Virginia Department of Radiology. Gia Ann DeAngelis, M.D., collected the ultrasound examinations and conducted the companion signal detection experiment to evaluate the effect of JPEG compression on ultrasound images of uterine fibroids. Bruce J. Hillman, M.D. (Chairman), and Samuel J. Dwyer 111, Ph.D., helped design the ROC study and data analysis. The Department of Radiology funded a portion of the equipment used in our prototype. References XTP Forum. Xpress Transport Protocol Specification Document, Version 4.0, February 1995. http://www.ca.sandia.gov/xtp/xtp.html. J. Grigsby, E. Sandberg, and et al. Analysis of Expansion of Access to Care Through Uae of Telemedicine and Mobile Health Services: Case Studies and Current Status of Telemedicine, May 1994. A. Lippman. Feature Sets for Interactive Images. Communications of the ACM, 34(4):92-101, April 1991. W. Pennebaker and J. Mitchell. JPEG StillImage Data Compression Standad. Van Nostrand Reinhold, 1993. Gammex RMI. Courier PC Teleradiology System Documentation, March 1995. Line Imaging Systems. WinRad Documentation, March 1995.