High-Definition Multimedia for Multiparty Low-Latency Interactive Communication

Transcription

1 High-Definition Multimedia for Multiparty Low-Latency Interactive Communication Petr Holub a,b, Luděk Matyska a,b,, Miloš Liška a,b, Lukáš Hejtmánek a,b, Jiří Denemark a,b, Tomáš Rebok a,b, Andrei Hutanu c, Ravi Paruchuri c, Jan Radil a, Eva Hladká a,b a CESNET z.s.p.o., Zikova 4, Praha, Czech Republic b Masaryk University, Botanická 68a, Brno, Czech Republic c Center for Computation and Technology, 302 Johnston Hall, Louisiana State University, Baton Rouge, LA Abstract We describe the high quality collaborative environment that uses High Definition (HD) video to achieve near realistic perception of remote sites. The capture part, consisting of a HD camera, HD-SDI capture Centaurus card and UltraGrid software produces a 1.5 Gbps UDP data stream of uncompressed HD video that is transferred over the 10GE network interface to the high speed IP network. The display part uses UltraGrid to down-sample the color depth and de-interlace the video, using either software solution or another Centaurus card to display the HD stream. Distribution to individual participants of the videoconference is achieved through user controlled UDP packet reflector based on the Active Element idea. The viability of this system has been demonstrated at the igrid 2005 conference for a three-way high quality videoconference among sites in Czech Republic, Louisiana and California. Key words: High Definition Video, Video Conference, Uncompressed HD Video, High Speed Multipoint Data Distribution Corresponding author. Address: Institute of Computer Science, Masaryk University, Botanická 68a, Brno, Czech Republic addresses: hopet@ics.muni.cz (Petr Holub), ludek@ics.muni.cz (Luděk Matyska), xliska@fi.muni.cz (Miloš Liška), xhejtman@fi.muni.cz (Lukáš Hejtmánek), jirka@ics.muni.cz (Jiří Denemark), xrebok@fi.muni.cz (Tomáš Rebok), ahutanu@cct.lsu.edu (Andrei Hutanu), ravi9@cct.lsu.edu (Ravi Paruchuri), radil@cesnet.cz (Jan Radil), eva@fi.muni.cz (Eva Hladká). Preprint submitted to Elsevier Science 31 October 2005

2 1 Introduction Evolution of collaborative environments, following the development of the highbandwidth low-latency networks brings new possibilities to the quality and extent of collaboration. The truly interactive communication requires high-resolution video and audio transmitted fast over the network, with end to end latency below 100 ms to avoid hearing artifacts. Even if captured and transmitted independently, the video and audio must be synchronized, and the low latency requirement holds true both for audio and video transmission. The perceived latency is generated on all the components of the end to end path, so use of uncompressed media is a way to minimize it. The buffering must be also minimized or completely removed, posing strict conditions on the network close to zero packet loss or reordering and minimal jitter are needed. Such a network can be efficiently constructed using a dedicated network circuits over fiber optic network. In this paper we describe a multiparty high-definition (HD) video quality videoconferencing system and the experience gained with its use during the igrid 2005 conference. In Section 2, we deal with the generation and multipoint distribution of HD video stream, including tools for video capture, transmission and display, and the UDP reflectors used to distribute the traffic. In Section 3 we compare the real-world behavior during igrid 2005 experiment with laboratory behavior. Section 4 briefs related work and Section 5 gives some concluding remarks and ideas for future development. 2 HD Video Transport and Distribution 2.1 Uncompressed HD Video Transmission The highest resolution for the HD video defined by the common HDTV norms [6,7] is the 1080i mode with resolution with interlaced line scanning. We use the uncompressed HD digital video as defined in SMPTE 292M [8], which uses the Serial Digital Interface (HD-SDI). The bandwidth for such a video-stream with 60 interlaced fields per second, 10 bits per color plane, and 4:2:2 color space sampling is 1,485,000,000 bits per second or Gbps. This payload is further encapsulated into UDP/IP stream (44 bytes per packet), leading to around 1.5 Gbps bandwidth requirement when transmitted over the network. To efficiently work with such a data stream, highly balanced components are required. The capture and display parts of the whole system are depicted in Fig. 1. 2

3 The capture part uses the DVS Centaurus 1 HD capture card a solution dictated by the selection of Linux operating system environment. We rewrote the UltraGrid software package [5] to support the 1080i mode 2. While the same card can be used also for video display, we extended the UltraGrid with support for software only display, including field de-interlace algorithm and color space down-sampling from 10 to 8 bits per color plane to avoid the use of expensive DVS Centaurus cards on both ends of the HD video paths. The computation and data manipulation intensive parts of the UltraGrid software were optimized for use on the AMD64 (Opteron) based computers. Capture part: PC (IA32/AMD64) HD-SDI DVS Centaurus Uncompressed raw HD UltraGrid 10GE NIC 10GE Display part: PC (IA32/AMD64) HD-SDI display device HD-SDI DVS Centaurus Uncompressed raw HD UltraGrid 10GE NIC 10GE network PC (AMD64) Display device (LCD, beamer) DVI-D/ VGA Graphics card Uncompressed raw HD UltraGrid 10GE NIC 10GE Fig. 1. Site connection scheme for HD transmission. The whole capture part started with the SONY HVR-Z1E camera whose analog output has been converted to HD-SDI using the AJA HD10A converter 3. The HD-SDI stream has been captured by the DVS Centaurus card, encapsulated into UDP/IP stream with the UltraGrid and sent over the Chelsio T110 LR 10GE card 4 to the network. On the receiving part, the IP stream has been captured by the same brand 10GE card, stripped of the IP header, color re-sampled and de-interlaced by the UltraGrid and sent to the graphic card. The computers used were dual AMD64 Opteron 250 machines running at 2.4 GHz with 4 GB RAM. The Centaurus and Chelsio cards were placed in the PCI-X 133 MHz slots. We used Linux kernel with drivers and manufacturer provided patches for both cards. The total end to end latency in a laboratory setup (both computers on the same 10 GE Cisco Catalyst 6506 switch) was 175±5 ms. We have also measured part The older releases available before igrid 2005 conference supported only the 720p mode

4 wise latencies: 4 fields are buffered on the capture card 5 (66.7 ms), 10 b to 8 b down-sampling (7±.5 ms), de-interlacing (7±.5 ms), software display (41±.5 ms), LCD display delay (25 ms). This counts up to 147 ms and we are attributing the remaining delay (28 ms) to camera and component to HD-SDI converter processing, sender processing, buffering on network interfaces and in networking stack of the operating system, and to the processing on the graphics card. It is worth noting that the duration of down-sampling and de-interlacing corresponds to memory copying speed (approx. 1 GBps on testbed machines, one frame has approx. 5 MB in 10 bit and 4 MB in 8 bit per color plane depth). On sender box, the CPU load was approximately 25%, while the receiver box showed 72% CPU load on average. While the audio is the main driver for the use of uncompressed video streams, it does not generate a comparable data flow. As we did not have embedded audio, we used a rat tool [1] from MBone Tools. The generated audio stream has been synchronized on the receiving machine via the UltraGrid capability to use time information from the RTP/RTCP packets. 2.2 HDV Transmission When the bandwidth or available hardware disables the use of uncompressed HD video, it can be transmitted in the HDV format. It is a proprietary MPEG-2 based compression scheme developed by SONY and it is transmitted in MPEG-TS envelope over the IEEE-1394 interface in a similar way to the DV format for standard definition resolution. It uses only resolution, 50 or 60 interlaced fields per second, 8-bit color space, 4:2:0 color space sampling, and interframe 60:1 compression resulting in approximately 25 Mbps video stream. We have analyzed how the HDV format is sent over IEEE-1394 interface and implemented a tool [3] for FreeBSD 5/6 to read it. The data are then rendered using VideoLAN Client (VLC) tool 6 and sent over the network either using VLC or some other tools like netcat 7. The VLC Linux implementation has serious problems (packet drop) with reading streams with data bandwidth above 10 Mbps. The Windows XP VLC implementation does not have this drawback and has been used instead. The measured end to end latency for the 60i HDV stream in the same laboratory setup as mentioned above is 1,907±13 ms. As the delay of camera compression is below 1 s, the highest latency is gathered in the VLC buffering, decompression and rendering process. The almost 2 s latency practically hinders any real time communication. If it must be used, the audio stream must be independent and should not be 5 The 2 fields reported by DVS did not work

5 Table 1 Reflector performance on the testbed. (a) gives maximum bandwidth of a single stream with packet loss <0.01% with respect to packet size in use, (b) gives packet loss with respect to bandwidth of a single stream given 8500 B packet size. packet size [B] max. bandwidth [Mbps] (a) bandwidth packet loss CPU load [Mbps] [%] [%] synchronized with the video. This way, acceptable interactivity level is guaranteed via audio and video visibly lags behind. (b) 2.3 Multipoint Data Distribution To distribute data among more parties in a multipoint videoconference, we used the generalized Active network Element [4] with distribution capabilities, based on the design of UDP packet reflector [2]. This gives us more control over the distribution than a network native scheme (multicast or broadcast), which also may not be available for very high speed networks. We have optimized the Active Element to provide a sustainable UDP packet replication rate of up to 2.0 Gbps on high end IA-32 or AMD64 workstation or server. The actual performance on the dual Opteron 250 computer (same as used for the video capture and display) is summarized in Table 1. Data stream was generated and captured by another dual Opteron machine. The results confirms the necessity to use Jumbo frames (long MTU) to achieve the highest performance. Also, the network card to memory bandwidth is important, as use of only 100 MHz PCI-X slot reduced the usable reflecting capability to 1.7 Gbps. 3 igrid 2005 Experiment The goal of the CZ101 igrid 2005 experiment was to provide low latency multiparty collaborative environment with HD video. The setup comprised three par- 5

6 ticipating sites: igrid premises at San Diego UCSD campus, Masaryk University/CESNET in Brno, Czech Republic, and Louisiana State University (LSU) in Baton Rouge. There were three networking circuits meeting at StarLight, Chicago, each coming from one participating site: igrid circuit from San Diego (round-trip time 78.2±.2 ms, 4 hops), NLR circuit from LSU (round-trip time 31.09±.04 ms, 2 hops) and a circuit from Brno spanning CzechLight and NetherLight (through Prague and Amsterdam, round-trip time 126.7±.3 ms, 2 hops). Though we originally requested L2 connectivity, we were given full L3 connectivity. The video capture and display setup described above has been used at all sites with the exception of LSU, where the unavailability of DVS Centaurus card 8 lead to the fallback HDV solution. The video was displayed on DELL UltraSharp 2405FPW 24 LCD (LSU, Brno), Sony SDM-P LCD (Brno) and on 63 Samsung PPM63H3 plasma screens (San Diego), all at resolution of San Diego StarLight Brno San Diego StarLight Brno LSU LSU (a) (b) Fig. 2. Connection scheme for (a) the first and (b) the second experiment. The full lines show uncompressed HD streams, dotted line shows compressed HDV streams. Empty circles denote sending computers, full circles stand for display computers, and full boxes are AEs. UDP packet reflectors were set up at StarLight (Chicago), running on dual Opteron computers as those used in laboratory testbed described above. Each reflector was replicating stream from one site to the remaining two sites. Because of situation described above, one of the reflectors was replicating only 30 Mbps stream, while the other two were replicating full 1.5 Gbps streams. To stress the infrastructure in the same way as if there were full HD stream coming from LSU, we changed the setup for a part of the demo and cascaded two of the reflectors so that Brno was receiving two identical streams from San Diego. This way we achieved total network flow of 4.5 Gbps on Brno StarLight circuit. This setup also proved the feasibility of AEs combination to provide scalable data distribution [4] even at very high data rates. Network statistics shown in Fig. 3 were gathered on Cisco Catalyst 6506 switch in Brno during the last demo when Brno was receiving two uncompressed HD streams 8 It has been stuck at customs because of hurricanes. 6

7 (a) in (b) out (c) in (d) out Fig. 3. Network statistics gathered on Brno uplink. The lines are five minute averages, while the surface is an envelope curve with min/max values within 5 minute interval. (a) gives in throughput in Gbps, (b) gives out throughput in Gbps, (c) gives in throughput in packets per second (pps), and (d) gives out throughput in pps. as described above. The statistics were gathered at the WAN-PHY interface, which is the uplink to the StarLight aggregating all the traffic from and to the Brno site. The outgoing traffic has been rather regular, while the incoming traffic displayed some burstiness. We attribute this behavior to the use of L3 network, where the packets are still buffered at some intermediate network elements and the jitter is still rather uneven. We expect the use of pure end to end optical network with no intermediate buffers will remove this problem (the video and audio flutter has been subjectively perceived during the demo). 3.1 Experiences and Problems The largest jumbo frames have 9216 bytes, but we used only 8500 byte frames due to a problem with the pre-production version of the WAN-PHY XENPAK on the Cisco Catalyst 6506 switch. As seen in Fig. 4, the packets are lost for frames larger than 8628 bytes packet loss [%] frame size [B] Fig. 4. Dependence of packet loss on frame size on pre-production WAN-PHY XENPAK in Cisco Catalyst

8 In contrast to the measurement probes used in the laboratory tests of the AEs duplicating capabilities, the UltraGrid produces very bursty traffic (see Fig. 5). Similar effect but at much lower bandwidth occurs also with the HDV transport using either VLC (Fig. 5c) or even worse using netcat (Fig. 5d). This severely reduced the duplicating capacity of the AE to some Mbps with large fluctuation but no packet loss. (a) (b) (c) (d) Fig. 5. Bursty traffic produced by uncompressed HD transport (a) compared to smooth traffic produced by the measurement probes (b). Bursty traffic produced by HDV transport with VLC (c) and netcat (d). The main reason for this slowdown has been the blocking character of the reading thread in such bursty environment. We modified the AE to use the non-blocking read() function giving precedence to the sending thread when no input data is available. This modification increased CPU load to 100%, but has been able to process even the bursty uncompressed HD streams at the necessary speed. The burstiness is very serious problem, as the unmodified AE has not been able to duplicate even the 25 Mbps stream fast enough because the burstiness in this case has been much higher and regularly overloaded the AE. Another source of problems encountered has been the overheating and instability of dual Opteron computers with Chelsio and especially Chelsio and Centaurus cards. The fast assemblage and setup of dual Opteron computers at San Diego even resulted in one of the machines unable to receive or send data with the Chelsio card on speed above 800 Mbps (the same card worked perfectly if moved into any of the other two machines). As this machines has been used to capture the HDV stream, this behavior did not have negative impact on the whole demonstration. 4 Related Work Our work follows development of UltraGrid [5], enhancing it with 1080i resolution support and extending its software display including de-interlacing and color space 8

9 scaling. We also further extend work on the use of Active Element based reflector for the efficient data distribution under strict user control (see discussion in [4]). During the igrid 2005, ResearchChannel had a similar demonstration [9] of multipoint HD videoconference. In contrast to their setup, we used the software display instead of SDI, we used independent audio stream (externally synchronized) and used Active Elements instead of multicast for data distribution. Also, we used Linux and not Windows operating system environment. 5 Conclusions and Future Work During the igrid 2005 conference we demonstrated that a high quality multiparty videoconference based on transmission of uncompressed HD streams is already achievable provided adequate networking resources are available. Although the use of L3 network instead of dedicated optical circuits together with software reflector lead to higher than theoretically minimal latency and jitter, the whole setup provided a realistic high quality collaborative environment. The same data distribution setup plus HD video transmission has also been used in another demonstration during the igrid 2005 conference. However, the experience also demonstrated deficiencies and open challenges for future research and development. At the network level, we plan to repeat the experiment over L2 and possibly even L1 network, to prove the hypothesis that it can reduce the rather high observed variation in delivered bandwidth. The rather complicated setup of the network for the demonstration also proved necessity for a special control plane over (optical) networks spanning several administrative domains if dedicated circuits are to be provided. If access to the L1 network is granted, we also plan to replace software based Active Elements with optical splitters that would further reduce the latency for multiparty transmission. Based on the theoretical research and experiments with cascading AEs we will work on a distributed AE for scalable replication of high speed data stream, as this is necessary for many-party videoconferencing but also for situation where each site provides several HD streams (e.g., visualization, stereo-transmission etc.). Acknowledgments We would like to thank Tom Košnar for gathering network performance statistics using his GTDMSIII system developed at CESNET, Martin Míchal and Josef 9

10 Vojtěch for preparation of CzechLight lines and experiments with optical splitters, and Alan Verlo, Pieter de Boer, Paola Grosso and other people at igrid 2005 NOC for network provisioning. Also, the help of Cisco (lending the pre-production WAN-PHY Xenpak) and especially Chelsio (lending two 10GE cards) is highly appreciated. This project has been supported by a research intent Optical Network of National Research and Its New Applications (MŠM ) and also by Parallel and Distributed Systems (MŠM ). References [1] V. Hardman, A. Sasse, M. Handley, and A. Watson. Reliable audio for use over the internet. In Proceedings of INET 95, Honolulu, Hawaii, June [2] E. Hladká, P. Holub, and J. Denemark. An active network architecture: Distributed computer or transport medium. In Proceedigns of 3rd International Conference on Networking (ICN 04), pages , Gosier, Guadeloupe, Mar [3] P. Holub. HDV capture for FreeBSD 5/6 operating system. hopet/hdv/ and 0+archive/2005/freebsd-firewire/ freebsd-firewire. [4] P. Holub, E. Hladká, and L. Matyska. Scalability and robustness of virtual multicast for synchronous multimedia distribution. In Networking - ICN 2005: 4th International Conference on Networking, Reunion Island, France, April 17-21, 2005, Proceedings, Part II, volume 3421/2005 of Lecture Notes in Computer Science, pages , La Réunion, France, Apr Springer-Verlag Heidelberg. [5] C. Perkins, L. Gharai, T. Lehman, and A. Mankin. Experiments with delivery of HDTV over IP networks. In 12th International Packet Video Workshop, Pittsburgh, PA, USA, Apr [6] Society of Motion Picture and Television Engineers. 1280x720 scanning, analog and digital representation and analog interfaces. SMPTE 296M [7] Society of Motion Picture and Television Engineers. 1920x1080 scanning and analog and parallel digital interfaces for multiple picture rates. SMPTE 274M [8] Society of Motion Picture and Television Engineers. Bit-serial digital interface for high-definition television systems. SMPTE 292M [9] M. Wellings, J. DeRoest, A. Philipson, J. Eveleth, J. Brown, C. Latham, R. Johnson, G. McLaughlin, A. Howard, J. O Callaghan, M. Lack, E. Verharen, D. Devereaux- Weber, and A. Kato. Global N-way interactive high-definition video conferencing over long-pathway, high-bandwidth networks. US118 igrid 2005 demo, igrid2005.org/program/applications/videoservices nwayconf.html. 10