Panoramic Video Transmission for Autonomous Water Surface Research Vessels

Transcription

1 S AARLAND U NIVERSITY FACULTY OF N ATURAL S CIENCES AND T ECHNOLOGY I D EPARTMENT OF C OMPUTER S CIENCE M ASTER T HESIS Panoramic Video Transmission for Autonomous Water Surface Research Vessels Submitted by Vinayak Hegde on October 04, 2016 Supervisor Prof. Dr.-Ing. Thorsten Herfet Advisors Daniel Schmitt Yongtao Shuai Reviewers Prof. Dr.-Ing. Thorsten Herfet Prof. Dr. Dietrich Klakow

2

3 Statement in Lieu of an Oath I hereby confirm that I have written this thesis on my own and that I have not used any other media or materials than the ones referred to in this thesis. Saarbruecken, October 04, 2016 Vinayak Hegde Declaration of Consent I agree to make both versions of my thesis (with a passing grade) accessible to the public by having them added to the library of the Computer Science Department. Saarbruecken, October 04, 2016 Vinayak Hegde

4 Abstract Autonomous or unmanned water surface vehicles (USV) play an important role in oceanology and environmental research. They carry sensors and measurement equipments to distant places without the restrictions imposed by personnel on board. Video surveillance is a valuable safety feature to avoid collisions with other watercrafts and swimmers. In this thesis, a video streaming module is designed and implemented for a small and lightweight USV performing long term monitoring of lakes and rivers. It is capable to transport a panoramic view from four cameras across the water over a distance of few hundred meters. A long range outdoor wireless network is established using the suitable antennas and radios. The setup is validated by measuring available bandwidth, signal strength and loss rate. Video streaming is evaluated in terms of video latency, error tolerance and bandwidth. Transmission of four videos is achieved up to a distance of 700 m over water.

5 Acknowledgements The journey of this Masters program has been a truly life-changing experience for me and it would not have been possible to do without the support and guidance that I received from many people. First and foremost, I would like express my sincere gratitude to Prof. Dr. -Ing. Thorsten Herfet for providing me this opportunity to work on my Master Thesis. His support and encouragement has been the driving force for me. I would like to thank my advisors Daniel Schmitt and Yongtao Shuai for the valuable feedback and discussions. I want to thank Dr. Holger Hewener for his suggestions. I am thankful Prof. Dr. Dietrich Klakow for taking time out of his busy schedule and reviewing my thesis. This thesis would not have come to a successful completion, without the help I received from the staff of the Ultrasound department of the Fraunhofer IBMT. A special thanks to Christian, Tobias, Jürgen, Markus, Martin and Leonora for lending me a helping hand in conducting the outdoor experiments. I want to thank all my fellow students of Fraunhofer IBMT for giving me a wonderful company. I owe a lot to my parents and sisters, who encouraged and helped me at every stage of my personal and academic life. Additionally I would like to thank all of my family members for the care and encouragement. I like to thank Sudi for the moral support and entertaining conversations. I would like to thank my friends Goutam and Praharsha. Finally, I would like to thank all people who have helped me during my master s study.

6 Contents 1 Introduction Project Background Unmanned Surface Vehicles Long-range Wireless Networks Thesis Motivation Related Work Synopsis Wireless Video Transmission Video Streaming System Video Acquisition Video Encoding and Decoding Streaming Protocols Transport Protocols Media Delivery and Control Protocols USV Video Transmission Frequency of Operation Range and Power Resolution and Bandwidth Video Transmission Module Server-side Client-side Equipment Implementation Hardware Implementation Software Implementation Server Machine Client Machine Evaluation Measurement Setup Test Location Test Setup Experimental Results Wireless Link Performance Video Transmission Performance Discussion

7 6 Conclusion and Future Work 41 Appendix A Equipment Specifications 43 A.1 RF units A.2 Antenna A.3 Camera A.4 Radio Appendix B Third Party Libraries 50 B.1 OpenCV B.2 FFMPEG Appendix C Measurement Tools 53 C.1 Iperf C.2 netem C.3 airos

8 List of Figures 1.1 Unmanned Surface Vehicle Region of Interest Video streaming system Group Of Pictures USV Video streaming module Radio USV Antenna Shore Antenna Camera Hardware setup USV Server Machine Encoding steps Client Machine Preliminary USV Test Location Antenna placement (Lake) Video frame Bandwidth vs Distance Signal Strength vs Distance RTT vs Distance Bandwidth for four videos A.1 Camera(Logitech C930e) A.2 Radio(Bullet M2) A.3 Power Over Ehernet A.4 Radio Main Page A.5 Bridge Mode

9 List of Tables 4.1 Operating System Third Party Libraries Encoder Parameters Loss rate vs Video Latency A.1 USV Antenna A.2 Shore Antenna A.3 Camera(Logitech C930e) A.4 Radio(Bullet M2)

10 List of Abbreviations IBMT GPS USV MPEG ITU-T GOP ISO TCP UDP RTP RTSP RTCP IP db SD HD USB RF LOS km PoE AC DC LAN V WLAN ROI Institut für Biomedizinische Technik Global Positioning System Unmanned Surface Vehicle Motion Picture Experts Group International Telecommunication Union-Telecommunications section Group of Pictures International Standards Organization Transmission Control Protocol User Datagram Protocol Real time Transport Protocol Real Time Streaming Protocol Real Time Control Protocol Internet Protocol DeciBel Standard Definition High Definition Universal Serial Bus Radio Frequency Line-of-sight Kilometer Power over Ethernet Alternating Current Direct Current Local Area Network Volts Wireless Local Area Network Region Of Interest

11 1 Chapter 1 Introduction Video is an important medium for communication and entertainment. The advent of video streaming techniques leverages us to make use of it in every possible application. These days we can find video streaming in scientific, security, military, traffic management and surveillance systems. Mobile video streaming systems for drones, unmanned aerial and unmanned water (surface) vehicles pose many challenges like weight, size, distance and possibly unfriendly environments. FIGURE 1.1: Unmanned Surface Vehicle A novel autonomous water surface research vessel is being developed at Fraunhofer Institut für Biomedizinische Technik (IBMT). This specific version of an unmanned surface vehicle (USV) will be deployed on a lake for mapping and sub-bottom profiling by using a sonar sensing technique. The prototype model of the USV which is under development is shown in Fig.1.1. The USV is equipped with a global positioning system (GPS) based navigation and a control system for planning and completing the autonomous mission. A region of interest (ROI) will be specified based on the geographical coordinates. The USV travels to that ROI and maneuvers across it to perform the sonar measurements. Fig.1.2 illustrates the sample ROI. Several of those ROI s of an approximate size

12 Chapter 1. Introduction m 2 may be investigated from a point at the shore. During such mission, the maximum distance between the shore and the USV is up to 1 km. The sonar data acquired by the USV during the process is used for further processing and analysis. FIGURE 1.2: Region of Interest During such a planned mission of the USV, a remote video surveillance system is a welcome feature to avoid the swimmers and the neighbouring water vehicles. Therefore, an appropriate video transmission and communication module for the USV is implemented. The module must be capable of doing the seamless video transmission for few hundred meters. In this thesis, a video transmission module is proposed. Multiple cameras are used to collect video data around the USV. The video data is encoded using the machine on the USV and the video is transmitted to the shore using a radio and an omnidirectional antenna. The video is received and shown at the shore such that an operator can control the USV as per surrounding scenario. The single camera video resolution should be in the range of 640x480p to fulfil this task. 1.1 Project Background The shores of many German lakes are subject to significant structural impairments. Embankments (e.g. seawalls and docks) and other man-made obstructions lead to changes in hydrodynamic conditions in the shallow water zone of the lake. The HyMoBio strategy project [5] motivation is to examine the changes in the shallow water zone over a period of time. Based on the findings of the HyMoBio project recommendations and solutions are provided for a

13 Chapter 1. Introduction 3 sustainable landscape. Fraunhofer IBMT is a HyMoBio strategy project partner. It is providing the sonar systems for sensing the state of a lakeshore. The sonar systems are transported on the USV as a payload. The USV is battery operated and the batteries are charged by solar power modules. The USV is well equipped with electromechanical systems. These systems facilitate USV s movement and manoeuvre during the autonomous mission. The USV has GPS antennas, which are helpful in guiding the USV in the lake as per the GPS coordinates. The goal of the mission is to acquire the sonar data in the specified locations and those locations are preloaded in the control machine of the USV. By using the GPS coordinates, the USV reaches the specified location. The USV does not have any defined heading direction of movement. 1.2 Unmanned Surface Vehicles The design and development of the USVs started in early 90 s. USVs are incorporated with GPS for assisting their movement during the autonomous operations, thus USVs have become more capable. Affordable long-range and high bandwidth wireless data systems also made significant contributions and they are helpful in remote controlling of missions [20] [22]. Today, USVs are being developed and demonstrated by research labs, academic institutes and government agencies for heterogeneous applications. USV s remote sensing platform enabled many research establishments to acquire the data at a low cost. The maneuverability feature of the USV creates an opportunity to have data from the regions, where reaching there could be dangerous to a human life and inaccessible for direct examination. 1.3 Long-range Wireless Networks Today, several applications demand outdoor wireless network deployments. The installation is done mostly in the regions where the cable based solutions like Optic Fiber or ADSL cannot be provided easily. The long-range wireless networks are beneficial for providing an Internet connection to the rural areas, semi-urban areas, distant education purpose, medical applications and much more. A wide range of long-range wireless products (e.g. antennas, routers) is available in the market. Those can be easily employed for establishing the long-range wireless networks. The unlicensed frequency bands (2.5 GHz, 5 GHz) additionally promises no extra communication charge. On the other hand, having an outdoor long-range wireless network is challenging. There are several issues while setting up the network and maintaining the long-range wireless network facility. Therefore, it is important to give

14 Chapter 1. Introduction 4 enough care from selecting the equipment to the final installation. Empirical studies [9][23] show that some obvious challenges must be addressed to our best ability, such that the long-range outdoor wireless networks will be more robust. Main obstacles are: The transmitted Radio Frequency (RF) signal would be degraded because of the path loss and other effects like reflection and refraction. Hence, the available signal strength at the receiver might not be sufficient always. As we operate at the outdoor, Equivalent Isotropically Radiated Power (EIRP) limits the maximum power in the transmitted signal. Therefore, it is must to have a good trade-off between the transmitted signal power and the minimum receiving sensitivity of the receiver. In the outdoor wireless networks, interference is a likely problem from other competent wireless networks, trees and the buildings. The weather conditions could affect the performance of the outdoor networks two ways. Firstly, adverse weather conditions could degrade the transmitted signal power. Secondly, if the installed equipment are not stable against the wind and the rain, then there would be some performance hindrance due to losing line-of-sight (LOS), if the alignment of the antenna is disturbed. There are logistical challenges for long-range wireless setup. First, providing the power to the equipment of operation using either the solar energy or a battery. Secondly, accessibility of the intended locations for the setup. LOS is directly linked to the wireless network performance. In an outdoor long-range wireless networks, it is better to have a good LOS for the best performance. Wireless networks over the water surface are a widespread topic in areas like the naval warfare, study vehicles like USV, fishermen boats, warning systems and various oversea communication applications. All the aforementioned challenges for the outdoor wireless networks are applicable to the wireless network over the water surface. In addition, on the water surface signal characteristics differs in some aspects like path loss characteristics compared to propagation on the ground [12] [21]. 1.4 Thesis Motivation The USV will be deployed in a lake close to the shore. Therefore, probable collision with other boats, study vehicles, swimmers is likely during it s ride to the specified position for the mission and at the time of maneuvering. Hence, it is necessary to have a suitable video streaming system, such that we can control

15 Chapter 1. Introduction 5 the USV s movement and maintain a safe distance to obstacles. No particular heading direction of movement is available for the USV. In addition, it is capable of taking the rotation in stand still position. Hence, movement of the USV is controlled from the shore by viewing the streamed videos in all four directions. These videos are expected to be transmitted in the real time to accomplish accurate control of the USV. The video streaming system of the USV must overcome some key challenges. The wireless network to be established for a distance up to 1 km and would faces the challenges mentioned in section 1.3. In addition, the established wireless network must provide sufficient data rate for transmitting four videos. With the above points in mind, necessary components are chosen and validated to establish a long-range wireless transmission network. Apart from setting up the long-range wireless network, the stringent requirements of the USV posed many constraints in setting up the video streaming module. The components chosen for mounting on the USV for establishing the wireless network must be compact, lightweight such that mounting and handling are easy and power efficient devices since power sources are limited in the USV. A "video and communication module" has been implemented to overcome all the requirements involved in surveillance and controlling of the USV in the lake. The established wireless network could be helpful for other functions like USV s gathered sonar data can be transferred to the machine on the shore, accessing and controlling the other machines on the USV for various activities. 1.5 Related Work Aust et al. [11] reviewed a number of outdoor long-range wireless networks and challenges. They briefed the results of long-range wireless network performance in the research work carried by many researchers. El-Sayed et al. [13] show that the variation of path loss and Round Trip Time (RTT) as per the link distance. It is found that the path loss and the RTT increases with increase of the link distance. Yang et al. [28] focused on the transmission over the sea at around 2 GHz and investigated the characteristic of received signal strength against the distance. Wideband channel characterization over the sea propagation is presented by Maliatsos et al. in [19]. Siddharth Unni et al. [25] explains a wireless network extension. They measured the received signal strength, the bandwidth against the distance and the effect of the antenna tilt is also observed. Lopes et al. [18] illustrates the Transmission Control Protocol (TCP) and User Datagram Protocol (UDP) bandwidth against the distance for a maritime environment.

16 Chapter 1. Introduction 6 The underlying video streaming concepts and systems presented in this work is inspired by the work of Apostolopoulos et al. [10] and Wu et al. [27]. They put forward the general video streaming architecture and the significance of each block of the architecture. They presented the video compression standards, transport protocols and challenges in video streaming are also reviewed. In this thesis work, the H264 video coding standard is employed for video compression. The video coding standard details are demonstrated in [26][24]. Challenges in video streaming, performance comparison is given in [29]. Sze-wing et al. [17] implemented an outdoor video streaming application for live monitoring of the construction work to keep an eye on progress and quality of the work. 1.6 Synopsis In chapter 2, a brief introduction of video streaming system, video compression standards, and transport protocols are given. Chapter 3 gives a detailed overview of a video transmission module for the USV along with the components. Chapter 4 describes the hardware and the software implementation of the video streaming module. Evaluation is given in Chapter 5, that includes the measurement setup and the results. Finally, Chapter 6 concludes this work and provides the possible future extension of the video streaming module.

17 7 Chapter 2 Wireless Video Transmission In this work, the video streaming module is designed by exploiting the wireless video transmission techniques. Therefore, in this chapter, a brief overview of the wireless video streaming system is presented. In the subsequent sections, each part of the video streaming system is given. Video streaming is divided into a broadcast, multicast and unicast, as per the number of hosts are on the network. Unicast : A message or a piece of information is sent from one point to the another point, e.g. a video conference with only two participants. Multicast : The piece of information is sent to a group of receivers on the network, e.g. multimedia content delivery networks. Broadcast : The piece of information is sent to all the receivers on the network, e.g. broadcasting on Ethernet. In our scenario, a video streaming happens from the server machine on the USV to the client machine on the shore. Therefore, it is considered as a point to point video streaming. 2.1 Video Streaming System Fig.2.1 shows a typical block diagram of video streaming over a wireless system. It can be divided into two parts as a server and a client. In a live video streaming, the video input is a video stream captured by the camera. The captured video stream is then fed to a video encoder, which compresses the video into the compressed stream. The compressed stream is sent to the client using appropriate protocols and wireless network. In the client, a reverse process is carried out. The received compressed streams are decompressed and rendered. In sections 2.2 and 2.3, each block is described comprehensively Video Acquisition In live streaming, the video content is delivered in real time once it is captured by a camera, e.g. streaming a soccer match. In stored video streaming, the video source is a file from the disk memory or from a stored location (e.g. Youtube videos).

18 Chapter 2. Wireless Video Transmission 8 FIGURE 2.1: Video streaming system There are several kinds of cameras available for as a source for live streaming. Most commonly used are IP cameras, web cameras, and professional cameras. The IP cameras are mainly used for the surveillance applications, they have a separate IP and they enable video streaming to a device which has Internet access. Web cameras are preferred in plenty of applications like video conferencing in telecommunications and telehealth care. Webcams are connected to the Internet via a machine they are being accessed and they are easy to use, plug and play solutions. Generally, through a USB cable they can be connected to a computer. The webcams do not need any additional requirement like hardware. They need neither an extra power source nor a video card. On the downside, they do not provide as high-quality video as the professional quality cameras. They normally do not have a storage space and few cameras won t provide access to their inbuilt video encoders (e.g. Logitech C930e, used in this thesis work). The professional cameras are used in producing television streams. They are capable of producing the higher quality videos, and other features like zooming, long distance coverage without degrading much in the quality, wide angle focus and a dedicated lens for various applications. On the other hand, they are not portable, handling is uneasy, and separate hardware like capture card is essential to convert a video signal into a signal format which can be streamed. Considering the above-mentioned advantages of webcams and since the USV is carrying a machine for generic control of sensors, webcams are used to capture the video data in our scenario and the details are given in Chapter 3.

19 Chapter 2. Wireless Video Transmission 9 FIGURE 2.2: Group Of Pictures Video Encoding and Decoding Video encoding converts raw video to a compressed format. The process of converting compressed format to raw video is called video decoding. Decoding might not reproduce the exact number of bits, but we can represent the video with an available number of bits. Uncompressed video data produces an enormous amount of data. Streaming uncompressed video data needs a lot of bandwidth and requires a huge disk memory for saving a copy. Uncompressed data demands a high computational power and a network bandwidth. These requirements can be reduced significantly by video encoding process. In general, the higher the compression rate, the lower the video quality. Encoding and decoding need some computational resources, but comparatively saving the bandwidth is essential than computational resources. The video is a sequence of images, which are encoded independently using joint photographic experts ground (JPEG), portable network graphics (PNG) or any other image compression standard is possible. However, in a video, lot of similarity can be found between the successive frames. By exploiting this similarity, we can achieve the great amount of video compression. In this process, the frame is predicted by previously coded frame. There are three kinds of frames namely Intra-coded or I-frames, Bidirectionally coded or B- frames and Predictively coded or P-frame. I-frames have the full information of the frame and they need more space than the P-frames and the B-frames. P-frames hold the changes in the previous frame, e.g. in a scene where the football is moving across the ground. In the above scenario, we need to encode the changes in the scene, background pixels remains same, hence encoder does not need to encode background pixels again and again. B-frames hold the difference between the current frame, preceding frame and the following frames to represent the details. Fig.2.2 illustrates the group of pictures (GOP). 1

20 Chapter 2. Wireless Video Transmission 10 Video compression is achieved by exploiting the redundancy between the successive frames. Unless there is a very fast movement in a produced video, the temporal redundancy between the frames is high. The compression methodology varies as per the application. In real video streaming non-scalable compression is employed and scalable is effective in stored video streaming [10]. Video compression standards Compression standards help to achieve interoperability between the consumers and the manufacturers. The standards define the rule for a bitstream and the decoding process. Compressed video is represented in bitstreams, and decoder processes them by using a set of rules as per the standard. Two standards organisations have played a major role in the development of the video compression standards : the International Telecommunications Union (ITU) and the Moving Pictures Experts Group (MPEG). The MPEG specializes in broadcast (television streams) whereas ITU focuses on telecommunications (Internet, phone). The two organisations work sideby-side since they have a common interest (Internet). The video compression standards under ITU are H.261, H.263, and H.264. The H.261 standard has a lower bitrate of 64 kbps and a higher bitrate of 2048 kbps. The H.261 [2]. frames are two types: the I frames and the P frames The H.263 standard supports lower target bitrate than H.261 standard. It uses PB (predicted, bi-directional) frame mode, which is helpful in increasing the frame rate without reducing the bit rate [3]. The H.264 [4] developed with the intention of reducing the bitrate and improving the video quality compared to the other two previous standards H.261 and H.263. The H.264 encodes both low and high-quality videos. H.264 has better compression efficiency than H.263. In this thesis, the H.264 video compression standard has been applied. 2.2 Streaming Protocols Communication protocols are rules for governing how data is communicated, by defining features like header format, data format, error handling, authentication and much more. Media streaming has acquired a lot of importance over the years, and many protocols have been defined, developed and standardized over the time. Many proprietary protocols also persist for media streaming. Media streaming protocols are divided broadly into following categories. Transport Layer Protocols Transmission control protocol (TCP) and the user datagram protocol (UDP)

21 Chapter 2. Wireless Video Transmission 11 are the two major protocols employed for the video streaming. TCP provides reliable transport via retransmission and acknowledgements. UDP is unreliable and connectionless. When TCP is used in the error-prone channels, due to a number of retransmissions delay would be unbounded. Therefore, in the error-prone channels, TCP is not suitable for video streaming. Whereas UDP is connectionless but delivery is not guaranteed. Hence for media streaming UDP is preferred and control information is sent using TCP. Media Delivery and Control Protocols Real time transfer protocol (RTP), real-time control protocol (RTCP) and real-time streaming protocol (RTSP) are designed to support media streaming. RTP is used over UDP. Main functionalities of RTP are time stamping, sequence numbering, source identification and payload type identification. RTCP is designed for control messages. It provides services like, receiver reports, sender reports, and source description. RTSP adds the functionalities like play, pause etc Transport Protocols Transmission Control Protocol is a connection-oriented transport protocol. TCP provides reliable data transfer by using the following features [15]. Connection Oriented : TCP requires a logical connection to be established between two processes before data is being exchanged. The connection must be maintained throughout the entire communication, released at the end. It provides acknowledgement after every successful delivery of data and automatic repeat request in case of byte errors in the received data. Flow Control : TCP has window-based flow control. The size of the window varies as per the receiver buffer size and the receiver window. The receiver window reflects the number of bytes the receiver can accept before the buffer overflow. The value of the receiver window is sent to the sender with an acknowledgement. Hence, the sender will get to know about the receiver status [15]. Congestion control : A congestion window is maintained at the sender. This window controls the sender rate at which the sender can send data to the network. TCP congestion control mechanism has three elements: slow start, congestion avoidance and congestion detection. Error Correction : error correction mechanisms makes the TCP reliable. The error correction includes finding the lost, corrupted, out of order and duplicate segments. TCP has three methods to detect and correct the errors: time out, checksum and acknowledgements. UDP does not deploy the acknowledgment, congestion control and flow control. The erroneous packet will be discarded. The point to point streaming does

22 Chapter 2. Wireless Video Transmission 12 not use UDP [15]. However, UDP is helpful for multicast and broadcast streaming. The advantage of the UDP is its simplicity. Features of UDP are, No connection establishment : TCP does three-way handshake before establishing the connection between two segments, however, UDP crashes away all this handshaking. Thus UDP does not introduce any delay. For example, DNS servers use UDP. No connection state : To manage connection state, TCP maintain connection state in the end systems. However UDP need not have this, hence UDP based servers can handle more clients compared to TCP. Small packet overhead :The header of UDP packet has only 8 bytes Media Delivery and Control Protocols RTP and RTCP together provide controlled delivery of the multimedia traffic. The media chuncks are encapsulated with RTP first, later it is given to the UDP for furthur transmission. RTCP provides feedback about the quality of the data delivery and information about session participants [16][27]. RTP mainly provides following functionalities to a multimedia streaming : sequence numbering, time stamping, payload identification, source identification. RTCP is used in collaboration with RTP. RTCP provides following service to the RTP session : Quality of service (QoS) feedback, participant identification, rate control and session control information.

23 13 Chapter 3 USV Video Transmission This chapter gives a detailed overview of the video streaming module for the USV. The video streaming module is based on the basic video streaming system architecture presented in chapter 2. Initially, the entities of the video streaming module like bandwidth, resolution, receiving side constraints are presented. The functionalities of the each block of the video streaming module is described. Finally, a brief overview of the equipment chosen for the video streaming module are given. 3.1 Frequency of Operation We have two unlicensed frequency bands for setting up the wireless networks. They are either in the 2.5 GHz or 5 GHz frequency band. The 5 GHz frequency band is preferred over the 2.5 GHz frequency band in terms of interference from the external sources. However, as the USV is deployed in a lake, we are not expecting many devices using the 2.5 GHz frequency band. Therefore, the extent of the external interference should not play a major role. If required, it is possible to add an extra router on the shore for frequency band switching from 2.5 GHz to 5 GHz. Secondly, the USV has stringent requirements for the equipment. As per the construction of the USV, the total weight of the all equipment together on the USV is limited. The compact equipment are preferred on the USV, in order to avoid the disturbances to the other equipment on the USV. Therefore, the antenna on the USV must be a lightweight and a compact one. From our survey for the antennas, it is found that the 5 GHz frequency band antennas are not suitable. Though the antennas have a sufficient gain, their size and the weight are not meeting the conditions to mount on the USV. 3.2 Range and Power For a proper set up of the system equipment, a link budget assessment has been performed. The involved parameter definitions and the calculations are given below.

24 Chapter 3. USV Video Transmission 14 db, dbm and dbi These are the basic units used in the radio frequency (RF) measurements. The definitions of the useful units are given in Appendix A. In each case db value is calculated with a reference, hence they are summable. Link Budget Calculation Link budget calculation is estimating the available signal power at the receiver by considering the transmit power, propagation losses and antenna gains. The communication between the transmitter and the receiver exists, when the received signal power is better than the receiver sensitivity of the receiver. Transmit power : It is the actual amount of power of the RF signal produced at the transmitter output. It is expressed either in milliwatts or in dbm. Antenna gain : The antenna gain describes how much power is transmitted in the desired direction with respect to the isotropic radiation. It is expressed in dbi. Free space path loss : It is the degradation of the signal strength happens to the transmitted electromagnetic signal as it propagates through space. It is expressed in dbm. This factor increases with the distance, other factors affecting the electromagnetic waves are obstacles such as walls, trees etc. Minimum path loss can be achieved by maintaining good LOS between the transmitting antenna and the receiving antenna. Free space path loss increases over the distance and the frequency. The free space path loss equation is given by [14] L p = 10 log ( 4πRf ) 2 10 (3.1) c In Eq.3.1. L p is the path loss in db, R is the distance in meters, f is the frequency in hertz and c is speed of light i.e Eq.3.1 can be rewritten as L p (db) = 20 log 10 (R) + 20 log 10 (4π) + 20 log 10 (f) 20 log 10 (c) (3.2) When the distance R in km and the frequency in MHz, Eq.3.2 can be written as, L p (db) = 20 log 10 (R) + 20 log 10 (f) (3.3) Receiver sensitivity : The sensitivity of the receiver shows, how well the receiver can detect minimum available RF signal and it is expressed in dbm. 1

25 Chapter 3. USV Video Transmission 15 For the first approximation, we consider a maximum distance of 1 km. The link budget calculations for the proposed setup includes, the receiving and the transmitting antenna gains, the free space path loss for the 1 km distance and the total transmitted power from the transmitting antenna is : P rx = P tx L p + G rx (3.4) In Eq.3.4, P rx : Received signal power in dbm. P tx : Transmitted power. The maximum limit for the emitted transmit power are described in EIRP limitations. The EIRP limit in our case is 20 dbm. 2 L p : Free space path loss. Frequency of operation is 2.5 GHz band, hence from Eq 3.3, for the distance of 1 km, we get free space path loss as 100 dbm. G rx : Receive antenna gain. It is the shore antenna in the setup, the gain is 15 dbi. The specifications of the shore antenna are given in Appendix A. Therefore, theoretical value for P rx is calculated by substituting the values in Eq 3.4 P rx = (3.5) = -65 dbm. 3.3 Resolution and Bandwidth Required bandwidth varies as per the video resolution, the target encoding quality, the encoding format, the frame rate and the constant/variable bit rate. For transmitting the SD (Standard Definition) videos we need average bandwidth of 2.2 Mbps, and for the HD (High Definition) videos 3.5 Mbps. In our case, a single video resolution of 640 x 480 p is sufficient. It needs the average bandwidth of 2.2 Mbps per video. We are combining and streaming the four video of 640 x 480 p resolution from the USV to the shore. Therefore, the approximated bandwidth required to stream the four videos at any instant of time is, In Eq. 3.6 B req represents the required bandwidth. B req = Mbps. (3.6) 2

26 Chapter 3. USV Video Transmission Video Transmission Module FIGURE 3.1: USV Video streaming module Henceforth, in this work, the whole setup on the USV is referred as a server-side and on the shore is referred as a client-side. The omnidirectional antenna on the USV is called as a USV antenna and the machine is named a server machine. The omnidirectional antenna on the shore is mentioned as a shore antenna and the machine is referred a client machine. Fig.3.1 shows the video streaming module with appropriate equipment for the USV. The video transmission module can be broadly divided into two parts, the server-side and the client-side. To establish a long-range video streaming, we have the antennas with sufficient gain to get the enough transmission power output and to establish a stable link between a server machine and a client machine. In the long-range wireless communication, it is common to use the directional antennas to focus the RF signals in the desired direction. However, the USV is capable of moving in all direction and it can take a rotation in an acquired position. Therefore, chances of losing the line of sight between the USV and the shore is highly likely, if omnidirectionality is not considered. This motivates us to use omni directional antennas with a suitable gain Server-side On the server-side, main equipment is the cameras, the server machine, USV antenna and the radio. The setup details of the server-side are given below.

27 Chapter 3. USV Video Transmission 17 Four web camera(logitech,c930e) are mounted in the desired position on the USV. With the cameras, we like to have 360 view. They are made water resistant by using a suitable housing. The cameras are connected to the server machine by an inbuilt USB 2.0 cable. The server machine encodes a video data of the camera. After encoding, the video data is sent to client-side via a radio(bullet M2) and the USV antenna. The OpenCV [8] libraries are used to access the connected USB cameras. H264 video encoding is implemented to encode the raw video data of the camera using the FFmpeg [1] libraries. More details about the camera accessing and the video encoding is given in chapter 4. A low loss RF cable is used to connect the radio and the USV antenna. For mounting the USV antenna on the USV, we consider the following precautions. First, the USV antenna won t get into a physical touch with the other moving equipment on the USV. Second, the USV antenna won t hurdle the normal operation of the other USV equipment during the mission Client-side At the shore, we have the setup to receive the possible weakest signal from the USV antenna. The high sensitive shore antenna on the shore receives the RF signals and amplifies it. It is possible to connect the shore antenna to the radio using a low loss RF cable or we can connect them directly. The radio on the clientside is same as on the server-side. The client machine is connected to the radio using the Ethernet cable. During the video streaming the shore antenna is placed such that, to get the LOS with the USV antenna. The client machine receives the encoded bit streams from the radio. Decoding and rendering of the video data take place on the client machine Equipment The main equipment of the video transmission modules are the radios, the antennas and the cameras. In this section, criteria followed to finalise each equipment are presented. The specifications of the equipment are given in Appendix A.2. Radio As per the applicable environment, a different kinds of routers are available. The radio has chosen for the video streaming module by looking at the various aspects and the attributes. We considered the following factors for the radio selection. Throughput : The maximum deliverable throughput of the radio should be sufficient to achieve the required data rate for the link. long-range link performance : Capability of the radio to support a longrange link.

28 Chapter 3. USV Video Transmission 18 Operating wireless mode : The router must be able to operate in the bridge mode. Power consumption : The power source for the USV is a battery or solar power. Hence, the power efficient equipment is preferred. Independent of the antenna : The selected router must be compatible with any antenna. Then only we can choose an antenna of our choice. The deployment complexity: The selected radio should not have too many settings to be done each time prior using. Plug and play integration Advanced software technology : It is better, if the observation of some parameters like a signal strength, the data rate are possible. Durability : The router is deployed in the outdoor. Therefore, the radio performance with respect to the weather conditions is also important. The radio (Bullet M2) provides distinctive attributes and a cost effective solution. Hence, it is chosen for our video streaming module and shown in Fig.3.2. FIGURE 3.2: Radio A point to point connection is needed between the server-side and the client-side. Therefore the radios are configured in the bridge mode. It creates Wi-Fi to Wi-Fi bridge between the server-side network and the client-side network. The procedure to configure the radios in the bridge mode is given in the Appendix A.2. Antenna USV antenna Along with the omnidirectionality of the antenna, there are few other important characteristics are considered for selecting the USV antenna. Those are:

29 Chapter 3. USV Video Transmission 19 The antenna gain : The selected antenna should have sufficient gain, such that we can achieve an enough transmit power. Compatibility: The antenna must be compatible with the radio. The antenna size : The USV is concerned with the weight of the equipment. Hence, the lightweight and a compactly designed antenna would be the best choice even in terms of mounting it on the USV. Weather Proof : The antenna is exposed to outdoor environments, hence weatherproof antenna is preferred. By considering the above-mentioned requirements, an omnidirectional antenna is selected. It is shown in Fig.3.3 and the technical specifications are given in Appendix A.3. FIGURE 3.3: USV Antenna Shore antenna For selecting the shore antenna, all the above said measures for the USV antenna are considered except the size requirements. The shore antenna shown in Fig.3.4. The technical specifications are given in Appendix A.3.

30 Chapter 3. USV Video Transmission 20 FIGURE 3.4: Shore Antenna Camera We have a very wide range of cameras for the selection. The main features of our interest are: Quality trade off: The professional cameras would provide the highest quality video. However, their weight is an issue and mounting those cameras on the USV is also going to be troublesome. On the other hand, the printed circuit board mounted CMOS cameras are compact and lightweight, but the mounting accessories are to be made newly. Field of view : Having a 90 field of view is suitable, such that 360 view will be covered with the four cameras. Supported resolution : Must support 640 x 480 p. FIGURE 3.5: Camera With the above requirements, we have selected a webcam (C930e, Logitech) and it is shown in Fig.3.5. It is made water resistant by a custom made housing. The technical specifications are given in Appendix A.

31 21 Chapter 4 Implementation In the hardware implementation, details of the modifications made to power on the radio on the USV is presented. The software implementation describes video streaming module for the server-side on the USV and for the client-side on the shore respectively. 4.1 Hardware Implementation As explained in section 3.5, the radio is powered up with the help of the PoE and the 230V AC input. However, on the USV, the power source is limited and only a +12 V DC power input is available. Therefore to power on the radio on the USV, a DC-DC converter is used. The power input to the DC-DC converter is +12V from the battery and it gives +24V DC power output as required by the radio. As we know, the video frame data from the server machine and the power input is provided to the radio through same Ethernet cable. Hence, an Ethernet cable with a special connector is used to integrate the video frame data from the server machine and the power input from the DC-DC converter. One end of the Ethernet cable is connected to the radio and the other end is connected to the DC power input, and to the server machine. The Fig.4.1 illustrates the arrangement. FIGURE 4.1: Hardware setup USV

32 Chapter 4. Implementation Software Implementation The details of the Operating System (OS) and the third party libraries used for the software implementations are given in table 4.1 and 4.2 respectively. Name Server machine Client machine Type Ubuntu Long Term Support (LTS) Ubuntu LTS TABLE 4.1: Operating System Name Version number OpenCV FFmpeg TABLE 4.2: Third Party Libraries The OpenCV and the FFmpeg library versions are latest available versions for the users at the time of implementation. An installation procedure of the third party libraries for the Ubuntu LTS OS is given in Appendix A. The implementation is tested only in the mentioned version number of the third party libraries and the OS Server Machine In the server machine, implementation process includes video acquisition from the cameras, concatenation of the frames from the different cameras, encoding of the video frames and the encoded frames are sent through the TCP socket. In the succeeding sections, the implementation steps are given in detail and Fig. 4.2 illustrates the server machine steps in the implementation. Video Acquisition : Cameras used for the video acquisition are Logitech Webcam C930e 1. The technical specifications and capabilities of the camera are listed in Appendix A.3. Necessary arrangements are done to mount the cameras on the USV. Four cameras are connected to the server machine by an inbuilt USB cable, it has USB type A connector. They can be connected to different USB 2.0 or USB 3.0 ports on the machine if a sufficient number of the USB ports are available. Otherwise, connecting all the four cameras to a single USB 2.0/3.0 port is also possible by employing a USB hub. Since USB 2.0 port has an effective signalling rate of 285 Mbps, multiplexing the cameras to a single USB port also works well. 1 http : //support.logitech.com/en u s/article/39606?product = a0qi v0maaq

33 Chapter 4. Implementation 23 FIGURE 4.2: Server Machine

34 Chapter 4. Implementation 24 The four cameras connected to the server machine are accessed by the OpenCV libraries, particularly by the VideoCapture class 2. It provides various options for adjusting the USB camera parameters like setting the frame resolution, frame rate etc. The VideoCapture class is used to check the camera accessing errors. Set Frame Resolution : Each of the cameras is able to provide a maximum resolution of 1080p. However, applicable video resolution in our scenario is 480p. Therefore, the cameras are set to take the video of 480p resolution. The VideoCapture class of the OpenCV library provides set property options. By using the concerned property Identification (ID) options, we can adjust the width and the height of the connected camera for the frame capturing. Frame Concatenation : Each of the obtained video frames from the cameras can be transported independently to the client machine. In that case, the problem might arise in synchronizing the videos at the client machine and the available bandwidth is shared among the videos. Therefore for the ease of the implementation, concatenation of the four videos are done at the server machine prior encoding. The frames from each of the camera are grabbed sequentially. In the first stage, frames of the first two cameras are horizontally concatenated, then frames of other two cameras are horizontally concatenated. Finally, both the horizontally concatenated frames are vertically concatenated to form a 1280 x 960p complete frame resolution. The OpenCV library provides the functions hconcat and vconcat for the horizontal frame and the vertical frame concatenation respectively. Step 1 : In the first step, frames from the camera 1 and the camera 2 are horizontally concatenated. Each frame of the cameras is 640 x 480p resolution. After the horizontal concatenation of the two frames resulting frame is 1280 x 480p resolution. We can observe that height of the frame is unchanged and the width of the new frame is double the single frame width. Step 2 : In the second step, frames from the camera 2 and the camera 3 are horizontally concatenated like said in step 1. Resolution after the concatenation is 1280 x 480p. 2 http : //docs.opencv.org/3.0 beta/modules/videoio/doc/reading a nd w riting v ideo.html

35 Chapter 4. Implementation 25 Step 3 : In the final step, horizontally concatenated frames from the step 1 and the step 2 are vertically concatenated. Resolution of the final concatenated frame is 1280 x 960p, here the width is unchanged and the frame height is doubled. Video Encoding : Even though the camera (Logitech,C930e) data sheet shows that the camera is able to provide the H264 encoded frames, the encoded frames can be accessed only by their collaborative partners. For instance, it is possible to get the h264 encoded frames in Skype 3 calling. One more bottleneck is, the OpenCV C++ API won t provide any access to get the encoded frames from the camera directly. As soon as we read the frames they are the raw frames. Therefore, in this work, the video encoding is done on the concatenated video frames. The video streaming implementation is done such a way that it is camera type independent. The camera type independent implementation has the following benefits. First, if required the implementation works with any of the available USB camera (which is different from the camera used in this work) in future. Secondly, the implementation does not require any camera libraries. Hence, a plug and play solution is provided with regard to the camera type. The H264 video encoding is implemented using the FFmpeg libraries. The H264 standard is widely used and is more efficient compared to MPEG 2 part 2 video encoding and an image compression techniques. The bandwidth requirements are higher in the MPEG 2 part 2 and in the image compression techniques. Steps in encoding are shown in the Fig.4.3. The encoder parameters like the bit rate, Quantization Parameters (QP), frame rate, pixel format are initialized accordingly before feeding the concatenated frame for encoding. 3

36 Chapter 4. Implementation 26 The QP affect the quality of the video. The FFmpeg library provides q min and q max for setting the quantization range. q min stands for minimum quantization and q max stands for maximum quantization. We need to assign values for q min and q max. Minimum value possible for q min is 0 and the maximum value of q max is 51. Hence, the QP together q min and q max can vary from 0 to 51 in FFmpeg library. For instance, q min 50 and q max 51 gives the lowest quality of the video and q min 0 and q max 1 gives the highest quality. FIGURE 4.3: Encoding steps The width and the height of the concatenated frames are set as the codec height and the width respectively. The video is acquired at 30 frames per second (FPS) from the camera. Therefore, the encoding frame rate is set as 30. The AVFrame structure of the FFmpeg library holds the raw video or audio data. While forming the AVFrame structure, we need to indicate the pixel format of the raw video frame data, raw frame height and raw frame width. In the frame formation step, the concatenated frame raw bits are represented in the AVFrame structure. The formed AVFrame structure has BGR24 pixel format, frame width is 1280 and the frame height is 960. The raw frames we get from the camera using the OpenCV library are in BGR24 pixel format. In BGR24 pixel format, each pixel is 8 bit long and has three channel components. In the frame scaling step, the pixel format from the BGR24 to YUV420 takes place. The input to the scaling is the AVFrame structured video frame, and the output is the destination frame. We have the AVFrame structure

37 Chapter 4. Implementation 27 with the required frame resolution i.e x 980p. Therefore, the frame scaling with respect to the frame resolution is not needed. The destination frame pixel format is YUV420. Finally, the scaled frame is encoded to get the encoded bit streams. The encoded bit streams are stored in AVPacket structure of the FFmpeg library. Video frame sending : Once the encoding is done, the frame data is sent to a client machine via the TCP socket. We can access the current encoded frame packet size and the packet data from the AVPacket structure. The sending of the video frame is done in two steps. First, the packet size is sent to the client machine, in order to create a buffer with the received packet size at the client machine. Secondly, the packet data is sent to the client machine. The created buffer of the current packet size at the client machine is employed to store the packet data. After the successful completion of sending the encoded concatenated frame, the next frames from the four cameras are fetched. The above said process repeats until the client machine is receiving the encoded data from the server machine. Once the client machine is turned off, the cameras are released by the VideoCapture class. When there is no connection between the server machine and the client machine, the server machine stays in the standby condition and listens to the connection. Once the connection is reestablishes between them again, the server machine starts fetching the video frames from the cameras and processes them Client Machine On the client machine, the video frame data is received through the TCP socket. The received video frame is decoded and displayed to the user. Fig.4.4 illustrates the flow diagram of the client machine and subsequent steps are explained below. The reception of video frame : The video frames are received via the TCP socket. It is carried out in two steps. First, the packet size is received and as per the packet size, the buffers are created. Secondly, the packet data is received until the intended packet size. The packet data is stored in the created buffers. Construction of a video packet : Decoding of the compressed data is done with the FFmpeg library. The compressed data must be in AVPacket structure format. We have the packet size information and the packet data from the previous step. In this step, the compressed data is represented in AVPacket format using the packet size and the packet data. Additionally, it is possible to save the compressed data on the local memory of the client machine. This can be played later using appropriate video players on the client machine. This enables us to view the video of the

38 Chapter 4. Implementation 28 FIGURE 4.4: Client Machine

39 Chapter 4. Implementation 29 USV s mission even in the future if required. Saving the compressed data on the client machine is an optional but not mandatory step. Decode the video packet : We know that on the server machine the H264 encoding is performed to encode the video frames. In the client machine, H264 decoding is performed on the compressed video frames. Before decoding, necessary initialization is done for the decoder. Initialization includes forming an empty AVFrame structure to hold the decoded data as per the bit pixel format, width and height of the decoded frame. If decoding is successful, then the decoded data will be AVFrame and it is given for frame scaling. In case of decoding failure, a decode failure message will pop up and it reads the next frame. The decoded frame includes the bit streams in YUV420 pixel format. We are using the OpenCV libraries for displaying the video frame. The OpenCV library needs the data to be in the BGR24 pixel format. Hence, in the scaling step, the pixel format conversion from YUV420 to BGR24 takes place. Video representation : Video is displayed in a concatenated format, it is done using imshow() function of OpenCV library. The imshow() function only accepts the video frame in MAT 4 structure. MAT is the OpenCV matrix structure. The scaled video frame data is pushed to MAT structure, such tha it can be displayed to the user using imshow() function of the OpenCV. 4 http : //docs.opencv.org/3.1.0/d3/d63/classcv 11 Mat.htmldetails

40 30 Chapter 5 Evaluation This chapter gives the details about the measurement setup, experimental results followed by the discussion. The measurement setup includes the test location and briefly about the equipment used in the experiment. The experimental results are presented in two parts. First, the WLAN link performance results are given and secondly, the video transmission performance are described. A hypothesis for the results are given in the discussion part. 5.1 Measurement Setup In the beginning of this work, it was planned to experimentally test the video streaming module performance along with the prototype of the USV. However, the prototype development of the USV was delayed due to the unavailability of crucial components. At the end of this thesis work, the USV s intermediate version was available as shown in Fig.5.1. It is constructed with wheels for movement on the ground. Therefore, testing of the video streaming module with the USV on the lake could not be done as planned initially. As an alternative, the video streaming module performance is tested for the long range on the ground and on the lake. Therefore, separate mechanical arrangements are made to mount the antennas during testing and a battery power source is employed as an equivalent to the USV s internal power supply Test Location Initially, the experiments are conducted in the court of the IBMT for distances up to 120 m. For longer distances on the ground a long and straight road has been chosen. It is easily accessible having the client machine and the server machine in two cars using the battery as a power source. For long distance lake measurements only a few lakes in the vicinity have been considered. The chosen test locations for the ground experiments and the lake experiments are shown in Fig.5.2a and Fig.5.2b respectively. The ground experiments are done near Kirkel(Saarland State, Germany). and the lake experiments are performed at Lake Würzbach(Saarland State, Germany).

41 Chapter 5. Evaluation 31 FIGURE 5.1: Preliminary USV (A) Ground (B) Lake FIGURE 5.2: Test Location Test Setup As stated before the video streaming module contains two endpoints the serverside and the client-side. The wireless bridge mode is established for long distance using the radios. The procedure to configure the radios in a bridge mode is given in Appendix A.4. Ground Scenario : At the test location, the client-side setup were stationary and a diesel generator was used as a power source. The server-side setup were moved to various approximate distances like 200 m, 400 m, 600 m and 800 m from the client-side. At each distance the measurements were made. A car battery and a DC-AC converter is used as a power source for the server setup. Lake Scenario : In the lake experiments, the client-side setup were made in a particular area at the shore. Due to unavailability of the USV, we could not take the measurements on the water surface within the lake.

42 Chapter 5. Evaluation 32 Instead, the server-side setup were moved to a different area of the shore, such that there was always a water surface between the server-side and the client-side. The antenna on the server-side (USV antenna) was kept at a height of 1 m to a water surface to simulate a real scenario to our best ability. Fig.5.3a shows the placing of the USV antenna in the lake scenario. The measurements were done at approximate distances 200 m and 700 m. The measurements were limited to two points since the shore of the Lake Würzbach is not accessible for our type of measurement at many places. During the measurements, the already existing 230V AC power source is extended to the client-side setup. A battery source and a DC-AC converter are employed for the server-side setup. Antenna placement : The minimum and the maximum height of the USV from a ground surface is 0.4 m and 1.2 m respectively. Therefore, the final antenna mounting on the USV can vary from m. During, both the ground and the lake experiments, the USV antenna is placed approximately at 1 m height from the ground/water surface. The shore antenna height itself is 1.72 m (table 3.3) and the bottom end of the shore antenna was about 0.5 m from the ground surface. Fig.5.3 shows the placement of the USV antenna and the shore antenna during the lake experiment. (A) USV antenna placement (B) Shore antenna placement FIGURE 5.3: Antenna placement (Lake) Video data : For the evaluation of the video transmission performance, four cameras were connected to the server machine and live streaming was observed in the client machine. A transmitted sample video frame is shown in Fig.5.4, which is from the initial setup at IBMT court.

43 Chapter 5. Evaluation 33 FIGURE 5.4: Video frame 5.2 Experimental Results In this section, experimental results are presented for the long range WLAN link and for the video streaming between the server machine and the client machine. Evaluation of the long range WLAN link is done with respect to the environment parameters like the received signal strength, link bandwidth, percentage loss rate and the Round Trip Time (RTT). The video streaming results contain the bandwidth required for transporting the four videos, the delay between producing the video in the server machine and displaying the video on the client machine, the maximum loss rate of the channel up to which the video streaming performance is satisfactory Wireless Link Performance The link performance is evaluated both on the ground surface and on the water surface. The following parameters are considered: the available bandwidth, the Received Signal Strength (RSS), the RTT and the loss rate against the distance. On the ground surface, an approximate measurement distances are 200 m, 400 m, 600 m and 800 m. On the water surface, approximate measurements points were, 200 m and 700 m, due to the limitations for the shore to shore setup (accessibility). Bandwidth vs Distance Fig.5.5 shows the variation of the bandwidth at various distances. The Iperf tool[6] is used to measure the bandwidth between the server and the client machine. A procedure to measure bandwidth using the Iperf tool is given in Appendix C.1. The window size of the TCP is set to 256K. Each traffic session lasts for 60 seconds and the values of the bandwidth are reported every second. In

44 Chapter 5. Evaluation 34 Fig.5.5, the averages of the bandwidth are plotted with respect to the distance of two endpoints. FIGURE 5.5: Bandwidth vs Distance The variation of the average bandwidth on the ground surface is observed from 73 Mbps to 32 Mbps with respect to the measured distance from 200 m to 800m. On the water surface, the average bandwidth at the distance of 200 m is about 24 Mbps and 13 Mbps at the distance of 700 m. Received Signal Strength vs Distance Received signal strength is observed using the radio web interface airos 1 from Ubiquiti networks. A brief description of airos is provided in Annexure B. Fig.5.6 shows the received signal strength variation at the different distances. On the ground surface, the received signal strength is observed from -53 dbm to -75 dbm as the measuring distance varied from 200 m to 800 m. On the water surface, it is -73 dbm at the distance of 200 m, and -82 dbm at the distance of 700 m. The variation of the Received Signal Strength (RSS) observed with varying the height of the antenna from 0.4 to 1m from the water surface. Loss Rate vs Distance To measure the approximate loss rate of the link, we produce a UDP traffic over the link via Iperf. The average percentage loss rate is about 1% when the data is transmitted at the rate 5 Mbps below the available bandwidth. It is up to 25% when the target bandwidth for the UDP traffic is set to the available bandwidth achieved by the TCP. 1

45 Chapter 5. Evaluation 35 FIGURE 5.6: Signal Strength vs Distance RTT vs Distance RTT is the time required for a packet to reach the specified destination from the source and get back to the source. It can be obtained by using the ping command on one of the machines. It is observed during the measurement that the RTT increases with the distance. The RTT variation as per the distance is given in Fig.5.7. FIGURE 5.7: RTT vs Distance

46 Chapter 5. Evaluation Video Transmission Performance The performance of the video streaming is assessed using the following three parameters : the actual bandwidth for transporting the video, the delay between video frame availability at the client machine with respect to the video capture at the server machine and the tolerable percentage packet loss rate for viewing the video at the shore. Actual Bandwidth The required bandwidth for the video transmission and the video quality varies with the encoder parameter settings. The encoder settings are given in table 5.1. The video recording from the camera is done at 30 FPS, therefore, the encoding frame rate is set to 30. The target bitrate and the GOP values are set as per the Youtube recommended encoder settings 2 3. The GOP is chosen as half of the FPS. Parameter Value set FPS 30 Bitrate 1 Mbps GOP 15 TABLE 5.1: Encoder Parameters The actual bandwidth is measured by using the system monitor tool for Ubuntu LTS OS on the server machine. Under the network history tab of the System Monitor tool, consumed bandwidth observation could be done for 60 sec. The bandwidth was varying from 2.5 Mbps to 4 Mbps, Fig.5.8 depicts the bandwidth variation. FIGURE 5.8: Bandwidth for four videos

47 Chapter 5. Evaluation 37 Video Latency The video latency is the total time taken from capturing the video frame in the server machine to displaying the same video frame in the client machine. It is calculated as below, T vl = T s + T rtt + T c (5.1) In Equation 5.1,T vl represents the end to end delay. T s indicates the time elapsed on the server machine, which includes the time for video frame capturing, encoding and sending. T rtt is the frame transmission time between the server machine and the client machine, we estimate it as RTT/2. T c indicates the time elapsed on the client machine, which includes the receiving, the decoding and the rendering of the video frame. The maximum video latency is calculated by taking the maximum observed value of T s and T c and T rtt for 200 m distance on the lake is, T vl = 180ms + 8ms + 400ms = 588ms (5.2) Apart from the above calculations, we observed the start-up delay of about 1.3 seconds. Video Streaming vs Packet Loss The packet loss in the channel is introduced by using the netem tool [7]. A brief overview of the netem tool is given in Appendix C.2. The video latency is calculated and tabulated against the percentage of packet loss rate in table 5.2. Loss rate percentage Video Latency 2% 695 ms 5% 696 ms 10% ms (occasionally observed) 12% ms (frequently observed) 15% ms TABLE 5.2: Loss rate vs Video Latency

48 Chapter 5. Evaluation Discussion RSS and Bandwidth It is evident from the Fig.5.5 and 5.6 that both on the ground and the lake experiments, the network bandwidth and the RSS are going down with the distance. In the ground experiment, the test location shown in the Fig.5.2a is a road and is surrounded by the greenery on both the sides. The continuous movement of the vehicles on the road and reflections from the trees is expected to have a significant effect on the results either positively or negatively. From Eq.3.3, the theoretical RSS for the distance of 800 m is -63 dbm for our setup. However, the Fig.5.6 shows that for 800 m distance the measured RSS is -73 dbm. In the lake experiment, the RSS is less compared to the ground experiment for the same measured distance. For instance, at 200 m distance between the shore antenna and the USV antenna, the RSS is approximately reduced by 20 dbm in the lake experiments. The reasons could be RF reflections on the water surface and RF absorption in the water. At 200 m distance on the lake, when the USV antenna height varied by a half meter from the water surface, up to 5 dbm improvements in the RSS is noticed. The RSS was around -81 dbm when the USV antenna is 50 cm to the water surface. We observed the best values when placing the USV antenna at a height of 1.5 to 2 m. The reason might be, when the USV antenna at the height 1.5 to 2 m, it had the best alignment with the shore antenna. In the real scenario, mounting the USV antenna at the height 1.5 to 2 m from the water surface could give the best results. Therefore, the optimal values are considered because placing the USV antenna at a height of 1.5 to 2 m from the water surface is practical for our real scenario (mounting of the antenna on the USV). Furthermore, at 700 m distance on the lake, the USV antenna height maintained was 1.5 to 2 m from the water surface. We considered placing the USV antenna at different places since the shore antenna was not visible. Up to 4 dbm variation in the RSS was found at distinct places. From the Fig.5.5, we can see that the bandwidth is going down with the distance both on the ground and the lake. The bandwidth is correlated with the RSS. For instance, consider the bandwidth and the RSS at 200m distance, from the Fig.5.5 and 5.6, the bandwidth values are 70 Mbps, 25 Mbps and the RSS values are -53 dbm, -73 dbm respectively. RTT and Video latency Estimating the loss rate and RTT against the distance is helpful in predicting the video transmission performance in terms of the video latency. When the

49 Chapter 5. Evaluation 39 required bandwidth is close to the available bandwidth of the WLAN, the loss rate is high. It is observed that when the transmission is done with 3 Mbps less than the maximum available bandwidth, the loss rate was low. The RTT directly affects the video latency. From the Fig.5.7, we can see that the RTT increases with the distance. Video Bandwidth The average bandwidth required for transmitting the four videos for our configuration is 3.5 Mbps. From the Fig 5.5, we can see that the available bandwidth for 800 m distance on the ground and 700 m on the lake are well above the required bandwidth. The four videos are transmitted without suffering from the high loss and large delay. The video latency sums up to 600 ms to 700 ms. The major delay of 350 ms to 400 ms is on the shore machine, "the reason could be a lower processing power of the client machine". The server machine had Intel core i5 processor and 4 gigabytes (GB) random access memory (RAM). The client machine had Intel Pentium 4 processor and 1 GB RAM. The received video could be viewed without much delay when the packet loss rate is up to 10%. In the case of the packet loss rate higher than 10%, the video latency is higher. The implementation is done on TCP, by using the UDP this delay would be less. Since there is no retransmission mechanism in UDP for the lost packets. Additionally, in case of unfavourable conditions for transmitting the four videos, a single video transmission is also possible since, it requires only 1 Mbps of bandwidth. In summary, we could find from the results that it is essential to have the sufficient RSS for transmitting a video from the USV to the shore. Therefore, further optimizing the existing setup or having more sophisticated equipment for the future version of the measurements are reasonable. For the current setup, the antenna height can be raised further with the help of a post, in order to get a better RSS. In a real scenario, antenna height could be adjusted to an optimal height by calibrating the link between the shore and the USV for a shorter distance. For example, acceptable RSS value could be -60 dbm for 50 m distance, it gives bandwidth value about 70 Mbps. For a shorter distance, if the link behaviour is as good as above, we can expect a sufficient performance for the long distance. Furthermore, when the USV is under the mission, the maximum coverage of the view angle is 180 from the shore regardless of the USV s position. Therefore using a sector antenna with an appropriate beam width is feasible. The sector antenna can witness better RF reception, this improves the RSS for the link. If the sector antenna has the beam width less than 180 or if the USV is located at the position which is not in a coverage area of the sector antenna, we can always

50 Chapter 5. Evaluation 40 adjust the sector antenna in the direction of the USV position for the better RF reception.

51 41 Chapter 6 Conclusion and Future Work In the beginning of this thesis work, an importance of the video surveillance for the USV was discussed. Then the requirements of the video streaming module is presented, it involves video resolution, degree of coverage and maximum range. The long-range wireless network was designed with suitable equipment for video streaming up to few hundred meters. The selection criteria for equipment are: quality, cost effectiveness, compatibility, complexity of handling and suitability for deploying in an outdoor environment. Validation of the equipment was done by conducting short distance tests. The software implementation for the video streaming includes the server machine and the client machine. The server machine does video capturing, video encoding and sending the video data via transport protocols. In the client machine, video decoding and video rendering are implemented. The implementation is independent of USB camera type and libraries. In addition, hardware implementation is done to enable the video streaming in the limited power resource USV s platform. Those are, a DC-DC converter for the radio, a suitable cable and connectors to integrate server machine data and the DC power together. The long-range video streaming measurements were conducted on the ground and the lake in the absence of the prototype USV. The experiment on the lake was conducted up to a distance of 700 m and on the ground up to 800 m. The suitability of the long-range wireless network is confirmed by checking the available link bandwidth, RTT and the loss rate. Later, we have transported a combined video of the four cameras and each video of 480p resolution. The actual bandwidth variation was 3 to 4 Mbps and a packet loss was introduced manually for further validation. Additionally, we found the approximate height for mounting the antenna on the USV for the best performance. The outcome of the experiments shows that the proposed module is suitable for a long distance communication and video streaming. While this thesis work demonstrated the video streaming for the USV, there are many opportunities for extending the scope of this thesis remain. If the WLAN link is upgraded as discussed in chapter 5, a high-definition (HD) video transport is achievable. Following features may be added to the video streaming

52 Chapter 6. Conclusion and Future Work 42 platform to cope with a severe link degradation. First, a graphical user interface can be provided at the shore for entering the resolution of the video, gray color video and frame rate of the video as per the link condition. Secondly, automatic change of the video resolution and frame rate by probing the link bandwidth. Adding an underwater camera is a welcome feature. It may be helpful in getting a more information about the underwater objects and in positioning the USV precisely for the measurements. In case of higher packet loss rate UDP transport performs better. By employing an error correction on the UDP transport a better bandwidth utilization also possible compared to TCP transport. Integrating the received video stream on the mobile devices is also worthwhile.

53 43 Appendix A Equipment Specifications A.1 RF units db (Decibel): It represents difference in the two signal power level. It is calculated by taking the logarithm (log) ratio of the received or measured power with respect to a reference power. Let P1 is a received or measured power, and P2 is the reference power, then db will be calculated as, db = 10 log 10 (P 1/P 2) (A.1) dbm (db milliwatt) : dbm represents the specified db value with respect to the 1 milliwatt reference level. Let P1 is the measured signal level then dbm is, dbm = 10 log 10 (P 1/1mW ) (A.2) dbi (db isotropic) : dbi represents the gain of an antenna with respect to the isotropic radiated power. Let P1 is the antenna radiated power and P2 is the isotropic antenna power then gain of the antenna is given by dbi = 10 log 10 (P 1/P 2) (A.3)

54 Appendix A. Equipment Specifications 44 A.2 Antenna USV antenna The technical specification of the USV antenna is given in table A.1. Name Specification Type ART5179 Gain 14 dbi Impedance 50 Ohm VSWR Max 1.5 Polarization Vertical(Linear) Beam width 360- Horizontal, 16- Vertical Dimensions 15cm x 10cm Weight 350g TABLE A.1: USV Antenna 1 Shore antenna The technical specifications of the shore antenna is given in tablea.2 A.2. Name Specification Type SAG24015 Gain 15 dbi Impedance 50 Ohm VSWR Max 1.5 Polarization Vertical(Linear) Beam width 360- Horizontal, 16- Vertical Dimensions 172cm x 2.5cm Max input power 150 watts Weight 950g TABLE A.2: Shore Antenna dBi 2 Rundstrahlantenne-Wetterfest-15dBi

55 Appendix A. Equipment Specifications 45 A.3 Camera The camera employed in this thesis work is a conventional webcam, shown in Fig.A.1. The camera technical and mechanical specifications are given in the table A.3. FIGURE A.1: Camera(Logitech C930e) Name Specification Type Logitech C930e Field of View 90 deg Supported highest resolution 1080 x 720 Focus Autofocus Frame rate 30 FPS Weight 393 g TABLE A.3: Camera(Logitech C930e) 3 3

56 Appendix A. Equipment Specifications 46 A.4 Radio The radio (Bullet M2) is shown in Fig.A.2. The technical specifications are given in the table A.4. Powering the radio, the bridge mode configuration between the radios is given in the subsequent sections. (A) Top view (B) Bottom view FIGURE A.2: Radio(Bullet M2) Name Type Operating Frequency Output Power RF connector Power rating Power Method Max Power Consumption Weight Specification Bullet M GHz 28 dbm N type Male jack Up to 24 V Passive Power Over Ethernet 7 Watts 180g TABLE A.4: Radio(Bullet M2) Radio Powering : The radio (Bullet M2) is powered by using the PoE adaptor as shown in the Fig.A.3. The input to the PoE is 230V AC, and it gives 24V DC output. It has two Ethernet ports namely PoE and LAN. Connect an Ethernet cable from the radio to the PoE (port name) of the PoE. Connect one more Ethernet cable from LAN port of the PoE to a machine. Radio Configuration Radio comes with a web interface application peripheral interface (API) called "airos". Following steps to be carried out for setting up the radio on a machine. 1. Configure the Ethernet adaptor on a computer with static IP address. For example IP address: and subnet mask :

57 Appendix A. Equipment Specifications 47 (A) PoE (B) PoE closeview FIGURE A.3: Power Over Ehernet 2. Launch the web browser. Enter and default IP address of the device (Default IP: ) 3. Upon initial login, Terms of Use appear on the login screen. Enter ubnt in the username and password fields, and select the appropriate choices from the Country and Language drop-down lists. Check the box next to I agree to these terms of use, and click Login. 4. Upon subsequent login, standard login screen appears. Enter ubnt in the User-name and Password fields, and click Login. 5. After login, Ubiquiti main page appears as shown in Fig.A Repeat the same step for the second radio also. FIGURE A.4: Radio Main Page Since we need a point to point connection between the USV and the shore, set the radios in a bridge mode.

58 Appendix A. Equipment Specifications 48 (A) Wireless Tab (B) Network Tab FIGURE A.5: Bridge Mode Bridge mode settings: Configuring the radio for the server machine and the client is given below. Server Machine : Open the Ubiquiti radio using web browser as stated before, after logging in, click on the "wireless" tab, it looks like as shown in Fig.A.5a and follow the steps below. 1. Select wireless mode as "station" in basic wireless settings. 2. Mark the check box for transparent wireless bridge. 3. Country code as applicable. 4. IEEE is set as B/G/N mixed by default. 5. SSID would be named as we want. 6. Channel width is "Auto 20/40 MHz". 7. Mark the "EIRP" enable check box. 8. Enter the antenna gain as per the connected antenna, the radio gain is adjusted as per antenna gain and EIRP limits of the country mentioned. Now go to "Network" tab, carry out the following. 1. under the network role section, select network mode as "Bridge". 2. Disable network to none. 3. Configuration Mode as simple. 4. Management network settings, set the Management IP address as static, enter the IP address, subnet mask address etc. 5. After the above said settings, click on the change option below, it asks for confirmation, click refresh and login again, now the server side bullet M2 radio is ready to be operated in bridge mode. Client Machine :