Dynamic Variations in the Speed of a Digital Video Stream due to Complexity of Algorithms and Entropy of Video Frames

Transcription

1 International Journal of Applied Environmental Sciences ISSN Volume 12, Number 2 (2017), pp Research India Publications Dynamic Variations in the Speed of a Digital Video Stream due to Complexity of Algorithms and Entropy of Video Frames S. Aparna 1 and M.Ekambaram Naidu 2 1 GITAM University, Hyderabad, India. SRK Institute of Technology, Vijayawada, India. Abstract This paper discusses the dynamic variations of frame rate in a live digital video due to complexity of video processing algorithms and entropy of frames. A number of video processing algorithms are applied on a live video captured by a web camera and frame rates are observed. It was observed that frame rate reduces not only due to complexity of an algorithm applied on a streaming video but also on the information content, that is, the entropy associated with each running frame. Keywords: Digital Video, Frame Rate, Complexity of Video Processing Algorithms 1. INTRODUCTION The three major parameters associated with video quality are (i) speed, (ii) size and (iii) clarity. Speed of a digital video stream is quantified by Frames per Second (FPS). This means the number of frames displayed or communicated each second. Usually, FPS is considered as a standard in a digital video display or communication. However, each user would have a different FPS, depending on the computer, camera type, video size and the internet connection speed of say each conference participant, especially in a video conference. The purpose of this paper is to highlight the effect of frame entropy[2] and algorithmic complexity on FPS, especially when digital video processing of main concern. Secondly, the size of the frame plays a significant role in maintaining FPS during display or communication.

2 242 S.Aparna and M.Ekambaram Naidu The term size refers to the number of pixels displayed in a frame which in turn quantifies its resolution defined by the frame width and frame height. Usually ispq makes use of 320x240 resolution by default for capturing video. The next important parameter that determines video quality is clarity. One would come across many factors that determine how crisp and clear a video image will appear. Most of the video conferencing facilities give top priority to image clarity even at the cost of FPS, which means that the video quality remains within acceptable limits, even if the FPS is reduced to an acceptable level. In a way one would have a trade-off between FPS and clarity. In this context, video image processing becomes an essential operation in order to maintain video quality. The problem faced, in such a case, is an additional overhead to maintain speed and quality. This paper addresses the problem of dynamic variations in the speed of streaming video while using various image processing algorithms to process a live video[3] stream frames. 2. EFFECTS OF ALGORITHMIC COMPLEXITIES ON FPS Complexity of an algorithm is evaluated based on the number of computations involved in processing a video frame. For example, let us consider an algorithm meant for detecting edges in a video frame. Algorithm: Rajan2-Cellular Logic Array Processing Based 2-Dimentional Edge Detection Input: 2-Dimensional image with 'T' Output: 2-Dimensional image after Edge detection Steps: Step 1: Read the pixels from 2-Dimensional image and place pixel values in a 1- Dimensional array called Input array. Step 2: Copy input array to output array Step 3: Repeat the steps 1 & 2 sliding the 5-neighborhood window over the image (input array) Start: Step 3(a): Find MAX(0,1); Step 3(b): Find MIN(0,1); Step 3(c):Find difference D = MAX(0,1)-MIN(0,1); Step 3(d): If(D <= T) Then Assign CP=0 in value output array;

3 Dynamic Variations in the Speed of a Digital Video Stream due to Complexity Else Find the slide the 5-neighborhood End: Repeat this process until the structuring element scans Step 4: Pass the output array to Display() the whole of the image Complexity Calculation In the Cellular Logic Array Processing[7] based edge detection algorithm Rajan2, steps 3(a), (b), (c) and (d) are executed a maximum number of times. The time complexity of this computation is evaluated as O(n 2 r 2 ). The repetition of steps 3(a),, (b), (c) and (d) takes place with the complexity of O(n 2 ). The total time complexity of 2Dimentional edge detection by using Cellular Logic Array Processing -Rajan2 method is an image of size nxn by a structuring element of size rxr is O(n 2 r 2 )+O(n 2 ). Let us also consider an algorithm meant for skeletonizing a video frame. Algorithm: Rajan 2-Cellular Logic Array Processing Based 2-Dimensional Skeletonization Input: 2-Dimensional image, threshold Output: 2-Dimensional image after Edge detection Steps: Step 1: Read the pixel from 2-Dimensional image and place in a 1-Dimensional array called input array. Step 2: Move the elements in input array to output array Step 3: Repeat the step 1 & 2 and slide the 5-neighborhood window over the image (input array) Start Step 3(a): Find the MAX(0,1); Step 3(b): Find the MIN(0,1); Step 3(c):Find the difference D = MAX(0,1)-MIN(0,1); Step 3(d): If(D <= T) Then Retain the CP as well as corner pixels and remove the boundary pixels in output array Else slide the 5-neighborhood

4 244 S.Aparna and M.Ekambaram Naidu End Repeat this step until the structuring element spans the whole of the image. Step 4: Copy output array to input array and repeat step 3 until there is no boundary left for removal. Step 5: Pass the output array to Display() Complexity Calculation In the Rajan2 Cellular Logic Array Processing based skeletonization, steps 3(a), (b), (c) and (d) are executed a maximum number of times. The time complexity is evaluated as O(n 3 r 2 ). Because the outer while loop is executed until both input buffer and output buffer are equal. In the worst case while loop executes for n times. That is, the time complexity of 2-D skeletonization by using a Cellular Logic Array Processing method on an image of size nxn by a structuring element of size rxr is O(n 3 r 2 ). With these we can conclude that the overall time complexity of an algorithm not only depends on the complexity of the algorithm but also on the number of instructions used to implement it. Here CP stands for Center Pixel, T stands for Threshold 3. EFFECTS OF VIDEO FRAME ENTROPY ON FPS Entropy of a digital image[8] is a statistical quantification of the information content in the image. Larger the entropy more the information contained in an image. Figure 1(a) shows a minimum entropy[5] image and figure 1(b) maximum entropy[4] image. The term entropy refers to the degree of randomness of information. Real time video communication employs an image quality parameter Transmitted Information T I which is briefly described here. Given events S1,..., Sn occurring with probabilities p(s1),..., p(sn), then the average uncertainty associated with each event is defined based on Shannon entropy[1] as Let x and y be the input and output random variables and their entropies H(x) and H(y), respectively. Joint entropy, H(x, y), is defined as, where Hx(y) and Hy(x) are conditional entropies. Hx(y) is the entropy of the output when the input is known and Hy(x) that of the input when the output is known. Now, the transmitted information TI could be computed as T(x; y), where:

5 Dynamic Variations in the Speed of a Digital Video Stream due to Complexity where pi=ni/n, pj=nj/n, and pij=nij/n. One can rewrite the above equations as given below. (a) Minimum entropy image; (b) Maximum entropy image Figure 1: Sample images Figure 2 shows a real time video frame captured by a web camera and its histogram Figure 2: A sample video frame captured by a web camera

6 246 S.Aparna and M.Ekambaram Naidu Complete video image frame statistics is given below. Histogram of the image clearly shows that the entropy of the image is small. One would visualize that the histogram of an image with maximum entropy would be sparsely distributed. Pixels Count Pixels without black Red Min 0 Red Max 255 Red Mean Red Standard Deviation Red Median 158 Red Total Count Green Min 5 Green Max 255 Green Mean Green Standard Deviation Green Median 157 Green Total Count Blue Min 3 Blue Max 255 Blue Mean Blue Standard Deviation Blue Median 151 Blue Total Count Saturation Min 0 Saturation Max 1 Saturation Mean Saturation Standard Deviation Saturation Median Luminance Min Luminance Max Luminance Mean Luminance Standard Deviation Luminance Median Y Min Y Max Y Mean Y Standard Deviation Y Median Cb Min Cb Max

7 Dynamic Variations in the Speed of a Digital Video Stream due to Complexity Cb Mean Cb Standard Deviation Cb Median Cr Min Cr Max Cr Mean Cr Standard Deviation Cr Median Red Min WB 0 Red Max WB 255 Red Mean WB Red Standard Deviation WB Red Median WB 158 Red Total Count WB Green Min WB 5 Green Max WB 255 Green Mean WB Green Standard Deviation WB Green Median WB 157 Green Total Count WB Blue Min WB 3 Blue Max WB 255 Blue Mean WB Blue Standard Deviation WB Blue Median WB 151 Blue Total Count WB Saturation Min WB 0 Saturation Max WB 1 Saturation Mean WB Saturation Standard Deviation WB Saturation Median WB Luminance Min WB Luminance Max WB Luminance Mean WB Luminance Standard Deviation WB Luminance Median WB Y Min WB Y Max WB Y Mean WB Y Standard Deviation WB

8 248 S.Aparna and M.Ekambaram Naidu Y Median WB Cb Min WB Cb Max WB Cb Mean WB Cb Standard Deviation WB Cb Median WB Cr Min WB Cr Max WB Cr Mean WB Cr Standard Deviation WB Cr Median WB Figure 3: A salt and pepper noise with its histogram 4. THE EFFECTS OF A VIDEO FRAME ENTROPY ON FPS An empirical study was undertaken to verify the effects of applying various algorithms on a streaming videos and results observed. Part of the results of the study is presented in figures 4(I) to (XXXV) and table 1. One may observe that Seven out of 69 Algorithms reduce the FPS.Among them few images are grey scale morpholgy[9] Rajan1 Filtered (13.78 FPS) (I) Rajan2 Filtered Th. 40 (7.31 FPS)

9 Dynamic Variations in the Speed of a Digital Video Stream due to Complexity Sobel Filtered (23.26FPS) Laplacian Filtered (22.92 FPS) (II) Prewit Filtered Kirsch Filtered (23.62 FPS) (III) Morphological Dilated (15.50 FPS) Morphological Eroded (15.99 FPS) (IV)

10 250 S.Aparna and M.Ekambaram Naidu Segmentation Mean Segmentation Median (3.01FPS) (V) High Pass Filter Mask1 High Pass Filter Mask2 (VI) High Pass Filter Mask3 (23.90 FPS) High Pass Filter Mask4 (16.49 FPS) (VII)

11 Dynamic Variations in the Speed of a Digital Video Stream due to Complexity Low Pass Filter Mask1 (15.76 FPS) Low Pass Filter Mask2 (VIII) Low Pass Filter Mask3 Low Pass Filter Mask4 (23.60 FPS) (IX) Low Pass Filter Mask5 (24.22 FPS) Faler Filtered Mask1 (23.62 FPS) (X)

12 252 S.Aparna and M.Ekambaram Naidu Faler Filtered Mask2 (24.25 FPS) Faler Filtered Mask3 (24.25 FPS) (XI) Faler Filtered Mask4 Faler Filtered Mask5 (XII) Kirsch Filtered Mask1 (24.63 FPS) Kirsch Filtered Mask2 (23.54 FPS) (XIII)

13 Dynamic Variations in the Speed of a Digital Video Stream due to Complexity Kirsch Filtered Mask3 Kirsch Filtered Mask4 (XIV) Kirsch Filtered Mask5 Kirsch Filtered Mask6 (XV) Kirsch Filtered Mask7 (23.90 FPS) Kirsch Filtered Mask8 (24.25 FPS) (XVI)

14 254 S.Aparna and M.Ekambaram Naidu Prewitt Filtered Mask1 (23.62 FPS) Prewitt Filtered Mask2 (22.92 FPS) (XVII) Prewitt Filtered Mask3 (22.64 FPS) Prewitt Filtered Mask4 (22.92 FPS) (XVIII) Prewitt Filtered Mask5 (22.92 FPS) Prewitt Filtered Mask6 (22.92 FPS) (XIX)

15 Dynamic Variations in the Speed of a Digital Video Stream due to Complexity Prewitt Filtered Mask7 (22.92 FPS) Prewitt Filtered Mask8 (22.92 FPS) (XX) Prewitt Filtered Mask9 (22.92 FPS) Sobel Filtered Mask1 (23.62 FPS) (XXI) Sobel Filtered Mask2 (20.37 FPS) Sobel Filtered Mask3 (22.94 FPS) (XXII)

16 256 S.Aparna and M.Ekambaram Naidu Sobel Filtered Mask4 (23.28 FPS) Sobel Filtered Mask5 (23.62 FPS) (XXIII) Sobel Filtered Mask6 (23.28 FPS) Sobel Filtered Mask7 (23.92 FPS) (XXIV) Sobel Filtered Mask8 (21.99 FPS) Laplatian Mask1 (22.66 FPS) (XXV)

17 Dynamic Variations in the Speed of a Digital Video Stream due to Complexity Laplatian Mask2 (22.29 FPS) Laplatian Mask3 (22.64 FPS) (XXVI) Laplatian Mask4 (24.25 FPS) Laplatian Mask5 (24.25 FPS) (XXVII) Robinson Mask1 (23.28 FPS) Robinson Mask2 (23.28 FPS) (XXVIII)

18 258 S.Aparna and M.Ekambaram Naidu Robinson Mask3 (23.28 FPS) Edge Enhancement East (24.25 FPS) (XXIX) Edge Enhancement west Edge Enhancement north (23.62 FPS) (XXX) Edge Enhancement South (24.25 FPS) Edge Enhancement north-east (24.25 FPS) (XXXI)

19 Dynamic Variations in the Speed of a Digital Video Stream due to Complexity Edge Enhancement north-west Edge Enhancement south- east (24.25 FPS) (XXXII) Edge Enhancement south- west Line Enhancement East -west (23.62FPS) (XXXIII) Line Enhancement North-south (22.64 FPS) LineEnhancement northeast-southwest (XXXIV)

20 260 S.Aparna and M.Ekambaram Naidu Line Enhancement northwest-southeast (23.62 FPS) (XXXV) Figure 4: Processed video image frames Table 1: Algorithms and FPS Sl. No. Algorithms FPS 1 rajan rajan Sobel Laplacian Prewit Kirsch morphological dilation morphological erosion segmentation mean segmentation median high pass filter mask high pass filter mask high pass filter mask high pass filter mask low pass filter mask

21 Dynamic Variations in the Speed of a Digital Video Stream due to Complexity low pass filter mask low pass filter mask low pass filter mask low pass filter mask fahler filter mask fahler filter mask fahler filter mask fahler filter mask fahler filter mask kirsch mask kirsch mask kirsch mask kirsch mask kirsch mask kirsch mask kirsch mask kirsch mask prewitt mask prewitt mask prewitt mask prewitt mask prewitt mask prewitt mask prewitt mask prewitt mask prewitt mask sobel mask sobel mask

22 262 S.Aparna and M.Ekambaram Naidu 44 sobel mask sobel mask sobel mask sobel mask sobel mask sobel mask laplacian mask laplacian mask laplacian mask laplacian mask laplacian mask robinson mask robinson mask robinson mask Edge enhancement East Edge enhancement West Edge enhancement north Edge enhancement south Edge enhancement north east Edge enhancement north west Edge enhancement south east Edge enhancement south west line enhancement east -west line enhancement north-south line enhancement northeast - southwest line enhancement northwest - southeast

23 Dynamic Variations in the Speed of a Digital Video Stream due to Complexity Figure 5: Graph showing the results of applying 69 algorithms on a video stream and the dynamic variations in FPS 5. CONCLUSIONS Sixty-nine algorithms were applied on various digital video streams and their effects on FPS were observed. It was observed that seven of these 69 algorithms reduce the FPS of a video stream. They are listed below along with the FPS Rajan Rajan Morphological dilation Morphological erosion Segmentation median 3.01 High pass filter mask Low pass filter mask The reason behind the reduction of FPS is twofold (i) algorithmic complexity and (ii) entropy of the video frames. It was also observed that the reduction in FPS is dynamic and it oscillates between tolerable limits. ACKNOWLEDGEMENTS The authors gratefully acknowledge the untiring support given to them by the research team of Pentagram Research Centre Private Limited, Hyderabad, Telangana State, India while carrying out research. REFERENCES [1] Shannon, Claude E(July-October 1948)."A Mathematical theory of communication" Technical Journal doi: /j tb01338.x.

24 264 S.Aparna and M.Ekambaram Naidu [2] C.Studholme, "An Overlap invariant entropy measure of 3d medical image alignment." Pattern Reconition, volume 32,issue1,January 1999,pages [3] John C.Crocker. "Methods of Digital Video Microscopy for Colloidal studies." Journal of colloid and interface Science. Volume 179, issue 1,15 April 1996.pages Elsevier. [4] S.F.Burch, "Image restoration by a powerful maximum entropy method". Computer vision, Graphics and image Processing.Volume 23,issue 2,August 1983,pages Elsevier [5] C.V.Angelino, E.Debreuve, M.Barlaud "A nonparametric minimum entropy image deblurring algorithm"2008 IEEE International conference on accoustics,speech and signal processing Pages ,DOI: /ICASSP [6] Vaddi chandra sekhar, satyajit bora, monalisa das, Pavan kumar manchi, S.JOsephine,Roy Paily "Design and Implementation of Blind Assistance system using real time stereo vision algorithms. Pages , DOI: /VLSID [7] Rajan, E.G., Cellular Logic Array Processing, Invited paper, World Congress for Nonlinear Analysts, organized by the international Federation of Nonlinear Analysts, Florida Institute of Technology, July 10-17, 1996, Athens, Greece [8] R.C. Gonzalez, R.E. Woods, Digital Image Processing. Addison Wesley, New York, 1992 [9] Stanley R. Sternberg. Gray scale morphology "Computer vision, graphics and image processing (1986) [10] Rajan, E.G., Medical Imaging in the framework of cellular Logic Array Processing, 15th International Conference of Biomedical Society of India,INCONBME 96, Coimbatore Institute of Technology, December 12-14, ABOUT THE AUTHORS S. Aparna is Assistant Professor from the Department of Computer Science and Engineering of the GITAM University, Hyderabad. My profound interests in research pulled me from industry and took to academics.my Research interests include Video Image Processing, Software Engineering, Data Base Management Systems. Dr. M. Ekambaram Naidu is the Principal of SRK College of Engineering and technology, Vijayawada.He is an avid Researcher and renowned professor. His research interests include image processing, Pattern Recognition and Analysis, Computer Networking, Software Engineering.