Dr.-Ing. Jan Bauer, TU Dresden,

Transcription

1 Automotive Infotainment & Telematics Funktionale Sicherheit für Videodatenübertragung im Fahrzeug am Beispiel von modernen Automotive Infotainment Systemen Dr.-Ing. Jan Bauer, TU Dresden,

2

3

4 Digitalisierung?

5 Benutzungssituationen werden vielfältiger vernetzte Mobilität individuelle Mobilität Reisen Transport Arbeiten Lebensraum

6 Systemgrenzen erweitern sich Fernzugriff Passagiere Fahrer Interaktion Fahrzeug Offboard- Inhalte Inhalte Dritter

7 Always On wird zentraler Fahrzeugestandteil Provider e.g. Traffic Service Vehicle Backend COM-Modul

8 Titel der Präsentation in CorpoS (Textkörper) 10pt Abteilung Datum 8 Das Fahrzeug ist damit Teil des Internets der Dinge. Kennzeichen: S-DD 1251 IP-Adresse: Kennzeichen: D-AG 321 IP-Adresse: Kennzeichen: S-OX 4841 IP-Adresse: Kennzeichen: B-AB 1255 IP-Adresse:

9 Navigation Services Remote Telediagnostics Concierge Service Vehicle setup Digital Vehicle Key in the Smartphone Personalization Entertainment & Media Office & Communication Remote Parking Pilot Vehicle monitoring

10 vor, integriert Inhalte und Dienste während, und nach der Reise.

11 Perfekte Bedienung High End Sound HD Displays Park System Navigation Spracherkennung Entertainment & Media

12 Video

13 Real Life Safety Ein Stern genügt

14 Motivation: ASIL Rückfahrkamera

15 ASIL-Rückfahrkamera: Verzögertes Bild im Stadtverkehr Ein-/Ausparksituation C3: Fahrer kann i.d.r. nicht erkennen, dass das Bild zeitlich verzögert und somit ein falscher Abstand zum Kollisionsobjekt angezeigt wird. Kinder sind nicht in der Lage entsprechend zu reagieren und auszuweichen. S2: Fußgänger wird angefahren und stürzt. Kopfverletzungen möglich. E2: Nur Parksituation mit Fußgänger in unmittelbarem Fahrzeugumfeld relevant.

16 Frame Buffers Frame Buffers Frame Buffers Challenges Modern video system architectures route safety relevant video streams (e.g. automotive backup camera) through processors that have multiple video frame buffers and often do not follow a safety relevant implementation like ASIL Due to the missing ASIL implementation, the processor routing the safety relevant video streams cannot be trusted and require further measures Example Video Distribution System CAMERA PROCESSOR DISPLAY ASIL NON-ASIL INPUT OUTPUT ASIL DISPLAY P NAV SOC

17 Brainstorming Possible Solutions Bypass Video processing unit by using a second video link Use-Video Blanking to send an ASIL-ID Embed a Signature and Counter within the RVC picture Use CAN signal to send an ASIL request to the display Use Control Chanel within the Videolink to send an ASIL request to the display Develop HU against ASIL-A

18 Watermarking: Overview Proposal: invisible, line-based image watermarking The watermark is inserted into each pixel to cover the complete content Safety relevant content is identified by the detection of the watermark Using alternating watermarks for each frame, frozen images can be detected Example Video Distribution System using Watermarking CAMERA PROCESSOR DISPLAY INPUT OUTPUT Frame Buffers P Frame Buffers Watermark Detection DISPLAY WM Generator NAV Frame Buffers FROZEN FRAME SOC

19 Frame Buffers Frame Buffers Frame Buffers Watermarking: Requirements CAMERA PROCESSOR INPUT OUTPUT DISPLAY DISPLAY P NAV Detect Frozen Images SOC Not relying on information apart from the image itself (e.g. blanking can be removed in the processing) Robust against frozen video streams with active video timing Robust against image manipulation (e.g. resizing, cropping, rotation, color compensation, flipping) Robust against image noise, obstruction and compression Low Probability of false positive / false negative No visible image distortion caused by the algorithm Fast change to a safe state to prevent an unsafe situation The ISO requires reaching the safe state in less than 500ms [1].

20 Watermark Design The watermark generation is based on i=4 basis functions The 4-Byte basis functions (W i ) are horizontally symmetric Example: To gain robustness against image stressors (manipulation, noise, compression, ) an oversampling of 10 is used (=320pixel watermark) With the small size the watermark is highly redundant in the picture (e.g. a Full-HD Frame contains 6480 watermarks, 6 Watermarks per Line) Example Watermark W 1 =0x5B 9B D9 DA W 2 =0xA W 3 =0xBB 8D B1 DD W 4 =0x E 22 Watermark Basis Functions

21 Watermark Generator To suppress the visibility the watermark is hidden in the chroma Cb of the image A gain K o is used to control detectability and robustness vs. the chance of visual detection A temporal error pattern LUT is used to spread the error by the watermark using FRC over subsequent frames With FRC the resulting average chroma after watermark removal is equal to the original value Watermark Generator Frame Cntr Input Video Stream RGB_2_YCbCr LSB Mask Ko Ex: 0xFC Cb Y, Cr Row Cntr + Column Cntr - fn S[2:0] Q[2:0] Cb F[2:0] Ko[1:0] H F [3:0] Temporal Error Pattern Look Up Table Ec_out Cb YCbCr_2_RGB Watermarked Video Stream 2 K 1 W k

22 Watermark Detection Watermarked Video Stream RGB_2_YCbCr Cb WM Extract > x(k) +/-1 N 1 Detection Bank Filter W > x k k Watermark Detector LSB Mask The watermark is detected by a convolution of the basis function (2 K-1-1) Th N 2 W x > k k Th Wm1 Frame Based Processing Frozen Frame Error W k with extracted watermark I: M k = I W k N m W x k k > A bank of filters is used to detect stretched and compressed watermarks due to image resizing or rotation A watermark detection occurs when the convolution result is higher than a threshold T With the threshold T the detection strictness can be adjusted A minimum number of watermark detections T F is necessary within a frame to activate safety relevant content algorithm Further all basis functions have to be detected every n frames Example Watermark Basis Function Wk and the Matched Filter s Mk output

23 Robustness Strategy Avoid False Negative Avoid False Positive 5-stage robustness strategy: 1. Similarity check of the basis function to the watermark, by using the threshold level T (e.g. 28) 2. High redundancy of the watermarks in one frame (e.g for FullHD) 3. High detection probability by watermark design, gain factor K o (number of LSBs used) oversampling and the use of the filterbank (robust against image stressors e.g. scale factors from 0.5 to 2) 4. A minimum Number of watermarks per frame T F (e.g. 16) needs to be detected to activate the ASIL algorithm 5. If the ASIL algorithm is active, every n (e.g. 10) frames (<<500ms) all the basis functions needs to be present, otherwise an ASIL error is generated, the display content is e.g. set to black (=safe state) and the ASIL main system is noticed

24 Results Example Image Before and After Watermark and DSC Processing.

25 Results Expert subjective evaluation: No artefacts due to the watermarking visible Objective: From literature a ΔE LAB 2.3 is visually undetectable. The results show for 10 typical automotive test picture that the average ΔE LAB after watermark removal is below this visual detection threshold For the given image stressors that zero false positive / negative detections can be achieved for the simulated content with a gain K O of 3 Bit average ΔE LAB Bits Watermark average ΔE LAB Bits Watermark Watermarked Image Quality Impact w/ and w/o DSC. All tested images a have an average ΔELAB below the perception threshold Watermark Detector Performance to Image Resizing (scale factor of 50, 75, 125, 175 and 200%) with a Gain Ko of 1, 2 and 3 bits

26 P Results Not relying on information apart from the image itself Robust against frozen video streams with active video timing Robust against image manipulation Robust against image noise, obstruction and compression No visible image distortion caused by the algorithm A Frozen picture of a backup camera can be avoided! Low Probability of false positive / false negative Fast change to a safe state to prevent an unsafe situation The ISO requires reaching the safe state in less than 500ms [1]

27 References [1] ISO: Road vehicles-functional safety", 2011 [2] VESA Display Stream Compression (DSC) Standard, Version 1.1, [3] J. Bauer, M. Kreuzer, T. Jung, D. Schäfer, "Increasing the Perceived Grey Value Resolu-tion by Combining Frame Rate Control and Error Diffusion to Reduce Visible Artefacts in Local Dimming Applications", electronic dis-plays Conference 2015, Nuremberg, Germa-ny, February 25-26, 2015 [4] M. Mahy, L. Van Eyckden, and A. Ooster-linck, Evaluation of uniform color spaces developed after the adoption of CIELAB and CIELUV, Color Res. Appl., vol. 19, no. 2, pp , Apr

28 Feel the Experience