INTEGRATION, PROCESSING AND RECORDING OF AIRBORNE HIGH RESOLUTION SENSOR IMAGES Hans Brandtberg Saab AB, Electronic Defence Systems, Avionics Division October 2010
INTRODUCTION Onboard high resolution sensors - background Need and requirements for onboard video processing to integrate sensors, displays and recorders Architecture examples - Sensor video processing and switching - Digital recording PAGE 2
AIRBORNE SENSORS More imaging sensors having an increasing resolution is being integrated for airborne applications, in both helicopters and fixed wing, manned and unmanned An aircraft could be equipped with three, four, five,. different sensors. Std and HD TV cameras Low light TV Infrared cameras Radars of different kinds Weapons with imaging sensors PAGE 3
AIRBORNE SENSORS All these sensors have a variety of image quality, resolution and video interfaces. The resolution is technology based (e.g. Color HDTV is standards but HDTV Flir is difficult) Example: Infrared: 640 x 512, 1280 x 720,. TV (normal/low light): 480 x 640, 576 x 720, Color HDTV: 720 or 1080 HD NTSC or PAL Search/imaging radar: normal high res Pod sensors: Infrared of high res color Infrared search and track: normal high res PAGE 4
GEO INFORMATION AND IMAGE LIBRARY AS A SENSOR A computer graphics function read the data base and generate out/in the terrain imagery (2D/3D maps, satellite photos eyc) PAGE 5
INFORMATION COORDINATION FOR THE COCKPIT Data information: -Symbols and text, pictorial information Database information: -2D/3D maps, aerial photos, satellite imagery Sensor imagery: -Radar -infra-red -TV / HDTV -Reconnaissance (e/o and ir) Computer Graphics Image generation Image processning Cockpit On-board and off-board sensors PAGE 6
AIRBORNE COCKPITS The focal point of a large amount of heterogeneous image information There is a also a large variation in cockpits: -Number of displays -Size of displays -Resolution of displays PAGE 7
SENSOR INTEGRATION AND IMAGE PROCESSING Recording, data links Video input processing Video input to output selection Video conversion (rotation, scaling, translation) Graphics overlay Basic adaptations. Different electrical interface Selection sensor -> display Different resolution, frame rate, gray scale adjustment, etc PAGE 8
SENSOR INTEGRATION AND IMAGE PROCESSING FOR RECORDING To store (record) and retrieve multiple video sources for replay during flight or in ground station To transmit and receive imagery over data link. Video from sensor Video compression Video decompression To the displays Mass storage Data link PAGE 9
VIDEO STANDARDS OVERVIEW Video standards for broadcast (television) Analogue: SMPTE 170M, PAL, NTSC, 720p, 1080i etc. Digital: SMPTE 259M, SMPTE 292, 720p, 1080i etc. Connectors: Coax, S-Video, HDMI Video standards for computer graphics VESA VGA, SVGA, XGA, SXGA, UXGA, WUXGA etc. Connectors: Coax, DVI, OpenLDI, DisplayPort Video standards used in avionics All of the above, plus additional standard and proprietary formats STANAG 3350, ARINC 817, ARINC 818 (includes supervision) Broadcast video standards usually from sensors Computer graphics video standards usually from graphics generators and in displays Video standards for broadcast and computer graphics are incompatible and needs to be converted between each other PAGE 10
VIDEO CONVERSION Video needs to be converted between different video standards, e.g. from sensor to display De-interlacing Scaling Colour space conversion Frame rate conversion and frame drift handling Physical encoding Video conversion will affect image quality, and is often a trade-off between cost, image quality and latency Commercial video conversion devices doesn t handle special video formats and are often not available for use in rugged environments or for small volume customers Video solutions needs to be flexible to handle many different formats, or custom made for each application PAGE 11
VIDEO TRENDS Each video frame can be seen as a set of data that belongs to a specific time slot The video data can be sent on an ordinary network link (e.g. Ethernet) or a simplified unidirectional data link (e.g. Fibre Channel-based as in ARINC 818) Time information can be included Transfer error detection built-in in network protocol Additional supervision data can be sent on the link Many sources can be sent on the same link if bandwidth allows PAGE 12
THE SOLUTION Equipment to interface, select, adapt and process sensor video imagery and sent the image for display, storage or data link transfer. Implementation: Cards in another equipment box (in Mission Computer?) Stand alone Digital Video Processor Vi see these equipment in a wide area of different aircraft: Airbus A380, A400M, Gripen, NH90/Hkp14, commonly on helicopters etc. Dedicated equipment Card(s) in a computer equipment PAGE 13
NH90 ARCHITECTURE EXAMPLE The SGM receives sensor video, performs format conversion to fit the cockpit display format, selects video for each display and generates and superimpose graphics. Various graphics display modes are also generated and presented PAGE 14
EXAMPLE OF GRAPHICS GENERATOR WITH VIDEO PROCESSING Analogue and digital video input Adaption & Switching CPU I/O Graphics Generators (4) Format Conversion Output Analogue and digital video output PAGE 15
GRIPEN ARCHITECTURE EXAMPLE Mission computer Avionic buses Display Processor Radar Weapon video Flir/LDP Recce IRST Integrated video management Mass storage Audio Camera PAGE 16
AIRCRAFT DATA MANAGEMENT AND STORAGE Airborne Recorder Radio Transmission Aircraft Data Man. Network Data Transfer Display Processor Cockpit Replay -Real time display -Onboard replay PAGE 17
IMAGE COMPRESSION Image information content from a high resolution sensor requires real-time image compression. Example: 1080i color = 180 MByte/sec 1 hour = 650 GByte JPEG MPEG2 MPEG4 JPEG2000 --- -- - High and flexible compression using JPEG 2000 frame coding combined with frame rate reduction. Full quality of every frame gives full flexibility in replay and evaluation in the ground station. JPEG 2000 is more error resilient than MPEG-2, since a bit error only affects one single frame. Selectable quality on frame by frame basis. PAGE 18
TYPICAL HELICOPTER ARCHITECTURE Video switch Cockpit display Sensors Multi-channel digital solid-state recorder Ground evaluation Panel to control onboard replay PAGE 19
SUMMARY High resolution sensors are increasingly common equipment onboard helicopters and fixed wing aircraft. The difference in interfaces, resolution etc will remain Today s electronic solutions enable cost efficient processing and information adaptation to intended use. PAGE 20
Saab AB, Electronic Defence Systems, 175 88 Järfälla, Sweden Tel: +468 580 84000 hans.brandtberg@saabgroup.com