WHITE PAPER Image Contrast Enhancement (ICE) The Defining Feature Author: J Schell, Product Manager DRS Technologies, Network and Imaging Systems Group
Image Contrast Enhancement (ICE): The Defining Feature Author: J Schell, Product Manager DRS Technologies, Network and Imaging Systems Group A technology once reserved for military applications and expensive government programs, infrared imaging systems are now being widely deployed in a variety of applications to address needs that traditional imaging technology cannot. Today, we are just scratching the surface of thermal imaging s capabilities and uses. As advances in thermal imaging technology and manufacturing efficiencies improve performance and drive down costs, we are experiencing a transition to thermal imaging as the preferred low-light solution for security and surveillance systems as well as the emergence of new, innovative products and markets. For example: Public safety applications continue to expand, and municipalities and universities are rolling out enhancements to monitor crowds and enhance public safety day and night; Predictive maintenance applications are monitoring temperatures of critical equipment to ensure minimal downtime; Manufacturing process controls are using thermal imaging to monitor proper heat transfer; Medical imaging providers are deploying the technology as an adjunctive screening for breast cancer; Meteorological and geological applications include mapping the earth s climate to correlate agriculture and ecological movements. In the majority of these applications, the effectiveness of the thermal imagery is highly dependent on the quality of the image it produces. This has led to an increasing demand for greater image detail. To address this need, DRS Technologies developed and introduced Image Contrast Enhancement (ICE) with the Tamarisk 320 and Tamarisk 640 thermal imaging modules as well as the WatchMaster IP 3000 and 6000 Series thermal camera systems. ICE provides additional capability to dial in image detail and present greater scene content when compared to other competitive offerings. ICE algorithms work with DRS' basic Automatic Gain Control (AGC) algorithm to produce enhanced imagery for many of today s common display technologies and formats. To understand the full capabilities of ICE, one must first understand the origin and use of traditional AGC. Traditional Automatic Gain Control Automatic Gain Control (AGC) was pioneered in the 1920s as part of audio volume control to automatically adjust recording levels (amplifying or gaining relatively weaker sounds or frequencies of sound while attenuating or limiting stronger ones) to condition audio signals in preparation for further audio processing. This concept also applies to shaping analog and digital images where certain intensities and frequencies of light are automatically scaled up or down to condition a video signal for display. As for thermal imagers, heat energy emitted by the objects in a particular scene is converted into a series of electrical signals which are processed and automatically gain corrected for display in either analog or digital formats. Initially, thermal imaging was most prevalently used for US military applications. AGC algorithms were tuned for night vision and used for detection, recognition and identification of battlefield equipment and soldiers. AGC automatically adjusts the image based on the relative thermal emissions of objects in the scene, making it easier to distinguish a hot vehicle or warm body from the generally cooler night air and/or vegetation around it. For this reason, AGC algorithms were tuned to automatically adjust gain settings to resolve warmer objects or targets, while suppressing background clutter (objects with less heat signature considered less relevant to the scene). This allowed the soldier to better focus on the warmer target. [1]
Today, AGC is a standard enhancement technique in thermal imaging systems. However, AGC is not an optimal solution in all cases. It is common for thermal imagers to lose some of the background content or scene detail when a hot object enters a relatively cooler scene. In scenes where objects have vastly different temperatures, AGC downplays or suppresses objects with intermediate temperatures or thermal emissivity in order to preserve objects at the colder and hotter extremes. This can be an issue because the human eye is only capable of discerning between 50 to 200 shades of gray, while the thermal imager itself can typically distinguish 16,384 levels (14-bit data) or shades. Therefore, when rendering an image for human vision, AGC algorithms compress the 16,384 shades down to 256 (8-bit data). In so doing, some detail is forfeited and vital elements may be lost. Referencing the AGC image (Figure 1), the inferno on the left side of the room is visibly bright white, representing the hottest object in the scene. Figure 1 AGC Since heat rises, the air and the masonry wall in the upper part of the room are warmer, making them appear brighter. However, there is a lack of detail outside the open window (outside temperature was approximately 35 F) and the cooler space in the lower right-hand corner of the room. The firefighter is visible but details of the surroundings are lost as AGC, by design, suppresses some of the intermediary temperatures to adjust for the extreme hot (fire) and extreme cold (outside air) elements of the scene. While AGC serves as an extremely valuable function for rendering a viewable image, it is not the optimal solution in every scenario. Many of today s applications demand greater image detail of the entire scene. With knowledge of these inherent gaps in AGC and fueled by market demand for greater image detail, DRS Technologies continued to refine its imaging technology. Today, DRS' proprietary Image Contrast Enhancement (ICE) provides a more comprehensive and robust imaging solution for a variety of applications. ICE Overview A Dynamic Filtering Process ICE works to deliver greater image detail by seamlessly integrating three separate processes: (1) Edge Enhancement, (2) Dynamic Contrast Thresholding and (3) Adaptive Rescaling. 1. Edge Enhancement: Edge Enhancement algorithms are commonly used in image processing and are routinely implemented to isolate the higher frequency components in the scene (regions with large thermal gradients), amplifying these differences in objects. As a standalone image enhancement algorithm, edge sharpening is best suited for scenes with a moderate amount of thermal contrast between objects. Neighboring objects with very little thermal differences (bland regions primarily composed of low frequency components) or objects with large thermal differences (well defined regions composed primarily of high frequency components) present challenges to edge sharpening. As the algorithm attempts to highlight bland elements, it often introduces unwanted gain noise into the image. Applying Edge Enhancement to regions already dominated by high frequency components tends to overexaggerate their distinction and thermal contours. DRS' ICE has addressed these shortcomings through multiple levels of frequency isolation, amplification and recombination. [2]
2. Dynamic Contrast Thresholding (DCT): To further enhance the thermal image, specific frequency components from the Edge Enhancement algorithm are passed through for additional processing. Additional image content is extracted by applying dynamic (scene dependent) scalar thresholds to identify the most significant content within this spatial frequency. Extracted data is then processed through compaction and stabilization algorithms. 3. Adaptive Rescaling: Finally, image data passed on from DCT processing undergoes additional scaling and smoothing operations. As before, this operation varies with scene content where distilled data is fused back in with the output of the Edge Enhancement algorithm to produce a hybrid image. The dynamic filtering sequence outlined above decomposes the image into three spatial frequency components: a) The high frequency component represents object edges. b) The mid frequency component captures the moderate thermal transitions in the scene. c) The low frequency components represent bland thermal transitions in the scene. After processing the essential components they are recombined to form an optimized image. The resulting image may then undergo additional optimization depending on the users preferred settings. ICE Adjustments In Figure 2, the application of ICE with moderate DCT has brought forth mid-frequency content. The firefighter is now clearly visible without losing detail of the fire (extreme hot) or the uniform dark space (air outside the window) as compared to Figure 1 (AGC) shown on page 2. Also, the low level of edge enhancement defines the outline of the fire, cinder blocks and detail around the window. In Figure 3, the ICE settings have been increased, further amplifying both DCT and Edge Enhancement algorithms, resulting in greater apparent detail in the entire scene. Both the higher frequency and lower frequency components in the scene (some of which were not noticeable in the AGC image) are given additional weighting. Edge details are further resolved as are the bland regions such as the dark corner of the room. The objects in the distance, through the window, have gained detail and outline, revealing the structure that was previously undetected. Figure 2 ICE Low Setting Figure 3 ICE High Setting [3]
Comparing the Value of DRS' ICE ICE is much more than simple edge enhancement. ICE is the summation of DRS' proprietary Edge Enhancement techniques coupled with Dynamic Contrast Thresholding and Adaptive Rescaling. Unlike other digital detail enhancement techniques on the market, ICE is adaptive to relative scene content, allowing for a degree of auto gain, making it more flexible over a wider range of thermal scenes. before pan DRS ICE High Setting Competitor digital image enhancement after pan DRS ICE High Setting Competitor digital image enhancement ICE vs. the competition: The images above compare two 17 μm, 640 x 480 resolution, uncooled microbolometers. In the top images, both systems were optimized to provide the greatest image detail while staring at the side of an office building (before pan). The cameras were then panned to the left, changing the scene, as pictured above in the bottom two images (after pan). With Dynamic Contrast thresholding and adaptive rescaling, the ICE algorithm maintains optimal detail through the panning motion. The competitor s digital detail enhancement lacks this capability, and the edge detail is excessively gained to the point of distorting the image. Summary Thermal imaging advances such as ICE continue to open the door for new applications that deliver the right images at the right time with the right information. Simple and Robust: When used in conjunction with proper gain and level settings, DRS' Image Contrast Enhancement (ICE) delivers a more robust solution for analytics engines. Autonomous: ICE operates seamlessly in the background, automatically adjusting image detail in real time, making it ideal for pan and tilt camera systems and surveillance applications. Flexible: The amount of ICE may be adjusted as focal points change and regions of interest move. The ICE function may be increased (ICE High) to gain greater edge detail or decreased (ICE Low) to end-user preference. DRS RSTA, Inc. 13544 N. Central Expwy, Dallas, TX 75243, Tel. 855-230-2372 www.drsinfrared.com This information is furnished in confidence with the understanding that it will not, without the permission of DRS Technologies, be reproduced, used, or disclosed for any purpose other than the purpose for which it was furnished. All Rights Reserved. MR-2013-06-664 06/13 [4]