Automatic Defect Recognition in Industrial Applications Klaus Bavendiek, Frank Herold, Uwe Heike YXLON International, Hamburg, Germany INDE 2007 YXLON. The reason why 1 Different Fields for Usage of ADR Systems ADR == Automatic Defect Recognition X-Ray other NDT methods Tires Food Electronics Castings Welds etc. Driving force: Automotive Industry (highest number of parts!) YXLON. The reason why 2
ADR for Tires Detector (LDA) Panoramic Tube Typical Defects: Body Cord (3): average Cord Spacing and Cord Diameter Sidewall (6): wide Cords (boubled), narrow, missing, broken / crossed Cords Belt and Turnup (2-5): Wander, Snaking, Scalloping, Necking, Flare Bead Area (8-10): loose Wires (Seat Side), Gross Weld Faults YXLON. The reason why 3 ADR for Tires YXLON. The reason why 4
ADR for Castings Weight is an important factor for cars (less weight: less fuel consumption, higher acceleration, better handling) More and more parts are made of Aluminium instead of steel. Aluminium parts are mainly casted. YXLON. The reason why 5 Typical Castings Typical smaller parts for automotive industry: Source: KSM Casting, Hildesheim YXLON. The reason why 6
Typical Castings Typical larger parts for automotive industry: YXLON. The reason why 7 Typical Defects in Aluminium Castings shrinkage sponge porosity (round) shrinkage cavity gas hole porosity (elongated) inclusion YXLON. The reason why 8
ADR for Welds Typical defects: Cracks, Porosity, Lag of Fusion YXLON. The reason why 9 ADR for special tasks Sorting of Vanadium / Aluminum Chips 0.5mm Defect Example 180mm Field of View YXLON. The reason why 11
Reasons for using ADR systems Advantages: Higher throughput than with visual inspection Inspection quality is independent of operator and time (no Monday morning) Reproducable results No x-ray expert necessary for inspection process Direct Feedback to casting process possible (control loop) Drawbacks: Human inspector knows how a defect looks like (in case of doubt...) Third dimension gets lost when the part is not moving in the X-ray beam Always some parts are rejected which are inside the specification Advantages in Costs? Depends on the amount of similar parts per day... YXLON. The reason why 12 When using ADR systems instead of visual inspection? It depends on the No. of similar parts per day Conditions: 12s / part [ADR] 40s / part [visual] 95% Uptime 360 days / 24h [ADR] 200 days / 7h [vis.] 2 years operation YXLON. The reason why 13
When using ADR systems instead of visual inspection? It depends on the # similar parts per day Conditions: 95% Uptime, 2 years operation 12s / part [ADR] 40s / part [visual] 360 days / 24h [ADR] 200 days / 7h [visual] YXLON. The reason why 14 Inspection with an ADR system X-Ray tube Object Manipulator X-Ray detector PC PLC high voltage generator 90kV 5mA X-Ray Controller YXLON. The reason why 18
Inspection with an ADR system X-Ray tube. Reject! Manipulator X-Ray detector PC PLC high voltage generator 96kV 5mA X-Ray Controller YXLON. The reason why 23 Main steps of an ADR system Transport part into the machine Identify part type and/or mould Move part into first position Load image processing parameter of first position Capture image(s) of first position Move part into next position Load image processing parameter of next position Start evaluation of next image Capture image(s) of next position NO Last position? YES Unload part NO Image IO? YES YXLON. The reason why 27
X-Ray Limitation Triangle: image acquisition time ==> production conditions ==> long term stability geometrical resolution ==> small focal spot size ==> little power limited No. of X-ray quants contrast resolution ==> high SNR YXLON. The reason why 28 $ First generation of ADR systems... starts 25 years ago. Conditions were: Very limited computing power and data transfer rate Special hardware (image processing algorithm in hardware) Little knowledge about the voids Simple solution with Golden Reference Image technology 1. Acquire image of acceptable part 2. This image is used as a reference (Golden Reference Standard) 3. Compare the golden standard image to the image acquired during inspection 4. Differences between the Golden Image and inspection image are discontinuities (Feature Extraction) YXLON. The reason why 29
First generation of ADR systems Simple Solution: Compare actual Image with Golden Reference Image Actual Image Reference-Greyvalue Actual Greyvalue Referenz-Edge Edge in actual image Deviation of 3 pixel only in Y and 0 Pixel in X direction Reference-Image Difference-Image Actual Image Reference Image (adapted in Greyvalues) YXLON. The reason why 30 First generation of ADR systems Different threshold for Binarisation results in (just one digit difference): YXLON. The reason why 31
First generation of ADR systems Golden Image Technology creates too many False Rejects because of Tolerance in Positioning (palette or robot gripper) Tolerance of the part itself (deviations in shape and in thickness) X-Ray quantum statistics Image capture statistics and produces larger anomalies than real defects in the part. Technology is not used any more in industrial casting industry. Second generation uses adapted filters to detect the flaws... YXLON. The reason why 32 Second generation of ADR systems Second generation uses adapted filters to detect the flaws... Process for automatic Defect Extraction 1. Adaption of contrast and brightness to reference values (grey-scaling) 2. Matching of region to be inspected to the actual Image 3. Filtering with special, adapted filters 4. Subtraction of original and filtered Image 5. Binarization of difference image 6. Classification of flaws according the specification; recognition of regular structures (Pseudos) YXLON. The reason why 33
Second generation of ADR systems Matching of Regions, Filtering of Images, Subtraction from Original A Matching of Regions Original Filter B A-B Filtered Image Spreading Difference YXLON. The reason why 34 Second generation of ADR systems Binarization of Difference, Matching of Defects to Image Threshold Difference Binarization C Result for computer A+C Result Image YXLON. The reason why 35
Second generation of ADR systems Result (for visual comparision) YXLON. The reason why 36 Second generation of ADR systems Classification of detections according to specification Single Defects (too small) Single Defect 3,5qmm 21 qmm Defect 9,3 qmm Removing of trained, regular structures Clustering of single defects Determination of defect size Comparison with specification (defect size dependend to region) regular Structures OK or Not OKDecission YXLON. The reason why 37
Second generation of ADR systems Main differences to the first generation: Flaw is filtered out of the actual image (reduces positioning variances) Image is separated in different regions and processed with different adapted filters (reduces influence of positioning variances) Several filters can be used for one region 2 step binarisation (marker for detection and segment for area) Additionally, some nice features were integrated into 2 nd generation ADR systems YXLON. The reason why 38 Features of second generation ADR systems Usage of Image Intensifier or DDA (flat panel detector) for best image quality Single User Interface for mechanic control and image processing Usage of standard computing hardware Good/Bad decision per image; image of one position: bad part: bad Large variety of different image processing functions... needs an expert to select the proper function for each situation Mask-oriented operation with special structures of the menues Only a few different parts per machine Verification of reliability of the inspection with test object (Bad Part or part with drilled holes) YXLON. The reason why 44
Advantages of second generation ADR systems Nearly all kinds of flaws can be detected Inspection with machine faster and more reliable compared to human operator Small No. of pseudos with good adjustment of the image processing parameter Drawbacks of second generation ADR systems Very high effort to train a part (week(s)) Expert necessary to select and adjust the different image processing parameter Permanent check of detection reliability necessary Only flaws, which fit into the area of the filter kernel, are detected reliably Position and part tolerances still increase the false reject rate YXLON. The reason why 45 Features of a third generation ADR system Very high effort to train a part (week(s)) Training of a new part should be very fast (< 1 hour) Expert necessary to select and adjust the different image processing parameter The ADR system should define the best image processing chain itself System can only run ONE part (or very very similar parts) System should inspect different parts in the same machine Only flaws which fits into the area of the filter kernel are detected System should overcome the limit of the filter kernel size Position and part tolerances still increase the false reject rate Automatic compensation and correction of positioning variances Additional requirements: Feedback to the casting process (not only sort out, prevent flaws...) Flaws should be classified in REAL defect types System should be adaptable to the requirements of the application / customer YXLON. The reason why 46
Limits of classical filter technique Local filter runs with a limited filter mask (kernel) through the image. Flaws larger as the kernel are not detectable with reliability. In most cases only the edges of the flaw are detected... YXLON. The reason why 48 Size independent Detection Algorithm Third generation ADR systems use complete Image as filter kernel size. During training system learns how a defect free image should look like. Information is generated from the complete image. Training by the computer Information Data Basis OK Images Analysis: - Statistical Information - Characteristical Performance Parameter References YXLON. The reason why 49
Perceptability of Third Generation ADR system with Big Flaws Filter: Difference Of Gaussians YXLON. The reason why 50 Perceptability of Third Generation ADR system with Big Flaws YXLON. The reason why 51
Perceptability of Third Generation ADR system with Big Flaws YXLON. The reason why 52 Perceptability of Third Generation ADR system with Big Flaws An other example: YXLON. The reason why 56
Contrast Sensitivity In the example the constrast difference is ~1.5%. Nevertheless the big flaw is detected! YXLON. The reason why 57 Moving the limits with a trainable system False Rejects [%] Application Range with new ADR technology 2 Possible with optimized Training Normal Application Range 1 2,5 5 7,5 10 Principle performance, not calculated Contrast [%] YXLON. The reason why 58
Third generation of ADR systems Position and part tolerances still increase the false reject rate Automatic compensation and correction of positioning variances using Image Registration The mask stays in position, the image in moved below the mask Now part deviation in size can be corrected Mis-Positioning (even Tilt and Gear) can be corrected Precise alignment of an edge for several image can be done Global Filter Algorithm is better adaptable to the image YXLON. The reason why 59 Classification in Real Defect Types Flaw type higher density lower density Inclusion Void Porosity (sharp edges) Round Elongated Round (single) Shrinkage Cavity Sponge YXLON. The reason why 62
Process Feedback with Forecast... by classification in real defect types the casting process can be optimized online YXLON. The reason why 63 Features of a third generation ADR system Very high effort to train a part (week(s)) Training of a new part should be very fast (< 1 hour) Expert necessary to select and adjust the different image processing parameter The ADR system should define the best image processing chain itself System can only run ONE part (or very very similar parts) System should inspect different parts in the same machine Only flaws which fits into the area of the filter kernel are detected System should overcome the limit of the filter kernel size Position and part tolerances still increase the false reject rate Automatic compensation and correction of positioning variances b b Additional requirements: Feedback to the casting process (not only sort out, prevent flaws...) b Flaws should be classified in REAL defect types System should be adaptable to the requirements of the application / customer b YXLON. The reason why 64
Third generation of ADR systems is individual configurable Marking Barcode ADR Statistic X-Ray Imaging Mechanic Control Framework YXLON. The reason why 65 Third generation of ADR systems is individual configurable Framework Process Review Automatic Marking ClassificationBarcode ADR Special FilterStatistic X-Ray Defect Detection Imaging Image Mechanic Registration Control TypeFramework Identification YXLON. The reason why 66
Easy to use requirements Training of a new part should be very fast (< 1 hour) The ADR system should define the best image processing chain itself System should inspect different parts in the same machine YXLON. The reason why 68 Quiet Office, 11:00AM YXLON. The reason why 69
Now the Part is trained. Run the part in Automatic Mode: YXLON. The reason why 94 Operator Panel at Machine YXLON. The reason why 95
YXLON. The reason why 96 Realization of an ADR machine YXLON. The reason why 108
Realization of an ADR machine YXLON. The reason why 109 Realization of an ADR machine... for cylinder heads YXLON. The reason why 110
Realization of an ADR machine... for aluminium wheels YXLON. The reason why 111 No ADR system is perfect. # of False Rejects # of False Accepts YXLON. The reason why 113
Reasons for false rejects Simple test with a real object and one 3,45mm 2 size Cavity: (done @KSM Castings) 1. 30 x evaluate the same image 2. 30 x take a new image with the object in the same position 3. 30 x take the part and move with the robot from the previous position; do not put it back to the palette 4. 30 x take a new image of the same object and do the complete mechanic handling of the ADR machine - take object from palette, - move it from position to position - put object back on the palette for next step YXLON. The reason why 114 Reasons for false rejects 1. 30 x evaluate the same image Always the same result: 3,55mm 2 No variance ; Image processing is deterministic! R = 3,55 mm 2 ; σ = 0,0 mm 2 YXLON. The reason why 115
Reasons for false rejects 2. 30 x take a new image with the object in the same position Variance through X-ray quantum noise! R = 3,59 mm 2 ; σ = 0,3185 mm 2 YXLON. The reason why 116 Reasons for false rejects 3. 30 x take the part and move with the robot; do not put it back to the palette Variance through X-ray quantum noise and positioning accuracy! R = 3,23 mm 2 ; σ = 0,3274 mm 2 YXLON. The reason why 117
Reasons for false rejects 4. 30 x take a new image of the same object and do the complete mechanic handling of the ADR machine - take object from palette, - move it from position to position - put object back on the palette for next step Variance through X-ray quantum noise and positioning accuracy and robot gripper + palette! R = 3,03 mm 2 ; σ = 0, 8584 mm 2 YXLON. The reason why 118 Why this large variance in the calculated defect size?... it depends on the shape and position of the defect in relation to the grid Perfect square defect of 16 pixel size; variance from -44% up to +56% in size YXLON. The reason why 119
Why this large variance in the calculated defect size?... it depends on the shape and position of the defect in relation to the grid Elongated defect (string defect) of 12 pixel size; variance from -58% up to +75% in size YXLON. The reason why 120 How many % is necessary to fullfill 6 Sigma? 400µm detector, magnification 1.25 YXLON. The reason why 121
How many % is necessary to fullfill 6 Sigma? 3,5mm 2 Defect 400µm detector, magnification 1.25 YXLON. The reason why 122 Example with well known round defect Testimages from Aluminium casting (stearing knuckle) Compare flat hole (penetrameter) with spherical void YXLON. The reason why 123
1st Version: Flat Area; compare flat and spherical hole YXLON. The reason why 124 2nd Version: with Gradient; compare flat and spherical hole YXLON. The reason why 125
Effect of position of a defect on the grid of the detector Deviation from theory YXLON. The reason why 126 Effect of position of a defect on the grid of the detector [%] Deviation to theory YXLON. The reason why 127
Reasons for false rejects To avoid false accepts you have to set a stronger level in the ADR setup. false rejects real measured defect sizes acceptance criteria limit in the ADR settings (to avoid false accepts) acceptance criteria limit Still the risk for false accepts defect size YXLON. The reason why 128 What can you do to reduce the amount of false rejects? Increase the No. of pixel in the object by use a higher magnification use higher basic spatial resolution (smaller effective pixel size) Increase grey value resolution to have more steps to distinguish between defect area and regular area. Increase the dose (No. of quants) for one image. YXLON. The reason why 130
False rejects and false accepts also with human operators Comparison of ADR system and 3 human operators under real conditions: 100 Wheels, 18 Clear Rejects Operator1 Operator2 Operator3 ADR Overall Correspondence Missed Rejects Reject Correspondence 96 94 94 100 4 6 6 0 78% 67% 67% 100% False Detections Time Requirements 0 2,4h 0 2,3h 0 2,5h 0 58min Human operators tend to say good even when the part is bad. YXLON. The reason why 131 ADR for welds... not yet realized because of the large variances in potential defects in a weld. But: A semiautomatic defect recognition can support the operator to detect the voids. YXLON. The reason why 144
Example: (Titanium) Ducts YXLON. The reason why 145 Example: (Titanium) Ducts YXLON. The reason why 146
Example: Small Titanium Duct YXLON. The reason why 147 Inspection of Ducts, potential for ADR? YXLON. The reason why 148
Inspection of Ducts, potential for ADR? Diameter 0.05mm YXLON. The reason why 149 High Speed System for Duct Inspection YXLON. The reason why 150
When should you use ADR? YXLON. The reason why 151 Application Range of Film and CR 300 250 Parts per Hour 200 150 Void Diameter [mm] 100 50 10 1.0 0.8 0.6 0.4 0.2 1.6 1.4 1.2 5.0 4.0 3.0 2.0 1.0 Contrast Sensitivity [%] YXLON. The reason why 152
Application Range for Visual Inspection 300 250 Parts per Hour 200 150 Void Diameter [mm] 100 50 10 1.0 0.8 0.6 0.4 0.2 1.6 1.4 1.2 5.0 4.0 3.0 2.0 1.0 Contrast Sensitivity [%] YXLON. The reason why 153 Application Range for ADR with Image Intensifier 300 250 Parts per Hour 200 150 Void Diameter [mm] 100 50 10 1.0 0.8 0.6 0.4 0.2 1.6 1.4 1.2 5.0 4.0 3.0 2.0 1.0 Contrast Sensitivity [%] YXLON. The reason why 154
Application Range for ADR with Digital Detector Array (DDA) 300 250 Parts per Hour 200 150 Void Diameter [mm] 100 50 10 1.6 1.4 1.2 1.0 0.8 0.6 5.0 4.0 3.0 2.0 1.0 Contrast Sensitivity [%] 0.4 0.2 limited No. of X-ray quants YXLON. The reason why 155 Application Range for ADR with CT System 300 250 Parts per Hour 200 150 Void Diameter [mm] 100 50 10 1.0 0.8 0.6 0.4 0.2 1.6 1.4 1.2 5.0 4.0 3.0 2.0 1.0 Contrast Sensitivity [%] YXLON. The reason why 156