TECHNOLOGY & STANDARDS COMMITTEE SUMMARY

Transcription

1 TECHNOLOGY & STANDARDS COMMITTEE EVIAN MEETING Commercial Technology Forum May 2004 Chairman: A.G.DE BOOS (Australia) Report No: CTF 02 The 2003 Australian Wool Innovation on-farm fibre measurement instrument evaluation trial - Part 2: Performance in objective classing and ranking for animal selection By Peter Baxter, Jim Marler SGS Wool Testing Services Australian Wool Testing Authority Ltd PO Box 15062, Wellington, New Zealand PO Box 190, Guildford, NSW 2161, Australia SUMMARY This report summarises the results of the 2 nd phase of a substantial trial carried out by Australian Wool Innovation Ltd. (AWI) in which the performance of the two common on-farm fibre diameter measurement technologies (OFDA2000 using midside samples and Sirolan Fleecescan using sampling of the whole skirted fleece) were compared with traditional midside sampling followed by laboratory fleece measurement, and with the outcomes of IWTO Certification of lots prepared using these technologies. In this phase, only Mean Fibre Diameter measurements were considered. The specific processes evaluated in this phase were the measurement-assisted classing of single mob wool clips, and the ranking of individual sheep. The evaluation was carried out on one superfine and one fine-wool property in Victoria, and a medium-wool property in South Australia. The work illustrated that these two specific on-farm sampling and measurement technologies (i.e. OFDA2000 measurement on midside samples and Fleecescan measurements on the skirted fleece) produced very similar outcomes with slightly lesser precision than laboratory fleece testing carried out on midside samples in accordance with AS/NZS There was a strong relationship between the two onfarm technologies and their ability to measure Mean Fibre Diameter compared with certificate testing, and when used for fleece classing, they both produced lines of wool of different diameters in the expected order (i.e. the line that was expected to be finest was the finest, etc). Biases were observed between the expected classed line diameters and the certified test results (from 1.1 µm to +0.9 µm), and these varied from property to property. The phenomenon sometimes called "Micron creep" was similarly observed for both technologies and would also be expected to exist if laboratory measurements were to be used for fleece classing. From both the fleece classing and sheep selection results it can be concluded that the precisions of the two on-farm technologies for mean fibre diameter are near equal. Finally, it was concluded that the larger the range in diameter within a flock, and the more precise the measurements, the more likely it is that classed lot measurements and animal selections using different instruments or systems will be consistent. BACKGROUND INTRODUCTION Since the late 1990s, both OFDA2000 (measurement based on imaging technology) and Sirolan Fleecescan /Sirolan Laserscan (measurement based on laser technology) instruments have been regularly used in Australia to measure Mean Fibre Diameter (MFD), but also other wool fibre traits, on samples taken from fleeces on-farm; in a race pre-shearing and in shearing sheds at shearing time. The 2003 Australian Wool Innovation on-farm fibre measurement instrument \ :- Page 1 of 39

2 In September 2001, Australian Wool Innovation Ltd. (AWI) initiated a series of research projects to gain a greater understanding of the issues involved with the development and application of the On-Farm Fibre Measurement (OFFM) technologies and systems. Nine research projects were initiated with respect to OFFM covering: market research, a world-wide search for new technologies, an assessment of potential enhancements to existing technologies, scoping of a Quality Assurance (QA) program, scoping of a wool grower extension program, a benefit cost analysis, an assessment of the potential for electronic identification, an assessment of existing and future decision support systems and a comparison of existing technologies. The market research commissioned by AWI (TQA 2002), which was completed in June 2002, indicated an increasing need by wool producers for an independent and objective assessment of the commercial application of the two current OFFM technologies. At that time, there had been little independent published information on the performance of the two instruments with respect to their use by woolgrowers for clip preparation, sheep breeding and/or flock management. There was also a scarcity of published data that used recognised and industry accepted statistical methods to determine the precision limits of each instrument and/or to compare their outcomes and their application methods. By June 2002, AWI had established a draft protocol for an extensive trial that would address the above issues. AWI also established an OFFM Expert Advisory Group (EAG) to provide assistance and guidance with the trial design, the implementation of the trial, and the analysis and interpretation of the trial results. Details of membership of the EAG, project management and consultancies are given in the Acknowledgements. OBJECTIVES OF THE OFFM INSTRUMENT EVALUATION PROJECT The key objectives of the instrument evaluation project (trial) were to quantify: 1. The precision/repeatability and accuracy of both instruments, with respect to an entire fleece; for the measured characteristics, principally, MFD; but including: Standard Deviation of Fibre Diameter (SDD), Coefficient of Variation of Fibre Diameter (CVD), Mean Fibre Curvature (MFC) and Comfort Factor (CF). 2. The ability of both instruments to class a single mob wool clip into lines of distinctive MFD categories (minimum category of one micrometre (µm)); and 3. The ability of both instruments to rank individual sheep on the basis of MFD. The above objectives relate to the instruments being a part of an OFFM system operated in an on-farm environment. Note, for the purposes of this document an OFFM system, which comprises the measurement instrument, any associated apparatus and application procedures, will be referred to as an instrument. The trial was specifically directed towards providing knowledge that could be used by wool producers (or their sheep and wool customers) to make informed decisions with respect to the use of these OFFM instruments for clip preparation, sheep breeding and/or flock management. The trial provided an opportunity to benchmark one instrument against the other; and against the commercially accepted method of fleece measurement using on-farm midside sampling and off-farm laboratory measurement; and against IWTO Certified results for on-farm prepared sale lots. Objective 1 of this project (phase 1) is described in the companion Part 1 paper (Marler & Baxter, 2004). Objectives 2 and 3 (phase 2) are described herein. SCOPE OF TRIAL DESIGN The trial was limited to providing information to wool producers specifically in relation to the classing and the ranking of the measures defined above. The trial did not assess: The other wool fibre measurements that may be provided by either instrument; The performance of either instrument in an in-store environment; or The two instruments in relation to issues such as ease of use, robustness, Occupational Health and Safety, after sales support, quality assurance, pricing, etc. The 2003 Australian Wool Innovation on-farm fibre measurement instrument \ :- Page 2 of 39

3 OBJECTIVES 2 AND 3 The objectives of Phase 2 of the project, reported herein, were to compare the performance of two OFFM instruments (OFDA2000 in conjunction with midside sampling and Fleecescan applied to skirted fleeces): in terms of classing of individual fleeces for the preparation of lines of wool (wool classing), and for the ranking the individual fleeces for the purposes of sheep selection. Comparisons were also made with midside sample measurements conducted at a commercial fleece testing laboratory using both OFDA100 (LabOFDA) and Sirolan-Laserscan (LabLSN) instruments. Phase 2 involved three properties, with approximately 800 sheep randomly chosen from a single mob from each property. At each property, an OFDA2000 instrument was used in a race prior to shearing and a Fleecescan instrument was operated in the shed in conjunction with shearing. Two properties were located in Victoria (one superfine and one fine-wool) and one in South Australia (medium wool). COMMON PROTOCOLS TRIAL PROTOCOL AND PROCEDURES The following summarises trial protocols that are common to both Phase 1 and Phase 2. The on-farm instruments evaluated in this trial were the OFDA2000 and the Sirolan Fleecescan /Sirolan Laserscan (hereafter called the Fleecescan); which are both used to measure wool fibre characteristics on-farm. The on-farm sampling and measurement environments were at a race in the shed prior to shearing and in the shearing shed in conjunction with shearing, respectively. Note, OFDA2000 instruments are often operated outside the shearing shed; however, for reasons of controlling the trial methodologies, the OFDA2000 testing was conducted inside the shearing shed. The measurements from the on-farm instruments were also compared with measurements derived from laboratory testing using the OFDA100 and the Sirolan Laserscan (hereafter called the Lab OFDA and Lab LSN) instruments. Other aspects of the trial methodology were: Each sheep was uniquely identified; with each fleece sampled, tested and shorn linked to the unique sheep identification. Commercial operators of the two instruments were engaged to provide the equipment, systems and operating personnel needed to carry out the trial in each location. Commercial rates were paid for the supply of these resources. The most recent standard operating procedures for the two instruments were used, with the instruments being operated to reflect normal commercial OFFM practice. An independent supervisor oversaw each trial on all ten properties. PHASE 2 PROTOCOL Phase 2 involved the following protocol: At each property approximately 800 sheep were randomly allocated to either OFDA2000 or Fleecescan fleece classing. One OFDA2000 and one Fleecescan, operated using standard commercial protocols, were used on each trial property. The trial supervisor attended all properties. Prior to measurement, each instrument was checked by the operator using the recommended operating procedures. Midside samples were taken from the left-hand side of all sheep. Normal commercial practice was employed; however, there was one key variation: The 2003 Australian Wool Innovation on-farm fibre measurement instrument \ :- Page 3 of 39

4 After sampling one tuft for OFDA2000 measurement, a midside sample was taken from each sheep for laboratory measurement; and as a result, each fleece had approximately 1% to 5% of its wool removed prior to skirting and measurement by Fleecescan. For Property 1(DO) the OFDA2000 and midside samples were taken whilst the sheep were standing. For Properties 2 and 3 (MS and JM), a VE machine was used to hold the sheep whilst the sampling was undertaken.. Fleeces were classed into one micron bin lines on Properties 1 and 2, and two micron bin lines on Property 3, based on the MFD measurements from the respective instruments. No attempt was made to optimise the number of bins or the bin size to target diameter price points in the market. If a fleece was classed out for quality reasons (e.g. coloured or tender), its identification was recorded and its test result removed from the on-farm instrument's dataset and subsequently from its bin line. Such fleeces were placed in an appropriate cast line. All bin (micron) lines were then baled. RANDOMISATION The sheep were randomised prior to sampling and measurement. Due to different yard and shed layouts, plus inclement weather at the Property 3, different randomising procedures were used on the three properties. Property 1 - Challicum Park (DO) The 800 sheep were initially run through a race and drafted alternately sheep by sheep into two mobs of 400. The sheep were ear-tagged with unique numbers. Table 1 shows the respective tag numbers (sheep) allocated for Fleecescan and OFDA2000 classing. Table 1: Property1 (DO) Tag numbers for sheep classed by Fleecescan and OFDA2000 Fleecescan Classed Sheep OFDA2000 Classed Sheep Tag No. 1 to to 800 Property-2 Mt Sturgeon (MS) It was not possible to separately draft and tag the sheep prior to testing, so groups of 100 or 200 sheep were split off as they came through the yards. The sheep were ear-tagged with unique numbers. Testing by the two instruments was undertaken according to the groups of tagged sheep shown in Table 2. Table 2: Property 2 (MS) Tag numbers for sheep classed by Fleecescan and OFDA2000 Fleecescan Classed Sheep No. Sheep OFDA2000 Classed Sheep No. Sheep Tag No. 811 to to Tag No to to Tag No to to Property 3 - Carlyle (JM) Persistent rain and a small shearing shed made randomisation of the sheep impossible. In addition, there were only 720 dry sheep in the shed. Consequently, the first 360 sheep were classed by Fleecescan and the second 320 classed by OFDA2000. All sheep were given an ear-tag with a unique number. This may have introduced a slight bias between the two data sets, as sheep may be different at the beginning of a mob compared to the end of the mob. Approximate tag numbers for the sheep measured by each instrument are shown in table 3. The 2003 Australian Wool Innovation on-farm fibre measurement instrument \ :- Page 4 of 39

5 Table 3: Property 3 (JM) Tag numbers for sheep classed by Fleecescan and OFDA2000 Fleecescan Classed Sheep OFDA2000 Classed Sheep Tag No to to 2500 Examination of the effectiveness of randomisation between the two classed groups by comparison of the combined certificate MFD values for the Fleecescan and OFDA2000 (midside) classed lines To assess whether the randomisation methods used on the three properties resulted in a biased allocation of sheep, the combined MFD was calculated for all the lines classified by the OFDA2000 (midside) and by the Fleecescan. Table 4 shows the MFD differences between the combined Certification results for each classing system for each of the properties. Table 4: Difference in combined certified MFD between Fleecescan and OFDA2000 classed lines for each property Property No. Difference in Combined Certified MFD (OFDA2000-classed minus Fleecescan-classed) Certification by Lab LSN Certification by Lab OFDA100 Average Certification by Lab LSN & OFDA100 1 (DO) (MS) (JM) Whilst there is very little difference in the Certified test results for Properties 1 and 2, for Property 3 the group of sheep classed by OFDA2000 is, on average, approximately 0.7 µm coarser that the group classed by Fleecescan. Therefore caution is required when comparing the results for OFDA2000 versus Fleecescan classing for this property. OFDA2000 MEASUREMENTS The grease correction factor (GCF) for OFDA2000 (midside) was determined using the standard commercial procedure. Based on greasy and clean measurements made on the first 30 sheep, one GCF was calculated for each property for sampling using the midside site. The OFDA2000 result was based on one test per sheep. The measurements of MFD are made as the technology scans the length of the micro-staple. For Merino wools in the range µm, it is estimated that between 600 and 3000 fibre segments are measured for diameter by the OFDA2000 during the measurement process, depending on staple length. FLEECESCAN At shearing, the ear-tag was recorded on paper which then remained with the fleece for the entire process. After shearing, each fleece was skirted and rolled, transferred to and measured once by Fleecescan. The Fleecescan was set to stop measuring when 600 fibre snippets had been counted. LABORATORY LASERSCAN AND OFDA100 For laboratory testing, a sample was taken from the left midside of each animal using an electric handpiece, after sampling for the OFDA2000 measurement. The midside sample was measured once using Lab LSN (1000 snippets) and once using Lab OFDA100 (one slide). This testing regime was in accordance Australian/New Zealand Standards, viz. AS/NZS :2000 (AS/NZS 2000a) and AS/NZS :2000 (AS/NZS 2000b). The 2003 Australian Wool Innovation on-farm fibre measurement instrument \ :- Page 5 of 39

6 PHASE 2 TRIAL PROPERTIES Table 5 details the properties, wool type, Mean Fibre Diameter and sheep numbers for Phase 2. Table 5: Summary of properties and sheep/wool used in Phase 2 trials Prop. No. 1 (DO) Date 25 Jun 2003 Property Details Challicum Park, Beaufort, VIC Sheep/Fleece Details MF4, adult fine-wool Merino ewes, full wool, 1% VM, 97 mm midside Approx. LSN Cert. MFD No. Sheep Analysed Fleece Classing Sheep Ranking (MS) 16 Jul 2003 Mt Sturgeon, Dunkeld, VIC MF4/3, 10 month superfine Merino hoggets, first shearing, 1.0% VM, 75 mm midside (JM) 14 Sep 2003 Carlyle, Angaston, SA MF5/4, mixed age medium wool Merino wethers, full wool, 1.5% VM, 100 mm midside PHASE 2 STATISTICAL ANALYSES For Phase 2, only MFD measured to one decimal place (µm) was analysed. Fleece Classing The data were edited to include only those fleeces for which all four measurements of MFD were available (i.e. OFDA2000 (midside), Fleecescan, Lab OFDA100 and Lab LSN). The lot means were computed from these fleece values only, so that all means for different methods were based on the same fleeces. As a result, some fleeces that contributed to the certificate results were not included in the predictions. The effect of this anomaly on the outcome was considered to be minor. The comparisons between predicted lot means and certificate results were calculated using an Excel spreadsheet. The likely spread of MFD across fleeces in the trial was determined after the first 100 sheep had been measured by the OFDA2000 on midside samples. The same spread of MFD, as measured by the OFDA2000 (midside), was used for the Fleecescan binning. The bin ranges (µm) chosen are shown in Table 6. One decimal point was used at Challicum Park whilst two decimal points were used at Mt Sturgeon and Carlyle. Table 6: Summary of bin ranges used for Phase 2 fleece classing Bin Challicum Park (DO) Mt Sturgeon (MS) Carlyle (JM) 1 <18.0 <15.95 < to to to to to to to 20.9 >17.94 > Sheep Selection The rankings were made using a Fortran program, written by Dr. John James, which calculated the number of co-selected fleeces for 20% and 80% selection rates based on low MFD. These two cut-offs are used as examples of possible selection criteria for rams and for breeding ewes respectively. These co-selected numbers were then expressed as percentages of the number of sheep selected by the specific system for each of the 20% and 80% cut-offs. All fleeces with four measurements were used for these computations, including those not forming part of the classed lots. In addition, correlations were The 2003 Australian Wool Innovation on-farm fibre measurement instrument \ :- Page 6 of 39

7 calculated for each pair of measurements using the statistical program ISUW (Interactive Statistical Unit for Windows). Analyses using IWTO-0 Appendix B procedures were made using an Excel spreadsheet. All fleeces with four measurements were included in these analyses. Comparisons were made between OFDA2000 and Fleecescan, and between each of these on-farm methods and the two laboratory methods (Lab OFDA100 and Lab LSN). ANALYSES AND RESULTS COMPARISON OF FLEECE CLASSING RESULTS Figure 1 presents for the three properties the relationship between the on-farm Fleecescan and OFDA2000 (midside) results for each bin line and the certified test results for each bin. The line of best fit (Y=X) is also presented. Figure 1: For three properties, comparison of on-farm MFD to Certificate Average MFD for all bin lines 27 Y=X 25 Certificate Average MFD MS DO MS DO JM JM Fleecescan OFDA MFD measured On-Farm by OFDA2000 or Fleecescan Figure 1 shows a strong relationship between the different measurement technologies and their ability to measure MFD compared to Certificate Testing. In other words, when used to class fleeces on-farm into classed lines of 1 µm bandwidths, both OFDA2000 (midside) and Fleecescan were able to produce lines of wool of different diameters in the direction to be expected (i.e. the line that was expected to be the finest was the finest, and so forth). Tables 7, 8 and 9 present the fleece classing results for Challicum Park (DO), Mt Sturgeon (MS) and Carlyle (JM) respectively. Columns with titles including "(m)" signify that the results were obtained on midside samples (as opposed to a random sampling of all fibres in the skirted fleece for the Fleecescan). For each classing method, the results are shown both in terms of the actual averages of the measurements, and as the differences from the average of the two certified results on the lots. Appendices 1, 2 and 3 show IWTO-0 Appendix B analyses for Challicum Park (DO), Mt Sturgeon (MS) and Carlyle (JM), respectively, in which each measurement system pair is compared. The 2003 Australian Wool Innovation on-farm fibre measurement instrument \ :- Page 7 of 39

8 Table 7: Fleece Classing for Property 1 (DO) OFDA2000 Classing Fleece Measurement Technologies Certificate Test Results Line Gsy Nett Wt Sch Dry Wt OFDA2000(m) FLCSCN LabLSN(m) LabOFDA(m) LabLSN LabOFDA100 Avg Combined Differences from Certified Averages Fleece Measurement Technologies Line Gsy Nett Wt Sch Dry Wt OFDA2000(m) FLCSCN LabLSN(m) LabOFDA(m) Combined Fleecescan Classing Fleece Measurement Technologies Certificate Test Results Line Gsy Nett Wt Sch Dry Wt OFDA2000(m) FLCSCN LabLSN(m) LabOFDA(m) LabLSN LabOFDA100 Avg Combined Differences from Certified Averages Fleece Measurement Technologies Line Gsy Nett Wt Sch Dry OFDA2000(m) FLCSCN LabLSN(m) LabOFDA(m) Combined The 2003 Australian Wool Innovation on-farm fibre measurement instrument \ :- Page 8 of 39

9 Table 8: Fleece Classing for Property 2 (MS) OFDA2000 Classing Fleece Measurement Technologies Certificate Test Results Line Gsy Nett Wt Sch Dry Wt OFDA2000(m) FLCSCN LabLSN(m) LabOFDA(m) LabLSN LabOFDA100 Avg Combined Differences from Certified Averages Fleece Measurement Technologies Line Gsy Nett Wt Sch Dry Wt OFDA2000(m) FLCSCN LabLSN(m) LabOFDA(m) Combined Fleecescan Classing Fleece Measurement Technologies Certificate Test Results Line Gsy Nett Wt Sch Dry Wt OFDA2000(m) FLCSCN LabLSN(m) LabOFDA(m) LabLSN LabOFDA100 Avg Combined Differences from Certified Averages Fleece Measurement Technologies Line Gsy Nett Wt Sch Dry Wt OFDA2000(m) FLCSCN LabLSN(m) LabOFDA(m) Combined The 2003 Australian Wool Innovation on-farm fibre measurement instrument \ :- Page 9 of 39

10 Table 9: Fleece Classing for Property 3(JM) OFDA2000 Classing Fleece Measurement Technologies Certificate Test Results Line Gsy Nett Wt Sch Dry Wt OFDA2000(m) FLCSCN LabLSN(m) LabOFDA(m) LabLSN LabOFDA100 Avg Combined Differences from Certified Averages Fleece Measurement Technologies Line Gsy Nett Wt Sch Dry Wt OFDA2000(m) FLCSCN LabLSN(m) LabOFDA(m) Combined Fleecescan Classing Fleece Measurement Technologies Certificate Test Results Line Gsy Nett Wt Sch Dry Wt OFDA2000(m) FLCSCN LabLSN(m) LabOFDA(m) LabLSN LabOFDA100 Avg Combined Differences from Certified Averages Fleece Measurement Technologies Line Gsy Nett Wt Sch Dry OFDA2000(m) FLCSCN LabLSN(m) LabOFDA(m) Combined The 2003 Australian Wool Innovation on-farm fibre measurement instrument \ :- Page 10 of 39

11 Figures 2, 3 and 4 present for the three properties the relationship between the classing decisions made using the on-farm OFDA2000 and Fleecescan instruments and their difference from the average Certified Test results for MFD. Figure 2: Property 1 (DO) - Fleecescan & OFDA2000 classing: Differences versus Certificate MFD Difference from Certificate Average Certificate Average MFD OFDA2000 Flcscn Linear (Flcscn) Linear (OFDA2000) Figure 3: Property 2 (MS) - Fleecescan & OFDA2000 classing: Differences versus Certificate MFD Difference from Certificate Average Certificate Average MFD OFDA2000 Flcscn Linear (Flcscn) Linear (OFDA2000) The 2003 Australian Wool Innovation on-farm fibre measurement instrument \ :- Page 11 of 39

12 Figure 4: Property 3 (JM) - Fleecescan & OFDA2000 classing: Differences versus Certificate MFD Difference from Certificate Average Certificate Average MFD OFDA2000 Flcscn Linear (Flcscn) Linear (OFDA2000) The data in Figures 2, 3 and 4 illustrate classing decisions made using the data for each of the three properties. The trend lines all exhibit a slope and in a number of cases an offset. The data supports the frequently observed phenomenon of micron creep (sometimes also called "micron shrinkage"). Micron creep is related to the measurement uncertainty. The finest and coarsest classed lines are affected by individual fleeces being binned in them that, for a perfect measurement system, with no error, would normally have been classed into a neighbouring bin. The magnitude of the deviation will be larger the higher the Confidence Limit of the measurement system. Micron creep is only related to situations at the extremes where the measurement data has been used to make selection decisions. That is, for OFDA2000(m) measurements during OFDA2000 Classing, and Fleecescan measurements during Fleecescan Classing. Measurements independent of any classing decisions include the following, which are not expected to be significantly affected: all the Laboratory (Lab-LSN(m) and Lab-OFDA(m)) measurements; the Fleecescan measurements when the fleeces are being classed on OFDA2000(m); and the OFDA2000(m) measurements when the fleeces are being classed on Fleecescan. As the slopes in Figs 2, 3 and 4 appear very similar, this would support the conclusion in Phase 1 that the precision of Fleecescan and OFDA2000 midside sampling are near equal (Atkins & Semple, 2003). The graphs also support the conclusion that there were biases between the OFFM systems and the Certified Test results, and that these varied from property to property. For Property 1 (DO), Figure 5 illustrates the relationship between the on-farm OFDA2000 classing decisions and the difference of OFDA2000(m), Fleecescan, Lab-LSN(m) and Lab-OFDA(m) results from the average Certified Test results for MFD. Figure 6 presents the comparable graph for Fleecescan classing. The 2003 Australian Wool Innovation on-farm fibre measurement instrument \ :- Page 12 of 39

13 Figure 5: Property 1 (DO) - OFDA2000 classing. Fleecescan, OFDA2000(m), Lab-LSN(m) and Lab- OFDA(m) results: Differences versus Certificate MFD Difference from Certificate Average Certificate Average MFD Linear (Flcscn) Linear (OFDA2000) Linear (LabLSN(m)) Linear (LabOFDA(m)) Figure 6: Property 1 (DO) Fleecescan classing. Fleecescan, OFDA2000(m), Lab-LSN(m) and Lab- OFDA(m) results: Differences versus Certificate MFD Difference from Certificate Average Certificate Average MFD Linear (Flcscn) Linear (OFDA2000) Linear (LabLSN(m)) Linear (LabOFDA(m)) For Property 1 (DO) the following trends were observed in Figures 5 and 6: The laboratory-measured midside was biased fine compared to the Certified Test results by 0.48 µm. (derived from the average of Lab-LSN(m) (-0.51, -0.70) and Lab-OFDA(m) (-0.28, -0.43)). The OFDA2000(m) was biased fine compared to the Certified results by 0.78 µm (-0.48 µm due to the sample site and µm due to the instrument). The Fleecescan was biased coarse compared to the Certified data by 0.27 µm. For Property 2 (MS), Figure 7 illustrates the relationship between the on-farm OFDA2000 classing decisions and the difference of OFDA2000(m), Fleecescan, Lab-LSN(m) and Lab-OFDA(m) results from the average Certified Test results for MFD. Figure 8 presents the comparable graph for Fleecescan classing. The 2003 Australian Wool Innovation on-farm fibre measurement instrument \ :- Page 13 of 39

14 Figure 7: Property 2 (MS) - OFDA2000 classing. Fleecescan, OFDA2000(m), Lab-LSN(m) and Lab- OFDA(m) results: Differences versus Certificate MFD Difference from Certificate Average Certificate Average MFD Linear (Flcscn) Linear (OFDA2000) Linear (LabLSN(m)) Linear (LabOFDA(m)) Figure 8: Property 2 (MS) Fleecescan classing. Fleecescan, OFDA2000(m), Lab-LSN(m) and Lab- OFDA(m) results: Differences versus Certificate MFD Difference from Certificate Average Certificate Average MFD Linear (Flcscn) Linear (OFDA2000) Linear (LabLSN(m)) Linear (LabOFDA(m)) For Property 2 (MS) the following trends were observed in Figures 7 and 8: The laboratory-measured midside was biased fine compared to the Certified Test results by 0.21 µm. The OFDA2000 was biased coarse compared to the Certified Test results by 0.24 µm (-0.21 µm due to the sample site and µm due to the instrument) The Fleecescan was biased coarse compared to the Certified Test results by 0.06 µm. For Property 3 (JM), Figure 9 illustrates the relationship between the on-farm OFDA2000 classing decisions and the difference of OFDA2000(m), Fleecescan, Lab-LSN(m) and Lab-OFDA(m) results from the average Certified Test results for MFD. Figure 10 presents the comparable graph for Fleecescan classing. The 2003 Australian Wool Innovation on-farm fibre measurement instrument \ :- Page 14 of 39

15 Figure 9: Property 3 (MS) - OFDA2000 classing. Fleecescan, OFDA2000(m), Lab-LSN(m) and Lab- OFDA(m) results: Differences versus Certificate MFD Difference from Certificate Average Certificate Average MFD Linear (Flcscn) Linear (OFDA2000) Linear (LabLSN(m)) Linear (LabOFDA(m)) Figure 10: Property 3 (JM) Fleecescan classing. Fleecescan, OFDA2000(m), Lab-LSN(m) and Lab- OFDA(m) results: Differences versus Certificate MFD Difference from Certificate Average Certificate Average MFD Linear (Flcscn) Linear (OFDA2000) Linear (LabLSN(m)) Linear (LabOFDA(m)) For Property 3 (JM) the following trends were observed in Figures 9 and 10: The laboratory-measured midside was biased fine compared to the Certified Test results by 0.48 µm. The OFDA2000(m) was biased fine compared to the Certified Test results by 0.28µm (-0.48 µm due to the sample site and µm due to the instrument). The Fleecescan was biased coarse compared to the Certified Test results by 0.35 µm. It is clear from Fig 9 and Fig 10 that the biases for property JM were not simple offsets, as was observed for the first two properties, and that the biases were dependent on the measured MFD. The 2003 Australian Wool Innovation on-farm fibre measurement instrument \ :- Page 15 of 39

16 CONCLUSIONS FOR FLEECE CLASSING RESULTS Evidence of Micron Creep exists for all three properties for both measurement technologies used On Farm. There can be no doubt that it would also exist for Laboratory measurements if they had been used for classing the fleeces. Despite the midside sampling site being recognised as the best site to take samples to represent the fleece, this should be considered to be a generalisation as this site will not always give an exact correspondence with the skirted fleece. This finding would not change any recommendation to use the midside sampling site but does have an impact on the potential differences that one could expect to see between OFFM performed on individual fleeces and the Certificate test results derived from the baled wool. COMPARISON OF SHEEP SELECTION RESULTS Table 10 presents for the three properties and four test instruments/methods: the mean, standard deviation, minimum and maximum values and range in MFD. Table 10: Summary statistics: MFD on 3 properties and 4 instruments for the sheep used for classing Property Test System Mean Std Dev Minimum Maximum Range Lab OFDA (DO) Lab LSN OFDA Fleecescan Lab OFDA (MS) Lab LSN OFDA Fleecescan Lab OFDA (JM) Lab LSN OFDA Fleecescan Tables 11, 13 and 15 show, for the three properties, the instrument correlation coefficients for MFD. Tables 12, 14 and 16 show, for the three properties, the number and percentage of co-selected sheep when one instrument is used as the criterion for selecting at both 20% and 80% cut-offs for MFD. Table 11: Property 1 (DO) - Instrument correlation coefficients for MFD OFDA2000 Fleecescan Lab OFDA100 Fleecescan 0.82 Lab OFDA Lab LSN The 2003 Australian Wool Innovation on-farm fibre measurement instrument \ :- Page 16 of 39

17 Table 12: Property 1 (DO) - Ranking for sheep selection at 20% and 80% cut-offs Variate Number at 20% Number at 80% PC at 20% PC at 80% Criterion is OFDA2000 OFDA FLCSCN LAB OFDA LAB LSN Criterion is Fleecescan OFDA FLCSCN LAB OFDA LAB LSN Criterion is Lab OFDA100 OFDA FLCSCN LAB OFDA LAB LSN Criterion is Lab LSN OFDA FLCSCN LAB OFDA LAB LSN Table 13: Property 2 (MS) - Instrument correlation coefficients for MFD OFDA2000 Fleecescan Lab OFDA100 Fleecescan 0.77 Lab OFDA Lab LSN The 2003 Australian Wool Innovation on-farm fibre measurement instrument \ :- Page 17 of 39

18 Table 14: Property 2 (MS) - Ranking for sheep selection at 20% and 80% cut-offs Variate Number at 20% Number at 80% PC at 20% PC at 80% 4Criterion is OFDA2000 OFDA FLCSCN LAB OFDA LAB LSN Criterion is Fleecescan OFDA FLCSCN LAB OFDA LAB LSN Criterion is Lab OFDA100 OFDA FLCSCN LAB OFDA LAB LSN Criterion is Lab LSN OFDA FLCSCN LAB OFDA LAB LSN Table 15: Property 3 (JM) - Instrument correlation coefficients for MFD OFDA2000 Fleecescan Lab OFDA100 Fleecescan 0.90 Lab OFDA Lab LSN The 2003 Australian Wool Innovation on-farm fibre measurement instrument \ :- Page 18 of 39

19 Table 16: Property 3 (JM) - Ranking for sheep selection at 20% and 80% cut-offs Variate Number at 20% Number at 80% PC at 20% PC at 80% Criterion is OFDA2000 OFDA FLCSCN LAB OFDA LAB LSN Criterion is Fleecescan OFDA FLCSCN LAB OFDA LAB LSN Criterion is Lab OFDA100 OFDA FLCSCN LAB OFDA LAB LSN Criterion is Lab LSN OFDA FLCSCN LAB OFDA LAB LSN CONCLUSIONS FOR SHEEP SELECTION RESULTS Whilst the OFDA2000 showed slightly higher correlation coefficients with the Lab OFDA100 and Lab LSN than the Fleecescan, this is to be expected since the OFDA2000, Lab OFDA100 and Lab LSN were all using samples obtained from the midside (they are comparing like with like), whereas the Fleecescan uses a random selection of fibres obtained from the skirted fleece. For all three properties the highest correlation between the measurements was between Lab-OFDA(m) and Lab-LSN(m), as would be expected given the slightly better precision of the laboratory measurements shown in Phase 1 of the project. The percentages of co-selected sheep vary with the ratios of the measurement precision of the two measurement systems to the range of MFD values in the selected group. Thus, for example, the percentage of co-selected sheep is higher in all cases for the 80% of sheep selected with the lowest MFD than it is for the 20% of sheep selected with the lowest MFD. This is to be expected since the ratio of the range to the measurement precision of MFD values in the 80% selection situation is significantly higher than for the 20% situation. In order to improve the agreement between measurement systems in the selection of the "top" 20% of a flock, it would be necessary to improve the precision of measurement so that the ratio of the range to the precision increases. This confirms the observation made in Phase 1 that growers using the technologies for ram selection should compare ways of improving precision of measurement against the benefits to be obtained. Similarly, the percentages in any category of comparison are higher for Property 3 than for Property 2. The former had a wider range of MFD values than the latter (approximately 15 µm compared to 6 µm - see Table 10). The results for Property 1 fell somewhere between the corresponding values for the other The 2003 Australian Wool Innovation on-farm fibre measurement instrument \ :- Page 19 of 39

20 two properties, as did the range of MFD values (approximately 9 µm). The percentages of co-selected sheep were highest when comparing the two laboratory midside measurements, and lowest when comparing the two on-farm technologies, with the corresponding comparisons between an on-farm technology and a laboratory measurement falling somewhere between. These relationships mirror those of the correlation coefficients. It can therefore be concluded that the greater the range in diameter within the flock, and the more precise the measurement, the more likely it is that animal selections made using different instruments will be consistent. CONCLUSIONS AND RECOMMENDATIONS The following conclusions were drawn based on the results presented for the three properties: Laboratory measurements on midside samples were slightly more precise than those made on-farm. The greater the range in diameter within the flock, and the more precise the measurements, the more likely it is that measurement assisted classed lot measurements and animal selections made using different instruments will be consistent. There was a strong relationship between the two on-farm measurement technologies and their ability to measure MFD compared to Certificate Testing. When used for fleece classing, both the OFDA2000 (when measuring midside samples) and the Fleecescan (when measuring skirted fleeces) produced lines of wool of different diameters in the direction to be expected (i.e. the line that was expected to be the finest was the finest, and so on). From fleece classing and sheep selection results it can be concluded that the precisions of Fleecescan and OFDA2000 midside sampling are near equal. Biases were observed between the OFFM systems and the Certified Test results, and these varied from property to property, ranging overall on individual lots from µm to +0.9 µm. A portion of these biases could be attributed to the midside sampling site (-0.5 µm to -0.2 µm based on the averages of the combined Lab results on the three properties), and the remainder to the individual OFFM instruments themselves (OFDA2000(m) -0.3 µm to +0.5 µm, and Fleecescan +0.1 µm to +0.4 µm). Micron Creep was observed for both measurement technologies used on farm. It would also exist for Laboratory measurements if they were to be used for classing the fleeces. There is no reason to suppose that these general conclusions obtained on 3 properties are not widely applicable. Clearly, however, as has been demonstrated, the magnitude of the effects on individual properties, and with different operators and instruments, will vary, depending on the circumstances prevailing at the time. The 2003 Australian Wool Innovation on-farm fibre measurement instrument \ :- Page 20 of 39

21 ACKNOWLEDGEMENTS This substantial trial was totally funded by Australian Wool Innovation on behalf of the wool producers of Australia. This report is based on the AWI Final Milestone Report on project EC 397, which details the following acknowledgements: This has been a large-scale trial, which would not have been possible without the help and support of a large number of individuals and companies. These people and their organisations are listed below. AWI established an Expert Advisory Group whose role was to establish the trial protocol; monitor the conduct of the trials, assist with and review the analyses and to prepare the Final Report to AWI. This Group s contribution has been significant. In particular, Jim Marler and Peter Baxter, with input from Ralph Behrendt, undertook an enormous amount of additional work to complete the protocol, analysis and draft of the final report. Their contribution and that of their companies for allowing them the time to do so is specifically acknowledged. Expert Advisory Group Members Mr Peter Baxter SGS Wool Testing Services Wellington, New Zealand Dr Ralph Behrendt Department of Primary Industries Hamilton, VIC Dr Kerry Hansford Teckel Consulting Pty Ltd Melbourne, VIC Dr Bill Humphries CSIRO Textile and Fibre Technology Geelong, VIC Dr John James Consultant Sydney, NSW Mr Jim Marler AWTA Ltd Sydney, NSW Dr Paul Swan AWI Ltd Sydney, NSW Mr George Waldthausen (Chair) AWI Ltd Sydney, NSW Woolgrowers Jack Donaldson Chatsworth House, Lake Bolac, VIC Bill Blackwell "Corea", Dunkeld, VIC Alan MacInnes Bernifay, Natimuk, VIC Bill & Vicki Webb Marbarrup, Kojunup WA Bevan McKnight Mt Sturgeon, Dunkeld, VIC Angus Jackson Corea Lane, Glenthompson, VIC Austin Eagle Glenara North, Horsham, VIC Ken Baldry Wallendbeen East, Cootamundra, NSW David Oddie Challicum Park, Beaufort, VIC Jim Mitchell Carlyle, Angaston, SA Instrument Owner/Operators Janelle Pearson Landmark, Bendigo, VIC Bill Crawford and Megan Jackson Sierra Park, Victoria Valley, VIC Russell Pitcher RD Pitcher, Skipton, VIC Paul Cocking Joyce and Leigh Gordon Bullawyn Pastoral Co, Dunkeld, VIC Colin Parker Dunkeld, VIC Sarah Lannan Landmark, Goulburn, NSW Paul Vallely The 2003 Australian Wool Innovation on-farm fibre measurement instrument \ :- Page 21 of 39

22 Riverina Wool Testers Pty Ltd Wagga Wagga, NSW Paul Cousins Cousins Merino Services Pty Ltd, Burra, SA Anita Eyres Doreenup Wool Testing, Kojonup, WA Ian Bradtke CH Bradtke & Son, Peterborough, SA Southern Tablelands Fibre Testing Crookwell, NSW Sydney East Multiplex Constructions Pty Ltd, Dongra, WA Andy Paton Mobile Microns, Deniliquin, NSW Service Providers Jock Simmons Aust. Cashmere Growers Assoc. Kellyville, NSW Bill Johnstone IWG, Fremantle, WA John Cheshire and Andrew Lindsay AWTA Ltd, Sydney, NSW David Cother AWEX, Melbourne, VIC Rob Mahoney Skipton, VIC Rod Agar and John Billing, AWTA Ltd., Melbourne, VIC Richard Manning IWG, Tamworth, NSW Eric Reiher AWTA Ltd, Newcastle, NSW Ross Bawden and Cam Cuthbertson Landmark, Brooklyn, VIC Bruno DeMattia Melbourne, VIC Additional Trial Staff Steve Harvey Malvern, VIC Jeff Wilson, Wallan, VIC Don Wight, Adelaide, SA Linda Venables Rippolea, VIC Peter Conrick, Turretfield, SA Cheryl Pope, Orange, NSW Project Management Program Manager: Mr George Waldthausen, AWI Project Manager: Mr Gary Macfarlane, GMAC Consulting Pty Ltd. Trial Manager: Mr Russell Pattinson, Miracle Dog Pty Ltd. Project Consultants Statistical Analysis: Dr John James Final Milestone Report Preparation: Dr Kerry Hansford, Teckel Consulting Pty Ltd The 2003 Australian Wool Innovation on-farm fibre measurement instrument \ :- Page 22 of 39

23 REFERENCES Marler, J and Baxter, P (2004) The 2003 Australian Wool Innovation on-farm fibre measurement instrument evaluation trial - Part 2: Performance in objective classing and ranking for animal selection. IWTO CTF, Evian, Paper CTF 01. BIBLIOGRAPHY 1. Andrews, R.N., Land, J.T.J. and Dodds, K.G. (1997) Classing of adult Merino fleeces using fibre diameters predicted from hogget measurements. Wool Tech. Sheep Breed. 45(1): AS/NZS (2000a) Australian Standard/New Zealand Standard :2000: Wool Fleece Testing and Measurement Measurement of mean fibre diameter and determination of fibre diameter distribution using the Sirolan-Laserscan measuring system. 3. AS/NZS (2000b) Australian Standard/New Zealand Standard :2000: Wool Fleece Testing and Measurement Measurement of mean fibre diameter and determination of fibre diameter distribution using the OFDA measuring system. 4. Atkins, K.D. and Semple, S.J. (2003) The economic benefits of on-farm fibre diameter testing in commercial Merino flocks: (1) Objective Clip Preparation. Wool Tech. Sheep Breed. 51(1): Baxter, P. (2000) Precision of measurement of diameter, and diameter length profile, of greasy wool staples on-farm, using the OFDA2000 instrument. 10th Intl. Wool Text. Res. Conf., Aachen, Paper RW Baxter, P. (2001) On-farm classing of animals and fleeces with the OFDA2000. Wool Tech. Sheep Breed. 49(2): Baxter, P. and Cottle, D.J. (1998) The use of midside fleece fibre diameter distribution measurements in sheep selection. Wool Tech. Sheep Breed. 46(2): Baxter, P. and Johnston, W. (2002). OFDA2000 proficiency trials. IWTO T & S Committee, Nice, December 2002, Report CTF Behrendt, R., Konstantinov, K., Brien, F., Ferguson, M. and Gloag, C. (2002) A comparison of estimates of mean fibre diameter, variation in fibre diameter and fibre curvature between OFDA2000 and conventional laboratory-based fibre testing. Wool Tech. Sheep Breed. 50(4): Brien, F.D, Ferguson, M.B. and Konstantinov, K. (2001) A comparison of fleece testing methods for the establishment of a ram breeding nucleus in a wool flock and for clip preparation. Proc. Assoc. Advmt. Anim. Breed. Genet. 14: Cottle, D.J., Almeida, C.D., Baxter, B.P. and Petrie, D.J. (1996) Precision of OFDA fibre diameter measurements on midside wool samples. Wool Tech. Sheep Breed. 44(4): Fish, V.E., Mahar T.J. and Crook, B.J. (2002) Sampling variation over a fleece for mean fibre diameter, standard deviation of fibre diameter and mean fibre curvature. Paper 32, Wool Industry Science and Technology Conference, October 2002, Hamilton, Victoria, Australia. (Also Wool Tech. Sheep Breed. 50(4): ). 13. Gillies, R.I. and Stadler, W. (1997) A four year study of the ongoing effectiveness of flock ranking following single testing for mean fibre diameter. Wool Tech. Sheep Breed. 45(1): Hansford, K.A., Marler, J.W. and McLachlan, I.M. (2002) Using OFDA2000 and Fleecescan to prepare lots for sale and sheep selection: a case study, Paper 35, Wool Industry Science and Technology Conference, October 2002, Hamilton, Victoria, Australia. (Also Wool Tech. Sheep Breed. 50(4): ). 15. IWTO (2001) IWTO-0-01 Introduction to IWTO Specifications. Procedures for development, review, progression or relegation of IWTO Test Methods and Draft Test Methods. The 2003 Australian Wool Innovation on-farm fibre measurement instrument \ :- Page 23 of 39