Colin Bird, Doosan Babcock , Glasgow, Scotland
Paper presented at 4th European - American Workshop on Reliability of NDE, Berlin, Germany, 24-26 June 2009.
Abstract
Many projects and studies have demonstrated that under strict control a wide range of components can be inspected reliably with ultrasonic phased array inspection, with better flaw sizing than conventional manual ultrasonic inspection. Phased array ultrasonic technology and instruments for non-destructive testing (NDT) are now widely available in the form of affordable/portable instruments from a number of manufacturers, as well as in the form of laboratory based, high performance systems. TWI has conducted a project, with nine sponsors representing nuclear utilities, oil and gas companies, military and regulatory authorities, to quantify the reliability for defect detection and sizing of manually applied phased array systems for ferritic welds. Five other companies assisted in the project by providing phased array operators, and a further company provided conventional manual ultrasonic operators.
The trial has generated 400 flaw size measurements from 10 phased array operators. The flaws included lack of fusion defects, cracks and volumetric flaws in butt welds ranging from 6mm to 50mm in thickness. This paper presents a statistical analysis of the results for defect sizing with respect to flaw size, flaw characterisation and operator qualifications. Furthermore this paper provides strong evidence and recommendations for the provision of good quality phased array inspections and operators.
1. Introduction
Phased array ultrasonic technology and instruments for non-destructive testing (NDT) have entered the second stage of their development. They are now widely available in the form of affordable/portable instruments from a number of manufacturers, as well as in the form of laboratory based, high performance systems. The cost of the portable equipment has reduced to the level where the technique is being used for widespread high quality applications. Projects (eg[1]) on a wide range of components have demonstrated that qualified phased array inspections can provide better flaw sizing than conventional manual ultrasonic testing (UT). This paper aims to provide industry with the objective technical information about the current standard of inspection reliability for manually applied phased array inspection.
To provide a base line for comparison with the phased array results, TWI also subjected the test blocks to manual UT, using experienced qualified level 2 operators.
This paper is based upon the results of a joint industry project and was designed to assess manually applied phased array inspection for flaw sizing. Manually applied techniques were deliberately chosen to enable a number of companies to take part without the expense of purpose-built manipulators. Although the techniques were manually applied, all of the data were recorded by the phased array systems, except in the case of the GE Phasor, which could not record data.
2. Project Approach
2.1 Project Outline
Six phased array companies, including TWI, and ten phased array operators were used for the blind trial. The volunteer companies were AGR, GE, M2M, Olympus and Sonatest (Harfang) plus five manual operators (four from British Energy/Oceaneering and one from TWI). These operators had a range of experience and training in phased array inspection and data analysis. TWI wrote and provided a generic inspection procedure to each company and operator. Each company followed this inspection procedure where possible. Where this was not possible (eg due to differences between 16-channel and 32-channel instruments), a modified procedure was applied. The base line procedure was provided in an attempt to distinguish variations in performance due to the operator and the type of equipment used from variations due to the procedure. However, the companies were also offered the opportunity to apply their own procedures, using separate operators; two of the volunteer companies chose to do this.
TWI designed and procured eight test blocks containing a broad spectrum of flaw sizes and types. Neither the operators nor the sponsor companies had any knowledge of the location, type and size of the flaws, the number of flaws in each block or the total number of flaws, ie the trials were 'blind'. The test blocks were manufactured by joining flat carbon steel plates with butt welds of various configurations. All plates were in the as-welded condition, with weld caps intact.
Each operator (phased array and manual) was presented with the eight test blocks and instructed to scan the blocks to the procedure and then analyse the data according to the inspection sensitivity specified in the procedure. The operators were allowed one week to inspect the test blocks and report the results. The operators were also briefed that critical flaw sizing was the primary objective of the trial and that the results were anonymous.
Following the blind examination of the blocks, they were re-examined with TOFD and sectioned where necessary to confirm the true defect sizes. This paper does not discuss the TOFD results. The results were collated and multivariate statistical analysis used to establish the factors governing the reliability of defect sizing.
2.2 Test block design
The eight test blocks were manufactured to contain a total of 40 flaws. The plate thicknesses ranged from 6mm to 50mm. The flaw types comprised:
- Smooth planar, to simulate lack of side wall fusion.
- Rough planar, to simulate cracks.
- Volumetric, to simulate porosity or slag.
The flaw locations, orientations and sizes comprised:
- Fusion face flaws (referred to in this project as tilted).
- Centre-line flaws (referred to in this project as non-tilted).
- Surface-breaking and sub-surface flaws.
- Through wall size from 1mm to 24mm.
To assist the statistical treatment of the results, a balanced experimental design with respect to flaw type, size and location within each test block size range was deployed.
2.3 Inspection procedure
The inspection procedure written by TWI was based on a single pass on each side of the weld with an azimuthal scan pattern in the through-wall direction. This procedure was designed for a 32/128 phased array instrument. Other procedures mixing linear and azimuthal scans were also used by the volunteer companies. Two of the volunteered procedures used 16/128 instruments; for these instruments the procedure and focal distances were modified.
The 6mm, 20mm and one of the 35mm plates were scanned from just one surface whereas the 35mm and 50mm plates with double V weld preparations were scanned from both surfaces. The scans were designed to provide half and full skip data from each side of the weld. The procedure required all flaw indications with a signal amplitude greater than DAC-26dB (ie 26dB more sensitive than 3mm side drilled hole DAC) to be investigated and all flaws with a length greater than 10mm to be sized and reported.
3. Analysis of results
3.1 Detection performance
Table 1. Detection performance
| | Missed calls | False calls | Detection % | Detection rate
excluding operators
6 & 7% |
|---|
| Phased array |
operator 1 |
4 |
2 |
90 |
90 |
| operator 2 |
1 |
1 |
97.5 |
97.5 |
| operator 4 |
1 |
4 |
97.5 |
97.5 |
| operator 15 |
1 |
1 |
97.5 |
97.5 |
| operator 5 |
0 |
3 |
100 |
100 |
| operator 6 |
10 |
2 |
75 |
NA |
| operator 7 |
13 |
3 |
67.5 |
NA |
| operator 8 |
0 |
6 |
100 |
100 |
| operator 9 |
1 |
2 |
97.5 |
97.5 |
| operator 10 |
3 |
4 |
92.5 |
92.5 |
| PA average |
3.4 |
2.8 |
91.5 |
96.6 |
| Std deviation |
|
|
11.3 |
3.5 |
| Manual UT |
operator 11 |
2 |
0 |
95 |
95 |
| operator 12 |
7 |
6 |
82.5 |
NA |
| operator 13 |
5 |
2 |
87.5 |
87.5 |
| operator 14 |
2 |
0 |
95 |
95 |
| operator 3 |
1 |
0 |
97.5 |
97.5 |
| Manual average |
3.4 |
1.6 |
91.5 |
93.8 |
| Std deviation |
|
|
6.3 |
4.3 |
Although it was not the prime driver for the project, the flaw detection rate for each operator (phased array and manual) was recorded and is summarised in Table 1. Detection of a flaw was defined as reporting the flaw in a location which physically overlapped the location (in both through wall and axial position) that was deemed to be 'correct' (based on design sizes, confirmed by supplementary NDT, and/or selective sectioning). It can be seen that two phased array operators and one manual operator had a substantially lower flaw detection rate than the average. For this reason they were excluded from this set of statistics. It should be noted that these operators also had the poorest flaw sizing ability and had no training in critical defect sizing. After exclusion of these operators' results, the overall flaw detection rate was 96.6% for phased array and 93.8% for manual UT.
3.2 Phased array sizing performance
Our chosen methodology for the statistical analysis of sizing accuracy was based on analysis of variance[2,3].
The main aim of the analysis was to identify which of the following factors have a statistically significant effect on sizing performance:
- Procedure/equipment
- Operator
- Operator experience and qualifications (eg whether or not the operator has a critical defect sizing qualification, such as CSWIP, in phased array)
- Wall thickness (including any asymmetry in the block)
- Weld shape (ie V, J or X)
- Flaw characteristics (eg size, type, location).
Further to the statistical treatment of the phased array data, a simple numerical analysis of the manual results was undertaken and a comparison made with that of the phased array results.
Fig.1. Observed phased array sizing errors for 6mm plate versus other plate thicknesses
Fig.2. Observed phased array sizing errors versus flaw height and operator for plates thicker than 6mm
Figure 1 shows the observed sizing errors for the 6mm plate versus those from the other plates. It can be seen from
Figure 1 that both the mean (or 'systematic') sizing error and the scatter about this mean (or 'random' error) is greater for the thicker plates than for the 6mm plate. But this is not unexpected as there is more scope to make larger errors in thicker plates.
Figure 2 shows the observed sizing errors broken down by operator and excluding the 6mm plate results. It can be clearly observed that different operators had a different pattern of errors and the best operators had the least variation of sizing error with respect to flaw size.
Because of the variations in sizing performance illustrated in Figures 1 and 2, it is clearly inappropriate to assign a single value to the flaw sizing error. Furthermore, due to the scatter in operator performance, it was difficult to distinguish variations with the procedure and the equipment from the variations due to the operator. However, there were no variations between different groups of operators that could be easily attributed to the different types of equipment. In particular, there was no evidence from this project that the 16-channel instruments performed any worse than the 32-channel instruments.
Those sizing errors larger than ~10mm in magnitude appeared not to belong to the same normal distribution as the rest of the data; it was surmised that these extreme sizing errors were, in fact, 'blunders' rather than true measurement errors.[4] These 'outliers' were removed from the data during the subsequent analysis. The distribution of the remaining sizing errors is approximately normal.
Despite the large scatter in the results, a number of conclusions can be drawn from the statistical analysis, based on formal statistical tests (each at the 5% significance level):
- In general, the following factors had a significant effect on the mean ('systematic') sizing error:
- Flaw height
- Operator
- Plate thickness
- The overall trend is for the systematic sizing error to decrease with flaw height, ie there is a greater tendency to undersize large flaws than small flaws. However, the trend is significantly different for different operators, egsome operators have a greater tendency to undersize large flaws than others (as illustrated in Figure 2).
- For a given flaw height, the systematic sizing errors are slightly larger (ie greater tendency towards oversizing) in the thicker blocks than in the thinner blocks. However, the effect of block thickness is much less pronouncedthan that of the operator or the flaw height. Also, there is no significant difference between the one asymmetric block and the symmetric block closest in thickness to it.
- In general, the random sizing errors also tend to increase with flaw height. A linear trend was fitted to the variance of these random errors.
- The random sizing errors in the plates thicker than 6mm show significant variations with flaw type. In particular, smooth non-tilted flaws appear to result in larger random errors than rough flaws (see Figure 3).
- The random errors are ~20% smaller for those operators who hold a phased array critical defect sizing qualification (CSWIP) compared to those who do not (see Figure 4). This latter effect is marginally significant.
- The random sizing errors in the 6mm plate show significant variations between different operators. It is unclear whether these variations reflect differences in procedure, equipment or the skill/experience of individual operators.Note, however, that the random errors are ~40% smaller for those operators with more than 5 years experience, as compared to those with less experience (see Figure 5).
Fig.3. Comparison of standard deviations of the (dimensionless) standardised random sizing errors for phased array versus flaw type for blocks thicker than 6mm (RNT = rough non-tilted; RT = rough tilted; SNT = smooth non-tilted; ST = smooth tilted; V = volumetric)
Fig.4. Standard box plot comparing the standard deviations of the (dimensionless) standardised random sizing errors for phased array versus phased array qualifications (plates thicker than 6mm)
Fig.5. Standard box plot comparing the standard deviations of the (dimensionless) standardised random sizing errors for phased array versus operator experience (6mm plate)
4. Discussion
TWI supplied a procedure which was believed to provide the optimum flaw sizing ability for the set of test blocks. Current industry practice is to use linear scanning with more than one beam angle for phased array inspection, one of the beam angles being chosen to be at near-normal incidence to the weld fusion face. Beam angles further away from normal incidence enable the procedure to use the maximum amplitude sizing technique, making use of echoes diffracted from the edges of the flaws. It was anticipated that, by choosing an azimuthal scan rather than a normal incidence linear scanning method, the flaw detection rate might be reduced. Further, the inspection sensitivity needs to be higher to cope with the oblique angles of incidence. Volunteer companies both followed the TWI procedure and provided additional inspection procedures. In particular, there were two clearly different procedures applied in addition to that of the TWI procedure (referred to as Procedure 1). Procedure 2 used both linear and azimuthal scans together with a 16-channel instrument. The flaw detection rate of 97.5% for this procedure was comparable to that achieved by the better phased array operators using Procedure 1. Procedure 3 was manually applied phased array, where the operator could perform a mixture of manual linear and phased array azimuthal scans. The detection frequency for these scans was also 97.5%. Thus, in this project, there was no clear evidence that the exclusive use of azimuthal scans by Procedure 1 resulted in poorer detection performance.
Fig.6. Phased array data and macro of a 24mm through wall flaw
The only significant variation (at the 5% level) in the sizing errors with flaw type was that smooth non-tilted flaws appeared to result in larger random errors than rough flaws (for plates >6mm thick). From a practical viewpoint, this is not unexpected. As can be seen from the analysis, the results from these thicker plates have been dominated by quite large sizing errors. In particular, the operators tend to misinterpret large flaws. TWI's review of the data indicates that sizing errors greater than 6mm arose not because of any inherent limitation of the sizing technique nor because the flaws were inherently misleading. Instead, it is believed that these were essentially data analysis errors. This is illustrated in Figure 6 which presents both the flaw and the phased array data. This data was correctly sized by some operators and incorrectly (by up to 20mm) by others. This analysis has revealed that all the flaws can be sized correctly in the through wall direction using the maximum amplitude technique and/or identification of the flaw tip diffraction signal. Smooth flaws lying on the fusion face (such as that shown in Figure 6) can be misinterpreted if the ultrasonic beams are not at perfectly normal incidence. Operators who have the critical defect sizing certificate have been taught to size from tip echoes; this may be why this set of operators achieve better overall sizing accuracy.
The total time taken for the phased array inspection was similar to that for the manual UT (5 days in each case). However, unlike for manual UT, the majority of the time taken for the phased array inspection was spent analysing data (which can be done 'off-line'); the scanning time constituted less than 20% of the overall inspection time.
5. Conclusions
This work has shown that there is a large variation in the ability of phased array operators to size flaws. There is also evidence that this sizing capability is linked with training, experience, size of flaw and whether the flaws are rough or smooth.
6. Recommendations
- To improve the quality of data interpretation it is recommended that phased array operators are trained to use crack tip diffraction signals and gain experience of large (>beam width) smooth planar flaws.
- To provide a better phased array service to industry, it is recommended that examination of operators include sizing of flaws greater than 6mm through wall and that the examination certificate states that the operators have been critically examined for defect sizing with stated sizing error tolerances.
7. Acknowledgements
The authors of this paper and TWI wish to thank the following volunteer companies and phased array equipment operators for the considerable work they put into the results presented in this paper: AGR, GE, M2M, Olympus and Sonatest (Harfang).
TWI also thanks the sponsors of the project for permission to release this paper, namely: BAE Systems, British Energy Generation Ltd, ESKOM, Exxon Mobil, Lloyds Register EMEA, Ministry of Defence (MOD), Petrobras, Shell, US Navy.
8. References
- Shipp R, Schneider C R A, Bird C R and Wood D A, 2002: 'Independent qualification of phased array inspection of fillet welds'. Proceedings of 2002 BINDT Conference.
- Cooper B E, 1969: 'Statistics for experimentalists'. Pergamon, Oxford. ISBN 0 08 012600 6.
- Minitab, 1998: 'Minitab reference manual - Release 12 for Windows'. Minitab Inc (USA), February.
- Chapman R K, 1993: 'Guidance document on the assessment of flaw measurement errors in the ultrasonic NDT of welds'. Nuclear Electric report TIGT/REP/0031/93 Issue 2, August.