Carmelo Mineo, Jonathan Riise and S Gareth Pierce
University of Strathclyde, Department of Electronic and Electrical Engineering Glasgow, G1 1XW, UK P Ian Nicholson and Ian Cooper
TWI Technology Centre (Wales) Port Talbot, SA13 1SB, UK
Paper presented at NDT 2015. The 54th annual conference of the British Institute of Non-Destructive Testing. 8-10 Sept. 2015, Telford, UK
The requirement to increase inspection speeds for non-destructive testing (NDT) is common to many manufacturers. The prevalence of complex curved surfaces in modern products provides motivation for the use of 6 axis robots in these inspections. The techniques and issues associated with conventional manual inspection techniques and automated systems for the inspection of large complex surfaces were reviewed. This paper presents a new MATLAB based software solution (RoboNDT), aiming to fulfil the requirements of robotized NDT inspection. RoboNDT enables flexible trajectory path planning to be accomplished for the inspection of complex curved surfaces. This newly developed software is capable of complex path planning, obstacle avoidance, and external synchronization between robots and connected NDT systems. Comparative accuracy experiments were undertaken to evaluate the path accuracy produced by the software when inspecting a curved 0.5 m2 and a 1.6 m2 surface using a KUKA KR16 L6-2 robot. The advantages of this software over conventional off-line-programming approaches are highlighted. By implementing full external control of the robotic hardware, it has been possible to synchronise the NDT data collection with positions at all points along the path. This approach facilitates the future development of additional functionality, suited to both fixed and mobile robot NDT inspection solutions.
In civil aerospace manufacturing, the increasing deployment of composite materials demands a high integrity and traceability of Non-Destructive Testing (NDT) measurements, combined with a rapid throughput of data. Modern components increasingly present challenging shapes and geometries for inspection. Using traditional manual inspection approaches produce a time-consuming bottleneck in the industrial production (1) and this limitation provides the fundamental motivation for increased automation.
Manual scanning requires trained technicians and results in a very slow inspection process for large samples. The repeatability of a test can be challenging in structures where complex setups are necessary to perform the inspection (e.g. orientation of the probe, constant standoff, etc.) (2). Developing reliable automated solutions has become an industry priority to drive down inspection times. The fundamental aims of automation within the inspection process are to minimize downtimes due to the higher achievable speed, and to minimise variability due to human factors.
Semi-automated inspection systems have been developed to overcome some of the shortcomings of manual inspection techniques, using both mobile and fixed robotic platforms (3, 4). In the spectrum of robot manipulators, some modern robots have suitable attributes to develop automated NDT systems and cope with the challenging situations seen in the aerospace industry (5). They present precise mechanical systems, the possibility to accurately master each joint, and the ability to export positional data at frequencies up to 1 kHz.
Exploring the current state of the art, RABIT is a group of robotic inspection systems developed by TECNATOM S.A., in collaboration with KUKA Robots Ibérica (5). Off-the-shelf robotic arms were also used in the LUCIE (Laser Ultrasound for Composite InspEction) system, addressed to inspect large curved surfaces such as the inside of aircraft fuselage, by means of ultrasound generated by laser (6). Genesis Systems Group has developed the NSpect family of Robotic Non-Destructive Inspection cells. General Electric (GE) has also investigated the integration of phased array UT with off-the-shelf industrial robots for the inspection of aerospace composites (7, 8).
Despite these previous efforts, there remain challenges to be addressed before fully automated NDT inspection of complex geometry composite parts becomes commonplace. The key challenges include generation and in-process modification of the robot tool-path, high speed NDT data collection, integration of surface metrology measurements, and overall visualisation of measurement results in a user friendly fashion. Collaborations driving this vision include TWI Technology Centre (Wales), which has carried out a 3-year project, called IntACom, on behalf of its sponsors; the objective of obtaining a fourfold increase in the throughput of aerospace components has been achieved (1). Additionally the UK RCNDE consortium conducts research into integration of metrology with NDT inspection (9, 10). Both these consortia have identified the requirement for optimal tool-path generation over complex curved surfaces, and the current paper describes joint work between these groups to develop a novel approach to a flexible robotic toolpath generation using a user friendly MATLAB based software application. The software, named RoboNDT, provides a low cost approach to robot path planning for NDT applications, and a platform for future solutions of highly specific NDT inspection challenges.
2. Existing robotic path-planning software
Six-axis robotic arms have traditionally been used in production lines to move the robot end-effector from one position to another for repetitive assembly and welding operations. In this scenario, where the exact trajectory between two points in space is not too important, the teach pendant of a robot is used to manually move the end-effector to the desired position and orientation at each stage of the robot task. Relevant robot configurations are recorded by the robot controller and a robot program is then written to command the robot to move through the recorded end-effector postures. More recently, accurate mechanical joints and control units have made industrial robotic arms flexible and precise enough for finishing tasks in manufacturing operations (11). Robotic manipulators are highly complex systems and the trajectory accuracy of a machining tool has a significant impact on the quality and tolerances of the finished surfaces. As a result, many software environments have been developed by manufacturers, academic researchers and also by the robot manufacturers themselves, in order to help technicians and engineers program complex robot tasks (12). The use of such software platforms to program robot movements is known as Off-Line Programming (OLP). It is based on the 3D virtual representation of the complete robot work cell, the robot end-effector and the samples to be manipulated or machined. Although some limited applications for inspection delivery have been demonstrated (13), in general conventional OLP is geared towards manufacturing applications where the task is the production of a specific component using conventional milling / drilling / trimming operations. In contrast, the result of an automated NDT inspection requires a flexible and extensible approach that has the possibility to allow changes in the path planning to accommodate requirements of future NDT inspections (e.g. fluid dynamics modelling of water-jets, compensation for part variability and conditional programming approaches).
Using current OLP software to generate appropriate tool-paths for NDT purposes can appear straightforward at first glance; however there are two specific inadequacies:
- Significant complications exist when two or more robotic arms need to be synchronized in order to perform a specific NDT inspection. The Ultrasonic Through-Transmission (UTT) technique, for example, uses two transducers: one emitter and one receiver; the receiver being placed on the opposite side of the component and facing the transmitting probe. Many commercial pieces of software (e.g. Delcam and Mastercam) do not offer any support for co-operating robots. FastSurf, an add-on from CENIT for Delmia (Dassault Systems), allows partial synchronization of robotic movements (e.g. at start or end points of complex paths) but not full synchronisation over the complete path, required for the UTT technique. Our software provides the capability for full point to point synchronisation between robots for situations where a change in material section is encountered – this is far more sophisticated than simple master-slave synchronisation implemented by typical robot equipment suppliers.
- Path-planning for automated NDT inspections is a very specific task. As previously mentioned, much commercial software for off-line robot programming draws its origin from the need to use the advantageous flexibility of general robotic manipulators to replace the more traditional and usual machining tools (milling machines, lathes, etc.). As a result, many commercial software applications for off-line robot programming are expensive tools, incorporating a number of functionalities specific for CAD/CAM purposes and machining features. Conventional OLP software does not have easy provision for the modification of tool-paths based on specific NDT inspection characteristics.
Contrary to currently available OLP software, RoboNDT provides a capability for full synchronisation (at all points on the path) with external instrumentation systems (in our case an ultrasonic NDT inspection system). Such synchronisation is fundamental (4) to building an accurate map of NDT results on an inspected part with the accuracy required (typically sub-millimetric).
3. RoboNDT software
Originally a MATLAB toolbox for robotic path planning targeted at ultrasonic NDT inspection was developed. A modular approach to the toolbox development was adopted throughout to allow for growth and progressive validation of a large-scale project. The toolbox was based on 4 main modules: start-up, path-planning, evaluation and outputs. The latest developments have led to a full software application, RoboNDT, equipped with a Graphical User Interface (GUI) to enhance ease-of-use. The latest executable version of the application can be downloaded from http://www.strath.ac.uk/eee/research/cue/downloads/. Figure 1 shows the schematic architecture of RoboNDT.
Figure 1 - Schematic architecture of RoboNDT.
Traditional commercial path-planning software generates specific robot language programmes that need to be transferred to the robot controller to be executed. RoboNDT generates output files suitable to be used through a C++ server application that has been developed to allow for external control of KUKA robots. This is a novel approach not seen in traditional OLP packages. The command packets of coordinates are sent from an external computer in real-time to the robot controller via Ethernet. Working with KUKA Robots, the external control is enabled by the KUKA Robot Sensor Interface (RSI) software add-on installed into the robot controllers (14). The C++ application manages the reception of robot feedback positions and NDT data, whilst transmitting the tool-path to the robot manipulating the NDT probe. This approach allows sending of command coordinates to multiple robots from the same external server computer and enables the path synchronization mismatch to be maintained within the distance covered by the robots in a single interpolation cycle. For example for robots controlled in a 12 milliseconds interpolation cycle running at 100 mm/s, the maximum path mismatch would be equal to 1.2 mm. This worst case scenario is much improved over commercial solutions that use digital I/O signals for synchronization purposes. These features make this approach more sophisticated than the simple master-slave synchronisation approaches developed by robot suppliers. Our solution allows for a constant change in the relative path to be encoded with ease. This covers the situation which arises in the ultrasonic through-transmission of materials with constantly changing thickness.
It was deemed that, in order to develop a flexible platform, the easy and appropriate definition of all elements involved in the path-planning operations had to be guaranteed. Five libraries have been implemented to allow the user to easily reproduce the real working environment for the robotic NDT inspection and use virtual models of the real equipment. The Libraries menu, accessible from the menu bar (Figure 2a), provides access to these important modules of the GUI. They are the robot library, the tool library (probes and sensors) the environment library and the appropriate contexts to manage the robot cells and the samples of interest.
Figure 2 – Libraries menu (a), Robot library (b), Environment library (c), Cell management (d), Sample management (e) and Tool library (f).
The list of available items is placed on the left hand side of each library. The user can select any of the items to display the corresponding Computer Aided Design (CAD) model. Graphical User Interface (GUI) buttons in the bottom left corner can be used to remove, edit or duplicate the selected robot model or to create a new item. All libraries, except the cell management context, allow loading of STL (Standard Tessellation Language) CAD files and the specification of key properties for robots, environments, samples and tools (kinematic features, coordinate reference systems, etc.). The cell management context allows the user to create a new robotic working environment through assembling one or more robots into one selected environment.
3.2 Start-up module
The triangular mesh of the sample, imported from the STL file, needs to be placed in the correct position within the virtual robotic cell. Existing software usually considers the CAD models strictly correspondent to the real parts; whilst this can be tolerated for well machined metallic samples, it is sometimes the source of unacceptable errors for large composite components. Therefore the sample’s position calibration mode implemented in RoboNDT uses a positioning algorithm originally proposed for 3D point cloud data registration (15). It calculates the optimum position of the STL mesh within the virtual robot cell, in order to minimize the square errors of the distances between at least four points selected on the real sample and the relative points in the CAD model. Figure 3a shows locating four reference points of a complex curved aerofoil sample in the virtual robot environment presented in RoboNDT at the end of the calibration.
Figure 3b shows a picture of the real setup; it shows the robotic hardware of an automatic inspection prototype system developed within the TWI led project IntACom (16). The system utilises two KUKA KR16 L6-2 robotic arms, with controllers that run KUKA System Software (KSS) 8.2. Mounted on the robot end-effectors are 3D-printed water jet nozzles, which encapsulate phased array ultrasonic probes. The water jets are used to transmit the ultrasonic waves to the specimen under inspection. The specimen under test in this work is a curved composite sample with 1.6m2 surface inspection area. The sample’s wide surfaces, spanning across most of the available robot working envelope and curving in different directions, were chosen for testing and validating the software.
Figure 3 – Real setup (a) and sample model in the virtual robot environment (b).
3.3 Path planning module
The easiest way to generate a tool-path following the contour of a meshed CAD surface would consist of approximating the mesh with a polynomial analytical surface. However, during initial development, this approach revealed its limitations. The approximation introduces an error by definition. The error can obviously be decreased by increasing the order of the polynomial fitting function. However, this would be at the expense of increased computation time.
More importantly, the approximation of a meshed surface with a polynomial surface is only possible when the surface can be mathematically described by a surjective function. A surjective function, z = f(x,y) with X-Y domain and codomain in Z, is surjective (or a surjection) if every element z in Z has a corresponding x-y couple such that z = f(x,y). The function f may map more than one X-Y couple to the same element of Z, but not vice-versa. The inverse of a surjective function is not surjective. As a result, the approximation of a meshed surface fails if the surface is not surjective and it is influenced by the orientation of the surface in the 3D Cartesian space.
Therefore a new approach was developed. Since the CAD files are imported as meshed objects, the new path-planning algorithms compute the tool-paths directly on the triangular mesh without need for an approximating analytical surface.
Figure 4 – Creation of a new task. First phase with the selection of the tool-path type and the definition of the tool footprints.
It is important to optimize the NDT coverage around surface voids and obstacles, and rule out any risk of collisions. The software is able to recognize holes and obstacles in the surface of interest. The footprint of the ultrasonic probe’s active area and its casing is definable in the GUI. The user can specify the footprints (active area if probe and housing) during the creation of a new inspection tool-path (Figure 4).
Particular attention has been paid to the development of the software functions responsible for applying kinematic features to the generated tool-paths. Acceleration and deceleration ramps characterize the robot speed pattern at the start and at the end point of each continuous portion of the tool-path.
3.4 Evaluation and output
Figure 5 shows the inspection tool-paths generated through RoboNDT for the experimental tests described in section 4. The tool-paths and the approaching and retracting trajectories are displayed relative to the virtual model of the robot arm. For the sake of testing the software with surfaces curving in different directions, the main skin of the winglet and the top surface of one of its back wall beams (Figure 3b) were considered for path-planning. The main skin surface has an area of 1.6 m2; the beam surface has an area of 0.5 m2. The generated tool-paths are raster scans with a 29.4 mm raster step. This step is suitable for phased-array ultrasonic inspection (PAUT) when a 64 element, 0.6 mm pitch phased-array probe is employed and its elements are fired with a focal law that uses a sub-aperture of 14 elements.
Figure 5 - Evaluation of generated tool-paths.
The output function of the software translates the generated tool-path into a set of command coordinates packets that can be interpreted by the robot controllers. Each robot pose is represented by a vector, p=[x, y, z, A, B, C]T , containing the three Cartesian coordinates of a given position and the roll (A), pitch (B) and yaw (C) angles of the end-effector orientation for that position.
The software generates two output text files: the first contains all command coordinates the robot needs to receive to inspect the target surface, and a second short log file containing the points to set the initial and final motion to approach the starting point of the inspection and to retract from the endpoint. These two files have very simple syntax; each line merely contains 6 coordinates (x, y, z, A, B, C) to drive the robotic arm to a specific pose. The two text files can be used by the aforementioned C++ server application and the packets of coordinates are sent one by one to the robot controller via Ethernet communication.
4. Validation experiments – Path accuracy and NDT results
Tests were carried out to validate the accuracy of the tool-paths generated through RoboNDT and prove the reliability of the new approach, based on external control of the robot motion and simultaneous collection of feedback coordinates and NDT data through a C++ server application. Since a single application manages the command and feedback coordinate packets, the new approach allows dynamic accuracy monitoring of the tool-paths. This type of investigation is not possible with traditional approaches.
Two sets of tool-paths were created to execute the NDT inspection at 100 mm/s and 300 mm/s, maintaining the same robot acceleration of 500mm/s2. The positional error is calculated as the distance between the commanded tool centre points (TCPs) and the reached points as measured by the robot encoders. The orientation error is calculated as the mismatch angle between the commanded rotation matrix and the rotation matrix computed from the feedback roll, pitch and yaw angles.
Table 1 reports the maximum and Root Mean Square (RMQ) errors. The maximum position error is equal to 2.70 mm and the maximum orientation error is equal to 0.29 degrees. As it was expected, faster speeds produce bigger errors because of the inertial effects affecting the robotic motion.
Table 1 - Maximum and Root Mean Square (RMQ) errors.
|Orientation error (degrees)
The variability of the standoff between the probe and the surfaces is shown in Table 2 with Time-Of-Flight (TOF) maps of the ultrasonic wave reflected from the scanned surface to the probe. The TOF values have been divided by the speed of sound in the sample to quantify the standoff variability in millimetres. The standoff relative to the RoboNDT tool-path is compared to that relative to the tool-path generated through leading aerospace commercial path-planning software based on the Dassault Delmia V5 platform. The comparison is made for the same travelling speed of 300 mm/s and acceleration of 500 mm/s2. The phased array probe focal law and the setting of the ultrasonic receiver (a Micropulse 5PA from PeakNDT) were set to acquire C-scans with resolution of 1.2 mm in all directions. The accuracy of the tool-paths is evaluated by comparing the feedback coordinates received from the robot encoders and the command coordinates as described earlier.
Table 2 – Maps of standoff between probe and scanned surface.
The variability of the standoff is within 10 mm for the tool-paths created with the commercial software and within 4.5 mm for the RoboNDT tool-paths.
The experimental data demonstrates that the path errors, achievable through externally controlled, robots are lower than the errors given by the traditional OLP approach. Further investigations demonstrated that the positional accuracy of a robot reaching a single point is good (less than 1mm for most places within the working envelope of the robot). However, if a robot program is created through the OLP approach to command the robot to follow a specific curve, the deviations from the ideal path are greater. The point accuracy is better than the curve-following accuracy. RoboNDT divides the path up into a long list of points, which are spaced 1 interpolation cycle apart, and removes the inherent inaccuracy of the robot. Therefore, as long as the kinematic model of the robot is good (calibrated), the external control leads to the best accuracy achievable with a robot without any external metrology systems.
The development of RoboNDT has enabled a new viable and innovative approach for robotic NDT inspections. However, the software is not yet optimized in terms of computation speed. For computing a path planning scan using the above parameters, the calculation time using RoboNDT took 6 minutes. The path-planning tasks executed by RoboNDT take around 5 times longer than the time taken by the commercial software. An Intel® Xeon® CPU computer with 24Gb of RAM, running a 64-bit Windows 7 operating system was used to test both approaches.
4.1 NDT results
Using the RoboNDT tool-paths, the raster scan of the main skin and of the beam surface took 205s and 38s respectively. For the commercial software generated tool-paths, the raster scans took 200s and 35s respectively. Previous manual scans of the same surfaces were completed in 2 and 0.4 hours respectively; this results in robotic inspections being around 40 times faster than manual inspection (in addition to being much more reliable, repeatable and accurate).
It is clear from Table 3 that the path accuracy of RoboNDT derived tool-paths exceeds those obtained from the commercial software. For the current application, the level of accuracy for both approaches is sufficient as the intended NDT delivery is accomplished using a water jet coupling approach (5, 16). The water path from water nozzle to sample surface can easily accommodate such tool-path inaccuracies. The bottom row of Table 3 shows the close-up of an array of artificial squared delaminations embedded within the thickness of the winglet main skin. The smallest delaminations have a size of 3 mm and are visible in both cases. For other NDT inspection applications the improvements in path accuracy are more significant, for example if implementing eddy current inspections where a tight control of standoff distances is required throughout the path to avoid false defect indications.
Traditional manually delivered NDT is time consuming and manufacturers are increasingly demanding smaller cycle times for the inspections undertaken. Although the geometries of some parts lend themselves to bespoke Cartesian (or Cartesian plus rotation) stage mechanical scanners, there are many instances of complex geometry that make the use of 6 axis robot positioners highly attractive. Most existing commercial off-line programming approaches are geared towards manufacturing processes, and lack the required flexibility for the delivery of NDT measurements. In particular the lack of full point by point synchronisation, between multiple robots and the external measurement system, is one of the key shortcomings of existing software. Future flexibility to accommodate part variability through conditional programming approaches, and ability to build additional path modification due to effects such as water jet orientation are also key attributes of the new software tool developed.
The software, named RoboNDT, has been tailored to the generation of raster scan paths for the inspection of curved surfaces by 6-axis industrial robots, and in its current form represents the first iteration of a system designed to overcome the issues with current OLP packages. RoboNDT is intended to be flexible and extendable to accommodate future system and robot developments. It has been explained how the execution of the calculated path by a robotic arm, externally controlled through a C++ server application, can be beneficial for NDT inspections. Comparative metrology experiments were undertaken to evaluate the real path accuracy of the toolbox when inspecting a curved 0.5 m2 and a 1.6 m2 surface using a KUKA KR16 L6-2 robot. The results have shown that the deviation of the distance between the commanded end effector position and feedback positions is within 2.7 mm. The variance of the standoff between the probe and the scanned surfaces was smaller than the variance obtainable via commercial path-planning software.
In the future, more versatile versions of the software with additional features could be realised. The ultimate goal of the authors remains the simultaneous management of command coordinates, robot positional feedback and NDT data by an integrated server application running on a single dedicated PC. This paves the way to introducing intelligent novelty factors to the robotic NDT inspections; on-line monitoring and data visualization, real-time path correction and versatile path amending approaches are just some of the possible opportunities.
This work was developed in partnership with TWI Technology Centre (Wales), University of Strathclyde (Glasgow), the Prince of Wales Innovation Scholarship Scheme (POWIS) and by IntACom, a project funded by Welsh Government, TWI, Rolls-Royce, Bombardier Aerospace and GKN Aerospace.
- I. Cooper, P. I. Nicholson, D. Yan, B. Wright, and C. Mineo, "Development of a Fast Inspection System for Aerospace Composite Materials - The IntACom Project," presented at the 9th International Conference on Composite Science and Technology (ICCST-9), Sorrento (Italy), 2013.
- T. Sattar, 'Robotic non-destructive testing', Industrial Robot: An International Journal, vol. 37, 2010.
- M. Schwabe, A. Maurer, and R. Koch, "Ultrasonic Testing Machines with Robot Mechanics - A New Approach to CFRP Component Testing," presented at the 2nd International Symposium on NDT in Aerospace, Germany, 2010.
- P. Louviot, A. Tachattahte, and D. Garnier, "Robotised UT Transmission NDT of Composite Complex Shaped Parts," presented at the 4th International Symposium on NDT in Aerospace, Berlin (Germany), 2012.
- E. Cuevas, M. López, and M. García, "Ultrasonic Techniques and Industrial Robots: Natural Evolution of Inspection Systems," presented at the 4th International Symposium on NDT in Aerospace, Berlin (Germany), 2012.
- F. Bentouhami, B. Campagne, E. Cuevas, T. Drake, M. Dubois, T. Fraslin, P. Piñeiro, J. Serrano, and H. Voillaume, "LUCIE - A flexible and powerful Laser Ultrasonic system for inspection of large CFRP components.," presented at the 2nd International Symposium on Laser Ultrasonics, Talence (France), 2010.
- A. Maurer, W. D. Odorico, R. Huber, and T. Laffont, "Aerospace composite testing solutions using industrial robots," presented at the 18th World Conference on Nondestructive Testing, Durban, South Africa, 2012.
- J. T. Stetson and W. D. Odorico, "Robotic inspection of fiber reinforced aerospace composites using phased array UT," presented at the 40th Annual Review of Progress in Quantitative NDE, Baltimore, Maryland, 2013.
- S. G. Pierce, G. Dobie, R. Summan, L. Mackenzie, J. Hensman, K. Worden, and G. Hayward, "Positioning challenges in reconfigurable semi-autonomous robotic NDE inspection," in SPIE 7650, Health Monitoring of structural and Biological Systems 2010, San Diego, California, USA, 2010, p. 76501C.
- C. Mineo, D. Herbert, M. Morozov, S. G. Pierce, P. I. Nicholson, and I. Cooper, "Robotic Non-Destructive Inspection," presented at the 51st Annual Conference of The British Institute of Non-Destructive Testing, Daventry (UK), 2012.
- R. Bogue, 'Finishing robots: a review of technologies and applications', Industrial Robot: An International Journal, vol. 36, pp. 6-12, 2009.
- Z. Pan, J. Polden, N. Larkin, S. Van Duin, and J. Norrish, 'Recent progress on programming methods for industrial robots', Robotics and Computer-Integrated Manufacturing, vol. 28, pp. 87-94, 2012.
- W. Haase, "Automated non-destructive examination of complex shapes," presented at the 14th Asia-Pacific Conference on NDT (APCNDT), Mumbai, India, 2013.
- KUKA, KUKA.RobotSensorInterface 3.1 Documentation vol.Version: KST_RSI_3.1_V1_en, 2010.
- P. J. Besl and N. D. McKay, "A method for registration of 3-D shapes," in Robotics-DL tentative, 1992, pp. 586-606.
- C. Mineo, S. Pierce, B. Wright, I. Cooper, and P. Nicholson, 'PAUT inspection of complex-shaped composite materials through six DOFs robotic manipulators', Insight-Non-Destructive Testing and Condition Monitoring, vol. 57, pp. 161-166, 2015.