In-Situ Support for Automotive Manufacturing Using Spatial Augmented Reality

In automotive manufacturing, quality inspection of spot welding demands excessive manual operations. Operators refer to a printed drawing of the testing body, with the inspection points marked on this drawing. Operators have to locate the matching spot on the drawing and the body manually to perform the inspection. Further more, different subsets of spots are inspected on different car bodies with a pre-determined sequence. This paper describes a system that projects visual data onto arbitrary surfaces for providing just-in-time information to a user in-situ within a physical work-cell. This system aims to reduce the inefficiencies and potential mistakes in manual inspection process. This paper discusses how spatial augmented reality and head-mount displays may be combined to display global information visible by all operators as well as personalized information to individuals. Further investigations on applying spatial augmented reality for spot welding inspections are explored, including four types of digital information projected onto the surfaces of car body parts under structured work environments: 1) Location of spot welds; 2) Inspection methods; 3) Operation Description Sheet (ODS) information; 4) Visualization of weld locating methods. Three visualization methods are used to attract operators’ attention to locate the position of spot welds efficiently. This paper also proposes a method to project augmentations on objects moving along an assembly line. The proposed system allows operators becoming more effective and efficient in performing proper inspections, by providing them the required information at the required time without the need to refer to paper-based manuals or computer terminals.

Abstract-In automotive manufacturing, quality inspection of spot welding demands excessive manual operations.Operators refer to a printed drawing of the testing body, with the inspection points marked on this drawing.Operators have to locate the matching spot on the drawing and the body manually to perform the inspection.Further more, different subsets of spots are inspected on different car bodies with a pre-determined sequence.This paper describes a system that projects visual data onto arbitrary surfaces for providing just-in-time information to a user in-situ within a physical work-cell.This system aims to reduce the inefficiencies and potential mistakes in manual inspection process.This paper discusses how spatial augmented reality and head-mount displays may be combined to display global information visible by all operators as well as personalized information to individuals.Further investigations on applying spatial augmented reality for spot welding inspections are explored, including four types of digital information projected onto the surfaces of car body parts under structured work environments: 1) Location of spot welds; 2) Inspection methods; 3) Operation Description Sheet (ODS) information; 4) Visualization of weld locating methods.Three visualization methods are used to attract operators' attention to locate the position of spot welds efficiently.This paper also proposes a method to project augmentations on objects moving along an assembly line.The proposed system allows operators becoming more effective and efficient in performing proper inspections, by providing them the required information at the required time without the need to refer to paper-based manuals or computer terminals.

Index Terms-Spatial Augmented Reality, Spot Welding
Inspection, Visualization Method.

I. INTRODUCTION
Augmented Reality (AR) is a technology which renders virtual objects into the real world [1], by registering computer-generated images over a user's view of the physical world at realtime.With this extra information presented to the user, the physical world can be enhanced or augmented beyond the user's normal experience.Additional information that is spatially located relative to the user can help improving their understanding of the world in situ [23].AR interfaces enable people to interact with the real world in ways that are easily acceptable and understandable by users.For example, doctors may use the AR system to allow an intuitive real-time intraoperative orientation in image-guided interstitial brachytherapy [8], and to guide the liver thermal ablation in interventional radiology [11].Doctors can also use projector based AR for the intraoperative visualization of preoperatively defined surgical planning data.The potential of AR for industrial processes is also increasingly being investigated [17].
Instead of body-attached displays such as a Head Mounted Display (HMD), Spatial Augmented Reality (SAR) detaches the technology from users and integrate it into the environment.SAR employs data projectors to superimpose computer generated virtual objects directly onto physical objects' surfaces.The user then views and interacts with the digital information directly projected onto the surfaces within a workspace in a natural manner [3].SAR has been employed in a number of application domains, such as design [14], entertainment [7], and museums [2].Due to the decrease in cost and improvements in availability of projection technology, personal computers, and graphics hardware, SAR is now a viable option for use in an industrial setting.This paper focuses on the projector-based SAR display and its applications in an industrial environment.
In automobile manufacturing, the quality of spot welding on car bodies needs to be inspected frequently.A typical car has thousands of individual spot welds.The spot welds are required to be checked in a predefined order, one individual spot weld at a time.For improving the productivity, operators often only check different subset of spots on different car bodies with a pre-determined sequence, instead of checking all spots on each body.When all spot welds are checked in a sequence, operators start a new spot sequence for checking.Currently, spot welding inspections rely on a printed drawing of the testing body, and the inspection points are marked on this drawing.The operator has to check the printed drawing and find the matching spot on the car body to perform the actual inspection.This manual inspection process suffers from potential problems: it is more likely for operators to make a mistake by checking wrong locations or wrong numbers of spot welding; it is also difficult for operators to remember the starting and finishing points on the checked body.This is especially true when the process is performed by an operator who does not routinely perform this task.
Furthermore, one significant problem that influences the effectiveness of SAR is the projector's viewport limitation.Multiple projectors are often used to extend the limited viewport of a single projector for augmenting large-scale environments.In practice, users may need to project digital information on objects moving along a linear track, for example, to project product information onto objects moving along an assembly line.In the case of an automotive assembly line, part locations are known relative to their position on the conveyor system, via the central automation system.Workers perform tasks while parts are moving in a linear fashion, and the part's position can be modeled as a projected ray from a fixed point that rotates about a single axis within the volume of the individual workspace.Because the linear track that the object moves along is often wider than the viewport of a projector, the projector cannot project digital information onto the object moved out of the viewport of the projector.
This paper describes a system that projects visual data onto arbitrary surfaces for providing just-in-time information to a user in-situ within a physical work-cell.The proposed SAR system provides a number of benefits to the spot welding inspection task in the automobile industry.SAR facilitates the presentation of projected digital AR information onto surfaces of car body parts.The projected AR provides where-to-act and what-to-do information, to assist operators performing spot welding inspections.Four types of digital AR information are projected on surfaces of car bodies in structured work environments: 1) Location of spot welds; 2) Inspection methods using various geometrical shapes, such as ultrasonic and destruction tests; 3) Operation description sheet (ODS) information; and 4) Visualization of weld find methods.
Various geometrical shapes are used to allow operators to identify the inspection method used for a specific spot weld.The animated visualizations guide operators to find the location of spot welds quickly and easily.Our system allows operators becoming more effective and efficient in performing the desired inspection task, by providing them the required information at the required time without the need to refer to paper manuals, terminals, or a laptop computer.The proposed system does not require additional special models in finding spot welds, but only needs knowledge of the location of spot welds on the part.
This paper also gets a projector to continuously track a moving object automatically when the object moves along a linear track.It utilizes a rotatable projector to project digital information on an object moving along a linear track.It analyzes the geometric relationship between the projector and the object, and presents a new projection matrix computation for the augmentation of the moving object.The proposed approach does not require any additional tracking devices such as cameras/light sensors, but only requires knowledge of the artifact's position relative to a linear track and a simple instrumented rotation platform.This setup effectively broadens the viewport of projectors, and has wide applications such as in automotive manufacturing.

II. SAR AND ITS INDUSTRIAL APPLICATIONS
SAR allows digital objects, images, and information to be added as real world artifacts by projecting them onto surfaces in the environment with digital projectors.This section outlines advantages of SAR as well as its employment in industrial applications.

Advantages of SAR
A key benefit of SAR is that users are not required to wear HMDs; therefore, they are unencumbered by the technology.The user can physically touch the objects at the same position as the virtual images are projected onto.In SAR, the field of view (FOV) of the overall system is the natural FOV of the user, allowing them to use their peripheral vision.The range of the SAR system's FOV can easily be extended by adding more projectors or by using rotatable projectors.Ultimately the FOV can emulate the full physical environment with a higher level of resolution with what is determined to be the correct number and position of projectors.Projector-based SAR allows possibly higher scalable resolution and bright images of virtual objects, text or fine details, than traditional HMD or handheld display solutions.Since virtual objects are typically rendered near their real-world locations, eye accommodation is easier to achieve [3,15].

SAR in Industrial Applications
AR technology was applied successfully in certain use cases in industries [16], and its major application areas include: servicing and maintenance, design and development, production support, and training.Similarly, SAR systems have the potential to improve processes in a variety of application domains [4,15].Bimber et al. [4] use conventional projectors that are calibrated in suitable locations to generate SAR scenes.They are able to show seemingly undistorted video and graphics on arbitrary surfaces in the environment, by means of pre-warping and color-adjusting the virtual data to counteract the reflection and perspective projection effects of the physical objects' surfaces.
In industries such as manufacturing, SAR may benefit a designer from the perceived ability to visually modify portions of a physically machined table-top model.The approach could also be used for product training or repair: one could set the product in the SAR environment and have the sys-tem render instructions directly on the product.Marner and Thomas [10] developed a SAR-based physical-virtual tool for industrial designers.Their system simultaneously models both the physical and virtual worlds.SAR is then used to project visualizations onto the physical object, allowing the system to digitally replicate the design process to produce a matching 3D virtual model.Olwal et al. [12] use SAR on industrial CNC-machines to provide operators with bright imagery and clear visibility of the tool and workpiece simultaneously.This helps to amplify the operator's understanding and simplify the machine's operation.Schwerdtfeger [19] uses HMD-based augmented reality to guide workers in a warehouse with pick information, which is named as pick-by-vision.
In the industry of automobiles, SAR can be used in quality assurance, material handling (e.g.bin picking and kitting of parts) and maintenance as well as other applications.The quality assurance of spot welding is one of typical applications in the automobile industry.Schwerdtfeger et al. [20], [21], [22] set up an AR system that uses laser projectors in the quality assurance of welding points.In this system, a hybrid information presentation approach is used: a laser projector is used to locate and display the position of welding points to be checked; an additional computer display is used to show complex what-to-do information to users.The system still required users to read the computer display while focusing on the welding points, thus affects the work efficiency.In addition, SAR can also be used for training in automotive manufacturing.HMD-based AR facilitates computer enhanced interaction and communication in the real world accessible anytime and everywhere [18].It offers the flexibility for information personalization, thus providing a customized AR visualization for each individual user.This section describes the use of SAR and HMD for portable collaboration in industrial applications.In this concept, a portable projector is used to project digital information onto 3D physical world.At the same time, each individual user uses HMD to view customized information rendered on top of projector-based SAR.Fig. 1 shows an illustration of this concept: SAR projects digital marks onto the physical object's surface.These marks not only provide a global physical world context for users to understand the problem space, but also serve as a fiducial marker for the HMD-based AR visualizations.Users wearing HMD can receive personalized information on the marked points.For example, in automotive welding applications, one user can review welding schedule parameters and another user can review weld specification and in different languages.

Combined SAR and HMD
Fig. 2 shows an example where two operators view the same SAR projected digital mark (light blue disk), with personalised welding and inspection information displayed by the HMD screen for each individual operator.The cyan marker indicates the physical position and type of weld, which is the common information for both users.Critical positions and information are physically common between the users, and therefore this removes any uncertainty due to tracking errors.The HMD AR displays are created using Tinmith Wearable Computer [13].The operator who is an English speaker receives operation information in English, while the operator who is a Chinese speaker receives operation information in Chinese.In this example, spot welding on a mechanical part are inspected.Each operator receives information on different aspects of the welding spot.The cyan marker is treated as a reference point for tracking.The marker-based AR is used to identify the marker as the reference point, and place the personalised instructions through the user's HMD according the reference point.

III. SAR IN SPOT WELDING INSPECTION
As mentioned above, SAR may be applied to improve industrial quality assurance.This section describes the pro-posed system which facilitates SAR for in-situ support of spot welding inspection in automobile manufacturing, where SAR is used to highlight the location of the spot welding and type of inspection on an unpainted metal car part.The use of SAR can help operators to improve the efficiency of spot welding inspection in an automobile industry.The approach aims to remove the need of holding a paper-based operation description sheet and relieve some of the user's cogitative load by displaying cross-referenced information readily, in a suitable location and at the required time, in order to improve the accurateness and efficiency of the inspection of spot welding.

Conventional Approach of Spot Welding Inspection
In a typical automotive manufacturing process, the quality of spot welding on car bodies needs to be inspected at regular intervals.For example, a typical car has thousands of individual spot welds.In the process of making a vehicle, sub-assemblies are made and these assemblies can have between 30-200 spot welds.The spots have to be checked with a randomised sequence, even if the same type of part is checkedthis has statistical reasons for dealing with the occurrence of false negatives.Instead of checking all spots on each body, operators only check different subsets of spots on different bodies.When all required spot welds are checked in a sequence, operators start a new spot sequence and repeat the inspection process.A variety of different methods are used to check spot welding: visual inspection, ultrasonic test, chisel test, and destruction test.
The current procedure that operators use to check spot welding is as follows (see Fig. 3): the operator has a drawing, which shows the welds to be checked on a sample body.First, the operator has to find the spot in the drawing; subsequently, the operator has to locate the matching point on the body; after this, he has to choose the corresponding control method to perform the inspection.
This manual inspection process suffers from potential problems: it is possible that the operator identifies wrong locations and wrong numbers of welding spots.The operator is required to take time to interpret the drawing and find the corresponding spot on the raw body frequently.This process is also difficult for the operator to remember where to start and where to finish the checking on the checked body.Skilled operators are therefore required for performing manual inspections.

Projector-Based SAR in Spot Welding Inspection
SAR can be applied to lower the skill-level requirement for spot welding inspection by projecting digital AR information onto surfaces in structured work environments.Specifically, SAR projects visual data onto arbitrary surfaces for the express purpose and providing just-in-time information to users in-situ within a physical work cell.In this paper, a projector mounted on a movable stand is employed to view and interact with digital information projected directly onto surfaces within a workspace.Fig. 4 shows the procedure of spot welding inspection using the SAR system described in this paper.In this system, the 3D CAD drawing of the test body part is used as an input of the SAR system, and it is stored in the SAR system as a geometrical model.The location of spot welds can be obtained from the geometrical model.The SAR system projects digital markers on top of spot welds with various geometrical shapes.The various geometrical shapes are used to inform operators what inspection method is used for a specific spot weld.As a result, operators are only required to perform the actual inspection in an inspection session, but not required to read the drawing of the test body from time to time to find spot welds on both the drawing and the test body.Fig. 3 and Fig. 4 illustrate how the SAR system effectively removes conventional operating steps before actual inspection.Fig. 5 shows the main components of the system in our configuration, projector, control pad and computer.In order to simplify user interactions with the system, a wireless control pad as shown in Fig. 6 was developed.The wireless control pad provides the operator to access the next/previous spot weld, find the current spot weld, go to the next page of spot welds, or go to the beginning of the spot welds list only by pushing one button on the pad.

IV. VISUALIZATION IN SPOT WELDING INSPECTION
SAR has the flexibility to provide various visualizations in the scene to augment information displayed to operators.This section presents different visualization methods to show the position of spot welds and assist operators locating spot welds easily.In our approach, the location of the spot weld is visualized directly on the physical surface of the test body part with a colorful geometrical shape.Meanwhile, the inspection method of the spot weld is encoded into the geometrical shape.Various geometrical shapes are used to indicate that a spot weld is to be tested using a specific inspection method as shown in Fig. 7.In addition, ODS information is displayed along with the spot weld marker together to assist operators performing the actual inspection (see Fig. 7(c)).

Visualization of Location and Inspection Method of Spot Weld
Fig. 8(a) shows an example of projecting AR information on top of a spot weld on the surface of a car part.The ODS information is also displayed with the AR information.In our system, various formats of digital information (e.g.text, image, video, geometrical shapes) can be projected on the surface of a car part.
In Fig. 8(b), multiple spot welds are identified with AR information at the same time.This allows operators to access and inspect multiple spot welds concurrently.It also shows the distribution of spot welds on the surface of the test body part.

Visualization of Weld-Find Method
If the test body part to be inspected is large or complex, operators may need to spend significant time to locate the spot weld during the inspection process.In order to assist operators to easily locate a spot weld to be inspected, this section describes a set of visualization approaches that project various dynamically changing geometrical shapes on the physical surface of the test body part.Three types of visualization approaches to assist in the task of weld-find are used in this paper (see Fig. 9):  Shrinking concentric circles: A size-changing circle centered on spot weld location is dynamically reduced from a radius about the size of the inspected part to the radius of the spot weld marker to guide operators locate the spot weld to be inspected. Moving arrow lines: Multiple arrow lines pointing to the spot weld are displayed.These arrow lines are initiated from approximately the edge of the part to the location of the spot weld dynamically to guide operators locate the position of the spot weld to be inspected. Arrow line between two spot welds: An arrow line pointing from the previous spot weld to the current spot weld is displayed.This arrow line directly guides operators to locate the next spot weld to be inspected along the arrow line.The process of rendering the SAR data requires 3D graphical objects to be created.In the case of spot weld identification marks, individual graphics files (in .objformat) were developed.This is required to obtain perspectively correct rendering which is very important for the readability of text.The weld-find animations were developed that did not require as accurate positioning and could be generated on the fly in real time.Each of the three weld-find visualizations only requires at most two 3D positions in the part's coordinate system.As illustrated in the examples above, SAR can provide effective guidance to attract the operators' attention to the next set of spot welds for inspection.The data items are projected onto the car body, providing instructions to operators.This removes the need to constantly refer to the instruction manual such as the ODS, thus speeding up the operation and reducing errors.There are benefits for providing in-situ data presentation for spot welding inspection.First is the reduction in cognitive load of forcing people to remember specific tasks and the order they are required in.Second, the vehicles coming down the line are individually built (each car is different as they come down the line), and this requires unique information for each vehicle.Third, changes to the production information can be directly sent to the production line and displayed to the user.Last but not the least, the use of SAR can improve the inspection accuracy and efficiency greatly.
A major advantage of our approach is that it can be used to assist operators in manual welding stations and to help the weld inspectors evaluate discrepant welds in automated welding stations.In the case of manual welding, a SAR based system can help the operator to ensure that all welds applied to the part are in the right sequence, in the right pattern, and at the right location.In cases of automated welding stations often time recovery from a welding related downtime is lengthened due to the lack of knowledge of what welds belong to what groups of welds to determine the proper corrective action.The SAR based system can help to quickly assess this condition.Another major advantage to a SAR based system is to help maintaining the fidelity between the welding databases and the actual welds that are being performed by the robots on the factory floor.Cycle time optimization process is usually continued for a long time after a production is launched.In many cases robot programmers move welds from robot to robot and from station to station in order to optimize cycle times.In many cases these welds are not reported properly.A periodic SAR based auditing system can help to facilitate this important check in order to assure the product design intent.
A major constraint of SAR based spot welding inspection system is the reliance of suitable projection areas on the object's surface.Limited amount and complexity of information that can be presented depend on the shape and size of the object [22].Meanwhile, since the diffuse reflection is very small from surfaces of car bodies, only a limited amount of light is reflected omni-directionally towards arbitrary viewer positions.Therefore, the projected and 3D aligned augmentations on the surface are sometimes not clear to viewers at specific angles; although, when this issue occurs, the viewer can usually adjust his/her positions slightly to find an improved viewing angle as a workaround.
When the system is used in a real-world environment, additional design factors need to be considered: 1) lighting condition of the work-cell; 2) tools for simplifying and automating the system operation (e.g.automatic projector calibration); 3) a control system which is able to identify and track moving objects and facilitate SAR projection on an assembly line.For the first issue, the allowed work-cell brightness is mainly determined by the physical properties of surfaces and the projector used in the system.The high luminous projectors could be used to project high contrast augmentation in a bright work-cell.Regarding the second issue, the project in our system is manually calibrated, and an automatic projector calibration approach will be a desirable feature for adopting our system in a real-world environment.Since this paper is mainly focused on the employment of SAR in the spot welding inspection, automated projector calibration will be considered as a future work.For the third issue, we developed an approach to project augmentations on objects moving on the assembly line as presented in Section V. Synchronization between the SAR system and the control of the assembly line could be used to recognize various parts on the assembly line as well as to manage the display information.Recent SAR researches focus more on moveable interactive augmentations.Dynamic content projection onto moveable objects has been explored by a number of researchers [15].Kimura et al. [6] employed a combination of projectors and cameras to augment objects, where the projectors and the cameras are fixed in the setup.Ehnes et al. [5] developed a system that keeps the augmentation displayed at the correct location while the object or the projector moves.
In order to augment moveable objects, the system has to -know‖ where these objects are.Positions of moveable objects have to be obtained in real-time [5,9].As mentioned previously, most prior research apply a combination of projector and camera-based tracking system to augment moveable objects.This section deals with a specific setup where objects moving along a linear track at a constant speed and are augmented in real-time.We aim to augment these kinds of objects by employing a projector without additional devices such as cameras or light sensors.

System Overview
A rotatable circular plate called LazySusan is developed to carry a projector, and LazySusan is capable of returning its rotation angles in real-time.The optical axis of the projector is placed to fit the axis of the circular plate (a line through the centre of the plate) as closely as possible.Objects are placed in front of the projector and moved linearly along a track.The configuration of the system is shown in Fig. 11.As illustrated in Fig. 11, O is the rotation centre of LazySusan, C is the projection centre of the projector, and D is the projection centre after the LazySusan rotates θ degrees.The distance between the rotation centre of LazySusan and the projection centre is r.This is the eccentricity of the projector.In this paper, we assume that the optical axis is fitted on the axis of LazySusan.Thus, LazySusan's rotation angle is equal to the projector's rotation angle.
In the proposed system, the projector is firstly calibrated manually at a position.Users also need to specify the linear track's direction by utilizing cursors.After these two initial steps, the projector can project digital information onto the moving object in the correct position.The object also automatically gets the right projection when the projector is rotating.In the proposed system, the moving object is augmented by the projector without any additional devices such as cameras/light sensors.This is achieved by analyzing geometrical relationships between the projector and the object.

Projection Matrix Computation
Generally, the 2D-3D projection model of a projector is similar to the concept of capturing photographs using a cam-era, with a difference in their projective directions.Therefore, projection from a projector onto a 3D object can be modeled as an inverse projection of pin-hole camera.Let matrix P denote the product of an intrinsic matrix and an extrinsic matrix (view matrix): In the intrinsic matrix, α and β are focal length with skew parameter, (Cx, Cy) is the optical centre.R and T are the extrinsic parameters denoting the coordinate system transformations from 3D world coordinates to 3D projector coordinates.The intrinsic matrix and extrinsic matrix are usually derived via a projector calibration process.Thus, the relationship between 2D data and 3D scene can be described as Because intrinsic matrix does not change for a specific projector in an application, the extrinsic matrix (view matrix) is manipulated to get the new projection matrix for the linearly moving object in this paper.The view matrix of the projector is represented as M. The projector rotates θ degrees (represented with a rotation matrix Rp) based on the LazySusan rotation.The movement of the object is represented with a translation matrix Tp.The definition of the linear track is a critical step for the right augmentation on the object.The world coordinates of the object are obtained through the projector calibration step by cursors.Furthermore, the object is moved on the experiment table linearly.So the cursor can also be used to define the position and direction of the linear track.These cursor positions are then mapped back and their corresponding world coordinates can be obtained similar to the projector calibration.
The new view matrix ' M for the linearly moving object is:

Implementation
In this paper, the projector is placed on the LazySusan plate.An axis line is drawn on the LazySusan plate so that the optical axis of the projector and the axis of LazySusan can be fitted as closely as possible.When the plate is rotated, LazySusan's control module feeds back the rotation angle to the SAR system.A paper with printed linear grid on it is placed on the experimental table, in order to guide the user to move objects linearly based on the grid.Fig. 12(a) shows the augmentation of an object at location A. When the object is moved linearly from A to B (see Fig. 12(b)), the object is continually augmented.When the object is moved out of the current viewport of the projector, the projector will be rotated according to the object's motion trajectory, such that the augmentation seamlessly follows the object.The user can define different moving tracks online so that the object can be moved along any linear track.
In practice, the optical axis of the projector may shift from the axis of the LazySusan.This misalignment of a camera/projector is related to the following factors [24]: 1) the eccentricity r as shown in Fig. 11; 2) the distance d from the projection centre to the object; 3) the projector rotation angle θ .It is found that the larger the eccentricity, the higher the misalignment; the larger the rotation angle (0 to 180 degrees), the higher the misalignment.As such, the following options can be considered to reduce the misalignment:  Minimize the eccentricity: This is difficult because the projection centre is not accessible by users.Users can reduce this affection by using the centering procedure as used in [24]. Minimize the rotation angle for each rotation operation: This is possible.In our approach, LazySusan can be rotated with a very slow speed to meet this requirement.The working viewport in the proposed approach depends on the distance between the projector and the linear track, as well as the rotation angle.This property allows users to get a wider working area, but not the limited viewport based on the projector.
The proposed approach facilitates a wide-range of practical applications.For example, in an automotive industry, SAR is used to highlight spot welds on an unpainted metal car part in a spot welding inspection process.The metal car part is moved on an assembly line, operators need to check each moving part one to the next.The proposed approach can project welding information on the moving part continuously along the assembly line.It enables operators to perform spot welding inspection efficiently and effectively.

VI. CONCLUSIONS
This paper described a SAR-based system for automotive spot welding inspection.SAR facilitated presentation of projected digital Augmented Reality information on surfaces of car bodies.The digital AR information including ODS information on surfaces of car bodies directly indicates the location of spot welds.SAR avoids the problem of operators taking time to read a drawing and find the corresponding spot on the raw body frequently as usually taken in manual inspection process.Various geometrical shapes are used to allow operators to identify the inspection method used for a specific spot weld.The animated visualizations in the system assisted operators to easily and quickly locate the spot welds.Our approach did not require special models but only required knowledge of location of spot welds on the part in finding spot welds.Besides the problems discussed in the previous section, the future work of this research will focus on the extensive evaluation of our approach.The operation information in other formats (e.g.movies, pictures) could also be projected onto physical surfaces of objects to assist operators.

Fig. 1 .
Fig. 1.Concept illustration of combining projector-based SAR and HMD-based AR for collaborations.

Fig. 2 .
Fig. 2. Concept illustration of combined projector-based SAR and HMD-based AR visualisation Combined SAR and HMD-based AR has various potential applications.Firstly, it can be used in a collaborative training environment which facilitates collaborations between trainees with different language backgrounds.Operators doing the same task in different countries need to be trained.Users training in their own languages are preferred to improve the effectiveness of the training.By using the collaborative training environment, one single training session is only used for all operators from different backgrounds.Secondly, the concept can be used in military, diplomatic and other similar situations where participants have different backgrounds and need to collaborate with each other.Thirdly, the concept benefits various industries in production support, designing and maintenance.Last but not least, the concept could be extended and used in remote collaborative tasks such as remote guiding, training and maintenance.

Fig. 5 .Fig. 6 .
Fig. 5. Projector, Control Pad and Computer are the main components of the system.

Fig. 7 .
Fig. 7. Visualization of location of spot welds: (a) Destruction test; (b) Ultrasonic test; (c) Spot weld marker with the ODS information.

Fig. 9 .
Fig. 9. Visualization of weld-find method: (a) Shrinking con-centric circles: A size-changing circle gathering from far to the centre of the spot; (b) Moving arrow lines: Arrow lines gathering from far to the centre of the spot; (c) An arrow line pointing from the previous spot to the current spot.

Fig. 11 .
Fig. 11.A top view of the rotatable projector and the moving object.
geometry in a captured image or projection pattern.