Design and Development of an Augmented Reality Robotic System for Large Tumor Ablation

An augmented reality based robotic system is proposed for treatment of large liver tumor using multiple overlapping ablations technique. Radiofrequency ablation is a non-resection interventional procedure that minimizes invasiveness and reduces trauma. A single ablation is however limited by the ablation volume coverage. In treatment of tumor with diameter that could be 150mm or more, a single needle insertion will not create sufficient ablation coverage to destroy the entire tumor. As such, multiple insertions are required. The precision and consistency associated with robot assisted surgery coupled with direct augmented reality projection system can better facilitate such treatment. The projection system is equipped with manipulator mechanism and robotic needle insertion arm for multiple insertions and needle deployment. The prototype augmented reality surgical system was evaluated on its abilities in ablating large tumor. Virtual models of the rectangular robot for projector manipulation and robotic needle insertion device were also used to assess the effectiveness of the proposed AR design.

good alternative to liver resection.While resection of tumor surgically is by far the best option [8] for liver tumor, only 20% of liver cancer patients are suitable for open surgery [9].Among the non-resectional procedures [7], RF ablation appears to be the most accepted treatment for liver tumors in terms of safety, ease of procedure and consistency [10,11].
However, complete annihilation of the tumor is dependent on the ablation volume coverage.In treatment of large tumor with diameter that could be as large as 150mm, a single RF application is insufficient to destroy the entire tumor.As such, multiple needle deployments and insertions are required.There have been many research efforts on treatment strategies using multiple overlapping ablations [12][13][14][15].Most of these researches revolve about planning and optimizing the ablation coverage to maximize effectiveness of the treatment.Although development significantly improves the effectiveness of pre-operative plans, achieving consistency and precision in intra-operative execution is still non-trivial.The technique of multiple needle insertions for overlapping ablation is difficult to perform manually.Surgeons are often deprived of visual information or have to rely highly on non-intuitive image guidance in executing the preplanned ablation model.Performing such operation manually will also be subjected to uncertainties and inconsistent outcomes [13].We proposed systematic multiple ablations through robotic procedure guided by an AR image overlay for large tumor treatment using RF ablation.
This paper focused on the design and development of an AR based robotic system for multiple needle insertion in treating large tumor.The projector based AR system also includes a robotic system equipped with needle deployment and insertion manipulator.

II. DESIGN AND DEVELOPMENT
The construction of the AR surgical system involves designing the mechanical architecture of the system and the development of AR features.Apart from design consideration such as safety and sterilizability, there exist other design specifications unique to an AR based robotic surgical system.The design needs to be conceptualized based on the geometric requirements specified by the system kinematic model.During design conceptualization, specific application mechanism is developed concurrently.The virtual models of the generated

Design and Development of an Augmented Reality Robotic System for Large Tumor Ablation
Liangjing Yang 1 , Chee-Kong Chui 1 , Stephen Chang 2 design concepts are constructed using Computer Aided Design (CAD) software for engineering visualization and subsequent development.Actual physical prototype is implemented to test the performance and robustness of the design.

Design considerations
Safety is an important design consideration.However, safety guidelines for industrial robot are not always applicable to surgical robot.It is a requirement for industrial robot to be isolated from human operator when it is powered.Such practice is however difficult to implement in a surgical environment where both human operators and subjects cannot be excluded from the machine's workspace [16].Therefore, there should be a reliable mechanism to cut off the power during an emergency as a form of engineering risk control.In an event of failure, the system has to be detachable from surgical tools easily to discontinue unexpected motion or enable manual operation so as to minimize interruption to operation.A modular approach for designing the robotic system will serve to facilitate the incorporation of such mechanism.
In addition, design should facilitate sterilization.Components having physical contact with patient need to be sterile.Even areas without contact should be covered with sterile sheet.In view of this, the modular approach is a good strategy.It will accommodate sterilization of components especially the end effecter of the needle manipulator which may contact the patient directly during an operation.Such component should be able to undergo sterilization in facilities like autoclave.The electromechanical manipulation system of the projector and the projector could all be covered with the sterile sheet.

Kinematic model
Interaction is an important concept in AR [2].For effective link between virtual models and the real world spatially, some form of registration or calibration is essential.The pre-requisite to this procedure is the establishment of a kinematic model to the various interacting elements.The main mechanical components comprise of the projection system and the robotic needle insertion device.In the surgical system, components like the projector, camera, manipulator and RF needles are interactive elements.As such kinematic models are derived to represent these components spatially.In addition, the kinematic models describe the spatial interaction between components within the system thus providing a means for path planning and evaluation of the planned path.
The kinematic configuration of the projector manipulation robot is depicted in Fig. 1.In essence, it consists of a Cartesian configuration with two axles of control to position the projector in a plane at a given height.In addition, there are two axles of control to orientate the projector in pitch and yaw fashion.
The design of the robotic needle insertion device is dependent on the kinematics configuration assuming a multi-body dynamic model.Schematics representing the kinematic configuration of the robotic system are analyzed.Frames and link parameters are assigned based on the Denavit-Hartenberg convention for the derivation of the governing kinematics represented by transformation matrix.Fig. 2 is a schematic overview of the kinematic analysis.While there is a need to deliver the treatment within a reasonable speed, the precision must not be compromised.The design concept where a sub-manipulator system is installed at the distal end of the main manipulator is implemented.This design concept allows fast motion to be carried out by the main manipulator and fine positioning executed by the sub-manipulator, hence achieving both swiftness and precision.A cylindrical manipulator configuration was proposed for the design of the main manipulator system.This main manipulator system will accommodate the proposed sub-manipulator system as illustrated in the figure.The sub-manipulator will consist of an x-y translation mechanism and a flexible arm for maneuvering the end effecter over the contour of the liver organ.The governing kinematics of the design can be represented by transformation matrix as shown in Equation 2.
where s j =sin (q i ), c ij =cos(q i ) , s ij =sin (q i +q j ), c ij =cos(q i +q j ), s ijk =sin (q i +q j +q k ), c ijk =cos(q i +q j + q k ).
where a R b is the rotational matrix from Frame a to b, a P b is the translational matrix from Frame a to b, and N is the number of segments.
Similarly, N T tip represents the orientation and position of the assigned frame at the needle tip with respect to Frame N.This part of the manipulator consists of only one degree of freedom.It can be expressed as follow where r is the displacement of insertion/withdrawal joint and d is the offset from grip to insertion joint axis.Based on current prototype, d has a value of 62mm.
A conservative estimation of the workspace for RF ablation treatment sufficient to cover the entire liver will be a planar area of 180mm by 180mm assuming insertion trajectory to be vertical.Hence any goal point must be within the system workspace.
Three dimensional virtual models of the overhead rectangular robot and serial robotic manipulator are built on CAD software, SolidWorks to prepare for prototyping.Fig. 3 and 4 shows the virtual model of the projector manipulation robot and the robotic needle insertion device for design visualization and simulation of the kinematic interactions with the surgical environment.It is designed such that the needle insertion device poses minimal obstruction to the projected image as well as the surgeon.

System Components
The overhead rectangular robotic projection system consists of aluminum profiles assembled structural frame and an adjustable projector housing.The upper frame can be dismantled from its base and mount onto ordinary workbench for development and testing of the AR system.Fig. 5 illustrates our current testing environment.The serial robotic manipulator for needle insertion is integrated into the system as featured in Fig. 6.The structural frame was designed to be assembled and dismantled easily.This is to facilitate future work in animal studies where the system will be set up in surgical theatre.Motion control is implemented with ball-screw translational stages and motorized mechanism driven by fine resolution stepper motors.Apart from motion control, the electromechanical manipulation system provides geometrical awareness to the system for tracking of the kinematics of the components.
Fig. 5: Projection system testing workbench Fig. 6: Incorporation of robotic needle insertion into the projection system Fig. 7 describes the prototype of the serial robotic manipulator for performing needle insertion.This prototype is a realization of the developed mechanical model described above.The robotic prototype consists of a 3 DOF manipulator system and a distal end sub-manipulator system of 48 DOF.The robotic system uses the design concept of sub-manipulator to deliver multiple insertions with consistency.The motorized sub-manipulator system is responsible for consistent and accurate deployment of the needle.Needle insertion is executed by a linear insertion actuator.This actuation system is guided by precise translation stage.The resolution of the linear translation is 0.25mm and the system is capable of a maximum insertion speed of 4mm/s.A needle guiding unit is attached to the insertion actuator to ensure linear trajectory during needle insertion.The needle guiding unit is a detachable module that stabilize linear trajectory of the needle shaft with elegant mechanical means.It rectifies any possible needle deflections by stabilizing the insertion with dual plane roller guide.This wheel also bears a knob that controls manual driven insertion.In event of system failure, the needle can be rapidly detached from the motor controlled grip via a quick release mechanism and resume operation using the manual knob.This targeting device consisting of a pair of laser pointers designed to create a visual marker on the surface or the projected image to indicate the point of insertion.

Operating Mechanism
The complete ablation of large tumor can be achieved by a systematic multiple needle deployment that plot out the required volume of ablation coverage.Dodd et al [13] derived various overlapping ablation models based on computer analysis of RF ablation thermal injury.One of them is the cylindrical model.Though it was found to be less efficient, the model offers ease of control.Most importantly, the systematic nature of the model enables intuitive and flexible execution planning.Further development was carried out by Chen et al [14] in a clinical study of 110 patients to establish the protocol for RF ablation treatment in large liver tumor.The protocol was developed based on a regular prism and polyhedron model.The developed serial robotic manipulator is capable of executing a trajectory for the cylindrical model.The model consists of spherical ablation volumes with uniform lateral intervals between centers of ablation spheres.This systematic row of ablation zone can eventually be extended to an array of larger ablation coverage.Our proposed operation mechanism applied the concept of sub-manipulator system at the distal end of the main manipulator system to execute precise deployment of needle under intra-operative AR guidance so as to cover accurately the entire volume of tumor as shown in Fig. 8.The kinematics involves simple translation within the specific space constraint depending on the surgery.Hence a sub-manipulator system consisting of precise x-y translation mechanism and flexible link is incorporated at the distal end for fine adjustment.Together with the high precision insertion actuator, the system is capable of positioning the needle at any point precisely and rapidly in three-dimensional free space.Fig. 9 is the process flow of an effective procedure that combines the consistency of the robotic system and visualization aid through AR.

Process 1: System initialization
This includes assigning a global frame of reference based on the geometrical model of the surgical field.The aim is to establish spatial reference to the surgical system and enable registration of the pre-operative planning to the intra-operative environment subsequently.

Process 2: System calibration
In essence, the serial robotic manipulator for needle insertion has to be calibrated to the overhead rectangular robot for projector manipulation.This includes the derivation of a geometrical transformation describing the spatial relationship between the two sub-systems.This can be done by processing the kinematic models discussed previously with geometric parameters.The transformation matrix will subsequently enable the expression of kinematics of the components to be referenced in the same domain.

Process 3: Registration
Registration in the AR context refers to the creation of relation awareness between the real environment and its corresponding virtual representations [1].The key difference between AR and VR lies in the fact that VR only involves virtual models existing in the virtual environment while AR complements the real world

Sub-manipulator system
environment with virtual model.Hence virtual models need to be reference with respect to real world coordinates.This can be done by a few developed techniques each with different system requirements [17].

Process 4: Image overlaying
This process includes application of the registration transformation obtained from process 3 to manipulate the image projection such that the targets corresponds to the tumors and that the image data is matched accurately to its corresponding features.The image coordinate of the targets should be derived from pre-operative computer processed imaging and surgical plan.

Process 5: Execution of overlapping ablation model
Multiple needle insertion and deployment of needle will be done under intra-operative guidance of the AR projection.

III. EXPERIMENT
The camera-projector system involves solving the homography to achieve registration of patient pre-operative and intra-operative images.There is an application to non-planar projection.Fig. 10 is an example of direct projection of image data onto the phantom object.The serial robotic manipulator for needle insertion can target all sectors of the liver organ.The first experiment is to test the proposed needle guiding concept.Needle insertion and withdrawal was done on the same point on a phantom liver from an insertion height of 160mm.A reference grid was overlaid on the phantom specimen to register the positions of five punctures.These five insertions are intended to puncture the same point.Hence it is desired that the five punctures are as close as possible to the intended point.The desired point is marked by a visual marker produced by targeting LED.Deviation of punctures from this theoretical point will be registered as the error.This experiment attempts to test the system accuracy in insertion path directed at target.All the insertions were done using RITA StarBurst XL needle.This is a commercially available RF needle used for tumor ablation during hepatobiliary surgery.The results are tabulated in Table 1.A motion consistency test was designed to assess the precision of the executed ablation model.In this experiment, the manipulator will execute the assigned trajectory as in actual intra-operative situation.The actual path taken by the needle tip will be recorded.In this experiment, we adopted the cylindrical model to be the pre-operatively derived plan.In order to plot out ablation pattern of this model, the needle tip will have to be deployed in uniform intervals as discussed in previous section.As such the virtual markers to be augmented on the operative field will be in a linear configuration.The positioning of the end effecter by x-y translation mechanism is tested.With the RF needle affixed, the serial robotic manipulator executes a cyclical series of sequential insertion, withdrawal and translation of the needle via a layer of phantom tissue resting on top of polystyrene with grid overlay.Fig 12 illustrates the execution path for the experiment.The deviation from the intended linear pattern is tabulated in Table 2.

IV. RESULTS
From Table 1 we see that the implementation of the needle guiding unit reduces the mean error by 0.21mm which is 22.1%.This suggests that the guiding unit effectively stabilize needle deflection during the insertion stroke.Therefore apart from its guiding function for manual insertion, the guiding unit is also important for accurate automated insertion.The performance of the system in this test is important as accurate targeting of a single target is the most important prerequisite for multiple needle insertion.Fortunately, this magnitude of error does not have significant clinical implication.According to the general practice for RF ablation treatment, a safety margin of 10mm in excess to tumor boundary is usually adopted [18].The performance test has shown that error of the developed robotic manipulator system is capable of achieving insertion and motion trajectory execution within an error of 1mm for the intended paths.This error is safely kept within the 10mm margin.The volumetric misalignment can be reflected by the amount of volume that fails to be covered in an actual insertion.Take the ablation spherical volume to be of diameter 40 mm which is the technically recommended standard for most RITA StarBurst XL needle.Mean error of insertion with insertion actuator and needle guiding unit is 0.71mm.By calculus derivation, the volumetric misalignment is approximately 891.8mm 3 .Comparing with the entire ablation volume of 33510mm 3 it is an error of 2.66%.
The result for the motion consistency test is also promising.The maximum deviation from the desire path is not more than 0.5mm.Fig. 13 illustrates the deviation of the actual ablation pattern from the intended path planned.With a resolution of 0.5mm, there is hardly any observable deviation except for a few occasions.The error in this experiment is very much due to needle deflection when it travels through the layers of agars.It has been explained in studies, that the unbalanced resistance load on needle tip during insertion resulted in needle deflection [19,20].V. DISCUSSION AND CONCLUSION An AR robotic system for multiple needle insertion is proposed and developed to successfully treat large tumor.It is capable of reliable performance in term of accurate visualization, precise motion execution capabilities.The kinematics and mechanics of the manipulators for projector and needle insertion mechanism are derived from first principle and geometrical approach mathematically.Prototype is constructed and experiments have been performed with the prototype system.Data have supported the feasibility of the AR surgical setup which yields promising results in terms of accuracy and precision.
The developed serial robotic manipulator is designed based on a geometrically efficient kinematic model.The key advantage of the developed mechanism is its straightforward governing kinematics that lay a foundation for an intuitive operation path as compared to conventionally sophisticated articulating manipulator system.Generally, it would have been argued that articulated robots are less workspace intrusive and dexterous [21].However through a problem specific approach, this study has shown that the proposed configuration can be more appropriate for multiple RF ablations, as well as displaying of image overlays on patient.The computation of the manipulator dynamics is less demanding because of the Cartesian configuration.In the case of the needle insertion serial robotic manipulator, the load sensor is not required for the dynamic control of the manipulator joints in contrast to the usual articulated manipulators.
The establishment of the manipulation mechanism also provides a platform for integration of existing research findings on soft tissue needle insertion with robotic devices.There have been a lot of research efforts directed toward understanding the needle-tissue interaction and modeling a realistic simulation of the interaction [22,23].This existing knowledge can be combined with the developed serial robotic manipulator mechanics for a more holistic model of robotic assisted needle insertion device.
Nevertheless, achieving accurate and realistic visual augmentation in the real-time surgical field remains challenging.The virtual anatomical models have been built from diagnostic imaging modules like MRI or CT.As seen in Fig. 14, the vasculature of the liver is displayed clearly to provide guidance in navigating surgical tools.We are developing effective algorithm for correction of geometric distortion characterized by high degree of elasticity transformation.To enhance the realism, radiometric compensation is also explored.

Fig. 1 :
Fig. 1: Kinematic of projector manipulation robot.It is a rectangular robot By multiplying the various transformation matrices, the homogenous matrix can be expressed in Equation 1.

Fig. 2 :
Fig. 2: Kinematic of serial robotic manipulator insertion device describes the spatial relationship between the globally fixed Frame 0 and the x-y translational stage.It is expressed as follow

Fig. 9 :
Fig. 9: Operation process flow of the AR robotic system

Fig. 10 :
Fig. 10: Image overlay on phantom patient for AR guidance surgery A series of tests were conducted to assess the performance of robotic needle insertion device.The reliability of the serial robotic manipulator is the key requirement for multiple overlapping ablations of our AR robotic system.An experiment test-bed was set up to carry out the experiments.Multi-layer phantom tissues made of agar were used.This tissue mimicking phantom model acts as the needle path medium to simulate realistic insertion strokes.Experiments are designed based on the established operation mechanism taking into account actual application and clinical factors.An illustration of the experiment set-up is shown in Fig. 11.

Fig. 11 :
Fig. 11: Experimental set up with phantom tissue model made of agar.

Fig. 13 :
Fig. 13: Deviation of actual and desired path

Fig. 14 :
Fig. 14: Image overlay on the phantom displaying vessels of the liver organ.

TABLE 1 :
RESULT OF ACCURACY TEST

TABLE 2 :
RESULT OF MOTION CONSISTENCY TEST