An integrated system for planning, navigation and robotic assistance for mandible reconstruction surgery

Abstract

Mandibular reconstruction is an extremely complex and high-risk surgery. The aim of the study was to integrate robotics technology into mandibular reconstruction surgery as it can reduce surgeons’ workload and improve the accuracy and quality of the surgery. In this study, we first introduced a mandibular reconstruction surgery robotic system, which includes integrated surgery planning, an optical navigation system, and a robot-assisted operation. Second, we addressed novel mandibular reconstruction surgery that is aided by a multi-arm robot with an optical navigation system. Finally, we conducted an accuracy test, a skull model experiment, and an animal experiment to evaluate the robotic system. Experiments showed that the robot’s mean placement error was 1.0205 mm, which was acceptable for clinical application. We will continue to study this point, however, and plan to reduce the error eventually to 1.00 mm. The robot ran smoothly and accurately. It was also able to perform fibular segment implantation, positioning, and auxiliary fixation during mandibular reconstruction surgery.

Introduction

The mandible, the largest and strongest bone of the face in the oral and maxillofacial region, plays an important role in maintaining the facial architecture as well as chewing, language, and other functions [1]. Mandibular defects are the most common oral and maxillofacial defects [2]. Mandibular defects not only destroy the appearance of patients but also affect facial functions [3].

Before the development of such techniques as medical navigation and imaging technology, the complex procedures required for mandibular reconstruction, including drilling, bone positioning and orientation, and bone grafting, among others, were performed manually by the surgeon [4, 5]. Even with advances in navigation and imaging, however, the surgery still presents many difficulties. First, because the structure of anatomic areas is intricate, the procedure is a complex, highly technical operation. Also, the operation cannot effectively ensure the quality of treatment. Even though computer-assisted navigation has been introduced into the operating room, there are still obstacles to the surgery, such as how to make the bone achieve the appointed position precisely by manual maneuvering [6, 7]. Second, the operation can easily last 8 h, or longer, which is fatiguing for the surgeons. Finally, if the surgery fails, the patient suffers serious consequences—hence, the need to attain good surgical results despite its difficulty [8, 9].

Fig. 1

System overview

Positioning accurately and moving stably—characteristics essential for this surgery—are the requirements of the robot. As a result, robots have been increasingly included to assist the surgeon, including the orthopedic robot RoboDoc (Curexo Technology, Fremont, CA, USA) [10], the robotic system ZEUS [11], and the robotic Da Vinci system [12] used for minimally invasive surgery, the endovascular micro robot [13], and the intestinal capsule robot [14, 15]. With the development of three-dimensional imaging, navigation, and medical robotics, robotic assistance for mandibular reconstructive surgery has been introduced.

The following lists some advantages concerning medical robot-assisted mandible reconstruction surgery.

• (1) The ability of robot positioning accurately can avoid injuring important vessels and nerves to reduce surgical risk. In addition, it can grip the implant bone with the certain correct orientation for a long time to surgeon’s operation.

• (2) The system can execute precisely preoperative surgical design and conducted interaction with the surgeon.

• (3) Using the accurate intraoperative navigation, the preoperative planning and surgery robot operations could be connected.

• (4) By establishing a safe monitoring platform with an input of more information, reliable manual interventional measures can be taken to ensure a safe operation [16–19].

Based on the requirements of mandibular reconstructive surgery, we developed a novel mandibular reconstructive surgery multi-arm robot (MRSMR) to assist surgeons performing these operations. We created a robotic assistance system for mandibular reconstruction by integrating preoperative three-dimensional (3D) reconstruction and surgical planning with an optical navigation system.

Materials and methods

The robotic system (Fig. 1) consists of the following major components: the MRSMR; an optical navigation system (NDI, Waterloo, Ontario, Canada); a workstation to run the 3D mandibular reconstruction surgery design software (MRSDS); and a main workstation to run the application and robot control.

In this paper, a novel robot system was designed with three arms to realize the holding and embedding motion. Two of the three arms, named left and right holding arms, realize holding the mandibular ramus, while another arm, named middle operating arm, completes to embed the new reconstructed mandible. During the surgery, the left and right arm aims to hold the head and keep the right position and the middle arm clamps the fibular to place the right position to operation for surgeons.

Fig. 2

Degrees of freedom configuration of the robot arm. a Left arm, b right arm, c middle arm

Fig. 3

Prototype of the mandibular reconstructive surgery multi-arm robot (MRSMR)

Mandibular reconstructive surgery multi-arm robot

Based on the requirements for mandibular reconstructive surgery, the MRSMR was designed with three arms. Each arm consists of six active degrees of freedom (DOFs) and one passive DOF. The first three DOFs (joints 1–3) achieve positioning within the working space, and the other three DOFs (joints 4–6) are used to make orientation adjustments. The passive DOF is used by the surgeon to assist with orientation adjustments. The two holding arms need to arrive near the mandibular rami. They then adjust and hold the rami in the appropriate position and orientation. The middle operating arm takes fibular implants to the surgical area being addressed.

Figure 2 shows the DOFs’ configuration of the middle arms. The first DOF of each arm is used to accomplish vertical movement, and joints 2 and 3 DOFs of each arm can rotate in the level plane. The first three DOFs decide the position of the end-effectors. The last three DOFs are in the robot’s wrist, which can control the required orientation of end-effectors. Figure 3 shows the prototype of the mandibular reconstructive surgery multi-arm robot.

With the requirements of safety, robustness, and real-time communication, the control system of the robot was designed based on the controller area network bus (CAN-bus). The whole control system structure is shown in Fig. 4. The joints of the arms are driven by a motor and a reducer. Each joint, which is equipped with an incremental encoder and a hall sensor for achieving the absolute position, has a digital servo-driver and controller. There is an emergency stop in each controller for the motor to ensure the safety of the system. All joint controllers communicate with a personal computer (PC)-based workstation via the CAN-bus.

In the robotic system, Omega.6 (Force Dimension, Nyon, Switzerland), which is designed by Force Dimension, is used as master manipulator. Three arms of the robot are employed as slave manipulators. As shown in Fig. 5, the structures of master and slave manipulator are heterogeneous. To achieve real-time control, the mapping relationship between master and slave manipulators in the world coordinate system should be built. Considering the configuration of master and slave manipulators, the first three joints (joints 1–3) between master and slave manipulators are mapped through Cartesian coordinate system, as well as the other three joints (joint 4–6) are mapped through Joint coordinate system. Since the joint space is ineffective, the research of control method in Cartesian space is pivotal. In Cartesian space, inverse kinematic is the most essential factor. To get unique solution in inverse kinematics and improve response rate between master and slave manipulators, a method based on theory of differential transform, which substitute displacement in very short period for instantaneous velocity of joints, is adopted.

Fig. 4

Control system of the robot

NDI navigation system

The NDI optical tracking system was used as the 3D coordinate measurement system for the real model of the patient and robot, as shown in Fig. 6. In the optical tracking system, a passive probe and passive rigid body were used for coordinate measurements. The optical tracker was used to position the robot and locate the position and orientation of the end-effectors. Optical tracker control was accomplished by the PC-based workstation.

Fig. 5

Heterogeneous structure for master–slave control

The optical navigation method was developed for the 3D reconstructed model, real patient model, and robot so the robot could be guided through coordinate transformation with target position input. Figure 7 shows the coordinate transformation. The registration between the 3D reconstructed image and the skull model utilizes point-based registration. The optical tracker is used to calibrate the position of the robot using a passive rigid body attached to the robot.

3D mandibular reconstructive surgery design software

A high-quality 3D craniofacial visualization system for surgical designing has been developed. The system is based on an open framework to allow improvement with new features, which can be easily plugged in. A variety of efficient tools (e.g., stoke-based direction volume object cutout) is provided to reduce the tedious user–robot interaction. Some solutions were introduced to improve the quality of 3D medical visualization. The system is already used for medical imaging data visualization, craniofacial surgical planning, and surgical navigation.

Application controller

The application control software, which includes robot control, runs on a workstation that contains a real-time interface. The application control includes the image display and operation, robot operation. The robot task communicates with the robot via CAN-bus and performs basic functions, such as receiving joint feedback. The control task implements the supervisory control layer. Its functions are to provide force and motion control. It also provides the interfaces to the force sensor and the navigation system. The robot and control tasks both require periodic, real-time execution. The main thread handles the graphic interface, which is implemented using the Fast Light Toolkit (www.fltk.org) [20] and drives the application procedural flow. It also sends the data to MRSDS for visualization.

Fig. 6

Optical tracking system

Fig. 7

Coordinate transformation

Registration and calibration

In this application, the surgeon uses MRSDS to design the surgery. Information is loaded into the application control software, which must ultimately use it for motion in the robot world frame. A complete set of transformations is shown in Fig. 7. Although the navigation system and MRSDS both read the imaging data, they use different conventions for the coordinate system. Therefore, we require a fixed transformation between the navigation frame and the MRSDS frame. The transformation between the two frames is obtained using registration methods provided by the navigation system. A point-based registration was used in the experiments, where a tracked, hand-held pointer probe is used to touch at least four features attached to the skull prior to obtaining the computed tomography (CT) data. The transformation between the robot world frame and the navigation system frame is obtained by moving the robot into six different positions, recording the end-effectors’ position in each coordinate system, and applying a standard point-based registration method. The robot kinematics already provides the location of the end-effectors with respect to the world frame. These offsets are obtained simultaneously via a standard pivot calibration method.

Fig. 8

The accuracy test

Experiments and results

Accuracy test of the robot

As shown in Fig. 8, the optical tracking system, including the optical tracker, passive probe and passive rigid body, is used as a measurement tool in this experiment. Its repeat measurement accuracy of the same point is less than 0.01 mm, absolute measurement accuracy with appropriate position and orientation is less than 0.15 mm. The spatial position of passive rigid body for location in the optical coordinate system, which fixed on the end-effector of robot, is treated as the actual position of the measurement point of the end-effector of robot.

• (1) Firstly, the rigid body for location fixed on the end-effector of the robot arm is treated as measurement point. Complete registration between robotic space and optical measurement space to obtain the mapping relationship, and get the coordinate value of the rigid body point determined by the rigid body for location in the robot coordinate system.

• (2) Secondly, give randomly the coordinate value $(x_A ,y_A, z_A )$ of a predetermined point (in the robot coordinate system) within robot workspace and control the rigid body point to the position of the predetermined point. Record the coordinate value $(x_{A^{\prime }} ,y_{A^{\prime }} ,z_{A^{\prime }} )$ of the rigid body point in the optical coordinate system and transform the optical coordinate system to the robot coordinate system to get the transformed coordinated value.

• (3) Then, get the absolute positioning accuracy by calculating the difference between the predetermined point and the actual point.

  $$\begin{aligned} \begin{aligned} \Delta x&=x_{A^{\prime }} -x_A \\ \Delta y&=y_{A^{\prime }} -y_A \\ \Delta z&=z_{A^{\prime }} -z_A \\ e_{rr}&=\sqrt{\Delta x^{2}+\Delta y^{2}+\Delta z^{2}} \\ \end{aligned} \end{aligned}$$

  (1)

• (4) Finally, repeat above procedures, in turn choose the different predetermined point and acquire the results of multiply measurements, then deal with these data.

The experiment was repeated 20 times with 7 different preset points. The following Table 1 shows the results of the positioning accuracy results of middle arm, and Fig. 9 shows the change of positioning accuracy test results of each arm.

Table 1 Position accuracy test results of middle arm

Fig. 9

Positioning accuracy test result

The results show that the maximum error of positioning accuracy is 1.3262 mm, the minimum error 0.7188 mm. The accuracy meets the requirement of the MRS so that the end-effector of the robot accurately could arrive at the designated position.

Skull model experiment

The skull model experiment was performed using a plastic skull model that represented the target anatomy. The experiments verified the robot-assisted surgical procedure. The work flow is shown in Fig. 10.

Fig. 10

Skull model experiment

The experiment was conducted to assist surgeons positioning and holding fibula reconstructed mandible defect implant. During experiment, surgeons operated robot and optical navigator with PC-based workstation, with which robot control software and reconstructed 3D model were shown on double displays. The work flow of skull model experiment is as follows.

• (1) According to the actual patient’s condition, manufacture the mandibular defect skull model.

• (2) Titanium screws, used for registration between skull model and reconstructed 3D image, were fixed on skull model as marker points.

• (3) High-precision CT scan of skull model.

• (4) Based on the CT data, complete 3D skull model reconstruction and surgery design of fibular reconstruction mandibular defect.

• (5) Designing, modeling, and printing the fibular implant, after that titanium screws were fixed on fibular implant to do its registration in robotic space.

• (6) Initialize navigation systems and robot. The rigid body was fixed on the skull as patient’s coordinate system.

• (7) Using probe to point to the titanium screw markers on the skull model, the registration between patient coordinate system and image coordinate system was finished.

• (8) The fibular implant was clamped by end-effector of middle arm. Likewise using probe to point to the titanium screws markers on the fibular implant, complete the registration of fibular implant in the robot coordinate system.

• (9) Based on the preoperative planning, the data of target position and orientation of fibular implant in image coordinate system were transferred to robot coordinate system.

• (10) After trajectory planning, control the end-effector of middle arm with fibular implant to target position with certain orientation.

• (11) At last, fibular implant was firmly held for surgeon operation

Fig. 11

The result of skull model experiment

Fig. 12

The goat skull experiment

Figure 11 shows the results of the skull model experiment. During the experiment, with the help of the navigation system the fibular implant was adjusted to the appropriate position and orientation by the robot’s middle arm according to the surgical design. The robot ran stably. The positioning error in this model experiment was acceptable for clinical application.

Animal experiment

The skull of an adult goat was used for the experimental study. Before the experiment, part of the goat’s mandible was excised. The aim of the experiment was to demonstrate that, with the help of the robot, surgeons could reposition the fibular implant to the target location according to the surgical plan. Then, another surgeon, with robotic help, refixed the mandibular bone of the goat using a titanium plate. The experiment is shown in Figs. 12 and 13.

After the experiment, we obtained the CT data for the goat skull and reconstructed a 3D goat skull model. The goat’s mandibular reconstruction was assessed by comparing the preoperative and postoperative 3D models using MRSDS. The 3D image showed that the position difference of the mandibular ramus on average was 1.17 mm and that of the contralateral mandibular ramus was 2.475 mm.

Fig. 13

The result of the goat skull experiment and measurement verification

This animal experiment showed that the robot successfully completed fibular segment implantation, positioning, and auxiliary fixation during mandibular reconstruction. The robotic arms ran smoothly and accurately. The comparative analysis showed that the basic shape of the goat’s mandible did not change after surgery.

Conclusion and discussion

Experiments have shown that robotic systems can quantify surgeons’ movements during operations to achieve precise surgery. Thus, with the help of robotic systems, the labor intensity of surgeons can be reduced. The surgery design software and navigation systems can assist surgeons with preoperative planning and ensure operative safety. We presented an integrated multi-arm surgical robotic system for mandibular reconstructive surgery and conducted experiments to verify the feasibility of using this robotic system. Experiments showed a mean placement error of 1.47 mm. There are many possible causes of placement error, including registration error, calibration error, and robot kinematic error. We expect to find corrective methods to improve the placement accuracy to about 1 mm. There is much work to be done, and inadequacies to be addressed.

More experiments will be done to measure the errors and evaluate the performance of the robotic system. The safety of robotic systems should be improved and optimized. More flexible robotic wrists and more easy-to-use end-effectors need to be designed according to clinical requirements. Future tests will be performed in human cadaver experiments until the system proves to be safe enough to conduct clinical testing.