Proposition and evaluation of a collision detection method for real time surgery simulation of opening a brain fissure
- 4.4k Downloads
In neurosurgery, opening a brain fissure allows an operator to access to the basis cranii and to treat a disease part safely, but the operator is required highly sophisticated techniques. In order to develop a neurosurgery simulator enabling surgeons to train themselves for opening a brain fissure procedure, it is required to detect collision between a thin instrument model and a deformable brain model and furthermore to compute feasible enforced displacements to deformable object. This paper proposes a new collision detection method which considers to the collision state of objects in previous loop of simulation. And simulation of opening a brain fissure is implemented using the proposed method which is accelerated by GPU (Graphics Processing Unit). Finally the performance of the proposed method is evaluated.
KeywordsCollision detection Operation simulator Software development ROBOMEC2013
1.1 Neurosurgery simulator
In recent years, due to the improvement of safety awareness and significant advances in medical technology, doctors have to spend more time in training themselves to learn up-to-date surgical techniques. In the preoperative training, surgical models and animals are used in spite of less reality or high cost. Surgical simulators based on Virtual Reality (VR) technology and haptic display offer high quality and cost-effective training. The surgical simulators can use a patient specific model and reproduce surgical situation repetitively. The surgical simulator benefits not only training but also preoperative efficient planning and have been actively studied in neurosurgery that requires particularly sophisticated techniques. NeuroTouch  provides training of tumor-debulking task and tumor caterization task. This simulator is introduced for neurosurgery training curriculums in Canada. By using this simulator, a novice doctor can learn surgery techniques for tumors in ensured view of the task-specific operating field. However, this simulator uses models of only the task-specific operating field to meet real-time computing requirement. ImmersiveTouch  is another surgical simulator which can be used for training to insert a neuroendoscope into a brain. This simulator supports the operator with a Computer Graphics (CG) view of internal objects in a skull. Using ImmersiveTouch surgeons can learn how to handle lesions located deep in the brain unlike in the case of NeuroTouch, but the surgical procedure which can be simulated by this simulator is limited to needle (neuroendoscope) insertion. These neurosurgery simulators to exist provide training environments which can display sight of operation or a perspective CG view. However, the operator needs to approach lesions located deep in the brain and ensure view of operation by opening a brain fissure using spatulas in some cases. During approaching, avoidance of any injury to a brain is essential for successful operation and difficult because internal tissues are composed intricately. It is necessary to develop neurosurgery simulator that realizes training to approach the depths of a brain by using a whole brain model.
1.2 Opening a brain fissure
Brain fissures form deep valleys, and the Sylvian fissure and the inter-hemispheric fissure lead to the depths of the brain.
1.3 Collision detection
Collision detection is the computational problem of detecting the intersection of objects and plays an important role in realizing interactive environments in physical simulation and video games . In robotics, collision detection is also a major topic to calculate time of impact and generate a path to avoid collision . In this paper, collision detection refers to detecting the intersection between a deformable object and a rigid object and handling responses of intersection of those objects. Collision detection for deformable models realizes many interactive simulations like a surgical simulation ,,,. Many methods of collision detection for rigid and deformable bodies have been proposed. Collision detection can be categorized with Discrete Collision Detection (DCD) and Continuous Collision Detection (CCD). Here we describe related works of DCD for deformable objects with some different methods and CCD that is recently studied intensively.
1.3.1 Discrete collision detection
In the neurosurgery simulation, thin surgical instrument models penetrate a deformable model. Then the surface-based method is not suitable.
Volume-based method Other major methods are based on intersection of the internal meshes of objects ,. Compared with surface-based method, this method can detect deep penetration between objects. The volume-based method can be classified roughly into image-based collision detection and intersection-based collision detection. Layered Depth Image (LDI) is a kind of image-based collision detection ,. This method can be accelerated with graphic hardware and is adopted in real time interactive simulators . LDI can detect fast collision between complex models. When objects penetrate each other deeply, LDI detects the contact volume as shown in Figure 3b. However the algorithm of response for collision detection is based on only geometrical relationship between models, therefore cannot handle correctly neither enforced displacements nor reaction forces. Other volume-based methods detect collision by intersection between internal meshes of objects ,. In this method, depth of penetration between objects is calculated and used for displacement for intersected objects. In mesh-to-mesh collision detection, volume-based method sometimes lead to inconsistent direction of penetration depth. Heidelberger et al. cleared this problem with direction detected by intersection of surface . However, in neurosurgery, direction of penetration depth does not always match with direction of ideal enforced displacements.
Ray-tracing-based method Ray-tracing-based method is one of volume-based methods. This method often uses surface data of an object and casts rays form a vertex of the surface meshes or internal meshes. This method also can detect intersection between objects with less tunneling ,. However displacement is sometimes infeasible because of difficulty in choosing feasible collision pairs.
1.3.2 Continuous collision detection
While DCD detects collision between objects at discrete time, CCD targets collision during a time step, and can compute the first time of collision and the contact point. CCD was proposed for collision between rigid objects ,. CCD is very high cost for deformable model because a large amount of elemental tests are needed. Recently many culling methods are studied and contribute to reduce computing time of CCD between deformable models ,. CCD can detect self-collision of a deformable object and is applied in clothes-simulation ,. Compared with DCD, CCD is robust for tunneling and can compute suitable enforced displacements for simulation of deformation . However, computing cost of CCD is larger than DCD and high speed test are required in an interactive simulator like a surgery simulator. Tang et al. showed a result of simulation that a liver is cut during a surgical operation . In this simulation, the model consists of 4338 tetrahedra, and the average CCD query time is 53.2 [ms] per frame. They achieved more than 10 times improvement over prior methods. This CCD query speed is not enough high because the scale of the brain model is larger and the required query time is less than 30 [ms] for real time simulation.
A real time simulator needs the contact state between objects for reactions and high speed DCD that performs test at small discrete time is suitable. DCD detects collision and enforced displacements using a geometrical relationship between objects. However enforced displacement as a result of collision detection sometimes cannot be detected with only the geometrical relationship like the depth of penetration. This paper proposes a DCD method realizing suitable deformation for opening a brain fissure.
2.1 Overview of the proposed method
Subsections from ‘Objects of collision detection’ to ‘Deformation with collision pairs’ explain how the proposed method detects collisions and how the results of the test are used for deformation of the brain. Then subsection ‘Preparation for culling collision pairs’ and ‘Culling collision pairs using history’ explain how removes false positive collision pairs by using history of collisions. Subsection ‘Merging the two intersection test’ and ‘Acceleration with GPGPU’ explain how the computational costs are decreased and how the computational speed is accelerated.
2.2 Objects of collision detection
Proposed method targets thin instrument models like a spatula (spatula model) and a deformable brain model. In the process of opening a brain fissure, the brain is retracted with two large planes of a spatula. In order to perform the collision detection fast, side planes of the spatula model are excluded from colliding planes. Thus, the algorithm can only detect collision on the largest two planes of the spatula.
2.3 Collision pair
What is collision pair Collision pairs are results of collision detection and present which elements contact each other. The collision pair found by the proposed method is combination of a node of the spatula model and a surface triangle of the brain model. Collision pairs are used for detection of enforced displacements for the deformable brain model.
If all collision pairs are used as displacement for the brain model, infeasible deformation will occur especially in the case where the spatula model penetrates the brain model deeply as shown in Figure 6b.
Handling pair To realize feasible deformation in deep penetration between models, the collision pair needs to be tested to see whether the intended pair is feasible or not. The geometric relationship between models and direction of moving model is not sufficient for testing which pair should be used for displacement in deep penetration. The proposed method can detect collision between deeply penetrated models and decide which pairs are feasible for deformation through two tests. In the first test, collision pairs are found with a ray-tracing-based method. In the second test, a history of the collision state finds infeasible pairs and displacement for the brain model is decided from feasible pairs.
2.4 Ray-triangle intersection
where P=D×E2 and Q=T×E1. A determinant of the matrix in the left side is calculated by scalar triple product. When 0<u<1,0<v<1 and u + v<1 are satisfied, the ray intersects the triangle. And when t is positive, the cross point between the ray and the triangle is in front of V0. When t is negative, the cross point between the ray and the triangle is behind V0. The positional relation between the source node and the triangle is important for detecting whether the intended collision pair is feasible or not.
2.5 Searching collision pairs using rays
To detect collision between a spatula and a brain fissure, a collision detection method based on the ray-triangle intersection test for a non-convex hull is proposed. The process of collision detection using rays is explained as follows.
A1. Search of nearest triangles Source nodes on the two planes of the spatula model cast a ray in the reverse direction of the normal vector N R of the plane, on which nodes are located. If the ray is casted in the reverse direction of the normal vector N R , the sign of t (the result of the ray-triangle intersection test) will change. It is not essential whether the direction of the ray is the reverse of N R or not. The contact planes approach to surface triangles of the brain objects. When collision between the plane and the surface triangle occurs, the triangle is behind the back side the plane. So the ray for searching collision pairs is casted in the reverse direction of the normal vector N R of the plane. Ray-triangle intersection test is pursued among rays from the source nodes and the surface triangle of the brain. The source node compares distances of intersected triangles and finds the nearest intersected triangle behind.
The proposed method can find a collision pair between a simple spatula model and a non-convex brain model.
2.6 Deformation with collision pairs
where V init is the initial position vector of the vertex, and V proj is the position of the vertex projected onto the collision plane. When the spatula model moves very small distance during simulation period and collision pairs are detected on only one side of the spatula model, all collision pairs can be used for enforced displacements. However, when the spatula model moves fast and penetrates the brain model deeply within the simulation period, collision pairs are detected on both two planes. If all collision pairs give the brain model the enforced displacement, the brain model deforms unsuccessfully as shown in Figure 6. Volume-based collision detection can detect collision pair of each vertex of surface or each triangle, and enforced displacements of the deformable object are computed using the depth of the collision pair. However suitable enforced displacements cannot be selected when there are some collision pairs with only geometrical relationship of collision pairs. In Figure 6b, the spatula moves to right and penetrates the brain. As a result of collision detection, some collision pairs on both two sides of brain are detected. However, a feasible collision plane of the spatula model is only in right side because the first contact side is the right side of the spatula and the left side should not give the enforced displacement to the right side of the brain. Then collision pairs need to be culled with not only the geometrical relationship but also with contact state between objects for realistic deformation. This paper shows a method of culling collision pairs using history of collisions in the following subsection.
2.7 Preparation for culling collision pairs
A point of difference from other previous works is culling collision pairs with a history of collision state. To cull successfully, two preparations are needed.
Arrangement of source nodes Second preparation is an arrangement of source nodes fixed on the two planes of spatula model. As previously mentioned, source nodes are discretely located on the two side of the spatula model. As additional assumption of the arrangement of source nodes, a source node on one side makes a pair with one on the other side. Pairs of source nodes are arranged so that pair source node is on same line parallel to the normal vector of the plane.
2.8 Culling collision pairs using history
Figure 10 shows an example of culling collision pair using His-flags. The process of culling collision pairs is as follows.
B1. Renewal of His-flags In (n-1)th loop of the simulation, some source nodes find collision pairs and the collision pairs are referred as displacements of surface triangles. Flags of collision nodes are raised. In n th loop of simulation, collision pairs are found again as a result of collision detection. Each source node on the two planes of spatula model has a collision pair.
B2. Search of infeasible pair for deformation If there are some source nodes whose His-flag is on, rays are cast from those nodes in the direction of the plane’s normal vector N R . As a result of ray-triangle test, the source nodes find the nearest triangle in front. The nearest triangle in front of the source node composes infeasible pair for deformation of the brain model as shown in Figure 10c.
B3. Detection of enforced displacements Triangles found in the process of B2 are infeasible for natural deformation. Then the collision pairs which contain infeasible triangle are canceled and there are only feasible collision pairs so that natural deformation is realized in deep penetration as shown in Figure 10d.
2.9 Merging the two intersection test
The proposed method carries out ray-triangle intersection test not only in collision detection but also culling collision pairs. If the intersection test in different process is carried out at once, computation time of collision detection will be decreased. Searching two kinds of triangle can be found in one ray-triangle intersection test. Triangles composing collision pairs are behind source nodes which cast rays. Triangles composing infeasible pairs are in front of the source nodes. In the previous subsection, the distance between a source node and a triangle can be measured with t in Eq. (5). So if rays which are cast with the same direction of N R intersect with surface triangles in the test, the minimum of the absolute value |t| is stored by each sign. As a result, source nodes find the nearest triangle in both front (t>0) and backside (t<0).
2.10 Acceleration with GPGPU
Bigger brain models and substantive source nodes increase computing time because the proposed method needs to try an intersection test for all combination of a source node and a surface triangle. Each ray-triangle intersection test has no data dependencies, so the test among all combination can be processed by parallel computing. As recent trends of parallel computing, General Purpose Graphic Processing Unit (GPGPU) contributes to acceleration using its many cores. GPGPU can be introduced into ray-triangle intersection during the proposed collision detection. For using GPGPU, we adopt Computing Unified Device Architecture (CUDA) that NVIDIA Inc. provides as an integrated development environment for Graphics Processing Unit (GPU). CUDA offers a hierarchical thread structure and various memory structures for parallel computing. The hierarchical thread structure consists of a grid, blocks and threads. Grid has some blocks and a block have some threads. Maximum number of block of each grid and maximum number of thread that each block has are limited. The source nodes are assigned to each block and the surface triangle to each thread so that the intersection test is performed in each combination of a block and a thread. When intersection test is performed by GPU, Central Processing Unit (CPU) sends the data sets of position of source nodes and all surface triangles to GPU by each side of the spatula. Using CPU, each result is sorted and the nearest surface triangle for a source node in both front and backside is determined.
3 Results and discussion
Specifications of CPU and GPUs
intel Core i7-3960X
2,496 CUDA Cores
2,688 CUDA Cores
3.1 Effectiveness of culling collision pairs
3.2 Simulation with collision on both sides of a spatula
3.3 Measurement of computation time
3.4 Simulation of opening a brain fissure
3.4.1 The condition of the simulation
3.4.2 The result of the simulation
Figure 14 shows the result of the simulation. Two spatula models exposed the insula of the brain model in Figure 14b and c. In the process of simulation, the spatula model was inserted and contacted the brain fissure with its both sides. In Figure 14d and e, a part of base of skull was exposed by excluding the brain. Proposed method can give realistic displacements to a deformable model in spite of long discrete time about 700 [ms] including computation of deformation.
3.4.3 Advantage of the simulation
By using this developing neurosurgery simulator, the operator can simulate opening a brain fissure with the reaction force from the deformed brain. This means that this simulator arrow surgeons to practice approaching to the lesion area in the depth of the brain. Comparing previous neurosurgery simulator ,, this simulator provides training environment more similar to real neurosurgery in the variety of operative procedures.
3.4.4 Advantage of the proposed method
The advantage of the proposed method is to detect enforced displacements for deformation of the brain. Other deformation simulations adopt force as the result of collision to meet real-time computing requirement ,,. The model of contact force is detected with the intersection volume and some parameters, but validity of physics in the contact model is not clear. In this paper, the simulator adopts enforced displacements as the reaction of collision, so that reaction force are computed based on FEM. Other advantage is robustness against false negative collision pairs. While the previous DCD method works in less than 80 [ms] , the proposed method realizes to simulate realistic deformation of the brain in spite of long discrete time (700 [ms]). This simulator is still developing, so there are rooms for improvement in respect of CG rendering, computational speed, interface of simulator, and so on. After improvement those elements, a quantitative evaluation by surgeons is desirable whether the proposed method is adoptable for movement of their hands.
The first limitation is the shape of instrument. Thickness of the spatula model is important to successful handling of the His-flags. When travel distance of the spatula is longer than the thickness of the spatula, His-flags cannot be updated in accordance with the change of collision states between the spatula and the brain. The thickness of spatula effects clerically in the first contact between the plane of the spatula and the surface of the brain as shown in Figure 10a. If the arrangement of source nodes that described in section ‘Preparation for culling collision pairs’ is satisfied, it is possible to extend the proposed method to curved or trapezoidal spatulas. However side planes of the spatulas are still not contactable.
The second limitation is length of a discrete time. While other real time simulation perform less than 30 [ms], the proposed method works in comparatively long discrete time about 700 [ms]. However, too long discrete times lead to false collision detection. And too thin spatula models cannot decide which side is make contact first.
The last limitation is density of source nodes. Low source nodes contribute to decrease in computing time of collision detection. But low density leads to fail detect collision and cannot give enforced displacement as the plane (Figure 15b).
This paper proposes a new collision detection method which uses ray-triangle intersection test and history of collision state. Using history information, proposed method can search and cull collision pairs and realize interactive simulation of opening a brain fissure. In order to accelerate computing time of the proposed method, decreasing number of source node is easy and useful, but there is limitation with respect of precision. So it is required to introduce some hierarchical data structures to our algorithm. To develop more realistic brain surgery simulator, it is required to introduce blood veins, nervous tissue and connective tissue into the brain model and collision detection which can handle contact with those complex inner structures. After improvement the physical model of brain, computational speed, interface and so on, a quantitative evaluation by surgeons is desirable whether the proposed method is adoptable for movement of their hands.
This works was partly supported by JSPS Grant-in-Aid for Challenging Exploratory Research(24650288) and next generation world-leading researchers(LR003).
Additional file 1:Simulation of opening a brain fissure. This is a movie file of demonstration of brain surgery simulator using the proposed method collision detection method. (MOV 18 MB)
- 2.(2014) Immersive Touch. . Accessed 10 August 2014., [http://www.immersivetouch.com/]
- 3.Yasargi MG: Microneurosurgery: operative treatment of CNS tumors. Thieme, New York; 1995.Google Scholar
- 4.Ericson C: Real-time collision detection. CRC Press, Boca Raton; 2004.Google Scholar
- 5.Balan L, Bone GM (2006) Real-time 3D collision avoidance method for safe human and robot coexistence. In: Intelligent Robots and Systems, 2006 IEEE/RSJ International Conference on, 276–282.. IEEE.Google Scholar
- 11.Heidelberger B, Teschner M, Keiser R, Müller M, Gross M: Consistent penetration depth estimation for deformable collision response. In Vision, Modeling, and Visualization 2004: Proceedings, November 16–18, 2004. IOS Press, Standford, USA; 2004:339–339.Google Scholar
- 12.Heidelberger B, Teschner M, Gross M: Real–time volumetric intersections of deforming objects. In Vision, Modeling, and Visualization: Proceedings. AKA, Berlin; 2003:461–461.Google Scholar
- 13.Faure F, Barbier S, Allard J, Falipou F: Image-based collision detection and response between arbitrary volume objects. In Proceedings of the 2008 ACM SIGGRAPH/Eurographics Symposium on Computer Animation. Eurographics Association, Switzerland; 2008:155–162.Google Scholar
- 14.Teschner M, Heidelberger B, Muller M, Gross M (2004) A versatile and robust model for geometrically complex deformable solids. In: Computer Graphics International, 2004. Proceedings, 312–319.. IEEE.Google Scholar
- 15.Hermann E, Faure F, Raffin B: Ray-traced collision detection for deformable bodies. In 3rd International Conference on Computer Graphics Theory and Applications, GRAPP 2008. INSTICC, Madeira, Portugal; 2008.Google Scholar
- 16.Kim Y, Koo SO, Lee D, Kim L, Park S (2010) Mesh-to-mesh collision detection by ray tracing for medical simulation with deformable bodies. In: Cyberworlds (CW) 2010 International Conference On, 60–66.. IEEE.Google Scholar
- 19.Shui Y, Zheng J, Ma X, Zhou H, Shen L (2012) Subspace culling for continuous collision detection In: Software Engineering (WCSE), 2012 Third World, Congress On, 83–86.. IEEE.Google Scholar
- 21.Govindaraju NK, Knott D, Jain N, Kabul I, Tamstorf R, Gayle R, Lin MC, Manocha D: Interactive collision detection between deformable models using chromatic decomposition. In ACM Transactions on Graphics (TOG), vol 24. ACM, New York, USA; 2005:991–999.Google Scholar
- 25.Sase K, Nakayama M, Satake Y, Abiko S, Jiang X, Tsujita T, Konno A, Uchiyama M: Evaluation of real-time preformance of the blunt dissection simulator using a real brain model. In Proceedings of the 2012 JSME Conference on Robotics and Mechatronics. JSME, Japan; 2012:2P-U02.Google Scholar
- 26.Fukuhara A, Tsujita T, Sase K, Konno A, Jiang X, Abiko S, Uchiyama M: Collision dtection for real time surgery simulation of opening a brain fissure. In Proceedings of the 2013 JSME Conference on Robotics and Mechatronics. JSME, Japan; 2013:2A1-L02.Google Scholar
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly credited.
Open AccessThis article is distributed under the terms of the Creative Commons Attribution 2.0 International License (https://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.