1 Introduction

Ensuring circular economy for Li-ion traction batteries in electric vehicles (EVs) requires one of several appropriate end-of-life scenarios, such as reuse, remanufacturing, or recycling, depending on their condition [19]. Disassembly is a crucial step in the recycling process, particularly when striving for maximum disassembly depth to streamline subsequent recycling steps [19]. The large number of battery packs necessitates automated disassembly for both economic and safety reasons. However, the disassembly of battery modules that make up a battery pack is challenging due to the presence of non-detachable joints, such as welded and riveted connections, as well as the close proximity of hazardous Li-ion cells [6]. Milling is a flexible and destructive process that can effectively separate mechanical non-detachable joints [19].

Due to component tolerances and cell breathing (during charging/discharging and over the service life due to lithium deposition) [17], Li-ion batteries exhibit an undefined change in thickness, which primarily leads to longitudinal contraction of the cell stack and thus, undefined displacement of cell connections. This can be illustrated by the example of VARTA LPP 402025CE cells (pouch cells) with a thickness of 4 mm: With a thickness tolerance margin of the cell of 15% according to the datasheet, a total thickness variance of 7.2 mm can occur in a stack of 12 cells in a row due to geometrical tolerance of the cells alone. The effect of length contraction of the cell stack with minimum and maximum accumulated thickness \(d_{all}^{-}\) and \(d_{all}^{+}\) is exemplified in Fig. 1. Thus, for a process-safe separation during automated disassembly, sensory detection of the displacement or the object to be dismantled is indispensable.

Fig. 1
figure 1

Exemplary illustration of the change in thickness of a cell stack

Full size image

3D cameras with a high information density (depth, color information...), as well as a relatively high accuracy, could be particularly suitable here. The computer vision (CV) pipeline for processing the generated point cloud consists of different tasks with parameters that should be fine-tuned for each application. Thus, a CV pipeline should be newly developed or adapted for each application. [15]

Fig. 2
figure 2

Exemplary illustration of welds/rivets and milling paths at battery module

Full size image

For the separation of non-detachable joints, this paper follows the approach of CAD-based milling path planning with subsequent registration with a 3D camera. The overall aim is the point localization on easily deformed objects in the context of Li-ion battery module disassembly based on a matching between the reference model (existing CAD model or self-recorded) and the real object. The localized points on the surface of the real object are used as support points for milling operations with a 6-axis articulated-arm robot. A 3D camera system serves as a sensor for capturing the surface of the real object. The validation took place on two different battery modules.

In the following, research work on computer vision applications for the disassembly of different products is presented.

Fig. 3
figure 3

Test setup with the relation of the different coordinate systems

Full size image

2 State of the art

In the context of dismantling, there are numerous works related to computer vision (CV) using 3D camera systems and CV pipelines. Most of the work concentrates on object detection and the localization of screws. There are different approaches to this, among other things:

  • Template matching [5, 7]

  • Contour based [1]

  • Feature-based (e.g., Haar Cascade) [2, 21]

  • Deep-learning approaches (e.g., Mask R-CNN or YOLO) [3, 12, 16]

The great advantage of the above approaches is that no virtual model such as CAD data of the object to be disassembled is required. However, a disadvantage is that significant features (e.g., contours, color gradients...) are required for detection. Thus, the shown approaches are only applicable to one specific type of connection like screws, for new types of connection a vision pipeline should be redeveloped or adapted. For milling operations to separate joint connections (e.g. welded connections on the housing, busbars between cells), the exact knowledge of milling paths (in the form of support points) on the surface of the real object (battery module) is required. Welding joints as well as riveted joints (red marked) and milling paths (marked black) at the battery module are exemplary shown in Fig. 2.

This is a problem of path generation on the (featureless) surface of objects. Object detection and localization approaches have limited applicability, as too specific in terms of connection type, and significant features are not always available. Contour-based approaches to path generation, such as in [18], where separating paths are extracted based on slots, are of limited use because significant features such as slots are required. [14] presents an approach where a product surface model is used. A surface manifold is generated by mapping subsets of the product surfaces into 2D space. The above approach was validated on the example of a mobile phone, which is a flat object; applicability to other disassembly objects was not demonstrated.

A generic approach to the automated milling process for joint separation uses computer-aided design (CAD). This method involves creating a strategic milling path in a virtual environment and then using registration to accurately align this plan with the actual physical object to be milled.

In the context of welding, the identification of surface paths for weld seams based on CAD data is already carried out in off-line programming (OLP) environments, an example being the RinasWeld software [20].

Registration involves aligning two sets of data, enabling a combination or comparison of information from different sources. This alignment, achieved through optimization-based methods, minimizes the differences between the data sets until the most suitable alignment is identified. This process particularly serves in point cloud registration, a specific application in computer vision and 3D reconstruction. Here, data points generated by 3D scanners are aligned as closely as possible. This alignment can be rigid, preserving the size and shape of point clouds for rigid bodies, or non-rigid, allowing for scaling and deformation, which is appropriate for flexible or varying objects.[11]

Work with registration algorithms in the context of Li-ion battery dismantling has been explored, as seen in [23]. The focus of this work, however, was primarily on the localization of individual components (such as BMS Slave Mounting) within the batteries, leveraging rigid registration methods (both global and local) for subsequent handling. Adding to this, [22] developed a unique transformation method. Using a graph-based system, they individually assessed transformation types, resulting in an optimized Truncated Least Squares (TLS) method. This method is highly robust against noisy correspondences and led to the faster TEASER++ algorithm. [22]

However, these methods do not adequately account for object deformation. This is important in the disassembly of battery modules, where there are significant geometric variations, especially in the stacking direction. Non-rigid registration methods that can handle these variations are therefore required. Furthermore, the high productivity demanded in industrial environments requires optimized approaches for computationally intensive operations. To address these needs, [9] introduced the Bayesian Coherent Point Drift (BCPD), a probability density function-based method offering improved handling of complex geometries. However, its subsequent iteration, BCPD++, while faster, still struggles with high execution times. [8]

A research gap in the context of disassembly is thus the non-rigid registration of the CAD file (based on which the milling path planning takes place) with the real object, which has geometric variations. In particular, the processing time factor should be considered for later industrial implementation.

In the following, we introduce a developed and tested data processing pipeline, which in this context refers to a systematic series of steps or procedures aimed at achieving the point localization of support points. This pipeline is specifically designed for the separation using milling of inseparable mechanical joints in Li-ion battery modules. The next chapter delves into the setup of the experimental plant and elaborates on the intricacies of the developed pipeline, placing particular emphasis on the chosen registration algorithm. The subsequent chapter showcases the results, establishing evidence of the pipeline’s functionality and offering a preliminary benchmark in terms of accuracy and overall processing duration.

3 Material and methods

Fig. 4
figure 4

General overview of the pipeline for point localization, using registration algorithms

Full size image

3.1 Test setup

The setup for the automated and flexible disassembly process as well as the different Cartesian Coordinate Systems (CCS) is shown in Fig. 3. The system uses a 6-axis articulated arm robot (Comau NJ290\(-\)3.0) with a milling spindle and a structured-light 3D scanner system (Zivid Two). A flexible clamping system which is responsible for clamping the battery module (cell stack is vertically aligned) allows 360\(^{\circ }\) accessibility for the milling spindle. At the same time, it serves as a turntable for the 3D camera through targeted rotation of the battery module. In the shown system four elementary CCS are used: Robot base CCS (\(CCS_{ROB}\)), Camera CCS (\(CCS_{CAM}\)), Object CCS including the rotation of the turntable (\(CCS_{OBJ}\)) and Tool center point CCS representing the milling tip (\(CCS_{TCP}\)). The relationship between the CCS is established by applying homogeneous transformation matrices, which include rotation and translation operations. This allows the transformation of points represented in \(CCS_{1}\) (\(x^{(1)}\)) to \(CCS_{2}\) (\(x^{(2)}\)) with the transformation matrix \(M_{1}^{2}\) by applying the formula:

$$\begin{aligned} x^{(2)}=M_{1}^{2}*x^{(1)} \end{aligned}$$

Important transformation matrix for the shown use case are:

  1. 1.

    \(M_{CAM}^{ROB}\) to transform points from Camera CCS to Robot base CCS

  2. 2.

    \(M_{OBJ}^{CAM}\) to transform points from Object CCS to Camera CCS

  3. 3.

    \(M_{ROB}^{TCP}\) to transform points from Robot base CCS to Tool center point CCS

\(M_{CAM}^{ROB}\) as well as \(M_{OBJ}^{CAM}\) can be determined by Eye-to-Hand Calibration. \(M_{ROB}^{TCP}\) is given by knowing the Denavit-Hartenberg parameters of the robot and the design data of the spindle.

3.2 Pipeline for point localization

We introduce a novel data processing pipeline tailored for point localization on deformed battery modules. This pipeline uniquely synergizes two distinct registration algorithms to bridge the gap between virtual CAD models and real battery modules, ensuring accurate point identification that guides the disassembly robot’s actions. Specifically:

  • TEASER++ by [22] serves as a rigid registration algorithm, providing a robust foundation by aligning broad structures and layouts between the virtual and real modules.

  • BCPD++ by [9] complements this by delving into non-rigid registration, accommodating fine-scale deformations and intricacies that may be present in the real battery module.

The innovative combination of these algorithms forms the crux of our pipeline’s novelty, offering both efficiency and unprecedented precision in point localization tasks.

TEASER++ is a rigid registration algorithm characterized by extreme robustness and certifiability, providing easily testable conditions to verify that the returned solution is optimal [22]. On the other hand, the method of [4] is a non-rigid registration approach that is particularly fast but requires multiple reference models. The BCPD++ algorithm uses a probability density approach, resulting in exceptional robustness even in the presence of complex geometries. [9].

The methodology consists of four main steps, as shown in Fig. 4. The first step is the generation of a reference model (see 3.2.1), which has to be done once for each new type of battery module. The second step is the 3D acquisition of the real battery module to be disassembled (see 3.2.2). In the third step, the reference model is registered with the point cloud of the real battery module (see 3.2.3), which forms the core of our methodology. Finally, in the last step of transformation (see 3.2.4), the selected points are transformed from the reference model into the robot coordinate system, making them accessible to the disassembly robot.

3.2.1 Reference model generation

Reference model generation plays a crucial role in disassembly planning, where a model is created based on the type of battery module that needs to be disassembled. During registration, the reference model is compared with the real battery module to ensure accuracy. To ensure flexibility in reference model generation, two methods have been developed. One method creates a point cloud from a CAD model, while the other method captures a 3D image of a preferably non-deformed real battery module using a 3D camera and uses it as the reference model. This way, disassembly can be performed even if no CAD model is available.

For the first method, the CAD model is transformed into a point cloud through point sampling. Using the hidden point removal algorithm developed by [13], all points that are not visible from the outside can be removed. The perspectives from which the points are visible are selected in a similar manner to the 3D camera capturing process, resulting in a comparable point cloud for registration. Normal vectors are then determined, which serve as an important basis for the registration algorithms. The orientation of the normal vectors can be determined using the starting points from the hidden point removal algorithm.

In addition to reference model generation using a CAD model, a real battery module can also be used. It can be captured using a 3D camera and directly used as a point cloud for disassembly planning. This method does not require the availability of a CAD model and allows for a more variant-rich disassembly of different battery modules. Furthermore, using a real battery module can result in a higher geometric match with the battery modules to be disassembled, leading to better results in point localization. However, CAD models have the advantage of having no disturbances or errors, making them useful for disassembly planning. The 3D acquisition procedure used for reference model generation is the same as the one used to capture the battery module to be disassembled in the pipeline (see 3.2.2).

3.2.2 3D acquisition

The acquisition of a 3D point cloud of a battery involves rotating the battery module on a turntable while a stationary 3D camera captures point clouds from different perspectives (as shown in Fig. 3). This process can be divided into two main steps: the acquisition of the 3D data from multiple perspectives, and the merging of these captures into a single point cloud.

To acquire the 3D data, the battery module is rotated about its vertical axis and captured from eight different sides using a 3D camera. This number of exposures provides a good balance between detail and acquisition time, as a higher number of exposures would result in a more detailed image but take longer to capture. To minimize shadowing and accurately capture the edge regions between the sides, each exposure is taken at a 45\(^{\circ }\) rotation angle. After the 3D data has been acquired, a coarse segmentation process is performed to remove points that are not located within the battery module. A statistical outlier removal algorithm is then applied to remove any outlier points that may have occurred during the acquisition process. The normal vectors of the points are then calculated using the origin of the camera coordinate system to point outwards, as these vectors are used in the registration process to determine the features of the point cloud. Next, downsampling is applied to reduce the point density of the point cloud, which is necessary to reduce computation time for subsequent algorithms. It is important to find a compromise between accuracy and computation time, as well as to maintain a constant point density for the selection of parameters for the following algorithms.

To merge the individual point clouds of the battery module, they must be rotated according to the movement of the turntable. This requires determining the rotation axis of the turntable, which can be achieved through a calibration process. A checkerboard pattern is rotated and point clouds are recorded, and then registered using an ICP algorithm to obtain the rotation matrix. This results in an Eye-to-Hand calibration problem that must be solved, which is achieved using the approach of [10]. This approach solves the general calibration problem and provides high accuracy, even in the application of a turntable.

The calibration process defines the relationship between the two coordinate systems of the camera and the turntable, resulting in the determination of the rotation axis of the turntable relative to the camera coordinate system. The individual point clouds can then be rotated to the correct position, and after merging, the point density is homogenized, and fine segmentation is performed using the DB-Scan algorithm. The final result is a point cloud that represents the entire battery module (see Fig. 5).

Fig. 5
figure 5

Result of the 3D-acquisition of a battery module (type 1)

Full size image

3.2.3 Registration

The objective of the registration process is to align the reference model with the actual battery module by locating each point of the reference model on the real battery module. This allows a robot to approach any selected point during the disassembly planning phase. To achieve high accuracy and robustness in the presence of geometrical deviations while maintaining an acceptable computation time, a concept that incorporates two different registration algorithms is proposed. The non-rigid registration algorithm (BCPD) offers high accuracy in registering geometric deviations in the point clouds, but its computational intensity and long run time make it less practical for large point clouds. To mitigate this issue, a rigid registration using the TEASER++ algorithm is performed prior to the BCPD algorithm. This roughly aligns the two point clouds, reducing the total computation time.

The computation time during the rigid registration is kept to a minimum by downsampling the point cloud, which reduces the point density and only changes during the computation. A FPFH descriptor is used to determine features, offering robustness and low computational time. Correspondences between the two point clouds are established using the nearest neighbor search in the feature space, which is accelerated by storing the features in a K-d tree. Once the correspondences are determined, the TEASER++ algorithm is employed to perform registration, determining and applying a transformation in the form of translation and rotation.

Following the rough alignment achieved through the rigid registration, the non-rigid registration is performed using the BCPD algorithm, providing the most accurate point localization possible in the presence of geometric deviations. The BCPD algorithm transforms each point individually, enabling the original shape to adapt to the target point cloud. This form of probability density-based registration offers high accuracy and robustness but comes at the cost of high computational time. To optimize the computation time, various approaches have been proposed, including using a K-d tree and the Nyström method, as demonstrated by [9]. However, these optimizations are still not sufficient for registering point clouds with several 100,000 points in a reasonable time. To address this issue, [8] proposed a faster variant, BCPD++, which uses a subset of points for computation and interpolates the result to save computation time without sacrificing accuracy. In the application of battery modules, it can be assumed that the deformations affecting the battery module have a holistic effect and can be described with a lower point density.

The BCPD++ algorithm requires the adaptation of certain parameters to achieve optimal results for the application, such as the convergence limit, which decides when the algorithm terminates. The improvement of the registration result is estimated at each iteration, and the algorithm terminates when the improvement falls below a certain threshold. To further reduce the computation time, the point cloud density is reduced by downsampling before applying the BCPD++ algorithm. A good registration result can be achieved with an acceptable computation time by selecting a sufficient minimum number of points for the downsampling process, as demonstrated by Fig. 6. Depending on the accuracy requirements of the disassembly process the point cloud density can be determined.

Fig. 6
figure 6

Relation between runtime and registration error of BCPD++ algorithm

Full size image

3.2.4 Transformation

In the disassembly process, an effective target point approach by the robot requires the conversion of points in the Camera Cartesian Coordinate System (CCS) to the Robot base CCS. This conversion requires determining a transformation matrix through an Eye-to-Hand calibration process.

This calibration involves capturing images of a calibration object, such as a distinctively featured chessboard, from various perspectives. Figure 7 displays a 3D image of the chessboard pattern with reference points (left) and a 3D image of the successfully calibrated system (right).

Fig. 7
figure 7

(Left) Chessboard with reference points. (Right) Successfully calibrated system

Full size image

The images obtained are processed to compute the transformation matrix \(M_{CAM}^{ROB}\), mapping points from the Camera CCS to the Robot base CCS. This transformation is governed by the equation (1), which can be solved using the method according to [10].

$$\begin{aligned} A*X = X*B \end{aligned}$$
(1)

where:

  • A and B represent transformations between successive robot and camera poses in their respective CCS.

  • X is the sought transformation \(M_{CAM}^{ROB}\) from the robot base to the camera in CCS.

With the computed transformation matrix, the Camera CCS points can be converted to the Robot base CCS through matrix multiplication, enabling the robot to accurately approach the target points. This method can also be used to determine the transformation matrix \(M_{OBJ}^{CAM}\), which maps points from the Object CCS to the Camera CCS.

4 Results

In this chapter, we delineate the evaluation results, underscoring the robustness and efficacy of our data processing pipeline in the precise determination of target points. Our results pinpoint an accuracy within the one to two-millimeter range when leveraging an AMD Ryzen 7 3800x 8-core 3.9 GHz processor, all within a swift runtime of mere seconds. The visual representation in Fig. 8 accentuates our process’s precision in registering and localizing points, even amidst pronounced geometric variations, such as displacements surpassing one centimeter.

It is important to define the scope of this work: our primary focus is on the accurate localization of target points for robotic applications. The trajectory planning of the robotic arm and its subsequent movement toward the identified target points is a separate research domain. While our methodology competently identifies these target locations, the nuanced task of the robotic arm to accurately act upon these points warrants dedicated exploration and is suggested as an avenue for future research in the concluding sections of this paper.

Fig. 8
figure 8

Localization of CAD points on a real battery module (type 2)

Full size image

A detailed overview of the hyperparameters used in our pipeline is provided in Table 1. For certain parameters such as kernel, the number of points used, and convergence limit, we created three distinct settings to highlight the relationship between speed and accuracy. These settings encompass a variant (A) prioritizing speed, a variant (C) aiming for maximum accuracy, and a balanced variant (B). Other parameters have been optimized and fixed across all variants, providing a stable foundation for the algorithm’s performance.

Table 1 Algorithm parameters
Full size table

We conducted a practical test by artificially increasing the distance between cells of a pouch cell, simulating the breathing effect-induced deformation on battery module type 1 (refer to Fig. 5). The resulting discrepancy in thickness between the reference model and the disassembled battery module exceeded one centimeter. Despite this substantial deformation, we achieved a remarkable accuracy of 1.35 mm.

The outcomes, documented in Table 2, were derived using a point cloud of approximately 300,000 points. Factors such as the size of the battery module and the hardware used significantly impacted the runtime, which directly correlates with the number of points. An interesting observation from our study was that a slight decrease in accuracy could significantly reduce runtime, particularly in the balanced variant (B). This was achieved by reducing the number of points utilized in the BCPD++ algorithm.

Table 2 Accuracy and runtime for three different parameter combinations in the registration process
Full size table

However, it’s important to note that our accuracy determination method, which is based on manually set ground truth points, likely incorporates an error tolerance. Consequently, we estimate the actual accuracy to be slightly less than one millimeter. Additionally, our approach cleverly incorporates preprocessing and merging during the battery module’s turning process, effectively reducing the actual required runtime by approximately two seconds.

5 Conclusion and outlook

The primary objective of this computer vision pipeline is to facilitate the automated disassembly of Li-ion battery modules with varying geometries, utilizing a 3D camera. The path planning for separation tasks, such as milling, is executed on the basis of a CAD model. To achieve precise localization of support points on the deformed surfaces for robot-assisted milling operations, an approach involving rigid and non-rigid registration techniques is employed. This method aligns the CAD model with the actual deformed battery module, enabling accurate and efficient milling operations.

A Bayesian Coherent Point Drift (BCPD) algorithm forms the core of the pipeline. This algorithm performs a non-rigid registration based on probability density and its runtime must be minimized to make it suitable for industrial disassembly applications. The TEASER++ algorithm was used for pre-registration, which increased the reliability for initial rotation differences and reduced runtime by several factors. Furthermore, the BCPD++ variant of the algorithm by [8] was employed, which reduces the point density during the calculation of the registration. The results indicated that the computation of the non-rigid registration for battery modules and their deformations can be performed with a limited number of points without significant loss in accuracy, resulting in further runtime savings.

It should be noted that there are certain deformations where a low point density may not be sufficient for precise localization. There is always a trade-off between accuracy and runtime, and this can be adjusted in the pipeline to meet specific requirements. In addition to the BCPD algorithm, successful registration requires 3D acquisition and reference model generation. To provide greater flexibility, two different methods for generating a reference model have been developed, eliminating the need for a CAD model.

The proposed data processing pipeline allows for localization with an accuracy of 1–2 mm in just a few seconds, without the need for additional models. This opens up potential applications in the industrial disassembly of deformed battery modules and other related fields. Further steps to optimize both accuracy and runtime can be taken, such as utilizing the position of the clamping unit to obtain boundary conditions for the rough registration and parallelizing computational processes on a graphics card. Another promising approach is to focus the non-rigid registration on the relevant parts of the battery module, such as those required for the current milling operation, to further improve the efficiency of the process. Furthermore, detailed benchmark tests to evaluate the accuracy under different boundary conditions (lighting conditions, additional battery modules...) can be performed with the given test setup. Another critical frontier to be explored is the precise engagement of the robotic arm with these localized points. Ensuring the seamless movement and interaction of the robotic arm based on the identified target points will form the basis of effective automated disassembly in real-world scenarios .