Low cost three-dimensional virtual model construction for remanufacturing industry
- 70 Downloads
Remanufactured products can save up to 80% of production and energy costs whilst generating lower CO2 emissions. The key success factors for remanufacturing are quality, lead-time and cost. Extensive work within the industry and the detailed analysis of the remanufacturing process has shown that component inspection has significant bearing on overall productivity. Remanufacturing lacks automation because activities are predominantly manual. Automation of remanufacturing process will not only decrease the number of non-remanufacturable components, through decreasing cost and increasing consistency in quality, but also attract industries to design for remanufacture. A digital model of the component is required to automate the disassembly process and move towards industry 4.0 and cyber physical systems. There are several expensive techniques to create a digital model, which are not feasible for the remanufacturing industry. The research paper aims to check feasibility of using Visual Structure for Motion (VFM), a relatively low cost method, to develop a 3D digital model, for automation of the automotive engine (in as received condition) disassembly process using industrial robots. These experiments assess the scientific feasibility of using Videogrammetry to acquire pre-disassembly 3D model of the engine. Multiple 2D images were acquired and processed to find matching common features. The location of the camera was calculated through the matching features, producing a three-dimensional digital representation of the captured volume. A sparse point cloud was initially created and was then converted into a dense 3D point cloud. The 3D point cloud was converted into a meshed model. 2D images were stitched together to create a virtual model of the engine with surface texture and colour. Small features were clearly visible in the 3D model.
KeywordsRemanufacturing 3D reconstruction Machine vision Digital manufacturing
Visual Structure for Motion
Cloud-based design manufacturing
Internet of things
Computer Aided Design
Structure for Motion
Open Multiple View Geometry
Scale-invariant feature transform
RANdom SAmple Consensus
Remanufactured products can save up to 80% in production and energy costs whilst generating lower CO2 emissions. Up to 85% of a remanufactured product’s weight can come from used parts thus reducing environmental and recycling impacts. The key remanufacturing success factors are quality, lead-time and cost . Currently, the remanufacturing process is performed manually. It is crucial to apply emerging technologies and digital manufacturing systems to remanufacturing, to exploit its full potential, whilst reducing lead time and cost [5, 6, 8, 16]. In current work, initial experiments are performed to assess the feasibility of Videogrammetry to acquire pre-disassembly three dimensional (3D) model of the engine. The application of this vision based technique will help automate the remanufacturing process. Integration of machine vision will help in metrology, machine learning, robot path planning and thus automate disassembly, quality control and other processes. 3D reconstructed model with colour texture will not only aid machine vision tools to be easily implemented, but also will pave way for the use of machine learning algorithms. Both these methodologies, machine vision and machine learning, will be used for robust identification and calculating location of features, for example, bolts type and location. This information will then be communicated to the robot to perform a particular operation, for example, selection of a particular tool to unscrew the bolt. Accuracy of the process will also be assessed in the second phase of research. 3D model of the engine, acquired using a conventional 3D scanner, will be compared to the 3D reconstructed model using VSFM technique.
This study discusses implementation of digital manufacturing in remanufacturing, whilst highlighting a new non-contact metrology technique. Suitability of several metrology techniques, currently being researched under the umbrella of digital manufacturing are discussed with respect to remanufacturing. VSFM, one of these emerging techniques is then selected because of low cost and efficiency. Future research is planned based on this initial study.
Implementation of digital manufacturing
The most important strategy to deal with the rising threat of climate change is to opt for a low carbon economy which will enable reduction in pollution, emission of greenhouse gases and energy consumption [2, 15]. Recent legislation and engineering accomplishments have reduced automotive emissions. Improved automotive component utilization, specifically the engine, is being achieved through remanufacturing. . It is very important to introduce systems of digital manufacturing into remanufacturing. This work is part of an ongoing research focusing on remanufacture of engines, using robots and digital technology. The implementation of cost effective digital technologies will help increase range of the remanufactured products.
Cloud-based design manufacturing (CBDM) systems incorporate communication and information infrastructure and employs the internet of things (IOT) to merge design and manufacturing related data . The concept of CBDM can be directly used to remanufacture products, in areas such as design for remanufacturing, reverse supply chain management, information management, automation, inspection and metrology. The essence of this system is to present physical objects (e.g. an engine) in digital form, and connect humans with machines . These systems can then be used to automate the remanufacturing process. Metrology has immense importance to automate the process, as it provides data which is used in robotic path planning such as feature size and location. Metrology has been widely used in many areas to capture the dimensional deviations of parts geometries .
Comparison of non-contact metrology techniques
Inspection range (m)
Speed (points per sec)
Structured Light Scanner
up to 2
up to 0.1
up to 1,300,000
up to 0.05
Laser tracked scanner
up to 35
up to 10,000
up to 0.016
up to 3.65
up to 450,000
up to 205,000
up to 0.1
3D Structure from motion
up to 1000
8688 × 5792 pixels
up to 255,045
up to 0.72
In general, 3D reconstruction pros are, its comparatively of low cost (£500), has higher inspection range (1000 m) and higher resolution. Cons are, it has relatively lower scanning speed (not overall process speed) (255,045 points per sec) and low information, on how to make scans more accurate. This study lays the basis of figuring out the influential parameters to fill this research gap.
Data acquisition through 3D reconstruction
It is now possible to reconstruct component’s geometry due to the advancements in metrology data acquisition and digitization techniques . 3D reconstruction is widely used for large scale buildings and cultural heritage, by means of photogrammetry methods that produce significant improvements in accuracy and scalability. In current study, Multi-view stereo (MVS) system is used to 3D reconstruct the engine geometry. MVS data is generally very detailed ([6, 17, 20, 23]; C. ). This technique has not been used in the context of manufacturing or remanufacturing. Appropriately reconstructed components can decrease the cost of remanufacturing by facilitating metrology and robot path-planning. Moreover, remanufacturers often do not have access to component’s original Computer Aided Design (CAD) files and technical drawings, required to remanufacture the components. 3D reconstruction techniques also provide additional data, such as surface texture and colour (C. ), which will help to digitize remanufacturing. This is another reason MVS and Structure for Motion (SFM) techniques have been preferred to other non-contact metrology techniques. These techniques were applied to reconstruct the surface geometry and texture of an “as received” automotive engine.
Structure for motion
SFM integrates point matching, feature extraction and existing knowledge of vision systems [4, 23]. Generally, SFM has been used either with limited number of images developed by academic researchers or using costly commercial software such as Autodesk remake , Agisoft  to combine large numbers of images. Currently, open-source Structure for Motion (SFM) packages such as Visual Structure for Motion (VSFM) (C. ) and Open Multiple View Geometry (OpenMVG)  are available. It is a low-cost method generally used in multi temporal surveys . These packages offer fast and free processing of several thousand images and produce results comparable to commercial software.
Currently, the disassembly of the engine, for remanufacturing is done manually. Labour and training cost are high to perform the disassembly process. Injury rate due to oil spillage and heavy components is quiet high as well. In the proposed setup, the disassembly process will be performed through an industrial robotic arm. Without digital model, the industrial robotic arm will not be able to identify and locate bolts, nuts and other components to be disassembled. After identifying a component (e.g. bolt), a particular end-effector (tool) will be selected and brought to the specified location (calculated through the 3D model). This low cost 3D digitization technique will enable the machine vision and artificial intelligence components of the system to select a tool and move it in a particular location.
Aim and methodology
The main purpose of this research work is to demonstrate the usability of SFM technique for remanufacturing, by 3D reconstructing an automotive engine. Effects of luminous intensity and number of 2D images was observed during the experiments. In the next step of this research, the authors intend to improve and integrate 3D reconstruction, machine vision and artificial intelligence to robotics for remanufacturing.
In this work, video-based photogrammetry (Videogrammetry) was used to acquire data. Videogrammetry is already applied in industrial manufacturing and robotics . SFM techniques are not fool proof; various parameters such as dynamic background, rapid movement of the camera, shiny surfaces and extreme changes in light can complicate the reconstruction process. Different types of features, background and different amounts of overlap between the pictures affect the reconstruction quality and time. Thus a video was taken with consistent background and lightning. A 3D model was reconstructed through the SFM technique using a sequence of images, without prior knowledge of the camera pose (location, orientation and field of view). Although it uses several modern techniques for detection of key points and dense reconstruction, it also borrows methods developed for classic photogrammetry, such as self-calibrating bundle adjustment  to automatically estimate the camera pose. Viewing parameters and 3D structure of a scene was simultaneously solved through SFM while using multiple feature recognition algorithms and computer vision techniques. The output meshed model was saved in Stereo-Lithography format (STL), which can be used for manufacturing and remanufacturing purposes.
VSFM compares every image in a set with every other one to find the best match. Although this technique guarantees to find the best match, the time taken to complete the 3D reconstruction increases exponentially with the number of pictures. Matches were not found with inadequate light on the engine and thus the 3D model was not created for this case. The advantage of using a video footage is that it restricts the comparison within neighbouring images. Thus, reconstruction time is reduced to a linear relationship, which allows for much larger datasets to be used in the reconstruction. This approach provided with necessary overlap between the pictures and did not compromise the performance of the Scale-invariant feature transform (SIFT) algorithm, as VSFM resize images and corrects radial distortion before processing.
Currently, the disassembly of the engine for remanufacturing is done manually. Labour and training cost are high to perform the disassembly process. Injury rate due to oil spillage and heavy components is quiet high as well. In the proposed setup, the disassembly process will be performed through an industrial robotic arm. Without digital model, the industrial robotic arm will not be able to identify and locate bolts, nuts and other components to be disassembled. Intellectual property right (IPR) issues might affect the relationship of the remanufacturer with the original equipment manufacturer (OEM). The 3D model generated from the VSFM technique is used for disassembly process only, which does not influence IP issues. Similar IP issues will be faced by other digitization techniques if used inappropriately. More research is required in the security of cyber physical system to avoid information leakages.
After identifying a component (e.g. bolt), a particular end-effector (tool) will be selected and brought to the specified location (calculated through the 3D model). This low cost 3D digitization technique will enable the machine vision and artificial intelligence components of the system to select a tool, perform metrology, inspection and robot path planning, to move it in a particular location.
Influential input parameters to 3D reconstruct the digital model were decided on the basis of current research. These parameters are luminous intensity and number of 2D images to create the model. For example, the current system was not able to generate a dense 3D point cloud and 3D model without proper light. Similarly, the reconstruction simulation process crashed with higher number of images. Further experiments were then planned and performed based on this information.
Currently, a machine learning software is being trained on a testing rig to identify different types of bolts. Research challenge is to identify damaged bolts. Both 3D reconstruction and machine learning codes will help to solve this problem. This information (type of bolt) will then be communicated to the robot to perform a particular operation, for example, selection of a particular tool to unscrew the bolt.
Generally, remanufacturers do not have access to the component’s technical drawings, CAD models, surface features and texture. Conventional optical metrology methods used in the manufacturing industry to acquire this data are expensive, time consuming and require expertise. Experiments were performed to check the feasibility of using Videogrammetry with a low-cost 3D reconstruction method VSFM and determine influential parameters effecting the process. A 3D Virtual model of an automotive engine including geometry, surface texture and colour was developed using a cell-phone camera. Future work will further evaluate these parameters and use this process for metrology, automated disassembly, robotic path planning and inspection purposes in order to enhance remanufacturing productivity.
This work was supported by the Engineering and Physical Sciences Research Council funding for Autonomous Inspection in Manufacturing & Remanufacturing (AIMaReM) [EP/N018427/1]. The authors will also like to thank Dr. Rahul Summan for his suggestions.
- 7.Crandall, D., Owens, A., Snavely, N., & Huttenlocher, D. (2011). Discrete-continuous optimization for large-scale structure from motion. Paper presented at the Computer Vision and Pattern Recognition (CVPR), 2011 IEEE Conference onGoogle Scholar
- 10.Flores D, Marcillo D, Pereira J (2017) 3D scanner based on an autonomous Wi-fi unmanned mini Quadcopter. In: Rocha Á, Correia AM, Adeli H, Reis LP, Costanzo S (eds) Recent advances in information systems and technologies: volume 3. Springer International Publishing, Cham, pp 551–558CrossRefGoogle Scholar
- 11.Foster I, Zhao Y, Raicu I, Lu S (2008) Cloud computing and grid computing 360-degree compared. Paper presented at the grid computing environments workshop, 2008. In: GCE'08Google Scholar
- 13.Francis CM (2016) Enhancing cultural resource documentation with terrestrial photogrammetry. Northern Arizona UniversityGoogle Scholar
- 17.Hugenholtz, C. H., Whitehead, K., Brown, O. W., Barchyn, T. E., Moorman, B. J., LeClair, A., . . . Hamilton, T. (2013). Geomorphological mapping with a small unmanned aircraft system (sUAS): feature detection and accuracy assessment of a photogrammetrically-derived digital terrain model. Geomorphology, 194, 16–24CrossRefGoogle Scholar
- 23.Shan Q, Wu C, Curless B, Furukawa Y, Hernandez C, Seitz SM (2014) Accurate geo-registration by ground-to-aerial image matching. Paper presented at the 3D vision (3DV). In: 2014 2nd international conference onGoogle Scholar
- 25.Stull, R. B. (2000). Meteorology for scientists and engineers: a technical companion book with Ahrens' Meteorology Today: brooks/ColeGoogle Scholar
- 26.Triggs, B., McLauchlan, P. F., Hartley, R. I., & Fitzgibbon, A. W. (2000). Bundle adjustment — a modern synthesis. In B. Triggs, A. Zisserman, & R. Szeliski (Eds.), Vision algorithms: theory and practice: international workshop on vision algorithms Corfu, Greece, September 21–22, 1999 proceedings (pp. 298–372). Berlin, Heidelberg: Springer Berlin HeidelbergCrossRefGoogle Scholar
- 28.Wu, C. (2011). VisualSFM: a visual structure from motion systemGoogle Scholar
- 29.Wu, C. (2013). Towards linear-time incremental structure from motion. Paper presented at the 3DTV-Conference, 2013 International Conference onGoogle Scholar
- 30.Wu C, Frahm J-M, Pollefeys M (2010) Detecting large repetitive structures with salient boundaries. Computer Vision–ECCV 2010:142–155Google Scholar
Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.