Skip to main content

Tolerance analysis for robotic pick-and-place operations

Abstract

Robotic workcell design is a complex process, especially in case of flexible (e.g., bin-picking) workcells. The numerous requirements and the need for continuous system validation on multiple levels place a huge burden on the designers. There are a number of tools for analyzing the different aspects of robotic workcells, such as CAD software, system modelers, or grasp and path planners. However, the precision aspect of the robotic operation is often overlooked and tackled only as a matter of manipulator repeatability. This paper proposes a designer tool to assess the precision feasibility of robotic pick-and-place workcells from the operation point of view. This means that not only the manufacturing tolerances of the workpiece and the placing environment are considered, but the tolerance characteristics of the manipulation and metrology process (in case of flexible applications) as well. Correspondingly, the contribution of the paper is a novel tolerance modeling approach, where the tolerance stack-up is set up as a transformation chain of low-order kinematic pairs between the workpiece, manipulator, and other workcell components, based on manipulation, seizing, releasing, manufacturing, and metrology tolerances. Using this representation, the fulfillment of functional requirements (e.g., picking or placing precision) can be validated based on the tolerance range of corresponding chain members. By having a generalized underlying model, the proposed method covers generic industrial pick-and-place applications, including both conventional and flexible ones. The application of the method is presented in a semi-structured pick-and-place scenario.

Introduction

Real manufactured and assembled products only match their nominal design within certain tolerances. Geometric and dimensional deviations are caused by imprecise manufacturing and assembly processes, resulting in imperfect workpiece shapes as well as inaccurate relative positions between them. These deviations influence the assemblability and functions of the product, and thereby the fulfillment of functional requirements (FRs). Therefore, the success of assembly processes and the resulting mechanical assemblies is significantly affected by the corresponding tolerance design. Tolerance analysis (or tolerance stack-up analysis) addresses this problem through the determination of the dimensional and geometrical variation of the final assembly from the given tolerances on individual components and on the created joints. Based on this, both the satisfaction of the defined FRs and the assemblability of the product can be verified [1, 2].

In case of manual assembly, proper geometric design and allocation of tolerances ensure that the worker will be able to assemble the product, while maintaining the product’s key characteristics (KCs) [3]. This concept is viable due to the assembly capabilities and dexterity of human workers (supported by suitable tools). On the other hand, as a manual assembly process becomes robotized, the robot capability needs to be taken into account when designing the assembly and corresponding robotic operation. However, by losing the dexterity and skills of the worker, the result of the assembly is determined not only by the robot repeatability (in addition to the tolerance of assembly components), but also by the tolerance of the involved cell components. Hence, the tolerance aspect of the whole operation needs to be addressed, which can affect the manipulator and equipment selection, or even the assembly design. Yet, the tolerance design for this scenario does not receive much support. Indeed, related issues are often overcome by applying much more precise equipment than necessary, or by trial and error.

The present research addresses the subject of tolerancing in robotic manipulation. Correspondingly, the main contribution of this paper is a novel tolerance analysis method for validating robotized operations in terms of tolerances, in order to overcome the above-mentioned issues. The proposed method aims to assess the feasibility of waypoint-based robotic applications, particularly the pick-and-place operation—including placement and insertion—which can be considered one of the most common assembly tasks [4]. The tolerance model is prepared for a general pick-and-place representation, capable of handling both conventional and flexible (e.g., bin-picking) tasks. Apart from mechanical tolerances, the latter involve the precision of the metrology system used to resolve uncertainties (such as the workpiece picking pose in case of bin-picking) and to realize visual servoing.

In the proposed model, the tolerance stack-up of the operation is set up on a transformation basis using low-order kinematic pairs (joints) [5]. The aggregated tolerances—corresponding to FRs—are formulated parametrically through the multiplication of parametric transformation matrices. As the design specifications (i.e., workholding, grasping, manipulation, servoing, and metrology characteristics as well as tolerances) are substituted into these formulae, the operation can be evaluated for feasibility.

This tolerance model provides a basis for Monte Carlo simulation and sensitivity analysis [6]. Using these, the particular workcell setup can be analyzed, allowing the designer to check the suitability of the selected equipment in the early design phase. This representation can also be used during the different planning (path and grasp planning) steps of the robotic workcell, as it can contain the robot kinematics and component relations besides the tolerance stack-up. Thereby, this method fits into the tolerance model of the generic pick-and-place workcell development methodology introduced in the authors’ previous work [7]. This aids developers in setting up feasible tolerance regions for Digital Twins of such robotic workcells and in assessing twin closeness, which indicates whether or not the virtually planned robotic operation can be executed feasibly in the real workcell.

The remainder of the paper is structured as follows. The related literature, regarding general tolerance modeling approaches, manufacturing operation-related approaches and the tolerance aspect of pick-and-place operations is overviewed in Section 2. In Section 3, the concept of the tolerance model is presented, together with the generalized pick-and-place operation, the basic structure of the transformation chain, fundamental FRs, and the pick-and-place-specific tolerance influencing factors. The complete model is formulated in Section 4, including the required input data, the deduction of the transformation chain, and the evaluation of FRs. In Section 5, the implementation of the proposed model is presented through a case study of an experimental, physical, pick-and-place workcell. Finally, conclusions are drawn, and the possible future research directions are presented in Section 6.

Related work

Tolerance modeling

During product design, the customer requirements define the desired product. These are then translated by the designer to FRs, which capture the functional intention of the designer in terms of dimensions and tolerances. The components of the critical FRs are KCs, which are defined as “the product, subassembly, part, and process features that significantly impact the final cost, performance, or safety of a product when the KCs vary from nominal. Special control should be applied to those KCs where the cost of variation justifies the cost of control.” by Thornton [3].

To achieve product feasibility (including assemblability and the fulfillment of FRs), the tolerancing problem needs to be solved [1]. Dimensional tolerances have been for long the primary means for expressing the allowable deviations of workpieces and products. Besides, geometrical tolerances also became formally defined and standardized by the introduction of Geometrical Dimensioning and Tolerancing (GD&T) [8]. Together, these allow the characterization of a variety of deviation types. The key approaches for tolerance analysis are the analytical worst-case and statistical analysis [9, 10], and Monte Carlo simulation [6]. The sought results can be the list of contributors, sensitivity, and effect of each contributor, as well as the tolerance stack-up.

Numerous research works were dedicated to defining new tolerance representation methods in order to establish mathematical models for the expression and representation of geometric deviations. These include the variational model [11], TTRS (technologically and topologically related surfaces) model [12], matrix model [13], vector-loop [14], torsor model [15], Jacobian-torsor model [16], GapSpace [17], T-Map model [18], deviation domain [19], polytopes [20], and Skin model shapes [21], among others. Furthermore, many commercial CAT (computer-aided tolerancing) software packages were developed on the basis of these models, e.g., 3-DCS, VisVSA, CETOL, FROOM, or CATIA.3D FDT [22,23,24]. Detailed summaries on tolerance representations and tolerance analysis approaches are given in [22, 25, 26].

From the above, the most relevant ones are the variational, vector-loop, matrix, and torsor models. These apply homogeneous transformations on an assembly graph to capture the geometric variations. The variational model [11] approach represents the deviations from the nominal geometry due to the tolerances and the assembly conditions through a parametric mathematical model. First, the nominal shape and dimensions of each assembly component are set up with respect to a local datum reference frame (DRF). Then, the components are assembled together using small kinematic adjustments to represent their relative location with respect to each other within the assembly. With this, each feature can be expressed in the same reference (i.e., the global DRF). The FRs can be solved by analytical approaches or Monte Carlo analysis.

The vector-loop model [14, 25] represents the workpiece geometric variability via chains of vectors. The vectors represent component dimensions or kinematical variable dimensions. Three types of deviations are considered: dimensional deviations, geometric feature deviations, and deviations originating from kinematic adjustments (from the assembly process). The vector-loop model results in a set of non-linear equations, which can be evaluated using a worst-case or statistical method when linearized, or using Monte Carlo simulation.

The matrix model [13, 26] is based on TTRS; it transforms the tolerance zones to establish the limit boundaries for tolerances. The aim of the matrix model is to derive an explicit mathematical representation of the boundary of the entire spatial region enclosing all possible displacements originating from the variability sources. The representation needs to be completed by an additional set of inequalities, which define the bounds for every component in the matrix. Being a point-based approach, the result of the matrix model is the variation of a point on a functional surface, and as the boundaries of the region of possible variations are defined (i.e., extreme values), intrinsically a worst-case approach is applied.

The small displacement torsor model [15, 25] is based on the first-order approximation of the matrix model [27]. The displacement of geometric elements is modeled as a translation vector and a linearized rotation matrix arranged into a torsor. Three kinds of torsors are defined: component, deviation, and gap torsor. The global behavior of the assembly can be obtained from the union of the torsors. For the evaluation of the model, a worst-case approach can be applied, similarly to the matrix model.

In the following, relevant tolerance modeling approaches are presented, which are related to manufacturing processes. In the field of precision engineering, a transformation chain–based approach is commonly applied for setting up the kinematic error model of machine tools, for estimating their achievable accuracy, and for analyzing the contributing tolerances [28]. A kinematic model–based approach is presented in [29] to investigate the contributing factors of the location deviations of holes in drilling operations. In [30], a positioning variation model is presented for an eye-in-hand drilling system, considering vision-based positioning error measurement and compensation. Further manufacturing process-related tolerance models include fixturing and the consideration of manufacturing signatures [31, 32]. In the field of robotic machining, a working precision analysis is presented in [33] using a robot-process model, to investigate the robot- and process-related influencing factors.

In case of robotic systems for workpiece manipulation, when considering the precision aspect, the focus is mostly on robot calibration. Tolerance modeling and analysis on the process side receive hardly any attention. In [34], the error model of the peg-in-hole assembly is set up for automated assembly, but it does not cover any errors before the insertion (placing) process. The relevant failure modes of automated assembly systems—involving precise grasping, releasing, and collision—are identified in [35]; however, only manufacturing errors and the robot repeatability are considered in the tolerances. Furthermore, no general model is provided, only the Monte Carlo simulation for error diagnosis.

Although numerous different tolerance representations and analysis approaches exist, to the best of the authors’ knowledge, there is no tolerance model specifically representing robotic manipulation, and particularly the pick-and-place operation, from the operation point of view. Inspired by this, the goal of the present research is to introduce a suitable approach for bridging the identified gap.

Tolerance factors in robotic pick-and-place

After overviewing the relevant tolerance models, in this section, the different contributing factors are gathered, which take part in the tolerance chain of pick-and-place operations. In case of conventional pick-and-place, apart from the cell component (e.g., workholders) and workpiece manufacturing tolerances, the robot positioning precision and the effect of the seizing and releasing processes need to be considered in the tolerance model. These can clearly affect the feasibility of the picking and/or the placing process of the operation.

Robot precision includes repeatability and accuracy, which are both standard robot characteristics [36]. Repeatability plays a key role in the tolerance stack of pick-and-place operations. Robot repeatability is generally provided by the manufacturers for positioning on the robot end flange. In this regard, the determination of position and orientation repeatability is presented in [37]. This characteristic in general can only be improved by using more precise components and component connections, as well as an internal measurement system with higher resolution [38]. On the other hand, accuracy is not as significant as repeatability in terms of tolerance, since in most cases, accuracy-related errors can be sufficiently compensated. Whenever suitable, traditional (lead-through) robot programming can neutralize poor robot accuracy. Moreover, robot calibration [39] is applicable for improving positioning accuracy. Calibration is especially important in case of applications utilizing model-based offline robot programming [7].

Workpiece seizing and releasing processes can also introduce geometric errors during the pick-and-place operation through the contact transitions between the components. This topic is much less studied in the literature, and only a few relevant papers were found. Uncertainty in case of grasping is studied in [40]. In this paper, the concept of self-alignment is introduced, which occurs during workpiece grasping. The phenomenon is studied for one particular workpiece and gripper finger geometry pair. The effect of grasping position on assembly success is analyzed in [41]. The focus of this paper is to investigate the alignment capability of a parallel finger gripper when seizing a cylindrical workpiece with initial position error, and to check whether or not the grasped workpiece can be successfully placed into a workholder. The geometrical conditions of the robotic peg-in-hole problem, as a placing task, are investigated by [42]. Here, the effect of chamfering on the placing side is taken into account as a source of self-alignment mechanism for the workpiece.

In more advanced applications, where metrology systems are applied to resolve uncertainties, other tolerance components are introduced in the tolerance chain. These are in form of equipment resolution, measurement errors, and the precision of data processing algorithms. Equipment resolution is provided by the manufacturers in general, and there are many papers in the literature about the accuracy of different approaches, such as camera calibration and 2D pattern detection [43], or model-based 3D pose estimation [44]. For robotic applications, fewer studies are available. An object recognition and pose estimation framework is presented in [45], including the accuracy measures of the estimated poses. Uncertainty in perception and grasping is taken into account for bin-picking in [46, 47]. The grasp is selected based on how likely its feasibility is, when simulating this uncertainty. Moreover, fine positioning is considered if placing precision is predicted to be insufficient; with a predetermined manipulator motion sequence, certain features of the workpiece are aligned after the placing process.

With the help of metrology systems, it is also possible to compensate positioning errors using servo techniques [48, 49] real-time, during operation. Using a closed-loop control cycle, the target point can be reached with continuous measurement and actuation of the manipulator, until the defined target condition is reached. Servoing can improve the positioning precision of the robot through closed-loop motion control in case of picking and/or placing, without the need for changing the equipment, or prescribing tighter tolerances (i.e., without increasing investment costs). This means that the precision influencing effects in the system are negated, and the resultant precision will depend on the manipulator positioning resolution, and the errors in the metrology system, measured data, and processing algorithms. Servo techniques, and visual servoing in particular, receive great attention in the literature, including robotic pick-and-place as well. Visual servoing is used for picking pose compensation and grasping in [50], and for placing pose compensation in an assembly cell in [51]. Furthermore, compliance control strategy is applied for peg-in-hole insertion in [52].

The listed tolerance factors, together with the corresponding literature, provide a basis for the tolerance model proposed in the present paper, and support the introduced tolerance influencing concepts. Although not all of these factors are thoroughly explored in the literature—especially, the tolerances introduced during contact transition between components, and characteristics of self-alignment while grasping and placing—the main ideas and mechanisms appear to be clear. This allows the preparation of the fundamental tolerance model for robotic pick-and-place considering the operation viewpoint, and more specific tools can be created as the related research progresses further.

Model overview

Concept and assumptions

In this paper, the tolerance stack-up of the pick-and-place operation is modeled as a sequence of transformations in a kinematic graph, which uses the same idea as the one behind machine tool accuracy estimation in the work of Slocum [28]. This concept resembles the definition of a mechanism using standard mechanism joints. The joints are considered as low-order kinematic pairs [5], which are essentially parametric transformations between the frames of components (rigid bodies). The idea is to represent different tolerance types as mechanism joints. In this way, the tolerance propagation can be modeled simply using matrix multiplication. These matrices contain nominal and tolerance values, as well as joint variables (in case of actual mechanism joints, such as the robot joints) as parameters for the underlying translational and rotational transformations. Since multiple frames can be within a single rigid body, connected through tolerances, the shape of the rigid body becomes determined and fixed as a particular tolerance value is selected from the defined tolerance interval.

From the general tolerance modeling methods, the proposed approach is most similar to the variational solid modeling approach [11]. However, in the proposed one, the kinematic joints are applied as matrix transformations (using homogeneous transformation matrices), and the number of constraint equations is minimized. This is done by first preparing the spanning tree of the kinematic graph, then by forming the constraint equations through loop closures (wherever necessary). Hence, open loops can be evaluated simply by substituting in the geometrical parameters, and for closed loops, only a minimum number of constraint equations need to be solved, before evaluating the FRs. Furthermore, this allows the deviation of both the position and orientation components of feature frames to stack up. Even though variation in orientation (e.g., perpendicularity or parallelism) is captured by tolerance zones for geometric tolerances in the standard, the representation of orientation as inclination is beneficial in scenarios such as robotic peg-hole insertion or grasping.

Considering the robotic assembly aspect of the pick-and-place operation, the term workpiece refers to the component, which is manipulated by the robot. Here, the model describes the pose of this workpiece throughout the operation. Hence, the workpiece needs to be attached in the transformation chain in a way to continuously represent its actual physical contacts with the other cell components. Therefore, the workpiece is detached and re-attached in the transformation chain according to the operation process steps (i.e., seizing and releasing). Correspondingly, the workpiece state is investigated before and after the seizing and the releasing actions.

The number of DoFs of the workpiece is restricted throughout the operation by the different components the workpiece is in contact with, namely, by the picking workholder, gripper, and placing workholder. In general, the workpiece does not move relative to the contacting component, except during the contact transition. However, the tolerance stack-up (including the contact transition) can result in deviations in the free directions between the aligned frames of the workpiece and other components. Therefore, these contacts should be captured through the proper parameterization of the transformations, i.e., according to the DoFs of the workpiece relative to the contacting components.

The tolerances appearing in the transformation chain need to be determined using existing tolerance analysis methods (as these are mostly resultant tolerances and not explicitly specified by the designer), or through measurements. It is noted that in case of individual workcell setup, some of the deviation sources—typically the component manufacturing tolerances and the robot accuracy—can be compensated through calibration. In the present paper, every input parameter or parameter interval (in case of tolerances) is assumed to be predetermined, along with the actual pick-and-place strategy (application of metrology and tolerance enhancing techniques). Also, since manufacturers specify the repeatability of the manipulators on their end flange, but not on their links, manipulators are presented as a single unit during tolerance analysis, and not as separate links. It is noted that in case of modular robots, the precision of individual modules could be taken into account, and it is possible to set up the corresponding kinematic chain automatically [53].

The presented model defines a set of possible frames to capture the FRs. To assess the fulfillment of FRs, inequalities need to be formed for each relevant general direction of each relevant frame. Alternatively, if required, inequalities can be formulated specifically for different artifacts of the workpiece or other components. These can be captured via additional frames, or even as geometric constraints. The FRs and included KCs need to be defined by the designer, and these are also assumed to be given in this paper.

Part of the applied functions (originating from tolerance influencing factors, see Section 3.5) throughout the tolerance stack are not continuously differentiable, and the extraction of orientation components has no closed-form solution. Consequently, the prepared tolerance chain is numerically evaluable. The evaluation of the model is carried out using Monte Carlo simulation, by substituting the nominal values, joint variables (if any), and tolerance set instances (sampled from the corresponding tolerance intervals) into the parametric tolerance chain.

The established transformation chain suits the family of robotic pick-and-place applications. When applied for a particular robotic workcell, the chain needs to be prepared by considering the operation conditions and kinematic joints present in the specific scenario. Then, by substituting the dimensional and tolerance parameters, the FRs can be evaluated in the different workpiece states (poses) along the operation sequence. The general structure of the tolerance model is applicable for most industrial pick-and-place tasks—where no humans are involved—in the presented form. For niche cases, the model might need slight adjustments to properly reflect on the particular scenario. The application of this model is most beneficial in case of tasks that are geometrically well defined and where the component relations are clear, and the component selection or design is still adjustable (i.e., in the early design phase).

Pick-and-place operation

A pick-and-place operation can be described as two subsequent (i.e., picking and placing) manipulator movement and gripper setting (or activating and deactivating) steps. From the control perspective, this can be defined using robot configurations; from the point of view of the workpiece, it can be described through poses. For clarification, poses and frames are defined in the robot task space as task space points; however, a pose corresponds to the reference frame of a rigid body, whereas frames correspond to a constant transformation with respect to rigid body references (can be multiple per rigid body). Configurations are defined in the robot configuration space as a robot joint vector.

The important robot configurations in the presented tolerance chain are the seizing and releasing configurations. On the path planning and control side, usually there are approach and retreat configurations for both of these configurations. Additionally, for flexible scenarios, metrology configurations can exist, for the execution of metrology and resolving the pose uncertainties. The seizing configuration corresponds to the workpiece picking pose. In the seizing configuration, the gripper is set to a seizing setup, therefore making contact with the workpiece. At this point, the workpiece gets into the seized pose, which is bound to the gripper. Then, the robot moves to the releasing configuration, transferring the workpiece to the releasing pose. This corresponds to the workpiece placing pose. Here, the gripper is set to a releasing setup; therefore, the workpiece breaks contact with the gripper, and establishes one with the placing workholder.

The component contacts can occur in multiple ways. The type of contact transient during seizing and releasing, as well as component compliances can have significant effect on the seizing and releasing action. If the contact break and establishment happen simultaneously, or one after the other (e.g., seizing with vacuum cup, or dropping workpiece when releasing), then the seizing and releasing action can introduce considerable additional deviations. On the other hand, if the first contact is maintained even after establishing the new contact (until the next robot movement), an overconstrained case occurs, where the components are prone to stuck or damage (e.g., peg-in-hole problem with a rigid gripper). In order to keep the paper more concise, here, the effects of the contact transients are simplified and are only taken into account in case of leading features (see Section 3.5). Nevertheless, these effects need to be considered on a case-by-case basis.

Transformation chain setup

There are relevant frames in a spatial and temporal basis as well. The workcell reference is the base frame, in which each component reference frame is defined. The relation of the frames can be represented on a kinematic graph, which is shown in Fig. 1. Here, the relevant frames are the manufacturing datum frames of the workpiece (wp,ref), picking workholder (wh1,ref), gripper (gr,ref), and placing workholder (wh2,ref).

Fig. 1
figure 1

Kinematic graph of the main workcell components

Component feature frames are defined relative to their reference frames. For the workpiece, these are the picking (wp,pick), grasp (wp,grasp), and placing (wp,place) frames. The pairs of these frames are on the contacting components. The picking workholder has the picking frame (wh1,pick), the placing workholder has the placing frame (wh2,place). The gripper has the grasp frame (gr,grasp); the grasp frame pair is usually the result of the grasp planning.

These frames are shown in case of a sample pick-and-place scenario in Figs. 2 and 3. As visible, the frames exist in different phases of the operation for the moving components. The workpiece poses are the picking (p1), seized (g1), releasing (g2), and placing poses (p2), while the gripper poses are the seizing (corresponding to seizing configuration, g1) and releasing poses (corresponding to the releasing configuration, g2).

Fig. 2
figure 2

Relevant frames in the workcell in a flexible scenario (in this particular case, the transformation between the picking frame pair is determined through metrology)

Fig. 3
figure 3

Relevant frames on the workpiece (in this particular case, the reference, picking and placing frames coincide) (a), and nominal and real grasp frame pairs in g1 or g2 pose (b)

Transformation matrices

Frames are described by homogeneous transformation matrices, parameterized with three translational and three rotational components. However, locally different representations can be selected (e.g., deviation in a cylindrical or Cartesian coordinate system). The tolerance chain includes manipulation, seizing, releasing, manufacturing, and metrology tolerances as parameters.

Homogeneous transformation matrices, realizing transformation from component c1, frame f1, pose o1 to component c2, frame f2, and pose o2, are denoted with \( {\boldsymbol{T}}_{\mathrm{c}2,\mathrm{f}2,\mathrm{o}2}^{\mathrm{c}1,\mathrm{f}1,\mathrm{o}1} \), or simply \( {\boldsymbol{T}}_{\mathrm{c}2,\mathrm{f}2,\mathrm{o}2}^{\mathrm{base}} \) if f1 is the base frame. For the sake of simplicity, pose indices are only noted at the pose changing components (i.e., in case of the workpiece and the gripper). Frame transformations are modeled as follows:

$$ {\boldsymbol{T}}_{\mathrm{c}2,\mathrm{f}2,\mathrm{o}2}^{\mathrm{c}1,\mathrm{f}1,\mathrm{o}1}\left(x,y,z,\xi, \eta, \zeta \right)=\left[\begin{array}{cc}\boldsymbol{R}\left(\xi, \eta, \zeta \right)& \boldsymbol{d}\left(x,y,z\right)\\ {}0& 1\end{array}\right] $$
(1)

where R is the rotation matrix and d is the translation vector containing both nominal (n) and tolerance (t) components:

$$ \boldsymbol{d}\left(x,y,z\right)={\left[x,y,z\right]}^{\mathrm{T}}={\left[{n}_x+{t}_x,{n}_y+{t}_y,{n}_z+{t}_z\right]}^{\mathrm{T}} $$
(2)

The rotational component can be constructed in multiple ways, depending on the order of rotations. Here, a sequence of yaw, pitch, and roll rotations is applied (this is suitable for typical tolerance values, but other conventions can also be applicable in case of singularity issues), which results in the following formula:

$$ \boldsymbol{R}\left(\xi, \eta, \zeta \right)=\left[\begin{array}{ccc}{\mathrm{c}}_{\zeta }{\mathrm{c}}_{\eta }& -{\mathrm{c}}_{\eta }{\mathrm{s}}_{\zeta }& {\mathrm{s}}_{\eta}\\ {}{\mathrm{c}}_{\xi }{\mathrm{s}}_{\zeta }+{\mathrm{c}}_{\zeta }{\mathrm{s}}_{\eta }{\mathrm{s}}_{\xi }& {\mathrm{c}}_{\zeta }{\mathrm{c}}_{\xi }-{\mathrm{s}}_{\zeta }{\mathrm{s}}_{\eta }{\mathrm{s}}_{\xi }& -{\mathrm{c}}_{\eta }{\mathrm{s}}_{\xi}\\ {}-{\mathrm{c}}_{\zeta }{\mathrm{c}}_{\xi }{\mathrm{s}}_{\eta }+{\mathrm{s}}_{\zeta }{\mathrm{s}}_{\xi }& {\mathrm{c}}_{\xi }{\mathrm{s}}_{\zeta }{\mathrm{s}}_{\eta }+{\mathrm{c}}_{\zeta }{\mathrm{s}}_{\xi }& {\mathrm{c}}_{\eta }{\mathrm{c}}_{\xi}\end{array}\right] $$
(3)

where cx stands for cos(x), and sx for sin(x). Each angle value contains a nominal and a tolerance component. The rotation angles (corresponding to the axes of the frame) are represented by the vector r:

$$ \boldsymbol{r}\left(\xi, \eta, \zeta \right)=\left[\xi, \eta, \zeta \right]=\left[{n}_{\xi }+{t}_{\xi },{n}_{\eta }+{t}_{\eta },{n}_{\zeta }+{t}_{\zeta}\right] $$
(4)

The tolerance and nominal parameters are summarized in arrays as:

$$ \boldsymbol{t}=\left({t}_x,{t}_y,{t}_z,{t}_{\xi },{t}_{\eta },{t}_{\zeta}\right),\boldsymbol{n}=\left({n}_x,{n}_y,{n}_z,{n}_{\xi },{n}_{\eta },{n}_{\zeta}\right) $$
(5)

This representation allows the extraction of each angle from the rotation matrix (numerically), their adjustment, and the re-creation of the adjusted rotation matrix, in a consistent way. The corresponding screw parameters of a transformation matrix are represented as an array:

$$ \mathbf{scr}\left(\boldsymbol{T}\left(x,y,z,\xi, \eta, \zeta \right)\right)=\left(x,y,z,\xi, \eta, \zeta \right) $$
(6)

In the following, for the sake of simplicity, the arguments of the transformations are only spelled out where relevant.

As an example, if a finger gripper with parallel finger planes seizes a slab-like feature on a workpiece (see Fig. 3); a planar kinematic joint needs to be set up to represent the tolerance of the seizing action. In this case, due to the planar contact, positioning tolerances from the gripper activation occur only in x and z direction, and orientation tolerance around the y axis (based on Fig. 3). The joint is formulated as follows:

$$ {\boldsymbol{T}}_{\mathrm{gr},\mathrm{g}\mathrm{rasp},\mathrm{g}1}^{\mathrm{wp},\mathrm{g}\mathrm{rasp},\mathrm{g}1}=\left[\begin{array}{cc}\boldsymbol{R}& \boldsymbol{d}\\ {}0& 1\end{array}\right]=\left[\begin{array}{cccc}{\mathrm{c}}_{\eta }& 0& {\mathrm{s}}_{\eta }& x\\ {}0& 1& 0& 0\\ {}-{\mathrm{s}}_{\eta }& 0& {\mathrm{c}}_{\eta }& z\\ {}0& 0& 0& 1\end{array}\right] $$
(7)

which contains both nominal and tolerance parameters, and can be evaluated to any tolerance set instance.

$$ {\boldsymbol{T}}_{\mathrm{gr},\mathrm{g}\mathrm{rasp},\mathrm{g}1}^{\mathrm{wp},\mathrm{g}\mathrm{rasp},\mathrm{g}1}=\left[\begin{array}{cccc}{\mathrm{c}}_{n_{\eta }+{t}_{\eta }}& 0& {\mathrm{s}}_{n_{\eta }+{t}_{\eta }}& {n}_x+{t}_x\\ {}0& 1& 0& 0\\ {}-{\mathrm{s}}_{n_{\eta }+{t}_{\eta }}& 0& {\mathrm{c}}_{n_{\eta }+{t}_{\eta }}& {n}_z+{t}_z\\ {}0& 0& 0& 1\end{array}\right] $$
(8)

Formulation of the transformation chain

Having prepared the construction of transformation matrices, the transformation chain can be formed. First, considering nominal transformations (denoted with T), an ideal case is assumed. The workpiece starts in the picking pose, where its picking frame is aligned with the picking frame of the picking workholder \( \left({{\boldsymbol{T}}^{\ast}}_{\mathrm{wp},\mathrm{p}\mathrm{ick},\mathrm{p}1}^{\mathrm{base}}={{\boldsymbol{T}}^{\ast}}_{\mathrm{wh}1,\mathrm{p}\mathrm{ick}}^{\mathrm{base}}\right) \).Thereby, the nominal grasp frame of the workpiece is determined through the workpiece reference frame \( \left({{\boldsymbol{T}}^{\ast}}_{\mathrm{wp},\mathrm{grasp},\mathrm{p}1}^{\mathrm{base}}\right) \). Then, the robot is commanded to align the gripper grasp frame with the grasp frame of the workpiece\( \left({{\boldsymbol{T}}^{\ast}}_{\mathrm{gr},\mathrm{g}\mathrm{rasp},\mathrm{g}1}^{\mathrm{base}}={{\boldsymbol{T}}^{\ast}}_{\mathrm{wp},\mathrm{g}\mathrm{rasp},\mathrm{p}1}^{\mathrm{base}}\right) \). The gripper is activated, and the workpiece is detached from the picking workholder and attached to the gripper with the grasp frames aligned, as it gets to the seized pose \( \left({{\boldsymbol{T}}^{\ast}}_{\mathrm{wp},\mathrm{g}\mathrm{rasp},\mathrm{g}1}^{\mathrm{base}}={{\boldsymbol{T}}^{\ast}}_{\mathrm{gr},\mathrm{g}\mathrm{rasp},\mathrm{g}1}^{\mathrm{base}}\right) \).

Next, the robot is commanded to the releasing configuration, where the placing frame of the now attached workpiece is aligned with the nominal placing frame of the placing workholder\( \left({{\boldsymbol{T}}^{\ast}}_{\mathrm{wp},\mathrm{place},\mathrm{g}2}^{\mathrm{base}}={{\boldsymbol{T}}^{\ast}}_{\mathrm{wh}2,\mathrm{place}}^{\mathrm{base}}\right) \). Here, the gripper is set to release the workpiece, detaching the workpiece from the gripper and attaching it to the placing workholder, as it gets to the placing pose. At this point, the workpiece is laid into the placing pose. The placing frame of the workpiece and the placing workholder meet \( \left({{\boldsymbol{T}}^{\ast}}_{\mathrm{wp},\mathrm{place},\mathrm{g}2}^{\mathrm{base}}={{\boldsymbol{T}}^{\ast}}_{\mathrm{wh}2,\mathrm{place}}^{\mathrm{base}}\right) \), and the pick-and-place operation is finished.

From here, assuming tolerances, the tolerance chain is formed on four branches, which are then connected. (i) The workpiece grasp frame for the picking pose is formed by the tolerance stack-up from the base frame through the picking workholder and the workpiece \( \left({\boldsymbol{T}}_{\mathrm{wp},\mathrm{grasp},\mathrm{p}1}^{\mathrm{base}}\right) \). (ii) The gripper grasp frame for the seizing pose is formed from the base through the robot links and gripper \( \left({\boldsymbol{T}}_{\mathrm{gr},\mathrm{g}\mathrm{rasp},\mathrm{g}1}^{\mathrm{base}}\right) \). (iii) The workpiece placing frame is formed for the releasing pose from the base frame through the robot links, gripper, and workpiece \( \left({\boldsymbol{T}}_{\mathrm{wp},\mathrm{grasp},\mathrm{p}2}^{\mathrm{base}}\right) \). And (iv) the placing frame of the placing workholder is formed from the base frame \( \left({\boldsymbol{T}}_{\mathrm{wh}2,\mathrm{place}}^{\mathrm{base}}\right) \). As the robot is commanded to the nominal workpiece grasp frame in the picking pose, and the gripper seizes the workpiece, the grasp frames (i–ii) are now misaligned because of the tolerances. These misalignments can further increase as the two frames are getting attached, due to the contact transition (seizing tolerance). This effect is not present in the nominal case; however, in reality, the workpiece pose changes during seizing (and also during releasing) due to the physical contact. The gripper-workpiece attachment (see Figs. 4 and 5) allows the formation of the workpiece placing frame when manipulated to the nominal placing pose.

Fig. 4
figure 4

Initial transformation chain before the workpiece seizing (a), and the workpiece laying in picking pose (p1) with aligned nominal grasp pair (b)

Fig. 5
figure 5

Transformation chain after seizing (a), and the workpiece in the seized pose (g1), detached from the picking workholder and attached to the gripper (b)

The workpiece is then manipulated to the releasing pose, introducing further deviations through the robot positioning precision. Finally, when releasing the workpiece, an additional releasing tolerance is considered. With this, the misaligned placing frames (iii–iv) are getting attached (see Figs. 6 and 7). The measure of misalignment in the placing frame pair \( \left({\boldsymbol{T}}_{\mathrm{wp},\mathrm{p}\mathrm{lace},\mathrm{p}2}^{\mathrm{wh}2,\mathrm{p}\mathrm{lace}}\right) \) thus becomes evaluable in tolerance set instances.

Fig. 6
figure 6

Transformation chain after the robot moves to the releasing configuration (a), and the workpiece in the releasing pose (g2) attached to the gripper (the releasing pose has an offset above the workholder for better visualization) (b)

Fig. 7
figure 7

Transformation chain after releasing (a), and the workpiece in the placing pose (p2), detached from the gripper and attached to the placing workholder (b)

It is noted that the robot is not necessarily commanded to reach exactly \( {{\boldsymbol{T}}^{\ast}}_{\mathrm{wp},\mathrm{grasp},\mathrm{p}1}^{\mathrm{base}} \) or \( {{\boldsymbol{T}}^{\ast}}_{\mathrm{wh}2,\mathrm{place}}^{\mathrm{base}} \). A constant offset can also be considered between the grasp and placing frame pairs, which is overcome by workpiece relative motion when being seized or dropped. However, for the sake of simplicity, no additional frames are introduced for these, and their effects are considered in the seizing and releasing tolerances.

Main functional requirements

To achieve a successful pick-and-place operation, multiple FRs need to be satisfied. FRs are arranged into arrays of feasible intervals, similarly to the nominal and tolerance parameters in formula (5) and are denoted with c. First and foremost, the workpiece needs to be placed into the placing pose within a given tolerance range in each direction (cplace). The allowed deviation for the placing pose is generally determined by the design of the assembly that contains the workpiece, or by the following process, for which the placed workpiece serves as an initial condition.

In addition, the FR for picking has to be met (cpick). Usually, this can be defined based on the gripping range within which the gripper can seize the workpiece. For example, in case of a vacuum gripper, the vacuum cup must be located within the specific planar surface; otherwise, the vacuum cup falls to the edge of this surface and the gripping will fail.

Depending on the task, additional frames (e.g., multiple feature frames for placing) and FRs can be defined. There can be different configurations and geometric artifacts, for which the feasibility needs to be checked (e.g., to avoid collision in different poses). Furthermore, FRs can be formulated not only as simple intervals, but also as geometric constraints. These need to be specified on a case-by-case basis, allowing the preparation of more detailed evaluation of operation feasibility.

Tolerance influencing factors

The tolerance chain does not only depend on the dimensional conditions of the pick-and-place scenario. Multiple other factors were identified that have a significant effect on the tolerance chain. These are defined in each direction separately, as follows:

  • Leading feature (i.e., self-alignment and self-location): when there is relative motion between the components (during seizing or releasing), depending on the physical contact between specific faces, edges and vertices, the workpiece can be either free or guided.

  • Pose specification: the picking and placing poses can be either known or not known with sufficient precision in the design phase.

  • Servoing: servo technique(s) can be either applied or not applied for robot positioning in case of the picking and placing poses.

The decisions on the tolerance influencing factors are arranged into arrays of Boolean values (similarly as in formula (5)) denoted with k, while the tolerance modifying functions are denoted with F(k, T), where T is the transformation matrix on which the modification is applied. The separate application of these functions to each translational and rotational direction is possible due to the selected rotational matrix formulation in equation (3). The effect of tolerance influencing factors on the operation sequence is shown in Fig. 8.

Fig. 8
figure 8

Simplified operation sequence (shown only in one direction) of flexible pick-and-place from the point of view of the tolerance influencing factors

Leading feature

Considering workpiece self-alignment and self-location, when being guided in a particular direction, a workpiece is limited to move between the guiding features in this direction. For example, if a workpiece is placed on a flat surface, it is free to move in the plane of the surface, but guided in the direction normal to the surface. Therefore, if the workpiece is dropped above the plane, its position normal to the plane will be limited by the plane as the two meet.

However, leading features are not exactly the same as a DoF restriction, as they have multiple effects. Leading features (i) guide the previous workpiece manipulation action, potentially reducing the accumulated tolerances up to the point of guiding, pose a new tolerance requirement to avoid workpiece wedging or damage while performing this previous action (ii), and potentially while performing the next action (iii).

For (i), the leading feature saturates the tolerance up to the following member of the tolerance chain, which improves the achievable precision (e.g., when using chamfers or cones). This allows worse precision while fitting, then improves the precision utilizing the geometrical, physical contact-based guidance of the fixture. Effects (ii) and (iii) pose new FRs in order to avoid overconstrained workpiece manipulation. As a simplified example, in order to utilize the self-locating capability of a chamfer, the deviation must be low enough to maintain the counterpart feature on the chamfer slope. It is noted that to determine actual self-alignment and self-locating ranges, as well as corresponding FRs, simulations or experiments are necessary, as their behavior is complex and depends on many factors (shape, material pair, forces, etc.) [36].

Leading features can be capable of handling different types of misalignments (axial, radial, lateral, or torsional) with a certain limit, depending on shape. Furthermore, these features can exist on the picking and placing side, on the gripper or on the workpiece itself. Leading features can be geometric (cones, chamfers) or kinematic (closing gripper fingers). Tolerance adjustment can happen during seizing and releasing, depending on the leading feature setting.

Although leading features are linked with the three effects above, these only manifest simultaneously during the seizing process. Before seizing, the previous workpiece manipulation step is not made as a part of the present assembly task (only (iii) applies), while at releasing, the following workpiece manipulation step is also not done as a part of the present assembly task (only (i) and (ii) apply). Moreover, as mentioned earlier, these effects are influenced by the contact transients and need to be considered accordingly.

The presence of leading features is represented with Boolean arrays. These are ka,wh1 when the workpiece lays on the picking workholder before seizing, ka,gr while the workpiece is being seized by the gripper, and ka,wh2 when it is being released onto the placing workholder. Each of these determines whether a corresponding FR is present or not. These are ca,wh1, ca,gr, and ca,wh2, which often overwrite the original requirements (cpick and cplace) to looser ones. On the other hand, only ka,gr and ka,wh2 have an effect on the aggregated tolerances. The corresponding functions are Fa,gr(ka,gr, T) and Fa,wh2(ka,wh2, T), which return the original transformation if there is no leading feature; however, they saturate the values in the guided directions to the corresponding ranges (agr, awh2).

For example, if a workpiece is guided during seizing in ξ and η but not in ζ, then the corresponding angle values are calculated from the original rotational matrix, and these get saturated, while the original angle value is kept in the ζ direction. Then, based on these values, the rotation and the whole transformation matrix are recalculated. When applied in a single direction (x), this can be formulated as:

$$ {\mathbf{F}}_{\mathrm{a},\mathrm{gr}}\left({\boldsymbol{k}}_{\mathrm{a},\mathrm{gr}},\boldsymbol{T}\left(x,\dots \right)\right)=\left\{\begin{array}{l}\boldsymbol{T}\left(x,\dots \right),\kern0.5em \mathrm{if}\ {k}_{\mathrm{a},\mathrm{gr},x}=\mathrm{false}\\ {}\boldsymbol{T}\left({a}_{\mathrm{gr},x}\cdot \mathrm{sat}\left(x/{a}_{\mathrm{gr},x}\right),\dots \right),\kern0.5em \mathrm{otherwise}\end{array}\right. $$
(9)

The formulation of Fa,wh2(ka,wh2, T) is completely analogous to formula (9).

Pose specification

Considering pose specification, there is a corresponding precision value for such a pose, which is sufficiently specified in the design phase. Thus, the tolerance of this pose is part of the geometrical tolerance stack-up and has to be considered accordingly, when checking feasibility. On the other hand, if the pose is not known precisely (typically in flexible scenarios), the design phase has to be carried out parametrically. Here, the pose has to be determined precisely during operation by means of metrology. In these cases, metrology precision needs to be considered when calculating the tolerance stack-up.

Whether or not the picking and placing pose are known with sufficient precision is captured with the Boolean arrays kpick and kplace, and the corresponding functions are Fpick(kpick, T) and Fplace(kplace, T). These functions return the input transformation matrix with the tolerance parameters corresponding to the designed pose (tp1 and twh2,place) and/or corresponding to the complete metrology process (tm,p1 and tm,p2). For example, if the workpiece picking position is only known in z direction but not in x and y directions, the translational part of the matrix will be set up using tm,p1,x, tm,p1,y and tp1,z. When applied in a single direction (x), this can be formulated as:

$$ {\mathbf{F}}_{\mathrm{p}\mathrm{ick}}\left({\boldsymbol{k}}_{\mathrm{p}\mathrm{ick}},\boldsymbol{T}\left(x,\dots \right)\right)=\left\{\begin{array}{l}\boldsymbol{T}\left({t}_{\mathrm{m},\mathrm{p}1,x},\dots \right),\kern0.5em \mathrm{if}\kern0.5em {k}_{\mathrm{p}\mathrm{ick},x}=\mathrm{false}\\ {}\boldsymbol{T}\left({t}_{\mathrm{p}1,x},\dots \right),\kern0.5em \mathrm{otherwise}\end{array}\right. $$
(10)

The formulation of Fplace(kplace, T) is completely analogous to formula (10).

Servoing

Lastly, applying servo techniques is a further possibility, during which certain elements of the tolerance chain can be improved or bypassed entirely, enhancing the overall precision capability of the system. Servo control allows the online correction of robot positioning by realizing a closed-loop motion control, until a certain condition is met (e.g., a certain positioning precision is achieved based on a camera system). In these scenarios, similarly to the case of poses that are not known precisely, the precision of the metrology has to be taken into account when evaluating the tolerance chain.

Whether or not servoing is applied in case of picking and in case of placing is captured with the Boolean arrays ks,g1 and ks,g2, and the corresponding functions are Fs,g1(ks,g1, T) and Fs,g2(ks,g2, T). These functions enable the substitution of the accumulated tolerances to servo tolerances (ts,g1 and ts,g2), returning the original transformation matrix if servoing is not applied, and constructing a new transformation otherwise. For example, in case there is servo motion on the picking side in x and y direction, then the x and y components of the translational transformation will be changed to ts,g1,x and ts,g1,y, while the z component remains that of the original. When applied in a single direction (x), this can be formulated as:

$$ {\mathbf{F}}_{\mathrm{s},\mathrm{g}1}\left({\boldsymbol{k}}_{\mathrm{s},\mathrm{g}1},\boldsymbol{T}\left(x,\dots \right)\right)=\left\{\begin{array}{l}\boldsymbol{T}\left(x,\dots \right),\kern0.5em \mathrm{if}\kern0.5em {k}_{\mathrm{s},\mathrm{g}1,x}=\mathrm{false}\\ {}\boldsymbol{T}\left({t}_{\mathrm{s},\mathrm{g}1,x},\dots \right),\kern0.5em \mathrm{otherwise}\end{array}\right. $$
(11)

The formulation of Fs,g2(ks,g2, T) is completely analogous to formula (11).

Model formulation

After the introduction of every necessary component for the preparation of the complete model, in this section, the detailed deduction of the tolerance analysis is presented. First, the necessary input data are overviewed. Then the transformation is constructed from the individual transformations between component frames and tolerance influencing functions. Finally, the evaluation of FRs is presented to assess the feasibility of the pick-and-place operation.

Input data

Both the requirement and the available capacity side need to be provided in order to draw conclusions about task-equipment compatibility. Based on the tolerance influencing factors, the following Boolean information needs to be decided for every direction:

  • is there a leading feature on the picking workholder, gripper, and placing workholder (ka,wh1, ka,gr and ka,wh2)?

  • is the workpiece picking and placing pose known with sufficient precision (kpick and kplace)?

  • is picking and placing servo applied (ks,g1 and ks,g2)?

Quantitative data needs to be defined in form of:

  • manufacturing tolerances: workpiece (twp,pick, twp,place, twp,grasp), gripper (tgr,grasp), picking (twh1,ref, twh1,pick), and placing workholder (twh2,ref, twh2,place),

  • location tolerance: workpiece picking pose on picking workholder (tp1),

  • tolerance of the complete metrology and servo metrology methods (tm,p1, tm,p2, ts,g1, ts,g2),

  • manipulator positioning tolerance (tr,g1, tr,g2),

  • seizing and releasing tolerance (tg1, tg2),

  • self-location and self-alignment ranges of the gripper and the placing workholder (agr, awh2)

Finally, FRs need to be formulated generally in form of feasible tolerance ranges:

  • the geometric relation between the gripper and the workpiece during seizing for successful picking (cpick); and between the workpiece and the placing workholder for successful placing (cplace),

  • additional FRs introduced by leading features for feasible picking (ca,wh1 and ca,gr) and placing (ca,wh2).

Tolerance model

The main goal of the generalized tolerance model is to determine the transformation between the corresponding placing frames of the placing workholder and workpiece \( \left({\boldsymbol{T}}_{\mathrm{wp},\mathrm{p}\mathrm{lace},\mathrm{p}2}^{\mathrm{wh}2,\mathrm{p}\mathrm{lace}}\right) \), for which the foremost tolerance requirement is defined. In order to achieve this parametric transformation, the tolerance chain is set up starting from the workpiece picking pose, up to the point when it settles on the placing workholder after releasing. The summary of transformation-related notations is given in Table 1.

Table 1 Summary of notations for homogeneous transformation matrices

The first step is to determine the grasp frame relation before seizing (see Fig. 4). This starts by determining the workpiece picking pose with respect to the base frame:

$$ {\boldsymbol{T}}_{\mathrm{wp},\mathrm{p}\mathrm{ick},\mathrm{p}1}^{\mathrm{base}}={\boldsymbol{T}}_{\mathrm{wh}1,\mathrm{ref}}^{\mathrm{base}}\cdotp {\boldsymbol{T}}_{\mathrm{wh}1,\mathrm{p}\mathrm{ick}}^{\mathrm{wh}1,\mathrm{ref}}\cdotp {\mathbf{F}}_{\mathrm{pick}}\left({\boldsymbol{k}}_{\mathrm{pick}},{\boldsymbol{T}}_{\mathrm{wp},\mathrm{p}\mathrm{ick},\mathrm{p}1}^{\mathrm{wh}1,\mathrm{p}\mathrm{ick}}\right) $$
(12)

where \( {\boldsymbol{T}}_{\mathrm{wh}1,\mathrm{ref}}^{\mathrm{base}} \) contains the workholder location tolerance (twh1,ref), \( {\boldsymbol{T}}_{\mathrm{wh}1,\mathrm{pick}}^{\mathrm{wh}1,\mathrm{ref}} \) contains the workholder machining tolerances (twh1,pick), and \( {\mathbf{F}}_{\mathrm{pick}}\left({\boldsymbol{k}}_{\mathrm{pick}},{\boldsymbol{T}}_{\mathrm{wp},\mathrm{p}\mathrm{ick},\mathrm{p}1}^{\mathrm{wh}1,\mathrm{p}\mathrm{ick}}\right) \) is the workpiece picking frame relative to the workholder picking frame including designed (tp1) and/or sensed parameters (tm,p1) depending on kpick. Next, the workpiece grasp frame is calculated:

$$ {\boldsymbol{T}}_{\mathrm{wp},\mathrm{grasp},\mathrm{p}1}^{\mathrm{base}}={\boldsymbol{T}}_{\mathrm{wp},\mathrm{p}\mathrm{ick},\mathrm{p}1}^{\mathrm{base}}\cdotp {{\boldsymbol{T}}_{\mathrm{wp},\mathrm{p}\mathrm{ick},\mathrm{p}1}^{\mathrm{wp},\mathrm{ref},\mathrm{p}1}}^{-1}\cdotp {\boldsymbol{T}}_{\mathrm{wp},\mathrm{grasp},\mathrm{p}1}^{\mathrm{wp},\mathrm{ref},\mathrm{p}1}, $$
(13)

where \( {\boldsymbol{T}}_{\mathrm{wp},\mathrm{p}\mathrm{ick},\mathrm{p}1}^{\mathrm{wp},\mathrm{ref},\mathrm{p}1} \) and \( {\boldsymbol{T}}_{\mathrm{wp},\mathrm{grasp},\mathrm{p}1}^{\mathrm{wp},\mathrm{ref},\mathrm{p}1} \) contain workpiece machining inaccuracies (twp,pick and twp,grasp, respectively). In the following, the gripper grasp frame is determined:

$$ {\boldsymbol{T}}_{\mathrm{gr},\mathrm{g}\mathrm{rasp},\mathrm{g}1}^{\mathrm{base}}={\boldsymbol{T}}_{\mathrm{gr},\mathrm{ref},\mathrm{g}1}^{\mathrm{base}}\cdotp {\boldsymbol{T}}_{\mathrm{gr},\mathrm{g}\mathrm{rasp},\mathrm{g}1}^{\mathrm{gr},\mathrm{ref},\mathrm{g}1}, $$
(14)

where \( {\boldsymbol{T}}_{\mathrm{gr},\mathrm{ref},\mathrm{g}1}^{\mathrm{base}} \) contains the robot positioning inaccuracies (tr,g1) and \( {\boldsymbol{T}}_{\mathrm{gr},\mathrm{g}\mathrm{rasp},\mathrm{g}1}^{\mathrm{gr},\mathrm{ref},\mathrm{g}1} \) contains gripper machining tolerances (tgr,grasp). Then, the relation of the grasp frame pair is determined, together with the servo technique at picking:

$$ {\boldsymbol{T}}_{\mathrm{wp},\mathrm{g}\mathrm{rasp},\mathrm{p}1}^{\mathrm{gr},\mathrm{g}\mathrm{rasp},\mathrm{g}1}={\mathbf{F}}_{\mathrm{s},\mathrm{g}1}\left({\boldsymbol{k}}_{\mathrm{s},\mathrm{g}1},{{\boldsymbol{T}}_{\mathrm{gr},\mathrm{g}\mathrm{rasp},\mathrm{g}1}^{\mathrm{base}}}^{-1}\cdotp {\boldsymbol{T}}_{\mathrm{wp},\mathrm{g}\mathrm{rasp},\mathrm{p}1}^{\mathrm{base}}\right), $$
(15)

where Fs,g1(ks,g1, T) enables the substitution of the accumulated tolerances to servo tolerances (ts,g1) based on ks,g1. The next step is the seizing process. At this point, the workpiece is detached from the picking surface:

$$ {{\boldsymbol{T}}^{\prime}}_{\mathrm{wp},\mathrm{g}\mathrm{rasp},\mathrm{g}1}^{\mathrm{gr},\mathrm{g}\mathrm{rasp},\mathrm{g}1}={\boldsymbol{T}}_{\mathrm{wp},\mathrm{g}\mathrm{rasp},\mathrm{p}1}^{\mathrm{gr},\mathrm{g}\mathrm{rasp},\mathrm{g}1}\cdotp {\boldsymbol{T}}_{\mathrm{wp},\mathrm{g}\mathrm{rasp},\mathrm{g}1}^{\mathrm{wp},\mathrm{g}\mathrm{rasp},\mathrm{p}1}, $$
(16)

where \( {\boldsymbol{T}}_{\mathrm{wp},\mathrm{g}\mathrm{rasp},\mathrm{g}1}^{\mathrm{wp},\mathrm{g}\mathrm{rasp},\mathrm{p}1}={\boldsymbol{T}}_{\mathrm{wp},\mathrm{ref},\mathrm{g}1}^{\mathrm{wp},\mathrm{ref},\mathrm{p}1} \), and it describes the seizing transient, including the misalignment (tg1), introduced by the seizing action. Then, the workpiece is getting attached to the gripper (see Fig. 5):

$$ {\boldsymbol{T}}_{\mathrm{wp},\mathrm{g}\mathrm{r}\mathrm{asp},\mathrm{g}1}^{\mathrm{gr},\mathrm{g}\mathrm{r}\mathrm{asp},\mathrm{g}1}={\mathbf{F}}_{\mathrm{a},\mathrm{g}\mathrm{r}}\left({\boldsymbol{k}}_{\mathrm{a},\mathrm{g}\mathrm{r}},{{\boldsymbol{T}}^{\prime}}_{\mathrm{wp},\mathrm{g}\mathrm{r}\mathrm{asp},\mathrm{g}1}^{\mathrm{gr},\mathrm{g}\mathrm{r}\mathrm{asp},\mathrm{g}1}\right) $$
(17)

Depending on the leading feature on the gripper (ka,gr), the misalignment between the grasp frames is reduced by the application of Fa,gr(ka,gr, T) to the corresponding self-alignment and self-locating range (agr). Next, the workpiece grasp frame is determined with respect to the gripper reference frame:

$$ {\boldsymbol{T}}_{\mathrm{wp},\mathrm{g}\mathrm{rasp},\mathrm{g}1}^{\mathrm{gr},\mathrm{ref},\mathrm{g}1}={\boldsymbol{T}}_{\mathrm{gr},\mathrm{g}\mathrm{rasp},\mathrm{g}1}^{\mathrm{gr},\mathrm{ref},\mathrm{g}1}\cdotp {\boldsymbol{T}}_{\mathrm{wp},\mathrm{g}\mathrm{rasp},\mathrm{g}1}^{\mathrm{gr},\mathrm{g}\mathrm{rasp},\mathrm{g}1} $$
(18)

The result is the transformation between the seized workpiece and the gripper. This needs to be applied at the end of the last manipulator link, after controlling it to the releasing configuration (see Fig. 6). Then, the workpiece grasp frame is to be determined with respect to the base frame in the releasing configuration:

$$ {\boldsymbol{T}}_{\mathrm{wp},\mathrm{g}\mathrm{rasp},\mathrm{g}2}^{\mathrm{base}}={\boldsymbol{T}}_{\mathrm{gr},\mathrm{ref},\mathrm{g}2}^{\mathrm{base}}\cdotp {\boldsymbol{T}}_{\mathrm{wp},\mathrm{g}\mathrm{rasp},\mathrm{g}2}^{\mathrm{gr},\mathrm{ref},\mathrm{g}2}, $$
(19)

where \( {\boldsymbol{T}}_{\mathrm{wp},\mathrm{g}\mathrm{rasp},\mathrm{g}2}^{\mathrm{gr},\mathrm{ref},\mathrm{g}2}={\boldsymbol{T}}_{\mathrm{wp},\mathrm{g}\mathrm{rasp},\mathrm{g}1}^{\mathrm{gr},\mathrm{ref},\mathrm{g}1} \) and \( {\boldsymbol{T}}_{\mathrm{gr},\mathrm{ref},\mathrm{g}2}^{\mathrm{base}} \) contains the manipulator positioning inaccuracies (tr,g2). Next, the workpiece is detached from the gripper, as it is being released (see Fig. 7):

$$ {\boldsymbol{T}}_{\mathrm{wp},\mathrm{g}\mathrm{rasp},\mathrm{p}2}^{\mathrm{base}}={\boldsymbol{T}}_{\mathrm{wp},\mathrm{g}\mathrm{rasp},\mathrm{g}2}^{\mathrm{base}}\cdotp {\boldsymbol{T}}_{\mathrm{wp},\mathrm{g}\mathrm{rasp},\mathrm{p}2}^{\mathrm{wp},\mathrm{g}\mathrm{rasp},\mathrm{g}2}, $$
(20)

where \( {\boldsymbol{T}}_{\mathrm{wp},\mathrm{g}\mathrm{rasp},\mathrm{p}2}^{\mathrm{wp},\mathrm{g}\mathrm{rasp},\mathrm{g}2} \) describes the releasing transient, including the misalignment (tg2), introduced by the releasing action. Then, as the workpiece arrives to the placing pose, the workpiece placing frame needs to be determined:

$$ {\boldsymbol{T}}_{\mathrm{wp},\mathrm{p}\mathrm{lace},\mathrm{p}2}^{\mathrm{base}}={\boldsymbol{T}}_{\mathrm{wp},\mathrm{grasp},\mathrm{p}2}^{\mathrm{base}}\cdotp {{\boldsymbol{T}}_{\mathrm{wp},\mathrm{grasp},\mathrm{p}2}^{\mathrm{wp},\mathrm{ref},\mathrm{p}2}}^{-1}\cdotp {\boldsymbol{T}}_{\mathrm{wp},\mathrm{p}\mathrm{lace},\mathrm{p}2}^{\mathrm{wp},\mathrm{ref},\mathrm{p}2}, $$
(21)

where \( {\boldsymbol{T}}_{\mathrm{wp},\mathrm{grasp},\mathrm{p}2}^{\mathrm{wp},\mathrm{ref},\mathrm{p}2}={\boldsymbol{T}}_{\mathrm{wp},\mathrm{grasp},\mathrm{p}1}^{\mathrm{wp},\mathrm{ref},\mathrm{p}1} \), and \( {\boldsymbol{T}}_{\mathrm{wp},\mathrm{p}\mathrm{lace},\mathrm{p}2}^{\mathrm{wp},\mathrm{ref},\mathrm{p}2} \) contains workpiece machining inaccuracies (twp,place). At this point, the placing workholder and the placing pose are to be introduced:

$$ {\boldsymbol{T}}_{\mathrm{wh}2,\mathrm{place}}^{\mathrm{base}}={\boldsymbol{T}}_{\mathrm{wh}2,\mathrm{ref}}^{\mathrm{base}}\cdotp {\mathbf{F}}_{\mathrm{place}}\left({\boldsymbol{k}}_{\mathrm{place}},{\boldsymbol{T}}_{\mathrm{wh}2,\mathrm{place}}^{\mathrm{wh}2,\mathrm{ref}}\right), $$
(22)

where \( {\boldsymbol{T}}_{\mathrm{wh}2,\mathrm{ref}}^{\mathrm{base}} \) contains the workholder location tolerances (twh2,ref), and \( {\mathbf{F}}_{\mathrm{place}}\left({\boldsymbol{k}}_{\mathrm{place}},{\boldsymbol{T}}_{\mathrm{wh}2,\mathrm{place}}^{\mathrm{wh}2,\mathrm{ref}}\right) \) contains the designed (twh2,place) and/or sensed parameters (tm,p2) based on kplace. Then, the placing frame pair relation is determined and the placing servo technique is applied (if any):

$$ {{\boldsymbol{T}}^{\prime}}_{\mathrm{wp},\mathrm{p}\mathrm{lace},\mathrm{p}2}^{\mathrm{wh}2,\mathrm{p}\mathrm{lace}}={\mathbf{F}}_{\mathrm{s},\mathrm{g}2}\left({\boldsymbol{k}}_{\mathrm{s},\mathrm{g}2},{{\boldsymbol{T}}_{\mathrm{wh}2,\mathrm{p}\mathrm{lace}}^{\mathrm{base}}}^{-1}\cdotp {\boldsymbol{T}}_{\mathrm{wp},\mathrm{p}\mathrm{lace},\mathrm{p}2}^{\mathrm{base}}\right), $$
(23)

where Fs,g2(ks,g2, T) corresponds to the placing servo, reducing the accumulated tolerances to the level of servo tolerance (ts,g2) based on ks,g2. Finally, the effect of the last leading feature is applied (if any):

$$ {\boldsymbol{T}}_{\mathrm{wp},\mathrm{p}\mathrm{lace},\mathrm{p}2}^{\mathrm{wh}2,\mathrm{p}\mathrm{lace}}={\mathbf{F}}_{\mathrm{a},\mathrm{wh}2}\left({\boldsymbol{k}}_{\mathrm{a},\mathrm{wh}2},{{\boldsymbol{T}}^{\prime}}_{\mathrm{wp},\mathrm{p}\mathrm{lace},\mathrm{p}2}^{\mathrm{wh}2,\mathrm{p}\mathrm{lace}}\right), $$
(24)

where Fa,wh2(ka,wh2, T) corresponds to the leading feature at placing, reducing the accumulated tolerances to the corresponding range (awh2) based on ka,wh2. With this, the placing frame pair and other geometric relations corresponding to FRs are set up parametrically.

Evaluation of functional requirements

To achieve a successful pick-and-place operation, all the actually considered FRs need to be simultaneously satisfied. First, the variation in the workpiece placing pose (relative to the placing workholder) is checked against the allowed deviation range cplace, in each direction. If this is not fulfilled, the application of leading features (e.g., chamfering), or precision enhancement is necessary in form of tolerance improvement or servo techniques. Here, cplace defines a feasible tolerance region. Assuming symmetric ranges, this can be written in a single direction (x) as:

$$ \left|\mathbf{scr}{\left({\boldsymbol{T}}_{\mathrm{wp},\mathrm{p}\mathrm{lace},\mathrm{p}2}^{\mathrm{wh}2,\mathrm{p}\mathrm{lace}}\right)}_x\right|\le {c}_{\mathrm{place},x} $$
(25)

Next, the FR for successful workpiece picking (cpick) is checked. This can be written in a single direction (x) as:

$$ \left|\mathbf{scr}{\left({\boldsymbol{T}}_{\mathrm{wp},\mathrm{g}\mathrm{rasp},\mathrm{p}1}^{\mathrm{gr},\mathrm{g}\mathrm{rasp},\mathrm{g}1}\right)}_x\right|\le {c}_{\mathrm{pick},x} $$
(26)

Then, the FRs posed by the leading features need to be assessed (ca,wh1, ca,gr and ca,wh2) to avoid overconstraining the system during contact transition. These can be formulated in a single direction (x) as follows:

$$ \left|\mathbf{scr}{\left({{\boldsymbol{T}}^{\prime}}_{\mathrm{wp},\mathrm{g}\mathrm{rasp},\mathrm{g}1}^{\mathrm{gr},\mathrm{g}\mathrm{rasp},\mathrm{g}1}\right)}_x\right|\le {c}_{\mathrm{a},\mathrm{wh}1,x},\kern0.5em \mathrm{if}\kern0.5em {k}_{\mathrm{a},\mathrm{wh}1,x}=\mathrm{true} $$
(27)
$$ \left|\mathbf{scr}{\left({{\boldsymbol{T}}^{\prime}}_{\mathrm{wp},\mathrm{g}\mathrm{r}\mathrm{asp},\mathrm{g}1}^{\mathrm{gr},\mathrm{g}\mathrm{r}\mathrm{asp},\mathrm{g}1}\right)}_x\right|\le {c}_{\mathrm{a},\mathrm{g}\mathrm{r},x},\mathrm{if}\kern0.5em {k}_{\mathrm{a},\mathrm{g}\mathrm{r},x}=\mathrm{true} $$
(28)
$$ \left|\mathbf{scr}{\left({{\boldsymbol{T}}^{\prime}}_{\mathrm{wp},\mathrm{p}\mathrm{lace},\mathrm{p}2}^{\mathrm{wh}2,\mathrm{p}\mathrm{lace}}\right)}_x\right|\le {c}_{\mathrm{a},\mathrm{wh}2,x},\mathrm{if}\kern0.5em {k}_{\mathrm{a},\mathrm{wh}2,x}=\mathrm{true} $$
(29)

The corresponding evaluations can be performed via Monte Carlo simulation. By substituting the input parameters with tolerance values sampled from the given intervals into the transformation chain, any transformation can be evaluated to check the fulfillment of FRs.

Implementation

The proposed tolerance model is presented through a flexible pick-and-place case study. A physical workcell was set up as an experimental demonstrator cell for palletizing workpieces laying in a semi-structured arrangement. Separated workpieces, laying in one of their stable poses, are manipulated from a light table to the corresponding workpiece pallet. The used manipulator is an UR5 robot arm equipped with a Robotiq 2f-85 finger gripper and an IDS XS camera. The actual workpiece picking poses are determined using this 2D camera, with the help of the backlighting capability of the light table. The workcell setup, with the applied frames, is shown in Figs. 2 and 3. The preparation and operation of the workcell is presented in more detail in [7].

Input data

For the sake of simplicity, the tolerance analysis is regarded as a planar problem, considering only the x, y, and ζ components. This consideration is viable, since the workpiece picking and placing pose correspond to the same workpiece stable pose, i.e., no re-gripping and no re-orientation of the workpiece is necessary, apart from rotating around the vertical axis. The screw parameters x, y, and ζ are represented as an array, where x and y components are translations parallel to the picking plane and ζ is the rotation around the axis normal to the picking plane (vertical axis). The decision parameters are Boolean, while dimensions, tolerance and FR ranges are given in mm for translational and in ° (deg) for rotational parameters. For the sake of conciseness, tolerance and FR ranges are considered symmetric, closed intervals with zero mean value, and are represented with a single value with a ± sign.

First, the decision parameters are defined. The workpieces lay on the light table in a semi-structured way; their pose is not known precisely in x, y and ζ directions. These have to be resolved using the camera fixed on the robot. However, the placing poses are known with sufficient precision in the design phase. Since the workpieces lay freely on a table, they are not guided in the plane. On the other hand, the finger gripper guides the workpiece in y and ζ directions (see Fig. 3) and the placing pose is guided in all three directions via a chamfered centering pin and chamfered columns (shown in Fig. 14). Lastly, servoing is applied neither in the picking nor in the placing phase. The corresponding factors are summarized in Table 2.

Table 2 Tolerance influencing factors

Next, the nominal and tolerance parameters are defined; these are listed in Table 3 (values with v1 and v2 subscripts are discussed in Section 5.2). The only relevant nominal dimension in the tolerance model corresponds to the workpiece grasp frame (nwp,grasp) relative to the workpiece reference. Other dimensions are either zero or they cancel out in this particular scenario. The positioning precision of the robot is given, together with the self-positioning and self-alignment ranges of the placing workholder, as well as the manufacturing tolerances of the workpiece, gripper, and picking and placing workholders. It is noted that the tolerance of the workpiece picking and placing frame (twp,pick and twp,place) relative to the workpiece reference is zero, as these all coincide. Furthermore, the tolerance of the picking and placing workholder references (twh1,ref and twh2,ref) are also considered zero, because the position of the corresponding picking and placing frames were measured relative to the workcell reference frame directly.

Table 3 Nominal values, self-location, self-alignment, and tolerance ranges

The remaining tolerance parameters (metrology, seizing, and releasing) were needed to be determined experimentally. The precise and exhaustive exploration of these parameters is not in the scope of this paper; nevertheless, simple measurement setups and overestimation of the parameters were carried out to have acceptable input data for the simulation. Correspondingly, to keep the paper more concise, only a brief overview is given about the experiments.

The combined tolerance of the metrology system and image processing algorithms (tm,p1) was measured by setting a workpiece in a pose on the light table using different 3D-printed temporary locators (see Fig. 9a, b), and capturing images of workpiece. The captured images have a resolution of 1280 × 720. After every captured image, the robot (with the mounted camera) was moved to a random picking configuration, then back to the image capturing configuration, to emulate the real operation and add disturbance to the vision system. The workpiece pose was evaluated on each image against the originally set up pose, using the image processing algorithm employed in the demonstrator (see Fig. 10). The vision-based localization employs feature recognition and homography transformation (the algorithm is described in detail in [7]). This process was repeated using multiple workpiece setups.

Fig. 9
figure 9

Workpiece setups on the light table with different temporary locators; setup for the measurement of tm,p1 (a, b), and setup for the measurement of tg1, agr, cpick, and ca,gr (c)

Fig. 10
figure 10

Recognized workpiece features, and localization based on the camera image (for different workpiece setups)

The seizing tolerance (tg1) was estimated by seizing a similarly positioned workpiece multiple times (using the temporary locator shown in Fig. 9c) with disturbed seizing configurations, and measuring the variation of the seized pose in the free direction with a digital caliper (with 0.01 mm resolution), relative to the gripper fingers. This process is shown in Fig. 11. The self-positioning and self-alignment range of the gripper (agr) was measured in the same setup in the guided directions, using the same camera system. However, the image processing algorithm was refined for more accurate results; the achieved precision of the camera-based measurement system on the same dataset is ±0.07 mm, ± 0.06 mm and ±0.17° in x, y,and ζ, respectively.

Fig. 11
figure 11

Measurement process of the seizing tolerance after the workpiece is set up, with a disturbed seizing configuration

Finally, the releasing tolerance (tg2) was measured by repeatedly fixing the workpiece manually into the gripper using a different temporary locator (see Fig. 12), then releasing it on the light table. The variation in the releasing pose was measured, again, using the camera-based measurement system.

Fig. 12
figure 12

Setting up the workpiece into the gripper using a temporary locator

FRs are defined symmetrically for picking and placing, and additionally for the leading features at the gripper and the placing workholder. These are listed in Table 4. The FR ranges were validated in the physical workcell. The requirement corresponding to seizing and the gripper alignment (cpick and ca,gr) were checked in the seizing tolerance measurement setup, to establish feasible extrema for the FR ranges. The leading feature on the placing side (awh2) is set so that it fulfills the placing requirement; therefore, cplace is automatically satisfied if ca,wh2 is fulfilled. That is, if after releasing the workpiece, it is guided to the placing pose by the gravity and the leading features, the placing pose is satisfactory. Otherwise, if the workpiece stays on top of the guiding features, does not reach the bottom support plane, or tilts out of the workholder, the placing pose is not satisfactory. Consequently, the analysis focuses on ca,wh2 instead of the original cplace requirement. The feasibility bounds were experimentally evaluated for ca,wh2 by manually fixing the workpiece into the gripper, once again, with a temporary locator (same way as in Fig. 12), moving the robot to a disturbed releasing configuration and releasing the workpiece. Two cases with different releasing configurations are shown in Fig. 13, one with correct, the other with a failed placement. In the determined deviation region, the releasing process was consistently successful for each sampled combination of deviations.

Table 4 Defined FRs
Fig. 13
figure 13

Experimental setup for the determination of the ca,wh2 feasibility bounds

Results

By applying the proposed tolerance model with the above input data, tolerance analysis can be performed for the pick-and-place scenario. To get a comprehensible result, a Monte Carlo simulation was carried out. The model was evaluated for 20,000 tolerance set instances, sampled in the given regions assuming normal distribution on 99.72% confidence level. To visualize the deviations, the 2D projection of the geometry was triangulated and transformed based on the results, the transformed shapes were overlaid, and their union was computed. In order to reduce the number of computations, the extreme displacements (with combined translations and rotation) were selected heuristically. The final union results in a boundary within which the actual workpiece will lay according to the tolerance data. Since there are two different aligning features, (chamfered orienting columns and chamfered positioning pin), two different 2D projection boundaries of the workpiece were calculated. Based on the resultant transformations, the worst tolerance stack-up case can be found in each direction. The results are shown in Table 5 and in Fig. 14. The calculation details are summarized in Table 6.

Table 5 Result of the Monte Carlo simulation on each FR
Fig. 14
figure 14

The top view of the workpiece pallet with the workpiece projection, worst-case boundary region, and geometric boundaries of the aligning features for the whole workpiece-chamfered columns (a), and the workpiece bottom hole-chamfered positioning pin relationships (b)

Table 6 Experimental results for the whole workpiece projection on an Intel I5-6440HQ CPU @ 2.60-GHz computer under Windows 10

According to the results, ca,wh2 is not satisfied in the worst deviation combination, which is also visible on Fig. 14, as the boundaries of the leading features and the deviation regions of the workpiece intersect in multiple places. When evaluating each deviation result of the Monte Carlo simulation, 2537 cases of the 20,000 do not satisfy ca,wh2, while the other FRs are always satisfied. With this, a failure rate of 12.69% is predicted for the placing part of the operation.

For comparison, a sequence of physical operations was executed to check the actual failure events. One pick-and-place cycle included the palletizing of 4 workpieces onto the 4 slotted workpiece pallet. Altogether 50 cycles were run, meaning 200 workpieces seizing and releasing processes. From the 200 cycles, releasing failure occurred in 7 cases (3.5%), where the workpiece did not fall into the placing pose, but stayed on top of the leading features or tilted in the wrong direction. Apart from these, there were no failure events during the operation, which matches the simulation results for the other FRs. Although the simulation and the experiment show a similar tendency in releasing failures, the gap between the results is significant. The reason behind this is that the combined leading features are represented with a single FR, which is a more conservative, more precautious approach. As this consideration in this particular case is not well suited, it is also possible to separate the FR to multiple checking points.

In this case, three different constraints can be formulated for feasible workpiece placing. Firstly, the workpiece-centering pin connection is described. Based on the diameter and chamfering of the workpiece and the workholder, the FR for the planar location (x and y) of the workpiece reference can be formulated as follows:

$$ \sqrt{\mathbf{scr}{\left({{\boldsymbol{T}}^{\prime}}_{\mathrm{wp},\mathrm{p}\mathrm{lace},\mathrm{p}2}^{\mathrm{wh}2,\mathrm{p}\mathrm{lace}}\right)}_x^2+\mathbf{scr}{\left({{\boldsymbol{T}}^{\prime}}_{\mathrm{wp},\mathrm{p}\mathrm{lace},\mathrm{p}2}^{\mathrm{wh}2,\mathrm{p}\mathrm{lace}}\right)}_y^2}\le \left(16.5-12.77\right)/2 $$
(30)

The remaining two FRs correspond to the chamfered columns of the workholder, and the corresponding vertices (designated as v1 and v2) on the workpiece that first make contact with the chamfers’ slope. These vertices also carry deviations relative to the workpiece reference. The corresponding parameters are listed in Table 3. The FRs can be formulated for the y-directional deviation (in the workpiece reference frame) of v1 and v2 by applying planar transformation to the vertices according to the deviation results from the simulation. This results in the following inequalities:

$$ \mid \left({n}_{\mathrm{wp},\mathrm{v}1,y}+{t}_{\mathrm{wp},\mathrm{v}1,y}\right)\cos \left(\mathbf{scr}{\left({{\boldsymbol{T}}^{\prime}}_{\mathrm{wp},\mathrm{p}\mathrm{lace},\mathrm{p}2}^{\mathrm{wh}2,\mathrm{p}\mathrm{lace}}\right)}_{\zeta}\right)+\left({n}_{\mathrm{wp},\mathrm{v}1,x}+{t}_{\mathrm{wp},\mathrm{v}1,x}\right)\sin \left(\mathbf{scr}{\left({{\boldsymbol{T}}^{\prime}}_{\mathrm{wp},\mathrm{p}\mathrm{lace},\mathrm{p}2}^{\mathrm{wh}2,\mathrm{p}\mathrm{lace}}\right)}_{\zeta}\right)\mid \le 13.25 $$
(31)
$$ \mid \left({n}_{\mathrm{wp},\mathrm{v}2,y}+{t}_{\mathrm{wp},\mathrm{v}2,y}\right)\cos \left(\mathbf{scr}{\left({{\boldsymbol{T}}^{\prime}}_{\mathrm{wp},\mathrm{p}\mathrm{lace},\mathrm{p}2}^{\mathrm{wh}2,\mathrm{p}\mathrm{lace}}\right)}_{\zeta}\right)+\left({n}_{\mathrm{wp},\mathrm{v}2,x}+{t}_{\mathrm{wp},\mathrm{v}2,x}\right)\sin \left(\mathbf{scr}{\left({{\boldsymbol{T}}^{\prime}}_{\mathrm{wp},\mathrm{p}\mathrm{lace},\mathrm{p}2}^{\mathrm{wh}2,\mathrm{p}\mathrm{lace}}\right)}_{\zeta}\right)\mid \le 13.25 $$
(32)

Now, the original FR (corresponding to ca,wh2) can be replaced with inequalities [30,31,32]. All three inequalities need to be satisfied simultaneously to achieve feasible workpiece placement. These requirements were also experimentally verified in the same setup as the measurement of ca,wh2, with different sampled deviations in the releasing configuration.

After checking the deviation results of the Monte Carlo simulation against the newly formulated FRs, only 961 error cases were observable, reducing the failure rate to 4.81%. This is much closer to the 3.5% failure rate obtained from the physical execution of the system. The remaining gap between the results is most probably due to the overestimated tolerance values and the intricacies of leading feature mechanics, contact transition, and dynamics of the released workpiece. Nevertheless, the experiments show close relationship with the simulation results, which validates the proposed method.

Conclusion and future work

This paper presented a novel tolerance model for evaluating the feasibility of robotic pick-and-place operations from the precision perspective. The proposed model aids robotic workcell developers in the selection of cell components, applied methods, and allocation of tolerances. Beyond the positioning precision of the manipulator and the manufacturing tolerances of the components, this tool allows the assessment of robotic workcell FRs considering tolerances introduced by the manipulation and metrology processes. The tolerance stack-up is set up in the form of a transformation chain. By substituting the tolerance values of the relevant frames along the tolerance chain, and comparing the aggregated tolerances to the feasible tolerance ranges, the realization of FRs can be validated.

First, the basic concept and main considerations were introduced, together with the structural setup of the transformation chain, the essential pick-and-place-related FRs, and the additional tolerance influencing factors. Then, the complete model formulation was presented, including the necessary input data, the composition of the tolerance chain, and the evaluation of the fulfillment of FRs. Finally, the approach was implemented and validated in case of a physical robotic demonstrator cell, through a flexible pick-and-place application.

Future work will include more exhaustive experimentation on real workcells to analyze and characterize specific tolerance inputs, such as seizing and releasing tolerances, investigate the effect of contact transitions, and identify further possible failure modes, which can potentially influence the computed tolerance chain. The other main goal is to generalize the model to different robotic operation types, like screwing or welding as well. A further possible research direction is to consider positioning tolerances on the individual robot links. If these parameters are available from the manufacturer or are measured, then the consideration of FRs along the robot trajectories could also be formulated (e.g., smallest distance between particular objects for offline path planners). Moreover, modular robots could also be considered in the tolerance model.

Availability of data and materials

The data that support the findings of this study are available from the corresponding author, upon reasonable request.

Code availability

The custom code that supports the findings of this study is available from the corresponding author, upon reasonable request.

References

  1. 1.

    Dantan J-Y, Qureshi A-J (2009) Worst-case and statistical tolerance analysis based on quantified constraint satisfaction problems and Monte Carlo simulation. Comput Aided Des 41:1–12. https://doi.org/10.1016/j.cad.2008.11.003

  2. 2.

    Shah JJ, Ameta G, Shen Z, Davidson J (2007) Navigating the tolerance analysis maze. Comput-Aided Des Appl 4:705–718. https://doi.org/10.1080/16864360.2007.10738504

  3. 3.

    Thornton AC (1999) A mathematical framework for the key characteristic process. Res Eng Des 11:145–157. https://doi.org/10.1007/s001630050011

  4. 4.

    Edmondson NF, Redford AH (2002) Generic flexible assembly system design. Assem Autom 22:139–152. https://doi.org/10.1108/01445150210423189

  5. 5.

    International Organization for Standardization (2019) ISO 10303-105:2019 - Industrial automation systems and integration - product data representation and exchange - part 105: integrated application resource: kinematics. Geneva, Switzerland

  6. 6.

    Yan H, Wu X, Yang J (2015) Application of Monte Carlo method in tolerance snalysis. Procedia CIRP 27:281–285. https://doi.org/10.1016/j.procir.2015.04.079

    Article  Google Scholar 

  7. 7.

    Tipary B, Erdős G (2021) Generic development methodology for flexible robotic pick-and-place workcells based on digital twin. Robot Comput-Integr Manuf 71:102140. https://doi.org/10.1016/j.rcim.2021.102140

    Article  Google Scholar 

  8. 8.

    Whitney DE (2004) Mechanical assemblies: their design, manufacture, and role in product development. Oxford University Press

  9. 9.

    Chase KW, Greenwood WH (1988) Design issues in mechanical tolerance analysis. Manuf Rev 1:50–59

  10. 10.

    Teeravaraprug J (2007) A comparative study of probabilistic and worst-case tolerance synthesis. Eng Lett 14

  11. 11.

    Marziale M, Polini W (2010) Tolerance analysis: a new model based on variational solid modelling. Am Soc Mech Eng Digit Collect 383–92. https://doi.org/10.1115/ESDA2010-24437.

  12. 12.

    Desrochers A (2003) A CAD/CAM representation model applied to tolerance transfer methods. J Mech Des 125:14–22. https://doi.org/10.1115/1.1543974

    Article  Google Scholar 

  13. 13.

    Desrochers A, Rivière A (1997) A matrix approach to the representation of tolerance zones and clearances. Int J Adv Manuf Technol 13:630–636. https://doi.org/10.1007/BF01350821

  14. 14.

    Chase KW, Gao J, Magleby SP (1995) General 2-D tolerance analysis of mechanical assemblies with small kinematic adjustments. J Des Manuf 5:263–274

    Article  Google Scholar 

  15. 15.

    Bourdet P, Mathieu L, Lartigue C, Ballu A (1996) The concept of small displacement Torsor in metrology. In: Ciarlini P, Cox MG, Pavese F, Richter D, (eds) Adv. Math. Tools Metrol. II, vol. 40, World Scientific Publishing Company, p. 110–22.

  16. 16.

    Ghie W, Laperrière L, Desrochers A (2003) A unified Jacobian-Torsor model for analysis in computer aided yolerancing. In: Gogu G, Coutellier D, Chedmail P, Ray P, (eds) Recent Adv. Integr. Des. Manuf. Mech. Eng., Dordrecht: Springer Netherlands, p. 63–72. https://doi.org/10.1007/978-94-017-0161-7_7

  17. 17.

    Zou Z, Morse EP (2002) Assembleability analysis using GapSpace model for 2D mechanical assembly. Proc. ASME 2002 Int. Des. Eng. Tech. Conf. Comput. Inf. Eng. Conf. 3:329–336. https://doi.org/10.1115/DETC2002/DFM-34187

  18. 18.

    Davidson JK, Mujezinovic A, Shah JJ (2002) A new mathematical model for geometric tolerances as applied to round faces. J Mech Des 124:609–622. https://doi.org/10.1115/1.1497362

  19. 19.

    Giordano M, Pairel E, Samper S (1999) Mathematical representation of tolerance zones. In: van Houten F, Kals H (eds) Glob. Consistency Toler. Dordrecht: Springer Netherlands, pp 177–186. https://doi.org/10.1007/978-94-017-1705-2_18

  20. 20.

    Teissandier D, Couetard Y, Delos V (1999) Operations on polytopes: application to tolerance analysis. In: van Houten F, Kals H (eds) Glob. Consistency Toler. Dordrecht: Springer Netherlands, pp 425–434. https://doi.org/10.1007/978-94-017-1705-2_43

  21. 21.

    Schleich B, Anwer N, Mathieu L, Wartzack S (2014) Skin model shapes: a new paradigm shift for geometric variations modelling in mechanical engineering. Comput Aided Des 50:1–15. https://doi.org/10.1016/j.cad.2014.01.001

    Article  Google Scholar 

  22. 22.

    Cao Y, Liu T, Yang J (2018) A comprehensive review of tolerance analysis models. Int J Adv Manuf Technol 97:3055–3085. https://doi.org/10.1007/s00170-018-1920-2

  23. 23.

    Schleich B, Anwer N, Zhu Z, Qiao L, Mathieu L, Wartzack S (2014) A comparative study on tolerance analysis approaches. In: Howard TJ, Eifler T (eds) Proc. Int. Symp. Robust Des. - ISoRD14. Copenhagen, Denmark, pp 29–39. https://doi.org/10.4122/dtu:2084

    Chapter  Google Scholar 

  24. 24.

    Qin Y, Qi Q, Lu W, Liu X, Scott PJ, Jiang X (2018) A review of representation models of tolerance information. Int J Adv Manuf Technol 95:2193–2206. https://doi.org/10.1007/s00170-017-1352-4

  25. 25.

    Schleich B, Wartzack S (2016) A quantitative comparison of tolerance analysis approaches for rigid mechanical assemblies. Procedia CIRP 43:172–177. https://doi.org/10.1016/j.procir.2016.02.013

    Article  Google Scholar 

  26. 26.

    Chen H, Jin S, Li Z, Lai X (2014) A comprehensive study of three dimensional tolerance analysis methods. Comput Aided Des 53:1–13. https://doi.org/10.1016/j.cad.2014.02.014

    Article  Google Scholar 

  27. 27.

    Desrochers A (1999) Modeling three dimensional tolerance zones using screw parameters. 25th Des Autom Conf vol. 1, Las Vegas, Nevada, USA, p. 895–903. https://doi.org/10.1115/DETC99/DAC-8587

  28. 28.

    Slocum AH. Precision machine design. Society of Manufacturing Engineers; 1992

  29. 29.

    Polini W, Corrado A (2019) A general model to estimate hole location deviation in drilling: the contribution of three error sources. Int J Adv Manuf Technol 102:545–557. https://doi.org/10.1007/s00170-018-03273-x

  30. 30.

    Mei B, Liang Z, Zhu W, Ke Y (2021) Positioning variation synthesis for an automated drilling system in wing assembly. Robot Comput-Integr Manuf 67:102044. https://doi.org/10.1016/j.rcim.2020.102044

    Article  Google Scholar 

  31. 31.

    Corrado A, Polini W (2020) Tolerance analysis tools for fixture design: a comparison. Procedia CIRP 92:112–117. https://doi.org/10.1016/j.procir.2020.05.174

    Article  Google Scholar 

  32. 32.

    Corrado A, Polini W (2017) Manufacturing signature in variational and vector-loop models for tolerance analysis of rigid parts. Int J Adv Manuf Technol 88:2153–2161. https://doi.org/10.1007/s00170-016-8947-z

  33. 33.

    Ferreras-Higuero E, Leal-Muñoz E, García de Jalón J, Chacón E, Vizán A (2020) Robot-process precision modelling for the improvement of productivity in flexible manufacturing cells. Robot Comput-Integr Manuf 65:101966. https://doi.org/10.1016/j.rcim.2020.101966

    Article  Google Scholar 

  34. 34.

    Doydum C, Duke PN (1991) Use of Monte Carlo simulation to select dimensions, tolerances, and precision for automated assembly. J Manuf Syst 10:209–222. https://doi.org/10.1016/0278-6125(91)90034-Y

  35. 35.

    Baydar C, Saitou K (2004) Off-line error prediction, diagnosis and recovery using virtual assembly systems. J Intell Manuf 15:679–692. https://doi.org/10.1023/B:JIMS.0000037716.69868.d0

  36. 36.

    International Organization for Standardization (1998) ISO 9283:1998 - Manipulating industrial robots - performance criteria and related test methods. Geneva, Switzerland

  37. 37.

    Mihelj M, Bajd T, Ude A, Lenarčič J, Stanovnik A, Munih M et al (2019) Accuracy and repeatability of industrial manipulators. In: Robotics, Cham. Springer International Publishing, pp 231–241. https://doi.org/10.1007/978-3-319-72911-4_15

  38. 38.

    Nubiola A (2014) Contribution to improving the accuracy of serial robots. PhD Thesis. École de Technologie Supérieure

  39. 39.

    Mooring B, Roth ZS, Driels MR (1991) Fundamentals of manipulator calibration. Wiley, New York

    Google Scholar 

  40. 40.

    Wagner M, Morehouse J, Melkote S (2009) Prediction of part orientation error tolerance of a robotic gripper. Robot Comput-Integr Manuf 25:449–459. https://doi.org/10.1016/j.rcim.2008.02.006

  41. 41.

    Suyama A, Aiyama Y (2016) Influence of grasping position to robot assembling task. Proc. ISR 2016 47st Int. Symp. Robot., Munich, Germany, p. 1–6

  42. 42.

    ElMaraghy HA, ElMaraghy WH, Knoll L (1988) Design specification of parts dimensional tolerance for robotic assembly. Comput Ind 10:47–59. https://doi.org/10.1016/0166-3615(88)90047-4

  43. 43.

    De la Escalera A, Armingol JM (2010) Automatic chessboard detection for intrinsic and extrinsic camera Parameter Calibration. Sensors 10:2027–2044. https://doi.org/10.3390/s100302027

  44. 44.

    Lourakis M, Zabulis X (2013) Model-based pose estimation for rigid objects. In: Chen M, Leibe B, Neumann B (eds) Comput. Vis. Syst., Berlin. Heidelberg: Springer, pp 83–92. https://doi.org/10.1007/978-3-642-39402-7_9

  45. 45.

    Collet A, Martinez M, Srinivasa SS (2011) The MOPED framework: object recognition and pose estimation for manipulation. Int J Robot Res 30:1284–1306. https://doi.org/10.1177/0278364911401765

  46. 46.

    Kumbla NB, Thakar S, Kaipa KN, Marvel J, Gupta SK (2018) Handling perception uncertainty in simulation-based singulation planning for robotic bin picking. J Comput Inf Sci Eng 18:1–16. https://doi.org/10.1115/1.4038954

  47. 47.

    Kaipa KN, Kankanhalli-Nagendra AS, Kumbla NB, Shriyam S, Thevendria-Karthic SS, Marvel JA, Gupta SK (2016) Addressing perception uncertainty induced failure modes in robotic bin-picking. Robot Comput-Integr Manuf 42:17–38. https://doi.org/10.1016/j.rcim.2016.05.002

    Article  Google Scholar 

  48. 48.

    Chaumette F, Hutchinson S (2007) Visual servo control. II. Advanced approaches [Tutorial]. IEEE Robot Autom Mag 14:109–118. https://doi.org/10.1109/MRA.2007.339609

  49. 49.

    Chaumette F, Hutchinson S (2006) Visual servo control. I. Basic approaches. IEEE Robot Autom Mag 13:82–90. https://doi.org/10.1109/MRA.2006.250573

  50. 50.

    Ma Y, Liu X, Zhang J, Xu D, Zhang D, Wu W (2020) Robotic grasping and alignment for small size components assembly based on visual servoing. Int J Adv Manuf Technol 106:4827–4843. https://doi.org/10.1007/s00170-019-04800-0

  51. 51.

    Chang W-C (2018) Robotic Assembly of smartphone back shells with eye-in-hand visual servoing. Robot Comput-Integr Manuf 50:102–113. https://doi.org/10.1016/j.rcim.2017.09.010

    Article  Google Scholar 

  52. 52.

    Song J, Chen Q, Li Z (2021) A peg-in-hole robot assembly system based on Gauss mixture model. Robot Comput-Integr Manuf 67:101996. https://doi.org/10.1016/j.rcim.2020.101996

    Article  Google Scholar 

  53. 53.

    Bi ZM, Zhang WJ, Chen I-M, Lang SYT (2007) Automated generation of the D–H parameters for configuration design of modular manipulators. Robot Comput-Integr Manuf 23:553–562. https://doi.org/10.1016/j.rcim.2006.02.014

Download references

Funding

Open access funding provided by ELKH Institute for Computer Science and Control. Work in this paper has been in part funded under project number ED_18-22018-0006, supported by the National Research, Development and Innovation Fund of Hungary, financed under the (publicly funded) funding scheme according to Section 13. §(2) of the Scientific Research, Development and Innovation Act, and in part by the Ministry for Innovation and Technology and the National Research, Development and Innovation Office within the framework of the National Lab for Autonomous Systems.

Author information

Affiliations

Authors

Contributions

B Tipary: conceptualization, code writing, manuscript writing and editing, methodology, investigation, visualization

G Erdős: conceptualization, manuscript reviewing, supervision

Corresponding author

Correspondence to Bence Tipary.

Ethics declarations

Ethics approval

Not applicable

Consent to participate

Not applicable

Consent for publication

Not applicable

Conflict of interest

The authors declare no competing interests.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Tipary, B., Erdős, G. Tolerance analysis for robotic pick-and-place operations. Int J Adv Manuf Technol 117, 1405–1426 (2021). https://doi.org/10.1007/s00170-021-07672-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00170-021-07672-5

Keywords

  • Design method
  • Tolerance analysis
  • Robot
  • Precision
  • Monte Carlo simulation