Abstract
Robotic workcell design is a complex process, especially in case of flexible (e.g., binpicking) workcells. The numerous requirements and the need for continuous system validation on multiple levels place a huge burden on the designers. There are a number of tools for analyzing the different aspects of robotic workcells, such as CAD software, system modelers, or grasp and path planners. However, the precision aspect of the robotic operation is often overlooked and tackled only as a matter of manipulator repeatability. This paper proposes a designer tool to assess the precision feasibility of robotic pickandplace workcells from the operation point of view. This means that not only the manufacturing tolerances of the workpiece and the placing environment are considered, but the tolerance characteristics of the manipulation and metrology process (in case of flexible applications) as well. Correspondingly, the contribution of the paper is a novel tolerance modeling approach, where the tolerance stackup is set up as a transformation chain of loworder kinematic pairs between the workpiece, manipulator, and other workcell components, based on manipulation, seizing, releasing, manufacturing, and metrology tolerances. Using this representation, the fulfillment of functional requirements (e.g., picking or placing precision) can be validated based on the tolerance range of corresponding chain members. By having a generalized underlying model, the proposed method covers generic industrial pickandplace applications, including both conventional and flexible ones. The application of the method is presented in a semistructured pickandplace scenario.
Introduction
Real manufactured and assembled products only match their nominal design within certain tolerances. Geometric and dimensional deviations are caused by imprecise manufacturing and assembly processes, resulting in imperfect workpiece shapes as well as inaccurate relative positions between them. These deviations influence the assemblability and functions of the product, and thereby the fulfillment of functional requirements (FRs). Therefore, the success of assembly processes and the resulting mechanical assemblies is significantly affected by the corresponding tolerance design. Tolerance analysis (or tolerance stackup analysis) addresses this problem through the determination of the dimensional and geometrical variation of the final assembly from the given tolerances on individual components and on the created joints. Based on this, both the satisfaction of the defined FRs and the assemblability of the product can be verified [1, 2].
In case of manual assembly, proper geometric design and allocation of tolerances ensure that the worker will be able to assemble the product, while maintaining the product’s key characteristics (KCs) [3]. This concept is viable due to the assembly capabilities and dexterity of human workers (supported by suitable tools). On the other hand, as a manual assembly process becomes robotized, the robot capability needs to be taken into account when designing the assembly and corresponding robotic operation. However, by losing the dexterity and skills of the worker, the result of the assembly is determined not only by the robot repeatability (in addition to the tolerance of assembly components), but also by the tolerance of the involved cell components. Hence, the tolerance aspect of the whole operation needs to be addressed, which can affect the manipulator and equipment selection, or even the assembly design. Yet, the tolerance design for this scenario does not receive much support. Indeed, related issues are often overcome by applying much more precise equipment than necessary, or by trial and error.
The present research addresses the subject of tolerancing in robotic manipulation. Correspondingly, the main contribution of this paper is a novel tolerance analysis method for validating robotized operations in terms of tolerances, in order to overcome the abovementioned issues. The proposed method aims to assess the feasibility of waypointbased robotic applications, particularly the pickandplace operation—including placement and insertion—which can be considered one of the most common assembly tasks [4]. The tolerance model is prepared for a general pickandplace representation, capable of handling both conventional and flexible (e.g., binpicking) tasks. Apart from mechanical tolerances, the latter involve the precision of the metrology system used to resolve uncertainties (such as the workpiece picking pose in case of binpicking) and to realize visual servoing.
In the proposed model, the tolerance stackup of the operation is set up on a transformation basis using loworder kinematic pairs (joints) [5]. The aggregated tolerances—corresponding to FRs—are formulated parametrically through the multiplication of parametric transformation matrices. As the design specifications (i.e., workholding, grasping, manipulation, servoing, and metrology characteristics as well as tolerances) are substituted into these formulae, the operation can be evaluated for feasibility.
This tolerance model provides a basis for Monte Carlo simulation and sensitivity analysis [6]. Using these, the particular workcell setup can be analyzed, allowing the designer to check the suitability of the selected equipment in the early design phase. This representation can also be used during the different planning (path and grasp planning) steps of the robotic workcell, as it can contain the robot kinematics and component relations besides the tolerance stackup. Thereby, this method fits into the tolerance model of the generic pickandplace workcell development methodology introduced in the authors’ previous work [7]. This aids developers in setting up feasible tolerance regions for Digital Twins of such robotic workcells and in assessing twin closeness, which indicates whether or not the virtually planned robotic operation can be executed feasibly in the real workcell.
The remainder of the paper is structured as follows. The related literature, regarding general tolerance modeling approaches, manufacturing operationrelated approaches and the tolerance aspect of pickandplace operations is overviewed in Section 2. In Section 3, the concept of the tolerance model is presented, together with the generalized pickandplace operation, the basic structure of the transformation chain, fundamental FRs, and the pickandplacespecific tolerance influencing factors. The complete model is formulated in Section 4, including the required input data, the deduction of the transformation chain, and the evaluation of FRs. In Section 5, the implementation of the proposed model is presented through a case study of an experimental, physical, pickandplace workcell. Finally, conclusions are drawn, and the possible future research directions are presented in Section 6.
Related work
Tolerance modeling
During product design, the customer requirements define the desired product. These are then translated by the designer to FRs, which capture the functional intention of the designer in terms of dimensions and tolerances. The components of the critical FRs are KCs, which are defined as “the product, subassembly, part, and process features that significantly impact the final cost, performance, or safety of a product when the KCs vary from nominal. Special control should be applied to those KCs where the cost of variation justifies the cost of control.” by Thornton [3].
To achieve product feasibility (including assemblability and the fulfillment of FRs), the tolerancing problem needs to be solved [1]. Dimensional tolerances have been for long the primary means for expressing the allowable deviations of workpieces and products. Besides, geometrical tolerances also became formally defined and standardized by the introduction of Geometrical Dimensioning and Tolerancing (GD&T) [8]. Together, these allow the characterization of a variety of deviation types. The key approaches for tolerance analysis are the analytical worstcase and statistical analysis [9, 10], and Monte Carlo simulation [6]. The sought results can be the list of contributors, sensitivity, and effect of each contributor, as well as the tolerance stackup.
Numerous research works were dedicated to defining new tolerance representation methods in order to establish mathematical models for the expression and representation of geometric deviations. These include the variational model [11], TTRS (technologically and topologically related surfaces) model [12], matrix model [13], vectorloop [14], torsor model [15], Jacobiantorsor model [16], GapSpace [17], TMap model [18], deviation domain [19], polytopes [20], and Skin model shapes [21], among others. Furthermore, many commercial CAT (computeraided tolerancing) software packages were developed on the basis of these models, e.g., 3DCS, VisVSA, CETOL, FROOM, or CATIA.3D FDT [22,23,24]. Detailed summaries on tolerance representations and tolerance analysis approaches are given in [22, 25, 26].
From the above, the most relevant ones are the variational, vectorloop, matrix, and torsor models. These apply homogeneous transformations on an assembly graph to capture the geometric variations. The variational model [11] approach represents the deviations from the nominal geometry due to the tolerances and the assembly conditions through a parametric mathematical model. First, the nominal shape and dimensions of each assembly component are set up with respect to a local datum reference frame (DRF). Then, the components are assembled together using small kinematic adjustments to represent their relative location with respect to each other within the assembly. With this, each feature can be expressed in the same reference (i.e., the global DRF). The FRs can be solved by analytical approaches or Monte Carlo analysis.
The vectorloop model [14, 25] represents the workpiece geometric variability via chains of vectors. The vectors represent component dimensions or kinematical variable dimensions. Three types of deviations are considered: dimensional deviations, geometric feature deviations, and deviations originating from kinematic adjustments (from the assembly process). The vectorloop model results in a set of nonlinear equations, which can be evaluated using a worstcase or statistical method when linearized, or using Monte Carlo simulation.
The matrix model [13, 26] is based on TTRS; it transforms the tolerance zones to establish the limit boundaries for tolerances. The aim of the matrix model is to derive an explicit mathematical representation of the boundary of the entire spatial region enclosing all possible displacements originating from the variability sources. The representation needs to be completed by an additional set of inequalities, which define the bounds for every component in the matrix. Being a pointbased approach, the result of the matrix model is the variation of a point on a functional surface, and as the boundaries of the region of possible variations are defined (i.e., extreme values), intrinsically a worstcase approach is applied.
The small displacement torsor model [15, 25] is based on the firstorder approximation of the matrix model [27]. The displacement of geometric elements is modeled as a translation vector and a linearized rotation matrix arranged into a torsor. Three kinds of torsors are defined: component, deviation, and gap torsor. The global behavior of the assembly can be obtained from the union of the torsors. For the evaluation of the model, a worstcase approach can be applied, similarly to the matrix model.
In the following, relevant tolerance modeling approaches are presented, which are related to manufacturing processes. In the field of precision engineering, a transformation chain–based approach is commonly applied for setting up the kinematic error model of machine tools, for estimating their achievable accuracy, and for analyzing the contributing tolerances [28]. A kinematic model–based approach is presented in [29] to investigate the contributing factors of the location deviations of holes in drilling operations. In [30], a positioning variation model is presented for an eyeinhand drilling system, considering visionbased positioning error measurement and compensation. Further manufacturing processrelated tolerance models include fixturing and the consideration of manufacturing signatures [31, 32]. In the field of robotic machining, a working precision analysis is presented in [33] using a robotprocess model, to investigate the robot and processrelated influencing factors.
In case of robotic systems for workpiece manipulation, when considering the precision aspect, the focus is mostly on robot calibration. Tolerance modeling and analysis on the process side receive hardly any attention. In [34], the error model of the peginhole assembly is set up for automated assembly, but it does not cover any errors before the insertion (placing) process. The relevant failure modes of automated assembly systems—involving precise grasping, releasing, and collision—are identified in [35]; however, only manufacturing errors and the robot repeatability are considered in the tolerances. Furthermore, no general model is provided, only the Monte Carlo simulation for error diagnosis.
Although numerous different tolerance representations and analysis approaches exist, to the best of the authors’ knowledge, there is no tolerance model specifically representing robotic manipulation, and particularly the pickandplace operation, from the operation point of view. Inspired by this, the goal of the present research is to introduce a suitable approach for bridging the identified gap.
Tolerance factors in robotic pickandplace
After overviewing the relevant tolerance models, in this section, the different contributing factors are gathered, which take part in the tolerance chain of pickandplace operations. In case of conventional pickandplace, apart from the cell component (e.g., workholders) and workpiece manufacturing tolerances, the robot positioning precision and the effect of the seizing and releasing processes need to be considered in the tolerance model. These can clearly affect the feasibility of the picking and/or the placing process of the operation.
Robot precision includes repeatability and accuracy, which are both standard robot characteristics [36]. Repeatability plays a key role in the tolerance stack of pickandplace operations. Robot repeatability is generally provided by the manufacturers for positioning on the robot end flange. In this regard, the determination of position and orientation repeatability is presented in [37]. This characteristic in general can only be improved by using more precise components and component connections, as well as an internal measurement system with higher resolution [38]. On the other hand, accuracy is not as significant as repeatability in terms of tolerance, since in most cases, accuracyrelated errors can be sufficiently compensated. Whenever suitable, traditional (leadthrough) robot programming can neutralize poor robot accuracy. Moreover, robot calibration [39] is applicable for improving positioning accuracy. Calibration is especially important in case of applications utilizing modelbased offline robot programming [7].
Workpiece seizing and releasing processes can also introduce geometric errors during the pickandplace operation through the contact transitions between the components. This topic is much less studied in the literature, and only a few relevant papers were found. Uncertainty in case of grasping is studied in [40]. In this paper, the concept of selfalignment is introduced, which occurs during workpiece grasping. The phenomenon is studied for one particular workpiece and gripper finger geometry pair. The effect of grasping position on assembly success is analyzed in [41]. The focus of this paper is to investigate the alignment capability of a parallel finger gripper when seizing a cylindrical workpiece with initial position error, and to check whether or not the grasped workpiece can be successfully placed into a workholder. The geometrical conditions of the robotic peginhole problem, as a placing task, are investigated by [42]. Here, the effect of chamfering on the placing side is taken into account as a source of selfalignment mechanism for the workpiece.
In more advanced applications, where metrology systems are applied to resolve uncertainties, other tolerance components are introduced in the tolerance chain. These are in form of equipment resolution, measurement errors, and the precision of data processing algorithms. Equipment resolution is provided by the manufacturers in general, and there are many papers in the literature about the accuracy of different approaches, such as camera calibration and 2D pattern detection [43], or modelbased 3D pose estimation [44]. For robotic applications, fewer studies are available. An object recognition and pose estimation framework is presented in [45], including the accuracy measures of the estimated poses. Uncertainty in perception and grasping is taken into account for binpicking in [46, 47]. The grasp is selected based on how likely its feasibility is, when simulating this uncertainty. Moreover, fine positioning is considered if placing precision is predicted to be insufficient; with a predetermined manipulator motion sequence, certain features of the workpiece are aligned after the placing process.
With the help of metrology systems, it is also possible to compensate positioning errors using servo techniques [48, 49] realtime, during operation. Using a closedloop control cycle, the target point can be reached with continuous measurement and actuation of the manipulator, until the defined target condition is reached. Servoing can improve the positioning precision of the robot through closedloop motion control in case of picking and/or placing, without the need for changing the equipment, or prescribing tighter tolerances (i.e., without increasing investment costs). This means that the precision influencing effects in the system are negated, and the resultant precision will depend on the manipulator positioning resolution, and the errors in the metrology system, measured data, and processing algorithms. Servo techniques, and visual servoing in particular, receive great attention in the literature, including robotic pickandplace as well. Visual servoing is used for picking pose compensation and grasping in [50], and for placing pose compensation in an assembly cell in [51]. Furthermore, compliance control strategy is applied for peginhole insertion in [52].
The listed tolerance factors, together with the corresponding literature, provide a basis for the tolerance model proposed in the present paper, and support the introduced tolerance influencing concepts. Although not all of these factors are thoroughly explored in the literature—especially, the tolerances introduced during contact transition between components, and characteristics of selfalignment while grasping and placing—the main ideas and mechanisms appear to be clear. This allows the preparation of the fundamental tolerance model for robotic pickandplace considering the operation viewpoint, and more specific tools can be created as the related research progresses further.
Model overview
Concept and assumptions
In this paper, the tolerance stackup of the pickandplace operation is modeled as a sequence of transformations in a kinematic graph, which uses the same idea as the one behind machine tool accuracy estimation in the work of Slocum [28]. This concept resembles the definition of a mechanism using standard mechanism joints. The joints are considered as loworder kinematic pairs [5], which are essentially parametric transformations between the frames of components (rigid bodies). The idea is to represent different tolerance types as mechanism joints. In this way, the tolerance propagation can be modeled simply using matrix multiplication. These matrices contain nominal and tolerance values, as well as joint variables (in case of actual mechanism joints, such as the robot joints) as parameters for the underlying translational and rotational transformations. Since multiple frames can be within a single rigid body, connected through tolerances, the shape of the rigid body becomes determined and fixed as a particular tolerance value is selected from the defined tolerance interval.
From the general tolerance modeling methods, the proposed approach is most similar to the variational solid modeling approach [11]. However, in the proposed one, the kinematic joints are applied as matrix transformations (using homogeneous transformation matrices), and the number of constraint equations is minimized. This is done by first preparing the spanning tree of the kinematic graph, then by forming the constraint equations through loop closures (wherever necessary). Hence, open loops can be evaluated simply by substituting in the geometrical parameters, and for closed loops, only a minimum number of constraint equations need to be solved, before evaluating the FRs. Furthermore, this allows the deviation of both the position and orientation components of feature frames to stack up. Even though variation in orientation (e.g., perpendicularity or parallelism) is captured by tolerance zones for geometric tolerances in the standard, the representation of orientation as inclination is beneficial in scenarios such as robotic peghole insertion or grasping.
Considering the robotic assembly aspect of the pickandplace operation, the term workpiece refers to the component, which is manipulated by the robot. Here, the model describes the pose of this workpiece throughout the operation. Hence, the workpiece needs to be attached in the transformation chain in a way to continuously represent its actual physical contacts with the other cell components. Therefore, the workpiece is detached and reattached in the transformation chain according to the operation process steps (i.e., seizing and releasing). Correspondingly, the workpiece state is investigated before and after the seizing and the releasing actions.
The number of DoFs of the workpiece is restricted throughout the operation by the different components the workpiece is in contact with, namely, by the picking workholder, gripper, and placing workholder. In general, the workpiece does not move relative to the contacting component, except during the contact transition. However, the tolerance stackup (including the contact transition) can result in deviations in the free directions between the aligned frames of the workpiece and other components. Therefore, these contacts should be captured through the proper parameterization of the transformations, i.e., according to the DoFs of the workpiece relative to the contacting components.
The tolerances appearing in the transformation chain need to be determined using existing tolerance analysis methods (as these are mostly resultant tolerances and not explicitly specified by the designer), or through measurements. It is noted that in case of individual workcell setup, some of the deviation sources—typically the component manufacturing tolerances and the robot accuracy—can be compensated through calibration. In the present paper, every input parameter or parameter interval (in case of tolerances) is assumed to be predetermined, along with the actual pickandplace strategy (application of metrology and tolerance enhancing techniques). Also, since manufacturers specify the repeatability of the manipulators on their end flange, but not on their links, manipulators are presented as a single unit during tolerance analysis, and not as separate links. It is noted that in case of modular robots, the precision of individual modules could be taken into account, and it is possible to set up the corresponding kinematic chain automatically [53].
The presented model defines a set of possible frames to capture the FRs. To assess the fulfillment of FRs, inequalities need to be formed for each relevant general direction of each relevant frame. Alternatively, if required, inequalities can be formulated specifically for different artifacts of the workpiece or other components. These can be captured via additional frames, or even as geometric constraints. The FRs and included KCs need to be defined by the designer, and these are also assumed to be given in this paper.
Part of the applied functions (originating from tolerance influencing factors, see Section 3.5) throughout the tolerance stack are not continuously differentiable, and the extraction of orientation components has no closedform solution. Consequently, the prepared tolerance chain is numerically evaluable. The evaluation of the model is carried out using Monte Carlo simulation, by substituting the nominal values, joint variables (if any), and tolerance set instances (sampled from the corresponding tolerance intervals) into the parametric tolerance chain.
The established transformation chain suits the family of robotic pickandplace applications. When applied for a particular robotic workcell, the chain needs to be prepared by considering the operation conditions and kinematic joints present in the specific scenario. Then, by substituting the dimensional and tolerance parameters, the FRs can be evaluated in the different workpiece states (poses) along the operation sequence. The general structure of the tolerance model is applicable for most industrial pickandplace tasks—where no humans are involved—in the presented form. For niche cases, the model might need slight adjustments to properly reflect on the particular scenario. The application of this model is most beneficial in case of tasks that are geometrically well defined and where the component relations are clear, and the component selection or design is still adjustable (i.e., in the early design phase).
Pickandplace operation
A pickandplace operation can be described as two subsequent (i.e., picking and placing) manipulator movement and gripper setting (or activating and deactivating) steps. From the control perspective, this can be defined using robot configurations; from the point of view of the workpiece, it can be described through poses. For clarification, poses and frames are defined in the robot task space as task space points; however, a pose corresponds to the reference frame of a rigid body, whereas frames correspond to a constant transformation with respect to rigid body references (can be multiple per rigid body). Configurations are defined in the robot configuration space as a robot joint vector.
The important robot configurations in the presented tolerance chain are the seizing and releasing configurations. On the path planning and control side, usually there are approach and retreat configurations for both of these configurations. Additionally, for flexible scenarios, metrology configurations can exist, for the execution of metrology and resolving the pose uncertainties. The seizing configuration corresponds to the workpiece picking pose. In the seizing configuration, the gripper is set to a seizing setup, therefore making contact with the workpiece. At this point, the workpiece gets into the seized pose, which is bound to the gripper. Then, the robot moves to the releasing configuration, transferring the workpiece to the releasing pose. This corresponds to the workpiece placing pose. Here, the gripper is set to a releasing setup; therefore, the workpiece breaks contact with the gripper, and establishes one with the placing workholder.
The component contacts can occur in multiple ways. The type of contact transient during seizing and releasing, as well as component compliances can have significant effect on the seizing and releasing action. If the contact break and establishment happen simultaneously, or one after the other (e.g., seizing with vacuum cup, or dropping workpiece when releasing), then the seizing and releasing action can introduce considerable additional deviations. On the other hand, if the first contact is maintained even after establishing the new contact (until the next robot movement), an overconstrained case occurs, where the components are prone to stuck or damage (e.g., peginhole problem with a rigid gripper). In order to keep the paper more concise, here, the effects of the contact transients are simplified and are only taken into account in case of leading features (see Section 3.5). Nevertheless, these effects need to be considered on a casebycase basis.
Transformation chain setup
There are relevant frames in a spatial and temporal basis as well. The workcell reference is the base frame, in which each component reference frame is defined. The relation of the frames can be represented on a kinematic graph, which is shown in Fig. 1. Here, the relevant frames are the manufacturing datum frames of the workpiece (wp,ref), picking workholder (wh1,ref), gripper (gr,ref), and placing workholder (wh2,ref).
Component feature frames are defined relative to their reference frames. For the workpiece, these are the picking (wp,pick), grasp (wp,grasp), and placing (wp,place) frames. The pairs of these frames are on the contacting components. The picking workholder has the picking frame (wh1,pick), the placing workholder has the placing frame (wh2,place). The gripper has the grasp frame (gr,grasp); the grasp frame pair is usually the result of the grasp planning.
These frames are shown in case of a sample pickandplace scenario in Figs. 2 and 3. As visible, the frames exist in different phases of the operation for the moving components. The workpiece poses are the picking (p1), seized (g1), releasing (g2), and placing poses (p2), while the gripper poses are the seizing (corresponding to seizing configuration, g1) and releasing poses (corresponding to the releasing configuration, g2).
Transformation matrices
Frames are described by homogeneous transformation matrices, parameterized with three translational and three rotational components. However, locally different representations can be selected (e.g., deviation in a cylindrical or Cartesian coordinate system). The tolerance chain includes manipulation, seizing, releasing, manufacturing, and metrology tolerances as parameters.
Homogeneous transformation matrices, realizing transformation from component c1, frame f1, pose o1 to component c2, frame f2, and pose o2, are denoted with \( {\boldsymbol{T}}_{\mathrm{c}2,\mathrm{f}2,\mathrm{o}2}^{\mathrm{c}1,\mathrm{f}1,\mathrm{o}1} \), or simply \( {\boldsymbol{T}}_{\mathrm{c}2,\mathrm{f}2,\mathrm{o}2}^{\mathrm{base}} \) if f1 is the base frame. For the sake of simplicity, pose indices are only noted at the pose changing components (i.e., in case of the workpiece and the gripper). Frame transformations are modeled as follows:
where R is the rotation matrix and d is the translation vector containing both nominal (n) and tolerance (t) components:
The rotational component can be constructed in multiple ways, depending on the order of rotations. Here, a sequence of yaw, pitch, and roll rotations is applied (this is suitable for typical tolerance values, but other conventions can also be applicable in case of singularity issues), which results in the following formula:
where c_{x} stands for cos(x), and s_{x} for sin(x). Each angle value contains a nominal and a tolerance component. The rotation angles (corresponding to the axes of the frame) are represented by the vector r:
The tolerance and nominal parameters are summarized in arrays as:
This representation allows the extraction of each angle from the rotation matrix (numerically), their adjustment, and the recreation of the adjusted rotation matrix, in a consistent way. The corresponding screw parameters of a transformation matrix are represented as an array:
In the following, for the sake of simplicity, the arguments of the transformations are only spelled out where relevant.
As an example, if a finger gripper with parallel finger planes seizes a slablike feature on a workpiece (see Fig. 3); a planar kinematic joint needs to be set up to represent the tolerance of the seizing action. In this case, due to the planar contact, positioning tolerances from the gripper activation occur only in x and z direction, and orientation tolerance around the y axis (based on Fig. 3). The joint is formulated as follows:
which contains both nominal and tolerance parameters, and can be evaluated to any tolerance set instance.
Formulation of the transformation chain
Having prepared the construction of transformation matrices, the transformation chain can be formed. First, considering nominal transformations (denoted with T^{∗}), an ideal case is assumed. The workpiece starts in the picking pose, where its picking frame is aligned with the picking frame of the picking workholder \( \left({{\boldsymbol{T}}^{\ast}}_{\mathrm{wp},\mathrm{p}\mathrm{ick},\mathrm{p}1}^{\mathrm{base}}={{\boldsymbol{T}}^{\ast}}_{\mathrm{wh}1,\mathrm{p}\mathrm{ick}}^{\mathrm{base}}\right) \).Thereby, the nominal grasp frame of the workpiece is determined through the workpiece reference frame \( \left({{\boldsymbol{T}}^{\ast}}_{\mathrm{wp},\mathrm{grasp},\mathrm{p}1}^{\mathrm{base}}\right) \). Then, the robot is commanded to align the gripper grasp frame with the grasp frame of the workpiece\( \left({{\boldsymbol{T}}^{\ast}}_{\mathrm{gr},\mathrm{g}\mathrm{rasp},\mathrm{g}1}^{\mathrm{base}}={{\boldsymbol{T}}^{\ast}}_{\mathrm{wp},\mathrm{g}\mathrm{rasp},\mathrm{p}1}^{\mathrm{base}}\right) \). The gripper is activated, and the workpiece is detached from the picking workholder and attached to the gripper with the grasp frames aligned, as it gets to the seized pose \( \left({{\boldsymbol{T}}^{\ast}}_{\mathrm{wp},\mathrm{g}\mathrm{rasp},\mathrm{g}1}^{\mathrm{base}}={{\boldsymbol{T}}^{\ast}}_{\mathrm{gr},\mathrm{g}\mathrm{rasp},\mathrm{g}1}^{\mathrm{base}}\right) \).
Next, the robot is commanded to the releasing configuration, where the placing frame of the now attached workpiece is aligned with the nominal placing frame of the placing workholder\( \left({{\boldsymbol{T}}^{\ast}}_{\mathrm{wp},\mathrm{place},\mathrm{g}2}^{\mathrm{base}}={{\boldsymbol{T}}^{\ast}}_{\mathrm{wh}2,\mathrm{place}}^{\mathrm{base}}\right) \). Here, the gripper is set to release the workpiece, detaching the workpiece from the gripper and attaching it to the placing workholder, as it gets to the placing pose. At this point, the workpiece is laid into the placing pose. The placing frame of the workpiece and the placing workholder meet \( \left({{\boldsymbol{T}}^{\ast}}_{\mathrm{wp},\mathrm{place},\mathrm{g}2}^{\mathrm{base}}={{\boldsymbol{T}}^{\ast}}_{\mathrm{wh}2,\mathrm{place}}^{\mathrm{base}}\right) \), and the pickandplace operation is finished.
From here, assuming tolerances, the tolerance chain is formed on four branches, which are then connected. (i) The workpiece grasp frame for the picking pose is formed by the tolerance stackup from the base frame through the picking workholder and the workpiece \( \left({\boldsymbol{T}}_{\mathrm{wp},\mathrm{grasp},\mathrm{p}1}^{\mathrm{base}}\right) \). (ii) The gripper grasp frame for the seizing pose is formed from the base through the robot links and gripper \( \left({\boldsymbol{T}}_{\mathrm{gr},\mathrm{g}\mathrm{rasp},\mathrm{g}1}^{\mathrm{base}}\right) \). (iii) The workpiece placing frame is formed for the releasing pose from the base frame through the robot links, gripper, and workpiece \( \left({\boldsymbol{T}}_{\mathrm{wp},\mathrm{grasp},\mathrm{p}2}^{\mathrm{base}}\right) \). And (iv) the placing frame of the placing workholder is formed from the base frame \( \left({\boldsymbol{T}}_{\mathrm{wh}2,\mathrm{place}}^{\mathrm{base}}\right) \). As the robot is commanded to the nominal workpiece grasp frame in the picking pose, and the gripper seizes the workpiece, the grasp frames (i–ii) are now misaligned because of the tolerances. These misalignments can further increase as the two frames are getting attached, due to the contact transition (seizing tolerance). This effect is not present in the nominal case; however, in reality, the workpiece pose changes during seizing (and also during releasing) due to the physical contact. The gripperworkpiece attachment (see Figs. 4 and 5) allows the formation of the workpiece placing frame when manipulated to the nominal placing pose.
The workpiece is then manipulated to the releasing pose, introducing further deviations through the robot positioning precision. Finally, when releasing the workpiece, an additional releasing tolerance is considered. With this, the misaligned placing frames (iii–iv) are getting attached (see Figs. 6 and 7). The measure of misalignment in the placing frame pair \( \left({\boldsymbol{T}}_{\mathrm{wp},\mathrm{p}\mathrm{lace},\mathrm{p}2}^{\mathrm{wh}2,\mathrm{p}\mathrm{lace}}\right) \) thus becomes evaluable in tolerance set instances.
It is noted that the robot is not necessarily commanded to reach exactly \( {{\boldsymbol{T}}^{\ast}}_{\mathrm{wp},\mathrm{grasp},\mathrm{p}1}^{\mathrm{base}} \) or \( {{\boldsymbol{T}}^{\ast}}_{\mathrm{wh}2,\mathrm{place}}^{\mathrm{base}} \). A constant offset can also be considered between the grasp and placing frame pairs, which is overcome by workpiece relative motion when being seized or dropped. However, for the sake of simplicity, no additional frames are introduced for these, and their effects are considered in the seizing and releasing tolerances.
Main functional requirements
To achieve a successful pickandplace operation, multiple FRs need to be satisfied. FRs are arranged into arrays of feasible intervals, similarly to the nominal and tolerance parameters in formula (5) and are denoted with c. First and foremost, the workpiece needs to be placed into the placing pose within a given tolerance range in each direction (c_{place}). The allowed deviation for the placing pose is generally determined by the design of the assembly that contains the workpiece, or by the following process, for which the placed workpiece serves as an initial condition.
In addition, the FR for picking has to be met (c_{pick}). Usually, this can be defined based on the gripping range within which the gripper can seize the workpiece. For example, in case of a vacuum gripper, the vacuum cup must be located within the specific planar surface; otherwise, the vacuum cup falls to the edge of this surface and the gripping will fail.
Depending on the task, additional frames (e.g., multiple feature frames for placing) and FRs can be defined. There can be different configurations and geometric artifacts, for which the feasibility needs to be checked (e.g., to avoid collision in different poses). Furthermore, FRs can be formulated not only as simple intervals, but also as geometric constraints. These need to be specified on a casebycase basis, allowing the preparation of more detailed evaluation of operation feasibility.
Tolerance influencing factors
The tolerance chain does not only depend on the dimensional conditions of the pickandplace scenario. Multiple other factors were identified that have a significant effect on the tolerance chain. These are defined in each direction separately, as follows:

Leading feature (i.e., selfalignment and selflocation): when there is relative motion between the components (during seizing or releasing), depending on the physical contact between specific faces, edges and vertices, the workpiece can be either free or guided.

Pose specification: the picking and placing poses can be either known or not known with sufficient precision in the design phase.

Servoing: servo technique(s) can be either applied or not applied for robot positioning in case of the picking and placing poses.
The decisions on the tolerance influencing factors are arranged into arrays of Boolean values (similarly as in formula (5)) denoted with k, while the tolerance modifying functions are denoted with F(k, T), where T is the transformation matrix on which the modification is applied. The separate application of these functions to each translational and rotational direction is possible due to the selected rotational matrix formulation in equation (3). The effect of tolerance influencing factors on the operation sequence is shown in Fig. 8.
Leading feature
Considering workpiece selfalignment and selflocation, when being guided in a particular direction, a workpiece is limited to move between the guiding features in this direction. For example, if a workpiece is placed on a flat surface, it is free to move in the plane of the surface, but guided in the direction normal to the surface. Therefore, if the workpiece is dropped above the plane, its position normal to the plane will be limited by the plane as the two meet.
However, leading features are not exactly the same as a DoF restriction, as they have multiple effects. Leading features (i) guide the previous workpiece manipulation action, potentially reducing the accumulated tolerances up to the point of guiding, pose a new tolerance requirement to avoid workpiece wedging or damage while performing this previous action (ii), and potentially while performing the next action (iii).
For (i), the leading feature saturates the tolerance up to the following member of the tolerance chain, which improves the achievable precision (e.g., when using chamfers or cones). This allows worse precision while fitting, then improves the precision utilizing the geometrical, physical contactbased guidance of the fixture. Effects (ii) and (iii) pose new FRs in order to avoid overconstrained workpiece manipulation. As a simplified example, in order to utilize the selflocating capability of a chamfer, the deviation must be low enough to maintain the counterpart feature on the chamfer slope. It is noted that to determine actual selfalignment and selflocating ranges, as well as corresponding FRs, simulations or experiments are necessary, as their behavior is complex and depends on many factors (shape, material pair, forces, etc.) [36].
Leading features can be capable of handling different types of misalignments (axial, radial, lateral, or torsional) with a certain limit, depending on shape. Furthermore, these features can exist on the picking and placing side, on the gripper or on the workpiece itself. Leading features can be geometric (cones, chamfers) or kinematic (closing gripper fingers). Tolerance adjustment can happen during seizing and releasing, depending on the leading feature setting.
Although leading features are linked with the three effects above, these only manifest simultaneously during the seizing process. Before seizing, the previous workpiece manipulation step is not made as a part of the present assembly task (only (iii) applies), while at releasing, the following workpiece manipulation step is also not done as a part of the present assembly task (only (i) and (ii) apply). Moreover, as mentioned earlier, these effects are influenced by the contact transients and need to be considered accordingly.
The presence of leading features is represented with Boolean arrays. These are k_{a,wh1} when the workpiece lays on the picking workholder before seizing, k_{a,gr} while the workpiece is being seized by the gripper, and k_{a,wh2} when it is being released onto the placing workholder. Each of these determines whether a corresponding FR is present or not. These are c_{a,wh1}, c_{a,gr}, and c_{a,wh2}, which often overwrite the original requirements (c_{pick} and c_{place}) to looser ones. On the other hand, only k_{a,gr} and k_{a,wh2} have an effect on the aggregated tolerances. The corresponding functions are F_{a,gr}(k_{a,gr}, T) and F_{a,wh2}(k_{a,wh2}, T), which return the original transformation if there is no leading feature; however, they saturate the values in the guided directions to the corresponding ranges (a_{gr}, a_{wh2}).
For example, if a workpiece is guided during seizing in ξ and η but not in ζ, then the corresponding angle values are calculated from the original rotational matrix, and these get saturated, while the original angle value is kept in the ζ direction. Then, based on these values, the rotation and the whole transformation matrix are recalculated. When applied in a single direction (x), this can be formulated as:
The formulation of F_{a,wh2}(k_{a,wh2}, T) is completely analogous to formula (9).
Pose specification
Considering pose specification, there is a corresponding precision value for such a pose, which is sufficiently specified in the design phase. Thus, the tolerance of this pose is part of the geometrical tolerance stackup and has to be considered accordingly, when checking feasibility. On the other hand, if the pose is not known precisely (typically in flexible scenarios), the design phase has to be carried out parametrically. Here, the pose has to be determined precisely during operation by means of metrology. In these cases, metrology precision needs to be considered when calculating the tolerance stackup.
Whether or not the picking and placing pose are known with sufficient precision is captured with the Boolean arrays k_{pick} and k_{place}, and the corresponding functions are F_{pick}(k_{pick}, T) and F_{place}(k_{place}, T). These functions return the input transformation matrix with the tolerance parameters corresponding to the designed pose (t_{p1} and t_{wh2,place}) and/or corresponding to the complete metrology process (t_{m,p1} and t_{m,p2}). For example, if the workpiece picking position is only known in z direction but not in x and y directions, the translational part of the matrix will be set up using t_{m,p1,x}, t_{m,p1,y} and t_{p1,z}. When applied in a single direction (x), this can be formulated as:
The formulation of F_{place}(k_{place}, T) is completely analogous to formula (10).
Servoing
Lastly, applying servo techniques is a further possibility, during which certain elements of the tolerance chain can be improved or bypassed entirely, enhancing the overall precision capability of the system. Servo control allows the online correction of robot positioning by realizing a closedloop motion control, until a certain condition is met (e.g., a certain positioning precision is achieved based on a camera system). In these scenarios, similarly to the case of poses that are not known precisely, the precision of the metrology has to be taken into account when evaluating the tolerance chain.
Whether or not servoing is applied in case of picking and in case of placing is captured with the Boolean arrays k_{s,g1} and k_{s,g2}, and the corresponding functions are F_{s,g1}(k_{s,g1}, T) and F_{s,g2}(k_{s,g2}, T). These functions enable the substitution of the accumulated tolerances to servo tolerances (t_{s,g1} and t_{s,g2}), returning the original transformation matrix if servoing is not applied, and constructing a new transformation otherwise. For example, in case there is servo motion on the picking side in x and y direction, then the x and y components of the translational transformation will be changed to t_{s,g1,x} and t_{s,g1,y}, while the z component remains that of the original. When applied in a single direction (x), this can be formulated as:
The formulation of F_{s,g2}(k_{s,g2}, T) is completely analogous to formula (11).
Model formulation
After the introduction of every necessary component for the preparation of the complete model, in this section, the detailed deduction of the tolerance analysis is presented. First, the necessary input data are overviewed. Then the transformation is constructed from the individual transformations between component frames and tolerance influencing functions. Finally, the evaluation of FRs is presented to assess the feasibility of the pickandplace operation.
Input data
Both the requirement and the available capacity side need to be provided in order to draw conclusions about taskequipment compatibility. Based on the tolerance influencing factors, the following Boolean information needs to be decided for every direction:

is there a leading feature on the picking workholder, gripper, and placing workholder (k_{a,wh1}, k_{a,gr} and k_{a,wh2})?

is the workpiece picking and placing pose known with sufficient precision (k_{pick} and k_{place})?

is picking and placing servo applied (k_{s,g1} and k_{s,g2})?
Quantitative data needs to be defined in form of:

manufacturing tolerances: workpiece (t_{wp,pick}, t_{wp,place}, t_{wp,grasp}), gripper (t_{gr,grasp}), picking (t_{wh1,ref}, t_{wh1,pick}), and placing workholder (t_{wh2,ref}, t_{wh2,place}),

location tolerance: workpiece picking pose on picking workholder (t_{p1}),

tolerance of the complete metrology and servo metrology methods (t_{m,p1}, t_{m,p2}, t_{s,g1}, t_{s,g2}),

manipulator positioning tolerance (t_{r,g1}, t_{r,g2}),

seizing and releasing tolerance (t_{g1}, t_{g2}),

selflocation and selfalignment ranges of the gripper and the placing workholder (a_{gr}, a_{wh2})
Finally, FRs need to be formulated generally in form of feasible tolerance ranges:

the geometric relation between the gripper and the workpiece during seizing for successful picking (c_{pick}); and between the workpiece and the placing workholder for successful placing (c_{place}),

additional FRs introduced by leading features for feasible picking (c_{a,wh1} and c_{a,gr}) and placing (c_{a,wh2}).
Tolerance model
The main goal of the generalized tolerance model is to determine the transformation between the corresponding placing frames of the placing workholder and workpiece \( \left({\boldsymbol{T}}_{\mathrm{wp},\mathrm{p}\mathrm{lace},\mathrm{p}2}^{\mathrm{wh}2,\mathrm{p}\mathrm{lace}}\right) \), for which the foremost tolerance requirement is defined. In order to achieve this parametric transformation, the tolerance chain is set up starting from the workpiece picking pose, up to the point when it settles on the placing workholder after releasing. The summary of transformationrelated notations is given in Table 1.
The first step is to determine the grasp frame relation before seizing (see Fig. 4). This starts by determining the workpiece picking pose with respect to the base frame:
where \( {\boldsymbol{T}}_{\mathrm{wh}1,\mathrm{ref}}^{\mathrm{base}} \) contains the workholder location tolerance (t_{wh1,ref}), \( {\boldsymbol{T}}_{\mathrm{wh}1,\mathrm{pick}}^{\mathrm{wh}1,\mathrm{ref}} \) contains the workholder machining tolerances (t_{wh1,pick}), and \( {\mathbf{F}}_{\mathrm{pick}}\left({\boldsymbol{k}}_{\mathrm{pick}},{\boldsymbol{T}}_{\mathrm{wp},\mathrm{p}\mathrm{ick},\mathrm{p}1}^{\mathrm{wh}1,\mathrm{p}\mathrm{ick}}\right) \) is the workpiece picking frame relative to the workholder picking frame including designed (t_{p1}) and/or sensed parameters (t_{m,p1}) depending on k_{pick}. Next, the workpiece grasp frame is calculated:
where \( {\boldsymbol{T}}_{\mathrm{wp},\mathrm{p}\mathrm{ick},\mathrm{p}1}^{\mathrm{wp},\mathrm{ref},\mathrm{p}1} \) and \( {\boldsymbol{T}}_{\mathrm{wp},\mathrm{grasp},\mathrm{p}1}^{\mathrm{wp},\mathrm{ref},\mathrm{p}1} \) contain workpiece machining inaccuracies (t_{wp,pick} and t_{wp,grasp}, respectively). In the following, the gripper grasp frame is determined:
where \( {\boldsymbol{T}}_{\mathrm{gr},\mathrm{ref},\mathrm{g}1}^{\mathrm{base}} \) contains the robot positioning inaccuracies (t_{r,g1}) and \( {\boldsymbol{T}}_{\mathrm{gr},\mathrm{g}\mathrm{rasp},\mathrm{g}1}^{\mathrm{gr},\mathrm{ref},\mathrm{g}1} \) contains gripper machining tolerances (t_{gr,grasp}). Then, the relation of the grasp frame pair is determined, together with the servo technique at picking:
where F_{s,g1}(k_{s,g1}, T) enables the substitution of the accumulated tolerances to servo tolerances (t_{s,g1}) based on k_{s,g1}. The next step is the seizing process. At this point, the workpiece is detached from the picking surface:
where \( {\boldsymbol{T}}_{\mathrm{wp},\mathrm{g}\mathrm{rasp},\mathrm{g}1}^{\mathrm{wp},\mathrm{g}\mathrm{rasp},\mathrm{p}1}={\boldsymbol{T}}_{\mathrm{wp},\mathrm{ref},\mathrm{g}1}^{\mathrm{wp},\mathrm{ref},\mathrm{p}1} \), and it describes the seizing transient, including the misalignment (t_{g1}), introduced by the seizing action. Then, the workpiece is getting attached to the gripper (see Fig. 5):
Depending on the leading feature on the gripper (k_{a,gr}), the misalignment between the grasp frames is reduced by the application of F_{a,gr}(k_{a,gr}, T) to the corresponding selfalignment and selflocating range (a_{gr}). Next, the workpiece grasp frame is determined with respect to the gripper reference frame:
The result is the transformation between the seized workpiece and the gripper. This needs to be applied at the end of the last manipulator link, after controlling it to the releasing configuration (see Fig. 6). Then, the workpiece grasp frame is to be determined with respect to the base frame in the releasing configuration:
where \( {\boldsymbol{T}}_{\mathrm{wp},\mathrm{g}\mathrm{rasp},\mathrm{g}2}^{\mathrm{gr},\mathrm{ref},\mathrm{g}2}={\boldsymbol{T}}_{\mathrm{wp},\mathrm{g}\mathrm{rasp},\mathrm{g}1}^{\mathrm{gr},\mathrm{ref},\mathrm{g}1} \) and \( {\boldsymbol{T}}_{\mathrm{gr},\mathrm{ref},\mathrm{g}2}^{\mathrm{base}} \) contains the manipulator positioning inaccuracies (t_{r,g2}). Next, the workpiece is detached from the gripper, as it is being released (see Fig. 7):
where \( {\boldsymbol{T}}_{\mathrm{wp},\mathrm{g}\mathrm{rasp},\mathrm{p}2}^{\mathrm{wp},\mathrm{g}\mathrm{rasp},\mathrm{g}2} \) describes the releasing transient, including the misalignment (t_{g2}), introduced by the releasing action. Then, as the workpiece arrives to the placing pose, the workpiece placing frame needs to be determined:
where \( {\boldsymbol{T}}_{\mathrm{wp},\mathrm{grasp},\mathrm{p}2}^{\mathrm{wp},\mathrm{ref},\mathrm{p}2}={\boldsymbol{T}}_{\mathrm{wp},\mathrm{grasp},\mathrm{p}1}^{\mathrm{wp},\mathrm{ref},\mathrm{p}1} \), and \( {\boldsymbol{T}}_{\mathrm{wp},\mathrm{p}\mathrm{lace},\mathrm{p}2}^{\mathrm{wp},\mathrm{ref},\mathrm{p}2} \) contains workpiece machining inaccuracies (t_{wp,place}). At this point, the placing workholder and the placing pose are to be introduced:
where \( {\boldsymbol{T}}_{\mathrm{wh}2,\mathrm{ref}}^{\mathrm{base}} \) contains the workholder location tolerances (t_{wh2,ref}), and \( {\mathbf{F}}_{\mathrm{place}}\left({\boldsymbol{k}}_{\mathrm{place}},{\boldsymbol{T}}_{\mathrm{wh}2,\mathrm{place}}^{\mathrm{wh}2,\mathrm{ref}}\right) \) contains the designed (t_{wh2,place}) and/or sensed parameters (t_{m,p2}) based on k_{place}. Then, the placing frame pair relation is determined and the placing servo technique is applied (if any):
where F_{s,g2}(k_{s,g2}, T) corresponds to the placing servo, reducing the accumulated tolerances to the level of servo tolerance (t_{s,g2}) based on k_{s,g2}. Finally, the effect of the last leading feature is applied (if any):
where F_{a,wh2}(k_{a,wh2}, T) corresponds to the leading feature at placing, reducing the accumulated tolerances to the corresponding range (a_{wh2}) based on k_{a,wh2}. With this, the placing frame pair and other geometric relations corresponding to FRs are set up parametrically.
Evaluation of functional requirements
To achieve a successful pickandplace operation, all the actually considered FRs need to be simultaneously satisfied. First, the variation in the workpiece placing pose (relative to the placing workholder) is checked against the allowed deviation range c_{place}, in each direction. If this is not fulfilled, the application of leading features (e.g., chamfering), or precision enhancement is necessary in form of tolerance improvement or servo techniques. Here, c_{place} defines a feasible tolerance region. Assuming symmetric ranges, this can be written in a single direction (x) as:
Next, the FR for successful workpiece picking (c_{pick}) is checked. This can be written in a single direction (x) as:
Then, the FRs posed by the leading features need to be assessed (c_{a,wh1}, c_{a,gr} and c_{a,wh2}) to avoid overconstraining the system during contact transition. These can be formulated in a single direction (x) as follows:
The corresponding evaluations can be performed via Monte Carlo simulation. By substituting the input parameters with tolerance values sampled from the given intervals into the transformation chain, any transformation can be evaluated to check the fulfillment of FRs.
Implementation
The proposed tolerance model is presented through a flexible pickandplace case study. A physical workcell was set up as an experimental demonstrator cell for palletizing workpieces laying in a semistructured arrangement. Separated workpieces, laying in one of their stable poses, are manipulated from a light table to the corresponding workpiece pallet. The used manipulator is an UR5 robot arm equipped with a Robotiq 2f85 finger gripper and an IDS XS camera. The actual workpiece picking poses are determined using this 2D camera, with the help of the backlighting capability of the light table. The workcell setup, with the applied frames, is shown in Figs. 2 and 3. The preparation and operation of the workcell is presented in more detail in [7].
Input data
For the sake of simplicity, the tolerance analysis is regarded as a planar problem, considering only the x, y, and ζ components. This consideration is viable, since the workpiece picking and placing pose correspond to the same workpiece stable pose, i.e., no regripping and no reorientation of the workpiece is necessary, apart from rotating around the vertical axis. The screw parameters x, y, and ζ are represented as an array, where x and y components are translations parallel to the picking plane and ζ is the rotation around the axis normal to the picking plane (vertical axis). The decision parameters are Boolean, while dimensions, tolerance and FR ranges are given in mm for translational and in ° (deg) for rotational parameters. For the sake of conciseness, tolerance and FR ranges are considered symmetric, closed intervals with zero mean value, and are represented with a single value with a ± sign.
First, the decision parameters are defined. The workpieces lay on the light table in a semistructured way; their pose is not known precisely in x, y and ζ directions. These have to be resolved using the camera fixed on the robot. However, the placing poses are known with sufficient precision in the design phase. Since the workpieces lay freely on a table, they are not guided in the plane. On the other hand, the finger gripper guides the workpiece in y and ζ directions (see Fig. 3) and the placing pose is guided in all three directions via a chamfered centering pin and chamfered columns (shown in Fig. 14). Lastly, servoing is applied neither in the picking nor in the placing phase. The corresponding factors are summarized in Table 2.
Next, the nominal and tolerance parameters are defined; these are listed in Table 3 (values with v1 and v2 subscripts are discussed in Section 5.2). The only relevant nominal dimension in the tolerance model corresponds to the workpiece grasp frame (n_{wp,grasp}) relative to the workpiece reference. Other dimensions are either zero or they cancel out in this particular scenario. The positioning precision of the robot is given, together with the selfpositioning and selfalignment ranges of the placing workholder, as well as the manufacturing tolerances of the workpiece, gripper, and picking and placing workholders. It is noted that the tolerance of the workpiece picking and placing frame (t_{wp,pick} and t_{wp,place}) relative to the workpiece reference is zero, as these all coincide. Furthermore, the tolerance of the picking and placing workholder references (t_{wh1,ref} and t_{wh2,ref}) are also considered zero, because the position of the corresponding picking and placing frames were measured relative to the workcell reference frame directly.
The remaining tolerance parameters (metrology, seizing, and releasing) were needed to be determined experimentally. The precise and exhaustive exploration of these parameters is not in the scope of this paper; nevertheless, simple measurement setups and overestimation of the parameters were carried out to have acceptable input data for the simulation. Correspondingly, to keep the paper more concise, only a brief overview is given about the experiments.
The combined tolerance of the metrology system and image processing algorithms (t_{m,p1}) was measured by setting a workpiece in a pose on the light table using different 3Dprinted temporary locators (see Fig. 9a, b), and capturing images of workpiece. The captured images have a resolution of 1280 × 720. After every captured image, the robot (with the mounted camera) was moved to a random picking configuration, then back to the image capturing configuration, to emulate the real operation and add disturbance to the vision system. The workpiece pose was evaluated on each image against the originally set up pose, using the image processing algorithm employed in the demonstrator (see Fig. 10). The visionbased localization employs feature recognition and homography transformation (the algorithm is described in detail in [7]). This process was repeated using multiple workpiece setups.
The seizing tolerance (t_{g1}) was estimated by seizing a similarly positioned workpiece multiple times (using the temporary locator shown in Fig. 9c) with disturbed seizing configurations, and measuring the variation of the seized pose in the free direction with a digital caliper (with 0.01 mm resolution), relative to the gripper fingers. This process is shown in Fig. 11. The selfpositioning and selfalignment range of the gripper (a_{gr}) was measured in the same setup in the guided directions, using the same camera system. However, the image processing algorithm was refined for more accurate results; the achieved precision of the camerabased measurement system on the same dataset is ±0.07 mm, ± 0.06 mm and ±0.17° in x, y,and ζ, respectively.
Finally, the releasing tolerance (t_{g2}) was measured by repeatedly fixing the workpiece manually into the gripper using a different temporary locator (see Fig. 12), then releasing it on the light table. The variation in the releasing pose was measured, again, using the camerabased measurement system.
FRs are defined symmetrically for picking and placing, and additionally for the leading features at the gripper and the placing workholder. These are listed in Table 4. The FR ranges were validated in the physical workcell. The requirement corresponding to seizing and the gripper alignment (c_{pick} and c_{a,gr}) were checked in the seizing tolerance measurement setup, to establish feasible extrema for the FR ranges. The leading feature on the placing side (a_{wh2}) is set so that it fulfills the placing requirement; therefore, c_{place} is automatically satisfied if c_{a,wh2} is fulfilled. That is, if after releasing the workpiece, it is guided to the placing pose by the gravity and the leading features, the placing pose is satisfactory. Otherwise, if the workpiece stays on top of the guiding features, does not reach the bottom support plane, or tilts out of the workholder, the placing pose is not satisfactory. Consequently, the analysis focuses on c_{a,wh2} instead of the original c_{place} requirement. The feasibility bounds were experimentally evaluated for c_{a,wh2} by manually fixing the workpiece into the gripper, once again, with a temporary locator (same way as in Fig. 12), moving the robot to a disturbed releasing configuration and releasing the workpiece. Two cases with different releasing configurations are shown in Fig. 13, one with correct, the other with a failed placement. In the determined deviation region, the releasing process was consistently successful for each sampled combination of deviations.
Results
By applying the proposed tolerance model with the above input data, tolerance analysis can be performed for the pickandplace scenario. To get a comprehensible result, a Monte Carlo simulation was carried out. The model was evaluated for 20,000 tolerance set instances, sampled in the given regions assuming normal distribution on 99.72% confidence level. To visualize the deviations, the 2D projection of the geometry was triangulated and transformed based on the results, the transformed shapes were overlaid, and their union was computed. In order to reduce the number of computations, the extreme displacements (with combined translations and rotation) were selected heuristically. The final union results in a boundary within which the actual workpiece will lay according to the tolerance data. Since there are two different aligning features, (chamfered orienting columns and chamfered positioning pin), two different 2D projection boundaries of the workpiece were calculated. Based on the resultant transformations, the worst tolerance stackup case can be found in each direction. The results are shown in Table 5 and in Fig. 14. The calculation details are summarized in Table 6.
According to the results, c_{a,wh2} is not satisfied in the worst deviation combination, which is also visible on Fig. 14, as the boundaries of the leading features and the deviation regions of the workpiece intersect in multiple places. When evaluating each deviation result of the Monte Carlo simulation, 2537 cases of the 20,000 do not satisfy c_{a,wh2}, while the other FRs are always satisfied. With this, a failure rate of 12.69% is predicted for the placing part of the operation.
For comparison, a sequence of physical operations was executed to check the actual failure events. One pickandplace cycle included the palletizing of 4 workpieces onto the 4 slotted workpiece pallet. Altogether 50 cycles were run, meaning 200 workpieces seizing and releasing processes. From the 200 cycles, releasing failure occurred in 7 cases (3.5%), where the workpiece did not fall into the placing pose, but stayed on top of the leading features or tilted in the wrong direction. Apart from these, there were no failure events during the operation, which matches the simulation results for the other FRs. Although the simulation and the experiment show a similar tendency in releasing failures, the gap between the results is significant. The reason behind this is that the combined leading features are represented with a single FR, which is a more conservative, more precautious approach. As this consideration in this particular case is not well suited, it is also possible to separate the FR to multiple checking points.
In this case, three different constraints can be formulated for feasible workpiece placing. Firstly, the workpiececentering pin connection is described. Based on the diameter and chamfering of the workpiece and the workholder, the FR for the planar location (x and y) of the workpiece reference can be formulated as follows:
The remaining two FRs correspond to the chamfered columns of the workholder, and the corresponding vertices (designated as v1 and v2) on the workpiece that first make contact with the chamfers’ slope. These vertices also carry deviations relative to the workpiece reference. The corresponding parameters are listed in Table 3. The FRs can be formulated for the ydirectional deviation (in the workpiece reference frame) of v1 and v2 by applying planar transformation to the vertices according to the deviation results from the simulation. This results in the following inequalities:
Now, the original FR (corresponding to c_{a,wh2}) can be replaced with inequalities [30,31,32]. All three inequalities need to be satisfied simultaneously to achieve feasible workpiece placement. These requirements were also experimentally verified in the same setup as the measurement of c_{a,wh2}, with different sampled deviations in the releasing configuration.
After checking the deviation results of the Monte Carlo simulation against the newly formulated FRs, only 961 error cases were observable, reducing the failure rate to 4.81%. This is much closer to the 3.5% failure rate obtained from the physical execution of the system. The remaining gap between the results is most probably due to the overestimated tolerance values and the intricacies of leading feature mechanics, contact transition, and dynamics of the released workpiece. Nevertheless, the experiments show close relationship with the simulation results, which validates the proposed method.
Conclusion and future work
This paper presented a novel tolerance model for evaluating the feasibility of robotic pickandplace operations from the precision perspective. The proposed model aids robotic workcell developers in the selection of cell components, applied methods, and allocation of tolerances. Beyond the positioning precision of the manipulator and the manufacturing tolerances of the components, this tool allows the assessment of robotic workcell FRs considering tolerances introduced by the manipulation and metrology processes. The tolerance stackup is set up in the form of a transformation chain. By substituting the tolerance values of the relevant frames along the tolerance chain, and comparing the aggregated tolerances to the feasible tolerance ranges, the realization of FRs can be validated.
First, the basic concept and main considerations were introduced, together with the structural setup of the transformation chain, the essential pickandplacerelated FRs, and the additional tolerance influencing factors. Then, the complete model formulation was presented, including the necessary input data, the composition of the tolerance chain, and the evaluation of the fulfillment of FRs. Finally, the approach was implemented and validated in case of a physical robotic demonstrator cell, through a flexible pickandplace application.
Future work will include more exhaustive experimentation on real workcells to analyze and characterize specific tolerance inputs, such as seizing and releasing tolerances, investigate the effect of contact transitions, and identify further possible failure modes, which can potentially influence the computed tolerance chain. The other main goal is to generalize the model to different robotic operation types, like screwing or welding as well. A further possible research direction is to consider positioning tolerances on the individual robot links. If these parameters are available from the manufacturer or are measured, then the consideration of FRs along the robot trajectories could also be formulated (e.g., smallest distance between particular objects for offline path planners). Moreover, modular robots could also be considered in the tolerance model.
Availability of data and materials
The data that support the findings of this study are available from the corresponding author, upon reasonable request.
Code availability
The custom code that supports the findings of this study is available from the corresponding author, upon reasonable request.
References
 1.
Dantan JY, Qureshi AJ (2009) Worstcase and statistical tolerance analysis based on quantified constraint satisfaction problems and Monte Carlo simulation. Comput Aided Des 41:1–12. https://doi.org/10.1016/j.cad.2008.11.003
 2.
Shah JJ, Ameta G, Shen Z, Davidson J (2007) Navigating the tolerance analysis maze. ComputAided Des Appl 4:705–718. https://doi.org/10.1080/16864360.2007.10738504
 3.
Thornton AC (1999) A mathematical framework for the key characteristic process. Res Eng Des 11:145–157. https://doi.org/10.1007/s001630050011
 4.
Edmondson NF, Redford AH (2002) Generic flexible assembly system design. Assem Autom 22:139–152. https://doi.org/10.1108/01445150210423189
 5.
International Organization for Standardization (2019) ISO 10303105:2019  Industrial automation systems and integration  product data representation and exchange  part 105: integrated application resource: kinematics. Geneva, Switzerland
 6.
Yan H, Wu X, Yang J (2015) Application of Monte Carlo method in tolerance snalysis. Procedia CIRP 27:281–285. https://doi.org/10.1016/j.procir.2015.04.079
 7.
Tipary B, Erdős G (2021) Generic development methodology for flexible robotic pickandplace workcells based on digital twin. Robot ComputIntegr Manuf 71:102140. https://doi.org/10.1016/j.rcim.2021.102140
 8.
Whitney DE (2004) Mechanical assemblies: their design, manufacture, and role in product development. Oxford University Press
 9.
Chase KW, Greenwood WH (1988) Design issues in mechanical tolerance analysis. Manuf Rev 1:50–59
 10.
Teeravaraprug J (2007) A comparative study of probabilistic and worstcase tolerance synthesis. Eng Lett 14
 11.
Marziale M, Polini W (2010) Tolerance analysis: a new model based on variational solid modelling. Am Soc Mech Eng Digit Collect 383–92. https://doi.org/10.1115/ESDA201024437.
 12.
Desrochers A (2003) A CAD/CAM representation model applied to tolerance transfer methods. J Mech Des 125:14–22. https://doi.org/10.1115/1.1543974
 13.
Desrochers A, Rivière A (1997) A matrix approach to the representation of tolerance zones and clearances. Int J Adv Manuf Technol 13:630–636. https://doi.org/10.1007/BF01350821
 14.
Chase KW, Gao J, Magleby SP (1995) General 2D tolerance analysis of mechanical assemblies with small kinematic adjustments. J Des Manuf 5:263–274
 15.
Bourdet P, Mathieu L, Lartigue C, Ballu A (1996) The concept of small displacement Torsor in metrology. In: Ciarlini P, Cox MG, Pavese F, Richter D, (eds) Adv. Math. Tools Metrol. II, vol. 40, World Scientific Publishing Company, p. 110–22.
 16.
Ghie W, Laperrière L, Desrochers A (2003) A unified JacobianTorsor model for analysis in computer aided yolerancing. In: Gogu G, Coutellier D, Chedmail P, Ray P, (eds) Recent Adv. Integr. Des. Manuf. Mech. Eng., Dordrecht: Springer Netherlands, p. 63–72. https://doi.org/10.1007/9789401701617_7
 17.
Zou Z, Morse EP (2002) Assembleability analysis using GapSpace model for 2D mechanical assembly. Proc. ASME 2002 Int. Des. Eng. Tech. Conf. Comput. Inf. Eng. Conf. 3:329–336. https://doi.org/10.1115/DETC2002/DFM34187
 18.
Davidson JK, Mujezinovic A, Shah JJ (2002) A new mathematical model for geometric tolerances as applied to round faces. J Mech Des 124:609–622. https://doi.org/10.1115/1.1497362
 19.
Giordano M, Pairel E, Samper S (1999) Mathematical representation of tolerance zones. In: van Houten F, Kals H (eds) Glob. Consistency Toler. Dordrecht: Springer Netherlands, pp 177–186. https://doi.org/10.1007/9789401717052_18
 20.
Teissandier D, Couetard Y, Delos V (1999) Operations on polytopes: application to tolerance analysis. In: van Houten F, Kals H (eds) Glob. Consistency Toler. Dordrecht: Springer Netherlands, pp 425–434. https://doi.org/10.1007/9789401717052_43
 21.
Schleich B, Anwer N, Mathieu L, Wartzack S (2014) Skin model shapes: a new paradigm shift for geometric variations modelling in mechanical engineering. Comput Aided Des 50:1–15. https://doi.org/10.1016/j.cad.2014.01.001
 22.
Cao Y, Liu T, Yang J (2018) A comprehensive review of tolerance analysis models. Int J Adv Manuf Technol 97:3055–3085. https://doi.org/10.1007/s0017001819202
 23.
Schleich B, Anwer N, Zhu Z, Qiao L, Mathieu L, Wartzack S (2014) A comparative study on tolerance analysis approaches. In: Howard TJ, Eifler T (eds) Proc. Int. Symp. Robust Des.  ISoRD14. Copenhagen, Denmark, pp 29–39. https://doi.org/10.4122/dtu:2084
 24.
Qin Y, Qi Q, Lu W, Liu X, Scott PJ, Jiang X (2018) A review of representation models of tolerance information. Int J Adv Manuf Technol 95:2193–2206. https://doi.org/10.1007/s0017001713524
 25.
Schleich B, Wartzack S (2016) A quantitative comparison of tolerance analysis approaches for rigid mechanical assemblies. Procedia CIRP 43:172–177. https://doi.org/10.1016/j.procir.2016.02.013
 26.
Chen H, Jin S, Li Z, Lai X (2014) A comprehensive study of three dimensional tolerance analysis methods. Comput Aided Des 53:1–13. https://doi.org/10.1016/j.cad.2014.02.014
 27.
Desrochers A (1999) Modeling three dimensional tolerance zones using screw parameters. 25th Des Autom Conf vol. 1, Las Vegas, Nevada, USA, p. 895–903. https://doi.org/10.1115/DETC99/DAC8587
 28.
Slocum AH. Precision machine design. Society of Manufacturing Engineers; 1992
 29.
Polini W, Corrado A (2019) A general model to estimate hole location deviation in drilling: the contribution of three error sources. Int J Adv Manuf Technol 102:545–557. https://doi.org/10.1007/s0017001803273x
 30.
Mei B, Liang Z, Zhu W, Ke Y (2021) Positioning variation synthesis for an automated drilling system in wing assembly. Robot ComputIntegr Manuf 67:102044. https://doi.org/10.1016/j.rcim.2020.102044
 31.
Corrado A, Polini W (2020) Tolerance analysis tools for fixture design: a comparison. Procedia CIRP 92:112–117. https://doi.org/10.1016/j.procir.2020.05.174
 32.
Corrado A, Polini W (2017) Manufacturing signature in variational and vectorloop models for tolerance analysis of rigid parts. Int J Adv Manuf Technol 88:2153–2161. https://doi.org/10.1007/s001700168947z
 33.
FerrerasHiguero E, LealMuñoz E, García de Jalón J, Chacón E, Vizán A (2020) Robotprocess precision modelling for the improvement of productivity in flexible manufacturing cells. Robot ComputIntegr Manuf 65:101966. https://doi.org/10.1016/j.rcim.2020.101966
 34.
Doydum C, Duke PN (1991) Use of Monte Carlo simulation to select dimensions, tolerances, and precision for automated assembly. J Manuf Syst 10:209–222. https://doi.org/10.1016/02786125(91)90034Y
 35.
Baydar C, Saitou K (2004) Offline error prediction, diagnosis and recovery using virtual assembly systems. J Intell Manuf 15:679–692. https://doi.org/10.1023/B:JIMS.0000037716.69868.d0
 36.
International Organization for Standardization (1998) ISO 9283:1998  Manipulating industrial robots  performance criteria and related test methods. Geneva, Switzerland
 37.
Mihelj M, Bajd T, Ude A, Lenarčič J, Stanovnik A, Munih M et al (2019) Accuracy and repeatability of industrial manipulators. In: Robotics, Cham. Springer International Publishing, pp 231–241. https://doi.org/10.1007/9783319729114_15
 38.
Nubiola A (2014) Contribution to improving the accuracy of serial robots. PhD Thesis. École de Technologie Supérieure
 39.
Mooring B, Roth ZS, Driels MR (1991) Fundamentals of manipulator calibration. Wiley, New York
 40.
Wagner M, Morehouse J, Melkote S (2009) Prediction of part orientation error tolerance of a robotic gripper. Robot ComputIntegr Manuf 25:449–459. https://doi.org/10.1016/j.rcim.2008.02.006
 41.
Suyama A, Aiyama Y (2016) Influence of grasping position to robot assembling task. Proc. ISR 2016 47st Int. Symp. Robot., Munich, Germany, p. 1–6
 42.
ElMaraghy HA, ElMaraghy WH, Knoll L (1988) Design specification of parts dimensional tolerance for robotic assembly. Comput Ind 10:47–59. https://doi.org/10.1016/01663615(88)900474
 43.
De la Escalera A, Armingol JM (2010) Automatic chessboard detection for intrinsic and extrinsic camera Parameter Calibration. Sensors 10:2027–2044. https://doi.org/10.3390/s100302027
 44.
Lourakis M, Zabulis X (2013) Modelbased pose estimation for rigid objects. In: Chen M, Leibe B, Neumann B (eds) Comput. Vis. Syst., Berlin. Heidelberg: Springer, pp 83–92. https://doi.org/10.1007/9783642394027_9
 45.
Collet A, Martinez M, Srinivasa SS (2011) The MOPED framework: object recognition and pose estimation for manipulation. Int J Robot Res 30:1284–1306. https://doi.org/10.1177/0278364911401765
 46.
Kumbla NB, Thakar S, Kaipa KN, Marvel J, Gupta SK (2018) Handling perception uncertainty in simulationbased singulation planning for robotic bin picking. J Comput Inf Sci Eng 18:1–16. https://doi.org/10.1115/1.4038954
 47.
Kaipa KN, KankanhalliNagendra AS, Kumbla NB, Shriyam S, ThevendriaKarthic SS, Marvel JA, Gupta SK (2016) Addressing perception uncertainty induced failure modes in robotic binpicking. Robot ComputIntegr Manuf 42:17–38. https://doi.org/10.1016/j.rcim.2016.05.002
 48.
Chaumette F, Hutchinson S (2007) Visual servo control. II. Advanced approaches [Tutorial]. IEEE Robot Autom Mag 14:109–118. https://doi.org/10.1109/MRA.2007.339609
 49.
Chaumette F, Hutchinson S (2006) Visual servo control. I. Basic approaches. IEEE Robot Autom Mag 13:82–90. https://doi.org/10.1109/MRA.2006.250573
 50.
Ma Y, Liu X, Zhang J, Xu D, Zhang D, Wu W (2020) Robotic grasping and alignment for small size components assembly based on visual servoing. Int J Adv Manuf Technol 106:4827–4843. https://doi.org/10.1007/s00170019048000
 51.
Chang WC (2018) Robotic Assembly of smartphone back shells with eyeinhand visual servoing. Robot ComputIntegr Manuf 50:102–113. https://doi.org/10.1016/j.rcim.2017.09.010
 52.
Song J, Chen Q, Li Z (2021) A peginhole robot assembly system based on Gauss mixture model. Robot ComputIntegr Manuf 67:101996. https://doi.org/10.1016/j.rcim.2020.101996
 53.
Bi ZM, Zhang WJ, Chen IM, Lang SYT (2007) Automated generation of the D–H parameters for configuration design of modular manipulators. Robot ComputIntegr Manuf 23:553–562. https://doi.org/10.1016/j.rcim.2006.02.014
Funding
Open access funding provided by ELKH Institute for Computer Science and Control. Work in this paper has been in part funded under project number ED_18220180006, supported by the National Research, Development and Innovation Fund of Hungary, financed under the (publicly funded) funding scheme according to Section 13. §(2) of the Scientific Research, Development and Innovation Act, and in part by the Ministry for Innovation and Technology and the National Research, Development and Innovation Office within the framework of the National Lab for Autonomous Systems.
Author information
Affiliations
Contributions
B Tipary: conceptualization, code writing, manuscript writing and editing, methodology, investigation, visualization
G Erdős: conceptualization, manuscript reviewing, supervision
Corresponding author
Ethics declarations
Ethics approval
Not applicable
Consent to participate
Not applicable
Consent for publication
Not applicable
Conflict of interest
The authors declare no competing interests.
Additional information
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Tipary, B., Erdős, G. Tolerance analysis for robotic pickandplace operations. Int J Adv Manuf Technol 117, 1405–1426 (2021). https://doi.org/10.1007/s00170021076725
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00170021076725
Keywords
 Design method
 Tolerance analysis
 Robot
 Precision
 Monte Carlo simulation