Introduction

It has been quite a few years since programmable paint spraying robots originally showed up in vehicle plants.1,2,3,4 Recently, spraying coating also finds its application in producing desired effects by painting textures in themed environments.5 Nonetheless, spray painting is not applicable for painting inner building walls and it is uneconomical to paint both interior and outside surfaces because of its need of high-voltage atomizers and the complicated control of paint spraying/deposition process related to the overlapping between paths.6,7,8,9,10,11 Also, a severe flaw of paint spraying robots is the inability to cover surfaces near boundaries.12,13 By contrast, rollers are popularly utilized in structural painting because of superior consistency of painting over sprayers, simpler adaptability of the holder frame size and nap materials, and easy reachability to corners.14,15 Despite the fact that rollers are convenient tools for everyday painting, paint can disturb the skin and paint fumes may cause cerebral pains, wooziness, and queasiness whenever breathed in for a lengthy period.16,17Sore backs and discomfort can additionally result from coating by hand over a prolonged time. With regularly improved mobility and maneuverability, robots are the ideal option and only choice in contrast to people for the conceivably dangerous coating assignment in hazardous settings (e.g., sites with radiation danger) and risky circumstances (e.g., battlefields).18 Therefore, application of paint through robots is an active and important area of research.15,19 Nonetheless, robotic painting with rollers has been rarely investigated compared with robotic painting by spraying, as it is difficult to embed force sensors in rollers due to the working environment, i.e., the roller must be repeatedly dipped into paints and the paint will solidify, reducing the accuracy of force sensors. Yet, embodied force sensors are required to ensure that the roller paints surfaces with a constant force at a proper orientation. Without force feedback, the robot roller either does not contact the surface seamlessly or is pushed so harshly that it will likely be destroyed. The orientation is also important because the roller might contact the surface by a point, failing to paint surfaces successfully. Our paper establishes the basis for autonomous coating as a substitute for hand painting by roller-equipped robots. Rather than constructing a new platform exclusively devoted to coating tasks in a specified environment,7,19,20,21,22 we use a multifunctional collaborative robot—Sawyer (Rethink Robotics)—to coat flat walls with different control methods; thereafter we examine the coating quality and explore the feasibility of replacing hand painting with robots.15,20,21,22 Empowering a universally useful hardware to coat objects, instead of creating specialized equipment that takes up extra space, is particularly significant for domestic robots. While PictoBot, a state-of-the-art robot for painting interior walls, aims to practically spray a relatively large indoor space, our robot targets coating surfaces with rollers as evenly and long-lasting (not easily peeling) as humans.23

Here, we compare the coating quality of hand coating with two different control strategies—pure position control and passive force control—on steel panels. Both the strategies are tested at various feedback periods (time interval between two corrections in the control system of the robot). Figure 1 displays a snapshot of the robot coating on planar metal sheets. Position control with sub-millimeter accuracy, common to almost all modern collaborative robots, relies on the built-in joint position control mode.24,25 The roller trajectory can be generated autonomously by the robot combined with the embedded camera on the robotic arm. With multiple tactile “sensors” on hand, humans are able to coat uneven surfaces with dents or bulges more or less consistently. This indicates that human hand coating is the result of an indispensable combination of position and force control. Purely relying on position (and neglecting the “feel” of the hand) is not adequate for coating rough surfaces. While almost all the robots are capable of controlling the position of their end-effector, robust force control is often unavailable, unreliable, or too expensive.19,26,27,28 To realize human-level performance cost-effectively, we design a compliant roller, incorporating linear springs and dampers. Along with an ultrasonic sensor fixed at the end-effector of the robot, this compliant roller provides a “passive” force control mechanism through position feedback control. We use the term “passive” because force is not directly measured but inferred from the position of the spring in which the force is proportional to the position. The compliant roller now can maintain the same force by maintaining the same distance from the target surface. Meanwhile, this also ensures that the roller contacts the surface seamlessly and continuously. It is difficult to satisfy this requirement using only position control and rollers with rigid handles. In addition, the frequency of the force adjustment is closely related to painting performance. The robot arm keeps jittering if it is adjusted too frequently according to the force feedback and cannot realize a constant force painting otherwise. The compliant applicator we developed is adaptable to general robotic arms.

Fig. 1
figure 1

(a) Sawyer paints the planar surface with a compliant roller exploiting position feedback control. (b) End-effector (compliant roller) comprised of (1) roller cover, (2) spring-damper system, and (3) the circuit including the microcontroller (Adafruit Pro Trinket 5 V) and an ultrasonic sensor

Full size image

Our previous work demonstrated that this framework could achieve even increase in coating thickness proportional to the number of painting layers, comparable to the coating of human hand.29 In this work, we investigate how even and long-lasting robotic painting can be achieved by measuring a list of metrics, including the force applied and its adjustment frequency. Experiments are conducted to compare the coating quality on a horizontal planar surface constructed by steel sheets with that of human hand painting. It is found that with appropriate force and frequency control the coating quality of the proposed robotic method is comparable to human coating performance. Moreover, in contrast to stepper motors used in other robots, the Sawyer motors used in our study are compliant to assure safety.19,26,27,28 Note that the mobility of the robot platform used in this work is enhanced with four wheels on the pedestal. With the addition of rangefinders to detect the distance between the wall and the pedestal and a microcontroller to control these wheels, the robot can navigate autonomously to walls. Also, the space our robot can reach will be enlarged greatly if a jack-up mechanism is implemented.23

The structure of this paper is as follows: we introduce the methodology and theoretical background first. Utilization of position feedback control to realize force control is the key theoretical basis. Next, we compare the force variation during painting using two different control regimes–pure position control and position feedback control–to test the validity of the latter control strategy. Finally, we exhibit painting quality comparison among human hand painting, pure position control-based robotic painting, and position feedback control-based painting. The adjustment frequency of position feedback is also explored.

Methodology

Virtually all collaborative robots are equipped with position control mode that, given the prescribed end-effector coordinates and orientations, calculates the robot joint positions and angles according to inverse kinematics.30 The precision of position control in contemporary collaborative robots reaches millimeter-scale. In order to implement a constant force on the roller based on position control, the nonlinear trajectory of the end-effector needs to be divided into small line segments. Start and end points of each segment need to be measured by dragging the robotic arm manually and then provided to the robot that will use these target points to plan its motion in MoveIt!.31 The length of these discrete segments must be small enough to ensure tight contact between the roller and the surface, largely lowering the coating efficiency and autonomy of the robot. An alternative approach is a force sensor of high resolution integrated with the robotic arm that will correct the motion of the robot when the force at the roller is larger or smaller than the prescribed force level. While this method may be effective, it is expensive and, in harsh working environment surrounded with paints, prone to damage. We developed a cost-effective spring system, i.e., a compliant roller working together with an ultrasonic sensor (LGDehome HC-SR04) attached at the robot end-effector, measuring the distance between the ultrasonic sensor and target plane simultaneously when the robot moves under position control. Figure 1b exhibits different components of the complaint roller. A roller cover is attached to a spring system (BQLZR 108,004 Aluminum Shock Absorber) which in turn is connected to the robot arm. The roller is specially designed so that it only complies (compresses or extends) along the normal direction of the surface. In the context of painting, when the robot contacts the surface and attempts to reach the desired position, the spring of the compliant roller is compressed, resulting in an external force F on the robotic arm. This force F can be approximated to be \(kx\), where \(k\) is the spring stiffness and \(x\) denotes the deformation of the spring. By maintaining a constant \(x\), we ensure a constant force F during each coating stroke. In order to monitor the distance in real time, the main controller of the robot, responsible for inverse kinematics, must collect the distance message instantly. Robot Operation System (ROS) is fast and is used for online operations in many robots such as Sawyer, the robot we are using. We incorporate the ultrasonic sensor as one ROS node and enable it to communicate with other nodes (sensors on the robot). The ultrasonic sensor measures the time difference between the transmitted and reflected sound wave and then the microcontroller (Adafruit Pro Trinket 5 V) calculates the distance.

Theoretical background

Automatic path planning

Overall, a complete movement realization of robot is divided into two sections, planning and action. When the robot is given a target point, the computing unit inside the robot controller will need to compute the corresponding joint angles to rotate to reach as close as possible to the target point. Then, the controller sends signals (rotation angles) to the digital motor in each joint to move. To enable a robot to autonomously coat a surface, we employ the OMPL real-time motion planner in MoveIt!, a freely and publicly available motion planning software.31 Users only need to input the coordinates of four corners of the surface to be coated. Our code for the whole project is available at https://github.com/duyayun/Vision-and-force-control-automonous-painting-with-rollers.

As shown in Fig. 2a, the trunk of the robot consists of seven revolute joints. In order to control the path of end-effector accurately, the robot needs to calculate the seven joint angles for the movement. In other words, the transformation matrix between two adjacent joints, J0-J6 in Fig. 2a, has to be calculated to get the transformation matrix between each joint to the end-effector, E; multiplying the transformation matrix, \({}_{i}^{i - 1} T\), and a vector denoting the position in a reference frame, {i-1}, yields another vector denoting the position in a different frame, as demonstrated in Fig. 2b.18 Quaternions are used for rotation representation; we refer the reader to a linear algebra textbook for details.32 This transformation matrix is

$$\begin{array}{*{20}c} {{}_{i}^{i - 1} T = R_{X} \left( {\alpha_{i - 1} } \right)D_{X} \left( {a_{i - 1} } \right)R_{Z} \left( {\theta_{i} } \right)D_{Z} \left( {d_{i} } \right)} \\ \end{array}$$
(1)

where, referring to Fig. 2b, \(R_{X} \left( {\alpha_{i - 1} } \right)\) represents rotation by an angle \(\alpha_{i - 1}\) about the x-axis of the (i-1)-th joint,

Fig. 2
figure 2

(a) Seven revolute joints, J0-J6, and end-effector, E, of Sawyer, (b) a schematic diagram to show the geometric connection between two adjacent revolute joints, \(i\) and \(i - 1\), where \(O_{i - 1}\) and \(O_{i}\) are the origins of the frames {\({ }i - 1\)} and {\({ }i{ }\)}, respectively, while \(\hat{X}_{i - 1} - \hat{Y}_{i - 1} - \hat{Z}_{i - 1}\) and \(\hat{X}_{i} - \hat{Y}_{i} - \hat{Z}_{i} { }\) are the corresponding frames. \(O_{i - 1}\) and \(O{^{\prime}}_{i}\) is the intersection point of the common normal to axis {\({ }i - 1\)} and axis {\({ }i{ }\)}. \(a_{i - 1}\) is the link length, i.e., the distance from \(\hat{Z}_{i - 1}\) to \(\hat{Z}_{i}\) measured along \(\hat{X}_{i - 1}\), \(\alpha_{i - 1}\) is referred to as the link twist, the angle between \(\hat{Z}_{i - 1}\) and \(\hat{Z}_{i}\) measured about \(\hat{X}_{i - 1}\), \(d_{{i{ }}}\) is the distance from \(\hat{X}_{i - 1}\) to \(\hat{X}_{i}\) measured along \(\hat{Z}_{i}\) and called link offset, \({\text{and }}\theta_{{i{ }}}\) is the joint angle, the angle between \(\hat{X}_{i - 1}\) to \(\hat{X}_{i}\) measured about \(\hat{Z}_{i}\). \({ }a_{i} \ge 0\),\({ }\alpha_{i} ,{ }d_{{i{ },}}\) and \(\theta_{{i{ }}}\) are signed quantities

Full size image

\(R_{Z} \left( {\theta_{i} } \right)\) represents rotation by an angle \(\theta_{i}\) about the z-axis of the i-th joint,

and \(D_{Z} \left( {d_{i} } \right)\) is translation by a distance \(d_{i}\) along the z-axis of the i-th joint.

The transformation matrix from the joint 6 to joint 0 is simply

$$\begin{array}{*{20}c} {{}_{6}^{0} T = {}_{1}^{0} T{}_{2}^{1} T{}_{3}^{2} T{}_{4}^{3} T{}_{5}^{4} T{}_{6}^{5} T.} \\ \end{array}$$
(2)

Referring to Fig. 1b and Fig. 2a, note that the target point (the endpoint of roller) is fixed at a distance from the end-effector. As long as the orientation of the roller relative to the plane to be painted is known, the Cartesian coordinate of the target point relative to J6 is available as \(X_{g} = \left[ {x,y,z} \right]^{T}\). In order to be compatible with the dimension (4 × 4) of the transformation matrix, \({}_{6}^{0} T\), one dimension is added to \(X_{g}\) such that \(X_{g} = \left[ {{\text{x}},{\text{y}},{\text{z}},1} \right]^{T}\) and the corresponding coordinate, \(X_{D}\), in the base frame (frame {0}) is

$$\begin{array}{*{20}c} {X_{D} = {}_{6}^{0} TX_{g} .} \\ \end{array}$$
(3)

The Denenvit-Hartenberg method is commonly used to solve it in robotics.30

In the case of Sawyer (the robot used in the painting experiments), all the limb lengths (\(a_{i - 1}\) and \(d_{i}\)) and relative orientations of the revolute joints, \(\alpha_{i - 1} \left( {i = 1, \ldots , 6} \right)\), are known and inverse kinematics are applied to solve for the joint angles \(\{ \theta_{i} \} \left( {i = 0, \ldots , 6} \right)\) for the desired target point \(X_{D}\). (Fig. 3).

Fig. 3
figure 3

The inner structure of series elastic actuator inside each joint of Sawyer

Full size image

Our study aims to evaluate the performance of robotic painting compared to human painting, in terms of evenness and durability. We do not cover the algorithm enabling our robot to recognize 3D objects and paint them using autonomous motion planning, as shown in Fig. 4a.29 For example, the 3D object to be painted is cuboid as exhibited in Fig. 4a. The point cloud data (PCD) scanned by a depth camera placed out of view is shown in Fig. 4b with background and outliers removed. Figure 4c is the reconstructed PCD based on our algorithm. Figures 4d1-d3 show the trajectory of robot roller with our autonomous motion planning algorithm implemented in MoveIt! for the three surfaces visible to the depth camera. Each red dashed rectangle whose width equals the width of the roller, and the moving directions of the roller are represented by dashed black arrows. The purpose of overlapped areas is to ensure that the surface is completely painted. Clearly, our motion planning algorithm is applicable regardless of the surface orientation.

Fig. 4
figure 4

(a) Three surfaces (denoted as 1, 2, and 3) on a cuboid box to be painted by Sawyer are scanned by a depth camera (not shown in the picture), (b) point cloud data (PCD) of the above box from an RGB-depth camera (Intel RealSense) with background and outliers removed from the original PCD, (c) reconstructed object, (d1-d3) motion of the roller on surface 1–3. Each stroke is represented by a red dashed rectangle whose moving direction is denoted by the black dashed lines and whose width is equal to the width of the roller

Full size image

Passive force control using position feedback on the end-effector

Note that the number of variables to describe the endpoint of the robotic arm is six, including three position coordinates and three orientation angles, smaller than the total degree of freedom (DOF) of Sawyer, which is seven. This is, therefore, a redundant system. With inverse kinematics applied, the joint angle solution might probably not be unique in some cases because of the redundancy. That is where energy minimization comes into play as the optimization criterion to choose the preferred ones. As shown in Fig. 3, inside every joint of Sawyer, there are series elastic actuators which contain embedded elastic elements–springs to actuate all joints. The energy stored in a spring is the elastic potential energy which is computed as \(E = \frac{1}{2}kx^{2}\), where \(k\) is the spring stiffness and \(x\) is the spring displacement. The robot will try to minimize this energy while planning its motion.

Position control is precise, but it is unreliable to accomplish a constant force by itself, especially with Sawyer robot that we are employing in this work, as it is known for safe interaction with humans. Inside each of its joints, there are a series of elastic actuators, as displayed in Fig. 3. The linear spring inside the series elastic actuator ensures the compliance of the robotic arm but deteriorates the ability of the robot to exert a constant large force onto the surface to be coated. Meanwhile, installing a force sensor near the roller is too expensive and difficult to integrate with the rest of the control system in our case. Passive force control using position feedback and compliant roller is a perfect option ensuring accuracy at a low cost. Figure 5 shows the general algorithm of realizing passive force control.

Fig. 5
figure 5

Position feedback control loop to realize force control during robotic painting

Full size image

In Fig. 5, there are multiple steps during robotic coating. Path planning is the first step. When the target surface is inspected by the robot, a coating trajectory will be planned, and waypoints \({X}_{m}\) on this prescribed trajectory are generated. Then inverse kinematics is used to compute the joint angles \({q}_{m}\) accordingly which are controlled by series elastic actuators inside the joints. Before starting experiments, we build a map from the distance between the ultrasonic sensor and surface, \(d\), to the force exerted onto the surface, \(F\), by the customized compliant roller. From the linear spring model, it is expected that the relation between \(\mathrm{F}\) and \(\mathrm{d}\) is approximately linear. In our experimental setup, if an 8 N force, measured by a push pull force gauge (Beslands NK-500), is expected to be applied, the distance is desired to be kept at 13.75 cm. The desired distance is indicated by the dashed line in Fig. 6b.

Fig. 6
figure 6

Force applied onto the target surface as a function of the distance between the robot end-effector and the target surface of different control regimes. The goal is to keep the force at 8 N when the robot paints. (a) Pure position control with no force control. (b) Force control through position feedback control utilizing an ultrasonic sensor attached at the end-effector. Here, \(\mathrm{y}\) is the coordinate along the length of the target surface

Full size image

Once the ultrasonic sensor gets a measurement of the distance, how much force applied onto the target surface is known. If the force is larger than the distance threshold, we move the robotic arm away and vice versa. Every control algorithm should have an adjustment frequency. On the one hand, if we listen to the ultrasonic sensor too frequently, the coating process will be unreasonably slow. On the other hand, if we do not get the feedback signal timely, the coating process is essentially governed by pure position control. Such control does not match the principle underlying human hand coating. Later, we will show the coating qualities with feedback control every 10 s and 2 s, where the passive force control seldom takes effect in the former while it shows obvious improvements in the latter. Force signal with different control strategies, i.e., pure position control and passive force control, is measured and exhibited to verify the validity of our proposed passive force control algorithm. Outcomes from one experimental trial are shown in Fig. 6 that shows pure position control is unable to maintain a constant force level (Fig. 6a), but passive force control maintains almost constant distance between the end-effector and the target surface (and thus constant force level) in Fig. 6b. Even if Fig. 6 is obtained from one trial, it is indicative of the most common issue of pure position control. Even if an 8 N force is initially applied onto the roller under pure position control mode, the robot arm often drifts away from the surface as shown in Fig. 6a or presses the roller too hard, potentially damaging the robot itself. By contrast, referring to Fig. 6b, the position feedback control can maintain the distance at the threshold, 13.75 cm, as mentioned above. Although the distance varies during painting, the variation is within 4 mm, 3% error in force, which is acceptable. Note that the safety of the robot arm is ensured throughout the entire control process by establishing a force threshold. For instance, when our ultrasonic sensor detects that the force exerted on the roller is larger than 13 N, which is excessive for painting, the robot arm will automatically conform to avoid being destroyed.

In summary, our economical method incorporating the ultrasonic sensor with the customized compliant roller enables the robot to paint as well as mimic human performance. Further experiments on coating quality will be detailed in the next section to show that the essence of hand painting is the collaboration between position and force control.

As stated previously, the objective of this study is not autonomous painting, as in our previous work, but human hand-level painting.29 Yet, we still explore the adaptability of our passive force control-based framework to a complicated 3D shape, a sphere for instance. We did not perform experiments on this, but experiments performed in our previous work show that our passive force-controlled robot can autonomously recognize how many faces a polyhedron has and finish painting surfaces thoroughly.29 We pointed out that the robot is incapable of autonomously recognizing and painting a 3D object with a continuously varying surface normal. This is due to the difficulty of point cloud data processing instead of the painting ability of the robot arm. The offset between the ultrasonic sensor and the compliant roller is 3.5 cm. If the shape of the 3D object is known and its radius of curvature is larger than 3.5 cm, the robot can still paint the object with a constant force (with a bit of mathematical calculation involved in the motion planning) if the direction of the last link on the robotic arm is aligned with the surface normal, as illustrated in three orientations in Fig. 7 (orientations 1, 2, and 3). More specifically, if a spherical object is to be painted with an 8 N force, the distance should be 13.75 cm plus the distance error caused by the 3.5 cm offset. The same reasoning applies to other objects with known curvatures.

Fig. 7
figure 7

The diagram depicting our roller-equipped robotic arm’s ability to paint a spherical object, with the direction of the last link aligned with the surface normal of the sphere

Full size image

Materials

BEHR MARQUEE 1 gal. (P440-3) Fishpond One-Coat Hide Ceiling Flat Interior Paint and Primer was purchased from Home Depot. Wooster 4 in. × 3/8 in. high-density pro woven roller cover was purchased from Home Depot. Steel panels (3′′ × 6′′ × 0.020′′) were purchased from Q-Lab Corporation (QD-36).

Coatings preparation and testing of coating properties

Coatings were deposited on steel panels with the help of roller by human as well as robot. Steel panels were arranged in a horizontal space to create horizontal area for painting. Layers of paint were deposited on the steel panel area one by one in both modes of painting. The coatings were allowed to air dry for 1 week before testing coating properties. A summary of time and force control parameters is mentioned in Table 1.

Table 1 Summary of methods and formulations
Full size table

Measurement of dry film thickness of painted panels

Dry film thickness was determined using a BykoTest 8500 film thickness gauge as described in ASTM D6132. This gauge uses electromagnetic induction to determine the thickness of a dry organic film. For each panel, 10 readings were taken on different points, and average along with standard deviation was determined and reported in the research.

Measurement of film hardness

Film hardness was determined using the König pendulum hardness method as described in ASTM D4366. In this test a panel was inserted into a Gardner pendulum hardness tester, and a pendulum was allowed to swing on the surface of the coating. The amount of time that the pendulum swings in a specified range is related to the dampening from the surface of the coating, and thus the coating’s hardness. Each panel was tested at three regions/spots, top, center, and bottom of the panel. From these readings, the average and standard deviation were determined.

Evaluation of coating flexibility

Coating flexibility was determined using a conical mandrel bending test as described in ASTM D522. In this test a panel is secured to a Gardner conical mandrel. The panel is then bent along the surface of the mandrel. If the surface of the coating is cracked, the length of the longest crack can be used to determine the percent elongation of the coating.

Measurement of impact resistance

Impact resistance of the coated films was determined using the method described in ASTM D2794. Films were placed under the guiding tube and a calibrated weight (3.92 lbs) is dropped from heights up to 43 inches to either the face of the sample, or through the backside of the sample substrate. The height at which a coated sample does not show signs of defects such as cracks, tears, or loss of adhesion is the recorded impact resistance.

Color scheme

Color was measured using a MacBeth ColorEye 7000 systems reflectance mode. Analysis was done using Lab color space with a D65 illuminant at a 10° observer angle. These are in accordance with the CIE L*a*b* color space system and in agreement to the ASTM E805 standard. This method comprises three coordinates that constitute the color space system: L*, a* and b*. Precisely, L* specifies lightness whereas the chromaticity is given by the coordinates a* and b*. Changes from − L* to + L* values mean modifications from darker to a brighter aspect. Similarly, a* and b* coordinates indicate color directions: + a* goes to the red and − a* goes to the green direction. The + b* goes to the yellow and − b* goes to the blue direction. Additionally, the total color variation (ΔE) can be estimated using equation (4):

$$\Delta E = \sqrt {\left( {\Delta L^{*} } \right)^{2} + \left( {\Delta a^{*} } \right)^{2} + \left( {\Delta b^{*} } \right)^{2} }$$
(4)

Organoleptic evaluation of coated panels was evaluated

Pictures were used to visually to compare the surfaces. Photographs were taken with a Samsung Galaxy S10, and Image J software was used for further analysis. Scale bars were added to these images using the program Image J.

Results and discussions

Coatings were applied on steel panels by varying the position and force control parameters and coating properties were studied to optimize the parameters. Commercial paint was chosen so that the difference in properties of the coatings were only due to the skill of the operator, in this case, between hand painted panels and robo-painted panels (with varying parameters). The major goal of this research is to mimic the painting ability of humans with the help of robots.

Dry film thickness

BykoTest 8500 film thickness gauge was used to measure dry film thickness (DFT) of hand and robo-painted panels. Figure 8a shows that DFT of hand painted panel with one layer of coating is 35 µm and it increases with increasing number of coatings. DFT of robo-painted coating 0R0 does not follow increasing or decreasing trend with an increasing number of layers. The standard deviation of DFT values for 0R0 is very high as compared to standard deviation values of hand painted coatings. DFT of robo-painted 8R2 coatings showed increasing trend with the increasing number of layers with very low standard deviation. Robo-painting 8R2 was able to coat consistently while mimicking hand painted panels. Though, DFT trends of 8R2 followed hand painted coatings but the increase in value of DFT with increase in number of layers was less for 8R2.

Fig. 8
figure 8

(a) Dry film thickness of hand painted, without force control robo-painted and 8 N force control and 2 s time control robo-painted coatings. (b) Dry film thickness of robo-painted panels with different force and time control parameters

Full size image

Figure 8b shows that DFT of robo-painted panels (2R10, 8R10, 13R10, and 8R2) with one layer of coating is 61.41 µm, 102.82 µm, 121.16 µm, 130.43 µm, and 28.86 µm, respectively. 2R10, 8R10 and 13R10 showed irregular trend with the increasing number of layers. We think that 10 s time control parameter is not appropriate (too slow feedback) for the robot to make uniform and consistent coatings. 8R2 coating was prepared by altering the time control parameter to 2 s instead of 10 s and this allowed the robot to coat consistently.

Pendulum hardness

Coating hardness is a materials property of the resin, as it is governed by inter- and intramolecular interactions operating within the resin components. However, the force applied on the substrate by the roller is a result of applicator’s perception. Hardness of the coatings depends upon the resin, formulation, and surface smoothness. In this research, panels were coated with the same interior wall paint and, therefore, the differences in hardness values are observed due to surface smoothness factor. Figure 9a shows that hardness value of hand painted (H) panels is consistent for 2-, 3-, 4-, and 5-time coated panels but the hardness value of 1-time coated panel was higher (39 s). The higher hardness value for 1-time coated panel can be attributed to the relatively small DFT of 35 µm; in this case, substrate hardness might have contributed toward higher value. 5-time coated panel of coating 8R2 has hardness value of 27 s and DFT of 61.94 µm and the hardness value of 2-times coated hand painted panel (H) was 26 secs and DFT was 61.41 µm. The robot was able to mimic hand painted coatings when appropriate control parameters were used.

Fig. 9
figure 9

(a) Pendulum hardness of hand painted, without force control robo-painted and 8 N force control and 2 s time control robo-painted coatings. (b) Pendulum hardness of robo-painted panels with different force and time control parameters

Full size image

The hardness values for robo-painted coatings 2R10, 8R10, and 13R10, shown in Fig. 9b, differ because the surface roughness varied with variation of control parameters and number of coats. Robo-painted coating 8R2 showed higher hardness values for 1-, 2-, 3-, and 4-times coated panels because the dry film thickness is low as compared to other panels.

Impact resistance

Figure 10 shows the impact resistance as a function of number of coating layers in various schemes. It is noticeable that human hand painting \(H\), pure position control scheme \({}_{0}^{{}} {\text{R}}_{0} ,\) and position feedback control \({}_{8}^{{}} {\text{R}}_{10}\), have the same trend, i.e., the impact resistance initially increases (at small values of number of layers) until reaching a certain threshold and then drops with the increase in number of painting layers even if the threshold is different. In both \({}_{0}^{{}} {\text{R}}_{0}\) and \({}_{8}^{{}} {\text{R}}_{10} ,\) the impact resistance reaches the maximum value when the panels are painted with three layers. This is reasonable as the control time period is 10 s in \({}_{8}^{{}} {\text{R}}_{10}\) and the robotic arm is controlled by pure position control while waiting for the next position correction 10 s later, which implies the force control is not even obvious during painting. In other words, passive force control with very slow corrections (every 10 s) is almost the same as pure position control; this also signifies the need of tuning the feedback frequency for correction. By contrast, \({}_{8}^{{}} {\text{R}}_{2}\) with correction every 2 s gives a different outcome from the three methods mentioned above. As the number of painting layers increases, the impact resistance decreases. Hand painting generally shows similar trend (except an increase in impact resistance as the number of layers increases from 1 to 2). In summary, comparison among \({\text{H}}\), \({}_{0}^{{}} {\text{R}}_{0}\), \({}_{8}^{{}} {\text{R}}_{10}\), and \({}_{8}^{{}} {\text{R}}_{2}\) indicates that feedback should be incorporated approximately every 2 s by the robot to match hand painting.

Fig. 10
figure 10

Impact resistance of hand and robo-painted panels

Full size image

Other than the methods analyzed, two control schemes are left unmentioned, \({}_{2}^{{}} {\text{R}}_{10}\) and \({}_{13}^{{}} {\text{R}}_{10}\), the results from which display no regular patterns. This is related to the robot controller algorithm. The position controller inside industrial robots is precise but, in the absence of force-based feedback, the robot may get damaged while working in unknown harsh environments. As such, in order to ensure the safety of the robot, we add collision detection into the whole control algorithm. Intuitively, the collision is defined as an unexpected load applied suddenly onto the robot. Therefore, the collision detection algorithm involves the detection of both magnitude and speed of force change. For \({}_{2}^{{}} {\text{R}}_{10}\), the force enforced onto the roller attached to the end-effector is 2 N and it will be recalibrated every 10 s, not obviously different from pure position control. When a 2 N force is required, the roller barely touches the plane to be painted so it will leave the plane from time to time, leading to unstable painting. In contrast to \({}_{2}^{{}} {\text{R}}_{10}\), \({}_{13}^{{}} {\text{R}}_{10}\) tries to perform the application of a 13 N force, which is potentially harmful to the robot if applied all of a sudden. Therefore, due to the vibration during movement, the force sometimes can be significantly larger than the expected force value; such spikes in the force signal is treated as an obstacle. As such, the roller in these two cases (\({}_{2}^{{}} {\text{R}}_{10}\) and \({}_{13}^{{}} {\text{R}}_{10}\)) will often push too hard or too mildly. In summary, when the force applied onto the painting plane is too large or too small, the impact resistance does not show any obvious pattern with the change of number of painting layers.

Conical mandrel bent test

Thermoplastic coatings are very flexible and have higher elongation at break (generally above 32%) but preexisting cracks in the coating matrix can reduce the flexibility and failure can occur at lower elongation percentages. Figure 11 shows that robo-painted coating 13R10 has 22% and 18% elongation at break for 2- and 3-time coated panels, respectively. Robo-painted coating 2R10 has lower flexibility when the panels were coated 4- and 5- times. Coatings H, 0R0, 8R10, and 8R2 have high flexibility (32%). Therefore, with appropriate control parameters robots can paint substrates with roller without altering the flexibility of coatings.

Fig. 11
figure 11

Conical mandrel bent test for hand and robo-painted panels

Full size image

Color difference due to difference in painting method

Color of the coating depends upon various factors including color of pigment, color of dye, surface roughness, light source, viewing angle and observer. Figure 12 shows that coatings H, 0R0, 8R10 and 8R2 have different colors.

Fig. 12
figure 12

Camera images of hand painted and robo-painted panels. Each panel has three layers of paint

Full size image

Hand painted panel was considered as control and ΔE values were calculated. The values obtained from color meter reading are reported in Table 2. ΔE values were calculated between hand and robo-painted panels. Color values of 3 times coated panels were recoded and ΔE values were calculated. ΔE values from Table 2 showed that least ΔE value of 1.0147 was observed for 8R2 when compared with control (H). The robo-painted panel 8R2 was able to mimic human painting closely because it has the least ΔE value among other robo-painted panels which were examined for color test.

Table 2 Color coordinates of the hand and robo-painted coatings
Full size table

Conclusions

This work suggests that with optimum position, force, and time control, it is possible for a robot to mimic a human applicator for coating surfaces using a roller. Using a wide variety of coating performance tests, we showed that physical and mechanical properties of robo-painted panels were equivalent to human-painted ones. Our work indicates that in near future it will be possible to use robots to paint substrates using roller applicator. This will open new avenues for coating substrates in hazardous environment high-rise architectural and engineering locations, and for accelerated household applications.