Delayed Teleoperation with Force Feedback of a Humanoid Robot

Teleoperation systems allow the extension of human capabilities to remote-control devices by providing the operator with conditions similar to those at the remote site through a communication channel that sends information from one site to the other. This article aims to present an analysis of the benefits of force feedback applied to the bilateral teleoperation of a humanoid robot with time-varying delay. As a control scheme, we link adaptive inverse dynamics compensation, balance control, and P+d like controllers. Finally, a test is performed where an operator simultaneously handles the locomotion (forward velocity and turn angle) and arm of a simulated 3D humanoid robot to do a pick-and-place task using two master devices with force feedback, where indexes such as time to complete the task, coordination errors, path tracking error, and percentage of successful tests are reported for different time-delays. We conclude with the results achieved.


Introduction
The methods for remotely controlling robots have evolved over time, and new research and developments have contributed to the fact that the common problems encountered in this area have been decreasing, while the efficiency and stability of communication between the human operator, the robot, and the environment have been improving considerably, allowing the human operator, through a variety of master devices, to explore remote environments and control a robot to complete a task while receiving many kinds of feedback through a bilateral communication channel, that links both sites [1] . Therefore, robot teleoperation allows a human operator, located at a local site, to send control commands to a robot at a remote site, which could be dangerous or inaccessible to a human, such as nuclear sites, fires, disaster scenarios, or even other planets while receiving important information that helps him to successfully complete the proposed task. However, the main problem with these systems is the presence of time delays which decrease the performance and transparency of the system [2,3] .
On the other hand, techniques to include the various nonlinearities and uncertainties in the teleoperation sys-tem to achieve system stability, such as adaptive control [4,5] , neural networks [6] , and sliding strategies can be applied to get a more robust compensation in front of parametric uncertainties and sensor noise, but the main studies in this area have been focused on manipulators, while the need for a robot capable of interacting with interfaces designed for humans is crucial. Teleoperated humanoid robots have been proposed as a potential solution in different applications such as nuclear plant inspection and telemedicine, among others. Designing controllers applicable to humanoid robots is challenging and time-consuming in the design of control systems for bipedal motion, such as zero moment point (ZMP) [7] , hybrid zero dynamics (HZD) [8] , artificial intelligence or adaptive control techniques [9] .
Teleoperation can be performed in two ways, either unilateral or bilateral. Our work is focused on bilateral teleoperation, especially on the use of the force feedback and its effects and advantages during the teleoperation. Many studies use force feedback in different ways to communicate data to the operator depending on the task on hand. The work presented in [10] uses a vibrotactile belt to inform the operator about the position of the ZMP within the humanoid support polygon. Another work that builds upon the previous paper was presented in [11], in which a haptic interface for humanoid teleoperation is introduced and the effects of the manipulation task on the balance of the robot are considered. The work presented in [12] uses a kinesthetic system that applies forces on the operator′s waist, this haptic force is based on the translation of the robot′s center of pressure (CoP), as a direct measurement of balancing stability using the support polygon. Another way to feedback force to the operator is through stiffness, as shown in [13], where a bilateral wearable device uses adjustable muscle actuator modules in order to control a robot, sense the external force, and transmit the motion of contact with the environment. Popular devices used to feedback force to the operator are haptic joysticks such as the PHANToM Omni haptic used in [14] to implement a general strategy for generating the force feedback based on the ZMP method for a walking robot. Considering operations in dynamically stable conditions, the ZMP coincides with the CoP. Thus, its location is used to predict the dynamic behavior of the system and define the feedback force. More recent studies such as [15] propose using force feedback as a proportional value to the relative instantaneous velocity between the operator and robot, and this force is applied to the torso of the operator.
As it has been shown in those papers, most studies use force feedback mainly to inform the human operator about the equilibrium of the robot using the ZMP criterion for static walks. Instead, our work uses force feedback during locomotion (dynamic walks) and manipulation tasks of a humanoid robot both based on the synchronism error, where the main contribution is to analyze how much the force feedback helps the delayed bilateral teleoperation system of a humanoid robot. The control scheme implemented is formed by an internal loop of adaptive reverse dynamics control plus a set of external loop algorithms based on controllers type P + d (Proportional plus damping), proportional differential (PD), and P + d + Fe (Fe is an external force), where the damping is established as a function of the delay time. Additionally, a balance dynamic control to hold the 3D posture adequately for the locomotion and manipulation tasks is developed. A test for simultaneous teleoperation of the locomotion and arm of a humanoid robot simulated in virtual robot experimentation platform (VREP) to perform a pick and place task was performed with a single operator driving two joysticks, the first for the lower body and the second for the humanoid arm. The human operator receives force feedback that improves the perception of the robot′s walk, and the control of the arm, helping him to regulate the commands he sends and preventing rapid movements of the master as the time delay increases. To report the achieved result, we evaluated the time to complete the task, coordination errors, path tracking error, and percentage of successful tests, comparing the use of force feedback against the non-force feedback case for delayed teleoperation of a humanoid robot.
This document is organized as follows. Section 2 presents the master and slave models. Section 3 presents the assumptions and properties used. In Section 4, a bilateral teleoperation of a humanoid robot is proposed and the stability analysis for the system of bilateral teleoperation is presented in Section 5. Then, Section 6 shows the simulations results obtained when a human operator handles a humanoid NAO in the VREP simulation envir-onment. And finally, the conclusions are presented in Section 7.

Models
The master can have a serial or a parallel type configuration. In this paper, a Novint Falcon is used, which has a kinematic model described in [16] and its model can be represented in the task space. Here, the master model for locomotion and manipulation are detailed: (ml) (mm)

Master for locomotion and master for manipulation
where the sub-index i can be equal to for locomotion or for manipulation, and are the master′s position and velocity for locomotion, and are the master′s position and velocity for manipulation, is the inertia matrix, is the matrix representing centripetal and coriolis forces, is the gravitational force, is the force caused by the human operator and is the control force applied to the masters which will be computed by the controllers.

Slave kinematic model
For the kinematic model of a humanoid robot, different revised techniques are presented [17,18] . This paper analyzes the model described in [18], where the humanoid has been modeled as the combination of four kinematic chains that share the same starting point called "ROOT". With this, it is possible to construct the Denavit Hantenberg algorithm of all the kinematic chains respect to the reference frame of the root [18] , as shown in Fig. 1.
The kinematic model is defined as where represents the velocities of the final effectors in Cartesian coordinates, is the Jacobian augmented and represents the joint speeds. The acronym is the is the stance leg, is the non-stance leg, is the active arm and is the passive arm. The joints of the humanoid robot are defined as: q r _ nsa is the roll nonstance ankle, q r _ sa is the roll stance ankle, q p _ nsa is the pitch non-stance ankle, q p _ sa is the pitch stance ankle, q p _ nsk is the pitch non-stance knee, q p _ sk is the pitch stance knee, q p _ nsh is the pitch non-stance hip, q p _ sh is the pitch stance hip, q y _ nsh is the yaw non-stance hip, q y _ sh is the yaw stance hip, q r _ nsh is the roll non-stance hip, q r _ sh is the roll stance hip, q r _ ash is the roll active shoulder, q r _ psh is the roll passive shoulder, q p _ ash is the pitch active shoulder, q p _ psh is the pitch passive shoulder, q y _ nel is the yaw active elbow, q y _ pel is the yaw passive elbow, q r _ ael is the roll active elbow and q r _ pel is the roll passive elbow.

Slave dynamic model
In the state of the art, there are different techniques for modeling a humanoid robot [19] . In this paper, we take the motion equation for a floating base humanoid robot described by [19,20] M (q)q + C (q,q)q + g = Sτ + τext.
representing the total number of joints of the robot, the variables are defined as: represents all degrees of freedom of the humanoid robot plus the degrees of freedom of the floating base (extended model), represents the inertia matrix of the extended model, represents the centripetal and Coriolis matrix of the extended model, is the gravity vector, is the driven joint selection matrix of the extended model, is the vector of powered torsion torques of the extended model, and are the generalization of external forces, where is the augmented Jacobian and represents the external forces. The dynamic humanoid model (4) between successive impacts can be represented in a state space asẋ , , is the external force of locomotion, and is the external force of manipulation. In general, the external force can be obtained using force sensors, or estimated with distributed tactile skin sensors [21] .

Assumptions and properties
Properties 1−3 and Assumptions 1−3 will be used in this paper [22,23] : The inertia matrices and are symmetric positive defined.
Property 2. The matrices and are skew-symmetric, ie., and .
x ml andẍ mlĊml xmm andẍmmĊmm Property 3. If are bounded, then is bounded. If are bounded, then is bounded.
The communication channel adds a forward time delay (from master to slave) and a backward time delay (from slave to master). These delays are time-varying, bounded and asymmetric. Therefore, there are positive values for and such as and for all . Assumption 2. The human operator has finite energy: Assumption 3. The environment is represented by a finite energy solution: where and are positive values and it is assumed that and .

Bilateral teleoperation of humanoid robot
This paper analyzes the effect of force feedback over the teleoperation of forwarding speed and turn angle of a humanoid robot (locomotion) and its active arm (manipulation). The implemented control scheme and their main parts are shown in Fig. 2.

NAO body
Virtual joints ROOT Section Non-stance leg Active arm Passive arm ID Joint 1 q r_nsa 2 q p_nsa 3 q p_nsk 4 q p_nsh 5 q r nsh 6 q y_nsh 7 q r_sa 8 q p _sa 9 q p_sk 10 q p_sh 11 q r_sh 12 q y _sh 13 q r_ash 14 q p_ash 15 q y _ael 16 q r_ael 17 q r_psh 18 q p_psh 19 q y_pel 20 1) On the local site, there is a human operator that handles two joysticks, one for locomotion control and the other for manipulation control, both have force feedback which improves the perception for teleoperating the entire humanoid robot, preventing rapid movements of the master depending on time delay, as well as leading the human operator hand to directions of fewer errors between the master and humanoid robot.
2) Walk and turn references: For the control of the forward speed, a foot analysis based on a standard human walk in Cartesian coordinates is performed to generate the human-inspired knee and hip references. The turn angle of the humanoid robot is set based on [24], where the turn is carried out during the double support phase.
3) Balance control of the torso is carried out depending on the hip speed and a static force analysis. 4) A cascade control structure is applied, using an adaptive inverse dynamics control internal loop plus a set of external loop algorithms based on P+d like controllers, where the damping is set depending on the time delay.

Closed-loop control error
First, it is necessary to define the errors between successive impacts to manage the outputs of the robot to be able to control the signals of forwarding velocity, turn angle, and Cartesian coordinates of the end effector of the active arm to follow the generated references by the human operator, whose errors are defined below.

Forward velocity error
The forward velocity error is defined bẏ is the forward velocity reference, given bẏ v hip q sf is the real velocity of the humanoid robot and it is obtained through a linearization-based procedure used in [25], where the hips position depends on the angle of the stance foot and the angle of the stance knee q p _ sk , and its derivative is computed by (10) a1 and a2 where are the leg link lengths.

Error angles for gait length
The gait length error is represented as For angles references , the standard human walk is analyzed where the foot trajectory in Cartesian coordinates is taken as a pattern. At all times, at least one of the two feet is in contact with the ground. According to [26], each limb spends approximately of the walking cycle as a single support leg, as part of the double support and as a swinging leg. Once the foot trajectory data has been analyzed, human-inspired references are obtained through an approximation by a Bezier curve plus a straight line. The Bezier function depends on the velocity reference, whose coefficients are found by optimization. The goal is to ensure that the trajectory of the robot′s foot is as close as possible to the typical trajectory of a person (scaled to the size of the robot used). It is also important to verify the constraints of the partial zero dynamics surface (PHZD), given in Section 4.4. To obtain the references of stance knee angle and stance hip angle during a walking cycle, inverse kinematics [27] is used, which is calculated from the generated Cartesian trajectory. From this, the angles q p _ sh and q p _ sk can be obtained. Taking into account the percentages with respect to the period of the single support and double phase and considering that the legs are symmetrical, angles q p _ nsh and q p _ nsk are calculated using a phase shift of the angles q p _ sh and q p _ sk . Therefore, the references are . And represents the real angles belonging to the sagittal plane, such as .

Turn angle error
The turn angle error is described as is the real angle of rotation Kd y3 (t) of the humanoid robot and the reference is given by This yaw rotation reference does not affect the control in the sagittal plane due to the invariance property given in Proposition 1 of [28] where the within-stride feedback does not depend on the yaw orientation of the robot. This reference is mapped to (q y _ sh , q y _ nsh ) using the proposal described in Fig. 15 of [24].

Manipulation error
The manipulation error in Cartesian coordinates is represented asỹ where is the manipulation reference of the final effector in Cartesian coordinates, given by is obtained using the direct kinematics to calculate the actual position of the final effector (active hand) concerning a coordinate system that is taken as a reference (shoulder) and knowing the values of the joints and the geometric parameters of the robot elements. Finally, the coordination error of the delayed bilateral teleoperation system is defined by

Balance control
For the balance control, the torso inclination is changed depending on the tasks of locomotion and manipulation.
where is calculated using a linear function that depends on the reference forward speed. This is done for the torso to compensate the dynamic forces caused by different robot forward velocities: This represents is a linear function of the reference forward velocity set by the human operator, where is the slope of the line, is the interjection with axis and is a proportional constant. is calculated from the static force′s analysis to keep the torso at an angle of inclination such that it compensates the forces caused by the position of the arms and the object gripped, as shown in Fig. 3.
aa nsl where the subscript is the active arm and the subcript indicates the non-stance leg. Therefore, the ankle references will be used to indirectly control the balance of the torso and these references are related to the other joints as qsa qnsa qsa = qnsa = θ ankle The references and are given by .

Closed loop slave control
The dynamic model (5) is used to represent the error dynamics through the Lie derivative notation as follows: where is based on the error vector (16) and this is adapted to work simultaneously with speed and position errors. Then, we define , and .
The feedback linearization controller based on [8] is stated as ) .
(22) (22) is that an accurate dynamic model of the system is required. If we consider uncertainties in the dynamics and assume that the functions of (5) are estimated, the controller must be designed based on the functions . Thus, the law of control (22) is reformulated as [9] u ) .

(23)
Therefore, by replacing (23) in (21), To compensate the uncertainties, a combined controller is proposed with , where the first part allows following the desired reference model considering a perfect knowledge of inverse dynamics and the second part compensates the nonlinear uncertainty . Next, from (24), the closed-loop dynamics of the error in the state space is represented aṡ  and are proportional gains, is a control gain, and are the inertia matrices of the reference model, defines the non-linear uncertainty. The forward speed and turning angle references are kept constant between impacts.
ϑ (t) From [9], we can find out , and such that: From (25), the state predictor is expressed as follows: whereθ =ψ (t) ∥ω∥ +φ (t) ∥Fe∥ +θ (t) is chosen as . Defining , its evolution can be written from (25) and (27) as follows: and . As a result, is estimated indirectly through , and , and the values of , and are computed by the following adaptations laws based on the projection operators [29] : is a symmetric positive defined matrix and the projection functions are defined as

Adaptive system stability analysis
To analyze the estimated errors, the following candidate Lyapunov function is considered: With a procedure similar to [9], we get:  [30] for a detailed proof of these properties). Thus, if , then . In addition, by having the adaptation gain sufficiently large, we can lower errors (33) to a bounded ball given by: (34) ω Next, to analyze the evolution of , the following candidate Lyapunov function is used: Deriving (35) and then inserting (25), it is obtained: , then is bounded. Through this, it can be concluded that the adaptation errors and of (34), and the state of (37) are bounded and therefore and becomes smaller as the gain is higher. Additionally with the values set in the controller, the convergence rate of the system can be arbitrarily set considering a perfect compensation of inverse dynamics. Instead, when we do not have an ideal estimation, the rate of convergence must be established sufficiently high with respect to the estimator errors to satisfy (37).

Slave for teleoperation
Because the humanoid robot has a hybrid behavior (continuous and discrete), changes between impacts, so the continuous virtual slave is used to limit this hybrid behavior. In addition, the error remains invariant during the impact and hence, a PHZD is defined like [25], where the condition holds because the foot arrives at the ground with a soft acceleration profile causing a null joint velocity reference at the time that the foot v hip δ ϵ real = kv hip 1 + σ real ρ > 0 reaches the ground. To teleoperate the forward velocity and turning angle , a virtual slave of closed-loop continuous dynamics will be defined in (39) that it bounds above the hybrid dynamic response, which stably converges if is sufficiently high (see Fig. 3 of [8]). From Proof Theorem 2 of [8], it is possible to infer the existence of , with T as a walking cycle time: Then, the hybrid dynamics tend to a stable limit cycle. It is important to point out that the walking cycle time has to be small enough to allow the robot walks at higher forward velocities, and big enough to ensure that the phases of the walk could be accomplished rapidly to retain stability. We propose a closed-loop virtual system that limits the response of stable hybrid dynamics, as shown in Fig. 4, where it is possible to appreciate qualitatively that the real robot convergence rate must be sufficiently greater than the convergence rate of the virtual system. The open-loops continuous slave for teleoperation is defined by (25)

Locomotion and manipulation teleoperation
f ml fmm f sl fsm fv As described in the literature, the P+d controllers are simple structures that have adequate performance in practice for bilateral teleoperation systems of manipulator robots [23,31] as well as bilateral teleoperation of mobile robots [32,33] , where a sufficiently high value of damping is injected to assure the stability of the delayed system. The bilateral loop is formed by force feedback applied to the master of locomotion and master of manipulation . The slave uses for locomotion and for manipulation. Finally, an elastic force (that links the virtual slave and the closed-loop humanoid robot) is applied. The controller equation are: . The parameters have positive   values, where  ,  , , and , represent positive gain matrixes, linearly maps the master position to a speed and turn angle references, linearly maps the master position of manipulation to the references of the end effector, and z represents a vector that links the acceleration and the angular velocity at an infinitesimal time before , i.e., with [32] . and are coefficients of damping injected in the master of locomotion and manipulation, is the damping injected in the virtual slave composed by the virtual damping of the forward speed and the damping of the turn of the real robot , is the damping of the active arm, is the elasticity coefficient and is the damping coefficient of the coupling impedance. The function has a bounded output that tends to zero for in order to assure that . This function represents a turn-off function after T seconds, i.e., the function is equal to one from 0 to T, and later it decays to zero in a finite duration time interval.

Stability analysis of the delayed bilateral teleoperation system
A positive definite functional with is built to evaluate the evolution of the locomotion and manipulation system from a finite initial condition. The functional is formed by Tẏ sm (ξ)dξdθ.
(52) V Following the procedure similar to the test described in [32] and [22], along the trajectories of the closed loop system, considering the virtual slave (39), locomotion master dynamics (1), time delay, the human operator (6), the slave (active arm) (40), manipulation master dynamics (1), and environment forces (7), is bounded by: International Journal of Automation and Computing 18(4), August 2021 (54) α ml σ sl αmm σsm λ ml , λ sl , λmm, λsm > 0 x ml , z,ẋmm,ẏsm ∈ L∞ λ ml , λ sl , λmm, λsṁ x ml , z,ẋmm,ẏsm Remark 1. If , , and are sufficiently high damping to fulfill of (54), then we can point out from (53) that the variables . As are higher, increasing the damping injected, then will remain in a smaller origin-centered ball. Next, if (53) is integrated with respect to time, we get: The term and are bounded since , (finite energy) and using the adaptive control stability demonstration of Section 4.3.1, it is concluded that the estimation errors are bounded . So, from (55), we infer that will be bounded for all t and therefore and .

Calibration of the proposed control scheme
The control parameters must be calibrated correctly to comply with stable behavior. The following guidelines to qualitatively calibrate the controller are established: 1) The parameters and are positive matrices calibrated with null delay. On the other hand, parameters set the desired level of force feedback while and establish the maximum speed or position, and are positive constants to set the coupling between the virtual robot and the real one.
2) The damping injected is set as a function of which in turn depends on the time delay (54) to hold a stable delayed bilateral teleoperation. In addition, the value of also depends on the errors of the adaptive control law. If adaptation errors increase, then must be higher to hold (37). Thus, if the time delay increases , then the master damping as well as the virtual damping must be raised too and therefore an increment of the real damping must be performed based on Section 4.4. This slows the convergence rate of the continuous part and therefore this procedure, retaining a constant cycle period, can be applied while is greater than a minimum bound [25] . If necessary, the designer could change the walking cycle time depend-ing on the real slave damping in order to assure a stable hybrid dynamic.

Simulations and results
In this section, different tests are presented to verify the theoretical results achieved. These tests were performed in a structural environment where a human operator drives a simulated humanoid robot in VREP. The task scenario as shown in Fig. 5 consists of the following phases: First, the forward speed and angle of the robot are controlled to achieve locomotion to the first black mark on the ground, which signals the place where the humanoid should be positioned to grab the red object of 50 grams of mass. Once in position, it takes the red object and transports the red object to the next black mark, where a manipulation task is performed to put the red object on the new location. Next, the NAO walks a straight line to the third black mark, and then it starts to turn to reach the final goal. During the whole task, the robot must follow the black path.  Fig. 6), two Novint Falcon robots are used as master devices, one for locomotion with 2 DOF and the other for manipulation with 3 DOF. The controllers for these devices were implemented on Matlab simulink. The right side of Fig. 6 shows the behavior of the simulated NAO during the different stages. A video of a test non-delay without force feedback and delayed with force feedback is presented in https:// youtu.be/8fk1gb3DAr4. Three variants of the experiment were performed: 1) non-delay, 2) delayed without force feedback, and 3) delayed with force feedback. The time delay is generated as a time variable signal plus a filtered Gaussian noise . The delays generated are and . These delays are incorporated into the communication channel. Each experiment is repeated five times with one human operator and the average value of the synchronization error for locomotion, path error, time to complete the task, and the number of successful tests are calculated using the following expressions: 1) Synchronization error: error between the position or reference speed (measured at the local site) and real sig- , which measures both velocity and turning errors, is computed as follows: 2) Path error is the average error between the reference path and the path followed by the robot. T task 3) Average time to complete the task is defined by where is the number of tests performed for each case.
4) The amount of successful tests is an indirect metric. It indicates that of the five tests that were carried out in each experiment, all of them were finished (all 5 phases). Here it can be observed that the test for the timedelayed without force feedback has the worst performance since the movements that are generated by the human operator are fast and more energy is injected into the system, then the robot moves without control. Then, when the force feedback is used in the delayed tests, the indexes improve and approach the values of the test without delay. This improvement is produced by a sufficiently high value of damping injected to assure stability  Also, Fig. 10 shows the different stages of the test, where the first locomotion phase goes from 0 to 50 s, the locomotion plus manipulation phases go from 50 s to 180 s, and the last locomotion phase go from 180 s to 360 s. In the locomotion phase, the active arm is teleoperated through the second joystick (Fig.6) to take the object and transport it to the next box. Throughout the experiment, the angle of inclination of the torso plays a very important role, as shown in Fig. 11, the torso adapts according to the different forward speeds and the torque produced by the arm when it is raised or/and takes the object (see Fig. 3). We remark that the use of force feedback helps to keep both manipulation and walking stable, since the human operator can feel the synchronism error as well as avoid rapid motions of a dual master when the time delay increases, via a controlled-damping injection.

Conclusions
In this paper, a control scheme for delayed bilateral teleoperation of a humanoid robot was performed to analyze the benefits of the use of force feedback. The proposed scheme links adaptative inverse dynamics compensation, P+d, PD, and P+d+Fe controllers, balance control, and coupling between a humanoid robot and a virtual slave. Using the parameter setting procedure proposed for the controller, which is based on delay-dependent in-   jected damping, we obtained a stable behavior of the teleoperated system in both manipulation and walking despite the communication delays between the master robot and the slave robot. Section 6 reported that the use of force feedback provides a better response from the system as opposed to not using it.
It is difficult to generalize the results obtained for any human operator (trained, expert, novice), therefore future researchers should be focused on a multiple operator analysis, experiments with real teleoperated humanoid ro-bots in front of several tasks, and different time delays to get more statistical information about the impact of the force feedback in multiple scenarios. However, we point out that the reason for performing human-in-the-loop simulations on simple tests was to verify the advantages of using force feedback in delayed teleoperation of humanoid robots, and with the indexes such as elapsed time to complete the task, synchronization error, and successful tests rate, this was corroborated in a quantitative manner.

Open Access
This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made.
The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not per- Locomotion + Manipulation θ torso Fig. 11 Balance control for locomotion and manipulation tasks mitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.