High Utility Teleoperation Framework for Legged Manipulators Through Leveraging Whole-Body Control

Legged manipulators are a prime candidate for reducing risk to human lives through completing tasks in hazardous environments. However, controlling these systems in real-world applications requires a highly functional teleoperation framework, capable of leveraging all utility of the robot to complete tasks. In this work, such a teleoperation framework is presented, where a wearable whole-body motion capture suit is integrated with a whole-body controller specialised for teleoperation and a set of teleoperation strategies that enable the control of all main frames of the robot along with additional functions. Within the whole-body controller, all tasks and constraints can be configured dynamically due to their modularity, hence enabling seamless transitions between each teleoperation strategy. As a result, this not only enables the realisation of trajectories outside the workspace without the whole-body controller but also the ability to complete tasks that would require an additional manipulator if just the gripper frames of the robot were controllable. To validate the presented framework, a set of real robot experiments have been completed to demonstrate all teleoperation strategies and analyse their proficiency.


Introduction
Within research and industry, legged manipulators are seen to be used in an increasing number of applications due to their ability to traverse rough terrain and complete manipulation tasks. This presents these systems to have great potential in being deployed to reduce risk to human lives through completing tasks in hazardous environments, such as hazardous substance disposal. However, these systems are typically highly redundant and require complex control frameworks to fully utilise them. In research, the most effective control frameworks feature an optimisation-based whole-body controller (WBC), as they specialise in leveraging redundancy to complete a primary task while simultaneously realising B Chengxu Zhou c.x.zhou@leeds.ac.uk 1 School of Mechanical Engineering, University of Leeds, Leeds, UK 2 State Key Laboratory of Robotics and System, Harbin Institute of Technology, Harbin, China several other objectives critical to robot stability and performance.
Optimisation-based WBCs tend to fall into two main types, in which the optimisation problem is either based on inverse kinematics (IK) or inverse dynamics (ID). IDbased WBCs see use in quadrupedal legged manipulators [1,2], wheeled bipedal robots [3] and humanoid robots [4,5] for the realisation of dynamic gaits and manipulation tasks. However, these ID-based WBCs are suited specifically for contact and disturbance-rich environments, resulting in a highly complex controller. In the case of teleoperation, the most common application is in structured hazardous environments, such as laboratories or warehouses, in which kinematic accuracy becomes a higher priority than traversing rough terrain. Hence an IK-based WBC is more suited to these applications as its formulation usually includes fewer tasks and constraints, and consequently, they tend to be more computationally efficient [6] (which is vital for low latency in teleoperation), while still being able to complete stable complex gaits [7] and dexterous manipulation tasks [8].
Although progress in AI has worked towards making such systems autonomous [13], they are not adept enough to deal with tasks of a sensitive nature where high-severity risks are present. Therefore, teleoperation is currently the optimal method of controlling legged manipulators as not only can the teleoperator be situated far away from hazards, but also they have the ability to intervene in an emergency. Furthermore, integrating a WBC with teleoperation into one framework, along with a set of teleoperation strategies, provides the depth of control required to generate the whole-body motions to complete tasks in hazardous environments. In literature presenting these frameworks, it is common to see the teleoperator use a joystick, such as in [14] for aerial manipulation and in [15] for quadruped manipulation. However, this leads to difficulties in aligning the end-effector for manipulation tasks due to either limited controllable Degrees of Freedom (DoF) [14] or the complexity [15] of the joystick. Other more radical designs of a teleoperation controller have been designed to utilise body motions of the teleoperator themselves, such as the work presented in [16] where data is collected from a force plate they stand on while strapped into a suit that can measure body tilt to control wheeled bipedal locomotion. However, the hardware of this controller severely restricts the teleoperator's mobility, while only being observed to control the robot in 1 DoF. Another such alternative teleoperation controller is developed in [17], where 2 DoF and 3 DoF controllers are used to control a wheeled base and arm respectively for simultaneous robot subsystem control. However, the reference trajectory generated by these controllers would not be suitable for dexterous manipulation tasks requiring 6 DoF trajectories.
In the effort of overcoming these limitations, motion capture suits are becoming more popular in research, where they are seen to control humanoid robots [18], quadrupeds [19], wheeled manipulators [20] and manipulators [21]. As such they have been seen to be utilised in research when coupled with a WBC to improve efficiency, through utilising redundancy, and stability. In [22] a 3D motion tracking device and muscle activity sensing wearable suit are used to teleoperate a wheeled manipulator. However, being a wheeled robot, this system has limited mobility. Another teleoperation-WBC framework that utilises a motion capture suit is presented in [23] which uses an IK retargeting method to control multiple robots. However, the centre of mass (CoM) cannot be directly controlled. Xsens body sensors are used in [24] to control a simulated humanoid robot with direct kinematic mapping between teleoperator and robot, although no enhancing teleoperation strategies were shown, such as fixing frames of the robot. These sensors are also used in [25] to map body motions to a wheeled manipulator featuring admittance control for safe interactions with humans. However, this framework only utilises one arm of the teleoperator for control, limiting the utility it can provide as utilising other limbs of the teleoperator would allow for additional functionalities to be introduced. Low and high-level control using both joysticks and a motion capture suit is shown in [26] for humanoid robots. However, much like most of these frameworks for humanoids, it can only be used for robots kinematically similar to humans. Consequently, these frameworks are sub-optimal for completing jobs in hazardous environments, as quadrupedal manipulators offer a naturally more kinematically stable system. This has been resolved in [19] and [27] where a teleoperation-WBC frameworks for quadrupedal manipulators are presented. However, in [19] only the manipulator end-effector and gait commands can be sent to the robot and only end-effector and trunk commands can be sent in [27], which would prove insufficient when dealing with tasks that involve manipulation using more than one end-effector.
Overall, these aforementioned frameworks only provide limited functionality and in turn the variety and complexity of the tasks they can complete is inherently limited as a result. The breadth of functionality a framework provides is the amount of utility they have, hence these frameworks of limited functionality have a low level of utility. This low level of utility seen in these works dramatically reduces their potential for deployment for use in hazardous environments, particularly when considering that completing these tasks using legged manipulators can often require utilising several frames of the robot and a fine level of control. Therefore, to develop a framework that can handle tasks in hazardous environments, they must have high utility to tackle the large variety and high complexity these tasks present.
Considering this, the work in this paper aims to produce such a framework for legged manipulators by leveraging a WBC within a teleoperation framework to achieve high utility that would allow a teleoperator to complete tasks efficiently via controlling the main frames of the robot, along with providing useful functionalities. This utility is heightened when considering that if the feet frames of the legged manipulator can be controlled for simple manipulation tasks, this mitigates the requirement of additional arms being added to the system, which also preserves payload capacity.
As such, by expending our previous work in [11,27], the contribution of this work is as follows: 1. The development of a teleoperation framework with high utility for legged manipulators that combines a specialised WBC with a whole-body motion capture suit. 2. Developing upon the work in [27], high utility is provided to the framework through the design of a set of teleoperation strategies that are realised through dynamically changing the tasks and constraints of the WBC. Additionally, an offline planner is manually designed and implemented within the framework for executing CoM adjustments and a static gait, with automatic static stepping functionality, to further enrich the utility of the framework. 3. Work in [11] is expanded upon for use in a teleoperation framework, introducing an additional task, the CoM task, and a set of constraints (halt, local halt and CoM stability). 4. The modularity of the framework is tested by implementing it in simulation on a range of different legged manipulators. 5. The capabilities are demonstrated on hardware (an Aliengo quadruped coupled with a ViperX 300 arm) in a set of general demonstrations and an object disposal experiment.
The utility of the framework mentioned in these contributions stems initially from the generality of the set of tasks and constraints that formulate the IK WBC featured in this work, detailed in Fig. 2. Then through leveraging these tasks and constraints, a set of teleoperation strategies are developed that each, in turn, deliver unique functionality and control features, providing the utility required to complete complex tasks; specifically, these strategies provide the teleoperator with control of all main frames of the robot, providing functionality to select which DoF to control frames in and a static gait. This is all packaged into the complete framework, detailed in Fig. 1, and as far as we are aware, this is the only teleoperation framework that provides this level of utility while requiring no additional hardware to be added to the robot and being manageable by a single teleoperator.
It should be noted that only a static gait offline planner is used within the framework for locomotion, as executing dynamic gaits is out of scope due to the focus of this paper being on evaluating the capabilities of an extended WBC for teleoperation control. Integrating an advanced planner, such as an MPC will form part of the proposed future work to expand the application of this framework. This paper is organised as follows. Section 2 describes the hardware used for the robot platform and the teleoperation control device. Section 3 outlines the optimisation tasks and constraints that forumulate the WBC. In Section 4, the set of teleoperation strategies is described. Section 5 details the simulation tests performed using the WBC with the proposed teleoperation strategies. Section 6 presents and discusses the experimental results from real robot tests of the teleoperation framework. In Section 7, the research is summarised and discussed.

System Description
The overall hardware system consists of an Aliengo quadruped, a ViperX 300 (a 5 DoF robotic arm) mounted to the Aliengo and the Noitom Perception motion capture suit as the teleoperation user interface. With the Aliengo having a maximum payload of 10 kg, the ViperX 300 arm has been optimised to have a mass of 2.5 kg to improve system performance [28]. The Noitom Perception motion capture suit is a wearable and inertial-based suit, with IMU devices placed at each link of the upper body that collects skeletal data. As the legs of the teleoperator are not used in this framework, only the upper body components of the suit are used. This is because, unlike the upper body, the lower body is difficult to use for both triggers and trajectories, and with the upper body being able to control all teleoperation strategies, outlined in Section 4, removing the lower body reduces the overall power consumption of the suit. Linking these systems together consists of a Ubuntu computer running the WBC and teleoperation strategies, a Windows computer collecting the motion capture suit data, and a 5 GHz Wi-Fi router for communication between them. This setup is illustrated in Fig. 1. A detailed block diagram of the WBC block within Fig. 1 is presented in Fig. 2, that details how the WBC utilises a model of the robot and

Formulation of Optimisation Problem
Following on from our previous work [11], the optimisation problem within the WBC is to be formulated as a QP problem based on the IK cost function, where it will optimise for joint velocitiesq (as represented by the inverse kinematics block in Fig. 2). Along with a set of constraints, the overall QP is formulated as where, expanding our previous work, A and b are used to stack all Cartesian, joint and CoM tasks so that the QP problem can realise several tasks simultaneously: specifically, the CoM task has been added to provide improved stability control to aid during teleoperation. n = 12 + 5 is the number of DoF of the legged manipulator (12 leg DoFs and 5 arm DoFs), m = 4 + 1 + 1 is the number of Cartesian tasks (four feet, one trunk and one gripper), and w Cart , w Jnt , and w CoM are the tasks weights used to prioritise certain tasks over others based on the control strategy. This task prioritisation method has been utilised within this work because it reduces the computational demand of the WBC as only one QP problem needs to be solved for each time step, as opposed to a hierarchical prioritisation method that needs to solve several sequential QP problems. Further, building upon our previous work [11], to enhance this QP for use in teleoperation, (1) is subject to i) a CoM stability constraint (2) which takes the full form of ii) a halt constraint (3), along with iii) joint velocity (4) and position (5) constraints (these joint constraints were adopted from [11]). It should be noted that within (4) a safety factor of 2 is added to protect against aggressive motion caused by latency. The specifics of the CoM stability constraint (7), halt constraint (3) and all tasks are detailed later in this section.

Cartesian Tasks
The Cartesian tasks that are used within the WBC reduce the residual between a target Cartesian position and the current Cartesian position of a frame by enforcing a velocity upon it. This is achieved by utilising the Jacobian matrix of a frame which can map the current joint velocities to a Cartesian velocity, which can be formulated into A and b as where bian matrix of all frames of interest and the target Cartesian task frame velocitẏ describes the required velocity for a frame of interest of the robot to execute a desired trajectory using the velocity control law presented in [29], whereẋ target i is the target frame velocity, x target i is the target frame configuration, K Cart is the task gain, and x fk i is the current frame configuration calculated through forward kinematics.
For the work in this project, the tracking of six frames of a legged manipulator will be used. This being, in reference to Fig. 2, all end-effector frames of the robot (four feet and one gripper) with the addition of the trunk frame. These frames have been selected to achieve a strong level of control can be achieved.

CoM Task
CoM trajectory planners see frequent use for quadrupedal robot locomotion, such as in [30,31]. Therefore, in order to allow this WBC to be compatible with these planners, a CoM tracking task is included.
A Jacobian for CoM can be found based on the robot configuration. Consequently, a CoM tracking task can be easily added to the QP problem, in the same manner as the Cartesian task, through defining A CoM and b CoM to be placed within (6) as in which J CoM is the Jacobian of the CoM andẋ CoM is the target CoM velocity.ẋ CoM is derived by applying the same velocity control law as seen in (9).

Joint Tasks
Within the WBC, several joint tasks are utilised to enforce specific behaviours on each joint. Two different joint tasks are imposed on the joints within this study, the manipulability gradient and damping joint tasks. The manipulability gradient task aims to reduce the likelihood of a singularity being produced, while the joint damping tasks aim to reduce high-frequency oscillations of the joints. Aligning with (6), these joint tasks take the following form, where w mnp , S mnp ∈ R n×n , and ∇ f (q) ∈ R n are the manipulability gradient joint task weight, selection matrix, and is the sum of all limbs' manipulabilities, w prev , S prev ∈ R n×n , andq * i prev ∈ R n are the damping joint task weight, selection matrix, and optimised joint velocities for the last time step respectively. It should be noted that w mnp + w prev = 1. Further joint tasks could be used within (11) such as a Tikhonov Regularization task, see [11] for details.

Constraints
To improve the solution space of the QP problem, several constraints are added to it. To promote the stability of the robot, a CoM stability constraint (7) has been defined. Through utilising the mapping between joint space and Cartesian space, the constraint restricts the position of the CoM to always lie within the bounds of the support polygon, while there are three or more points of contact. For (7), x lb and x ub are the lower and upper bounds of the support polygon respectively, x fk CoM is the estimated position of the CoM, J CoM is the CoM Jacobian, and δt is the time step.
Another constraint that has been applied to the QP problem is the halt constraint (3) is the Jacobian of the frame to be constrained, and c is the number of these constraints. This constraint enforces zero velocity at a frame and can be applied to the frames in contact with the ground to enforce a non-slip condition. This constraint can also be adapted to halt a frame locally to another frame in position, where J local is the Jacobian of the frame that the halted frame has zero velocity in respect to.

WBC Applications for Teleoperation
Through this formulation of the WBC, several tasks and constraints can be combined to improve the utility of the teleoperation framework.

Gaze Control
For the teleoperator to fully control the camera view of the robot, the orientation of the frame of the robot that the camera module is placed upon should be controlled by generating a rotational trajectory that matches the orientation of the teleoperator's head, and realised through (9). This will enable the teleoperator to change their field of view with ease, improving operability. This task would have a relatively low weight as it should not hinder more critical tasks from being met.

Frame Locking
Quadrupedal legged manipulators will often complete manipulation tasks as instructed by the teleoperator. Once the manipulator has picked up an object, it could be desirable to keep that object stationary in space. An example of this would be when the robot has picked up an object but must move the quadruped section of the robot out of the way of hazards. This would involve enforcing zero velocity in selected DoFs of the gripper frame by applying a halt constraint on it, as detailed in (3) where any component within J halt related to a DoF not to be constrained set to zero. This implementation is included in Section 4 across various teleoperation strategies and is achievable as (3) can be added and removed dynamically.

Static Gait
The static gait allows for the controlled locomotion of quadruped-like robots, making it ideal for manipulation tasks. As such, this gait should be included in this framework. A simple static gait can be realised by assigning a Cartesian task a suitable trajectory for each foot frame and a CoM task, with the CoM task being controlled using either an offline planner or automatically due to the teleoperator only using high-level control. In the case of using an offline planner, a trajectory would be supplied to the CoM task to ensure that it moves to a position within the dynamic support polygon.
For the WBC to control the CoM automatically, the target CoM position is set as the centre of the support polygon, as illustrated in Fig. 3.

Automatically Static Stepping
During operation, the teleoperator may not consider or have a full understanding of how their commands may violate the WBC constraints, such as trying to move the trunk frame to a position that would result in the CoM leaving the area bound by the CoM constraint; without intervention, the input command would not be realised. To mitigate this limitation, the framework tracks the residual, D, between the target CoM position and the current CoM position, If a component of D surpasses a defined limit, a command would be sent to a walking pattern generator [31]. However, this is out of the scope of this paper, as the work in this paper aims to investigate the pairing of teleoperation and a WBC, and consequently, the simple static gait planner introduced in Section 3.2.3 is used to execute a manually designed gait. Furthermore, the direction of this static gait can be determined by the component of D that breached the limit. Note that the vertical DoF is not considered here. Overall, this results in moving the bounds of the support polygon and workspace in the direction of the reference trajectory.

Foot Manipulation
During manipulation tasks, often the manipulator will be grasping an object making it unavailable for further manipulation tasks. In humanoid robots, this issue is easily mitigated by utilising the other arm of the robot, however, legged manipulators typically do not feature an additional manipulator due to the additional payload limiting performance. Consequently, using the feet frames of the robot to complete simple manipulation tasks, such as pushing obstructing obstacles, is the favoured option over adding an additional manipulator to the system. This is achieved through utilising a Cartesian task for any of the feet frames.

Teleoperation Strategies
To enable the teleoperator to utilise all main frames of the robot to complete tasks despite the kinematic dissimilarities between themselves and the legged manipulator, and expanding upon our work in [27], a set of teleoperation strategies is developed. Each strategy is selected based on the hand posture of the teleoperator, as detailed in Table 1 along with their functionalities. Once a strategy is entered, the motion capture suit provides the current pose of the right hand in the hip frame. This is set as the start of the reference trajectory, with any subsequent movement from this point being sent as a reference trajectory to the WBC at 500 Hz, as illustrated in Fig. 2. To map human motions to the robot, a scaled relative pose relationship at each time step t is derived as, where x = [p x ; p y ; P z ; θ r ; θ p ; θ y ] is a vector of displacements in position and rotation, in which p x , p y , p z , θ r , θ p and θ y correspond to x, y, z, roll, pitch and yaw respectively, and µ is the scaling factor for the mapped motion. Superscripts m and target refer to master, and slave target trajectory respectively. Subscript 0 depicts the initial state once a strategy is entered. This enables the teleoperator to exit a strategy, using TS0, and then re-enter with all sub-    1,1,1,1 1,0,0,1,1

TS5 n/a
• Resets all frames to a home position. 1,1,1,1 1,1,1,1,1 sequent movements starting from that point, allowing the teleoperator to either readjust their posture when they are at the limits of their movement or pause operation safely.

Simulations
Before the teleoperation framework was tested on hardware, a set of simulations were completed in order to tune the WBC task weights and gains, test the framework's modularity, and ensure it was safe to deploy on the real robot. These simulations consisted of a Pybullet physics base simulation where three different quadruped robots (A1, Laikago and Aliengo all from Unitree) paired with the ViperX 300 arm were tested using pre-recorded motion capture suit data that involved utilising all teleoperation strategies. Tuning the task weights and gains involved a simple trial and error method, where an initial estimate was made, and then they were either increased or decreased by a factor of Fig. 4 Snapshots of TS1c being used across a range of different robots to test framework modularity 10 or 2 for the weights and gains respectively. As a result, similar whole-body motions were observed across all three robots to realise the reference trajectory, as illustrated in Fig. 4. Furthermore, as illustrated in Fig. 4, an increase in z displacement and pitch rotation is observed before any x displacement. This is a direct result of the CoM stability constraint (7) causing the WBC to prioritise trajectories in DoF that do not cause a shift in the CoM. The effectiveness of the WBC being able to realise reference trajectories that would otherwise be impossible is highlighted in Fig. 5, where without the WBC activated, the input of the reference trajectory results in a singularity being reached. Additionally, in Fig. 5 the WBC's ability to preserve stability using a combination of (7) is highlighted, where in the case where a foot is lifted from the ground, the WBC causes the CoM to shift within the new support polygon; with the WBC inactive, the robot is observed to fall.

Experimental Results
To test and validate the effectiveness of the teleoperation framework for use with hardware in real-time, two experiments were completed. One was a general demonstration of the majority of the teleoperation strategies, and the other Fig. 5 Comparing the performance with and without the WBC enabled was a real-world task of disposing of an object, representing a hazardous substance, in a push pedal bin. To analyse the performance of the framework across all experiments, the joint positions of the robot were recorded and were then passed to a simulated robot in PyBullet, from which the positions and orientations of all frames of interest were extracted and presented in Figs. 8 and 10. Furthermore, in all experiments, no joint feedback was used due to the real robot's joints not being able to perfectly realise the WBC output. Consequently, this highlights the advantages of utilising a human-in-the-loop, as through visual feedback they can compensate for the limitations of the robot and the imperfect realisation of reference trajectories. A supplementary video of these experiments can be found at https://youtu.be/iL_K_CVA0pU.

General Demonstration
In this experiment, one of three different movement types was used to provide a reference trajectory to the WBC, using different strategies to evaluate their performance. This being in x, y, z, or yaw, which will be referred to as M1, M2, M3 and M4. It should be noted that with the trunk being aligned with the world frame, +x was in the direction the trunk is facing, +y was to the left of the trunk, and +z was vertically upwards. The strategies used and analysed in this test are TS0, TS1, TS2 and TS3.

TS1
Initially, TS1b was entered and an M4 was generated using the motion capture suit to be realised by the gripper frame. As such, the gripper frame rotates in θ y while the trunk frame was observed to adjust its pose to aid in the realisation of this trajectory, as seen in the data in Fig. 8 and demonstrated in Fig. 6. However, a small displacement of the gripper was seen in p y despite the use of a halt constraint applied to its position. This was due to the WBC not accounting for the imperfect joint motors of the robot hence not minimising error when a significant shift in the CoM tending towards an individual leg occurs, which was further exacerbated by the weight of the arm, this is highlighted in Fig. 9 where the y component of the centre of pressure (CoP) approaches the boundary. This issue could potentially be resolved by building the WBC around ID instead of IK, as not only would it better consider the weight of the arm, but even though the Aliengo motors can be position controlled they operate through torque control. Switching to TS1c, a M1 and a subsequent M3 trajectory were generated and executed by the gripper frame. Again for both trajectories whole-body motion was observed in the robot to help the gripper realise the reference trajectory, as presented in Figs. 6 and 8, while also demonstrating accurate realisation of the WBC output. Furthermore, the orientation halt constraint proves to be effective in TS1b, as a minimal rotation of the gripper frame was observed.

TS3
For all strategies of TS3 (TS3a, TS3b, TS3c and TS3d), before each foot was lifted the offline planner provided a reference trajectory for a CoM Cartesian task to ensure the CoM lies within the new support polygon to preserve stability. This stability is supported by Fig. 9 where even with a foot lifted from the ground the CoP always lies within the stability bounds of the support polygon. Foot frames FR and RL completed an M1 trajectory, while frames FL and RR completed an M2 trajectory. Pose adjustments were observed by the trunk to assist the foot frames to achieve their reference trajectories, however, these are only minor adjustments, due to the CoM constraint, so stability was maintained while there is a reduced support polygon, as shown in Figs. 7 and 8. Furthermore, the gripper frame was observed to stay almost completely fixed in position while the foot frames completed their tasks, demonstrating the effectiveness of the halt constraint, with the only caveat of the imperfect joint motors which caused the gripper frame to displace slightly in p y .

TS2
In TS2a, a reference trajectory of M2 and then M3 were generated by the motion capture suit and sent to the WBC to realise these trajectories using a trunk Cartesian task. As there was a halt constraint in place for the gripper's position, as the trunk completed both reference trajectories, in the WBC output the gripper remained locked in position, as presented in Fig. 8. However, minor displacement was observed in the real robot in p y due to a combination of imperfect motors unable to perfectly compensate for the movement of the trunk and the IK-based WBC not considering the dynamics of the robot. This displacement could be reduced through the implementation of a state estimator, such as [32], which considers the dynamics and contact state of the robot to provide an accurate estimation of the trunk's position and orientation. After completion of these trajectories, TS2b was entered and an M1 trajectory was realised by the trunk while the gripper frame was locked in position and orientation, using a halt constraint, as shown in the data presented in Fig. 8. This significantly shifted the CoM and, which breached the limit described in Section 3.2.4. This caused the offline planner to automatically generate the reference trajectories for a static stepping gait. During this gait, the was gripper locked locally to the trunk, using a local halt constraint, while its orientation was still locked in the world frame. The gait was observed to complete successfully; overall forward displacement can be observed in Fig. 8 and its stability is observed in Fig. 9 where the x and y components of the CoP never leave the stability limits, demonstrating the effectiveness of the CoM task (10).

Object Disposal
The objective of this test was to dispose of a small box in a push pedal bin. This task requires the robot to simultaneously hold open the bin with the pedal while dropping the box in the bin, for which a range of frames need to be controlled in select DoF at a time (for dexterity) all while stability is maintained. Therefore, utilising the high utility of the tele-   TS1c  TS3a  TS3b  TS3c  TS3d  TS2a  TS2b CoP x CoP y Stability Limits x Stability Limits y operation framework is critical to the success of this test. The box had a width of 64.57 mm while the gripper had a maximum gripping width of 75.82 mm; the completion of this task also demonstrates the high level of dexterity achieved by this framework. Initially, TS1a was used to utilise both position and orientation trajectories, generated by the motion capture suit, to position the gripper frame close to the box and roughly align with its orientation, all the while the WBC produced whole-body motion to aid in the realisation of this task. Next, TS1c was used to for finer movement to position the gripper ready to pick up the box, while it is locked in orientation, as shown in Fig. 10. The gripper then closed and picks up the box using TS1c, after which TS5 was used to bring the robot back into the home configuration for quick repositioning of the gripper and box. TS3b was then used to pick up the FL foot and push the pedal on the bin, lifting the lid. Due to the imperfect motors, the trunk rotates in θ r when the FL foot lifts off the ground. This in turn causes the real robot to experience error from the WBC output, in the gripper and FL frames, as detailed in Fig. 10. However, through observation, the teleoperator compensated for this, which lead to the successful opening of the bin. While the gripper and foot were locked in position using a halt constraint, ensuring the box was stable, and the foot remained on the pedal, TS2a was used to orientate the trunk to look into the bin to simulate the scenario outlined in Section 3.2.1 and the teleoperator only having   Fig. 10, even while it is on the unstable surface of the bin pedal and while the trunk and gripper frames complete trajectories. This demonstrates the effectiveness of the halt constraint (3) and the stability of the framework enforced by the CoM constraint (7), which is further supported by Fig. 11 where the CoP is observed to stay within the bounds of the stability limits. TS3b was then re-entered to lift the FL foot, releasing the pedal. Finally, TS5 returned the robot to the home position. This experiment is presented in Fig. 12, where it can be observed that not only does the framework enable the successful execution of a complex experiment, where three frames of the robot are used in conjunction to complete tasks, but also the robot remained stable even when one foot was lifted and the trunk was completing a trajectory (further evidence is provided in the supplementary video). In turn, this demonstrates the effectiveness of this framework as without the ability to control a range of frames of the robot this task would prove to be impossible without adding another manipulator to the robot, although doing so would only be a detriment to its performance and payload capacity.
Despite latency being a common issue in teleoperation, in terms of both time delay and data loss which can both result in robots exhibiting dangerous behaviour, no latency issues were encountered during both this experiment and the general demonstrations. This was primarily due to the joint velocity dampening tasks (Tikhonov Regularization and joint dampening) that penalize large changes in joint velocities and the CoM constraint (7) which together both ensured that any delay or intermittent references passed to the framework due to latency would neither cause unstable or dangerous behaviour in the robot. Additionally, as mentioned in Section 3.1, to add further protection against aggressive motions the joint velocity limits have a safety factor of 2 applied to them.
The only limitation experienced through deploying the framework to complete this task was the high mental load on the teleoperator, where completing the task required the use of many of the control options, which could have potentially led to increased time in completing this complex task. One method of alleviating this mental load on the teleoperator would be to add further automation to shift more of the workload onto the framework, such as automating which DoF a frame is being controlled in through using computer vision t=0 t=51 t=69 t=77 t=99 t=116 t=133 t=140 Fig. 12 Snapshots of the teleoperator controlling the robot during the object disposal experiment and machine learning techniques to infer the task requirements.

Conclusion
In this paper, a teleoperation framework of high utility, which utilises a tailor-made WBC, a range of teleoperation strategies, and a motion capture suit to control legged manipulators has been developed. Through implementing a set of tasks and constraints specialised for teleoperation with a QP optimisation problem, forming the core of the WBC, the teleoperation framework can take input reference trajectories generated by a motion capture suit to control a range of functionalities and all main frames of the legged manipulator. This work has demonstrated the effectiveness of each teleoperation strategy through a general demonstration of the strategies and through completing an object disposal experiment that would be impossible to complete without the use of these strategies and no modifications to the robot's hardware. Future work to further improve the system would involve utilising a gait pattern generator and online planner to aid in realising highly dynamic motions and gaits, consequently enabling the use of the framework for applications where rough terrain is present. Furthermore, as discussed in Section 6.2, further automation will be added to the system to reduce the mental load on the teleoperator to improve efficiency while completing complex tasks. Other future work could also involve testing the implementation of an ID-based WBC, as it is able to consider the dynamics of the robot and handle rough terrain.

Declarations
Competing interest The authors have no relevant financial or nonfinancial interests to disclose.
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecomm ons.org/licenses/by/4.0/.