Smart ArM: a customizable and versatile robotic arm prosthesis platform for Cybathlon and research

Abstract

Background

In the last decade, notable progress in mechatronics paved the way for a new generation of arm prostheses, expanding motor capabilities thanks to their multiple active joints. Yet, the design of control schemes for these advanced devices still poses a challenge, especially with the limited availability of command signals for higher levels of arm impairment. When addressing this challenge, current commercial devices lack versatility and customizing options to be employed as test-beds for developing novel control schemes. As a consequence, researchers resort to using lab-specific experimental apparatuses on which to deploy their innovations, such as virtual reality setups or mock prosthetic devices worn by unimpaired participants.

Methods

To meet this need for a test-bed, we developed the Smart Arm platform, a human-like, multi-articulated robotic arm that can be worn as a trans-humeral arm prosthesis. The design process followed three principles: provide a reprogrammable embedded system allowing in-depth customization of control schemes, favor easy-to-buy parts rather than custom-made components, and guarantee compatibility with industrial standards in prosthetics.

Results

The Smart ArM platform includes motorized elbow and wrist joints while being compatible with commercial prosthetic hands. Its software and electronic architecture can be easily adapted to build devices with a wide variety of sensors and actuators. This platform was employed in several experiments studying arm prosthesis control and sensory feedback. We also report our participation in Cybathlon, where our pilot with forearm agenesia successfully drives the Smart Arm prosthesis to perform activities of daily living requiring both strength and dexterity.

Conclusion

These application scenarios illustrate the versatility and adaptability of the proposed platform, for research purposes as well as outside the lab. The Smart Arm platform offers a test-bed for experimenting with prosthetic control laws and command signals, suitable for running tests in lifelike settings where impaired participants wear it as a prosthetic device. In this way, we aim at bridging a critical gap in the field of upper limb prosthetics: the need for realistic, ecological test conditions to assess the actual benefit of a technological innovation for the end-users.

Introduction

In the field of upper-limb prosthetics, the last decade saw great progress in mechatronics, leading to the introduction of more advanced upper limb robotic prostheses in the market. These devices, in particular prosthetic hands (e.g. i-Limb hands from Össur, BeBionic and Michaelangelo from OttoBock, Taska hands) typically feature more Degrees of Freedom (DoF) than their older counterparts, and offer more accuracy as well as finer actuation. However, the main challenge in upper-limb prosthetics still lies in the wearer’s control of such DoFs as their number increases. In particular, although methods to retrieve command signals from the wearer have also improved along with mechatronics, most commercially available prosthetic arms still make use of dual-site electromyographic (EMG) measurements to detect movement intentions, despite this method’s limited bandwidth. Additionally, myoelectric prosthesis wearers may lack options to customize its control in depth when attempting to tailor its behavior to suit their needs [1].

Indeed, commercially available prostheses are typically “closed” devices that can only be configured with proprietary tools provided by the manufacturer. Such tools, which may exist in different versions intended for either end-users (i.e. prosthesis wearers) or clinicians, restrict access to the prosthesis’s parameters and state variables. For instance, accessible EMG-related settings are often limited to electrode measurement gains, as well as threshold values and raise times. Additionally, these settings affect joint activation and switching signals such as co-contraction, but the underlying control law cannot be changed, to perform torque or current control for example. Finally, even though industrial standards in the market offer a certain level of compatibility, inter-operability between brands is generally limited to basic analog signals relaying EMG measurements to the more distal parts.

As these commercial devices do not allow for in-depth customization of their motor behavior, research groups in the field of arm prosthetics need to resort to substitutes on which to deploy their own control laws for testing and validation. To this end, several approaches have been employed by researchers. Firstly, one solution consists in designing and making in-house their own ad hoc prosthetic device [2,3,4,5,6], over which they can have total freedom, in particular with respect to the control laws that can be implemented. The main drawback of this approach is the significant amount of development and production that is required as up-front work, before a prototype is available for carrying out experiments. Additionally, custom-made devices may be difficult to reproduce by other labs willing to employ them in their own research, especially if most of their parts are not readily available as manufactured goods. Besides, such ad hoc robotic test-beds tend to lack versatility when they include case-specific components (either software or hardware) restricting the possible application scenarios to those that were originally envisioned during development.

A second solution consists in employing life-sized human-like robotic arms, either developed in-house [7, 8] or reproduced from open-source designs [9,10,11,12]. With respect to commercial prostheses, such robots offer more freedom over their motor behavior, as well as their connections with external devices. This allows researchers to customize the control laws these robots follow, by interfacing them with the relevant sources of input signals and defining the policies that translate these signals into motor commands. The key limitation of this approach is that these robotic arms are stand-alone devices, whereas actual prostheses are body-worn by nature. As a result, even though such devices have been used for myoelectric control training or studies on sensory feedback, they are not suitable as is for carrying out experiments in lifelike settings with differently-limbed persons.

A third solution consists in setting up a virtual reality (VR) environment, within which participants control a virtual prosthesis [13,14,15,16,17,18,19]. Following this approach, researchers can focus on designing control laws without having to solve the underlying engineering challenges that their deployment and operation on a physical robot would imply. It also allows researchers to account for inter-individual morphological variability in their protocols, without involving additional material costs. However, due to the virtual nature of such an experimental setup, this type of prosthesis substitute does not accurately render the dynamics of a physical body-worn device. Indeed, a participant controlling a simulated arm in a VR environment would not experience the same physical interactions in terms of weight distribution and inertia, along with joint movement limitations due to socket or strap constraints. Additionally, a VR apparatus would not allow to take into account the residual limb’s tendency to slightly rotate or shift inside the socket depending on the arm’s posture, which is one of the common causes for disturbances in arm prosthesis control. Lastly, virtual joints usually follow simplified models, unaffected by mechanical play nor friction, and capable of unrealistically high instantaneous speeds and accelerations.

It is also worth noting that prototypes of 3D-printed “Do-It-Yourself” (DIY) arm prostheses [20], while inexpensive and easy to reproduce, are not designed to provide the same level of motor performance as their industrial counterparts. As a consequence, they are not capable enough for most functional use cases involving finger dexterity or mechanical loads [21].

To this date, most research works in the field of upper limb prosthetics have focused on designing and controlling prosthetic hands, with the aim to provide sufficient finger dexterity for a wide range of Activities of Daily Living (ADL). This led to a large number of prototypes of prosthetic hands being reported in the recent literature [10, 12, 22,23,24,25,26]. In contrast, little research has focused on prosthetic elbows and their control, especially as the latter typically adds up with wrist and hand control for trans-humeral prosthesis wearers. Yet, the elbow joint plays a critical role in the execution of reaching movements and also influences the orientation of the end effector [27]. In this context, we commit to expand the body of literature dedicated to the challenges specific to trans-humeral prosthetic wearers, and develop prototype devices with which to experimentally investigate these challenges.

The goal of the present work is to provide researchers in over-the-elbow prosthetics with a versatile and customizable robotic test-bed, in the form of a trans-humeral prosthesis. This control-agnostic platform was designed for prototyping, experimenting and developing novel control approaches involving multiple joints from elbow to end-effector. Its elbow joint was designed in order to accommodate for all sizes of above-the-elbow residual limbs, including cases of forearm agenesia with long humerus. This paper introduces the prosthesis platform (illustrated in Fig. 1) that was developed with this aim by our research group. We start with a description of the platform’s design from an engineering standpoint, in terms of electronic, mechanical and software characteristics. Then, we detail its main features and measures of performance, and put them in perspective to existing prosthetic devices. We also illustrate its versatility as a research tool with examples of application scenarios, including several studies where this platform was part of the experimental apparatus, as well as our long-lasting participation in Cybathlon’s “Arm Prosthesis” race¹. Finally, we discuss the platform’s limitations and possible solutions to overcome them.

Fig. 1

The Smart ArM prosthesis in its current iteration, fitted with a Taska prosthetic hand

Robot design

Design principles

The development of the Smart ArM prosthesis was driven by several design principles. Firstly, the main goal was to develop a prosthetic arm test-bed, whose control schemes should be customizable in depth and at all steps of the control loop. In particular, researchers should be able to reprogram the prosthesis’s embedded system and design their own control laws by implementing whatever formulas or rules they devise to translate input signals into motor commands. The ability to customize the control schemes would also play a role in the prosthesis’s modularity, in that the embedded system should be able to manage a variable number of actuators in order to accommodate for different levels of impairment. Additionally, the embedded system being reprogrammable should allow the interfacing of a wide range of external devices, including measuring tools, body-worn accessories and sensory feedback devices.

Secondly, the prosthesis should be compatible with current industrial standards in upper-limb prosthetics, in order to increase modularity. In particular, to ensure compatibility with the largest number of commercial prosthetic hands, the prosthesis should include a Quick Disconnect Wrist (QDW) assembly, capable of achieving both mechanical coupling and analog signal transmission, through a four-ring coaxial plug. Additionally, the prosthesis should be operated with the same type of surface electrodes that are fitted on commercial arm prostheses, and provided by manufacturers such as OttoBock and Steeper. With this aim, the prosthesis should provide the electronic hardware required to connect these electrodes with the embedded system, as well the software tools required to process the myoelectric signals. Besides, the prosthesis’s electric power supply should be consistent with the sizes and specifications of batteries typically found in commercial prostheses. In terms of voltage, current standards correspond to two- and three-cell Lithium-Polymer (LiPo) batteries, corresponding respectively to 7.4 V and 11.1 V in nominal voltage supply. The three-cell standard was recently introduced in upper limb prosthetics by OttoBock with the Michaelangelo hand and Axon-Bus system [29]. Furthermore, with respect to the device’s mechanical structure, its chassis should allow for durable and reliable fixation to conventional prosthesis sockets.

Finally, the prosthesis should employ, to the largest extent possible, off-the-shelf components that are easy to procure and to replace, in order to foster easy integration within research setups while reducing costs. This principle should condition the design choices with respect to materials, machining, mechanical parts, electronic components and computing-capable devices. In particular, electronic components (with the exception of prosthetics-specific hardware such as surface electrodes) should be available from spare parts sellers. Moreover, components at a risk of getting obsolete or discontinued should be easy to substitute with other products with similar features. Besides, inexpensive methods such as additive manufacturing should be preferred when making parts that do not play a critical role in the prosthesis’s mechanical integrity. Lastly, the embedded system’s architecture and features should favor freely available libraries and tools, while relying as little as possible on proprietary software.

System overview

As an introduction to the following sections, which describe the platform’s hardware and software, here we provide the reader with an overview of the Smart ArM prosthesis’s general structure and main features. Most of the platform’s components, are concentrated in its forearm structure (see Fig. 2). The elbow actuation assembly links the trans-humeral socket with the forearm’s proximal end, while the prosthetic hand (not shown in Fig. 2) is coupled to the foream’s distal end through a dedicated connector.

Fig. 2

Close-up picture of the prosthesis’s forearm, with upper shell open. LED light-emitting diode, NGIMU new generation inertial measurement unit (from x-IO Technologies)

The platform’s forearm is built on an aluminium structure (see Section "Forearm"), which houses the batteries and all the embedded electronics (see Section "Electronics") under a shell. On the outside, several user interface elements are located on the wrist, to allow the wearer to interact with their prosthesis (see Section "User interface").

With respect to the platform’s internal structure, Fig. 3 describes how its components (i.e. sensors, actuators, user interface elements) connect to the main board, which runs the program managing the prosthesis control (hereafter referred to as “control program”).

Fig. 3

Diagram of connections between the components of the platform in its current iteration. I2C Inter-Integrated Circuit, MQTT message queuing telemetry transport (obsolete meaning), SPI Serial Peripheral Interface, UART universal asynchronous receiver-transmitter, USB Universal Serial Bus, WLAN wireless local area network

With respect to connections with external devices, Fig. 4 illustrates peripheral systems that were successfully interfaced with the platform, as described in Section "Interfaces with external devices" and Appendix D "Compatible prosthetic hands".

Fig. 4

Examples of peripheral devices compatible with the Smart Arm platform

Electronics

The Smart ArM platform’s electronics include four electronic boards (see Fig. 5), which are described in the following sections, as well as two removable batteries and various user interface elements.

Fig. 5

Disassembled view of the prosthesis’s essential electronic components

Main board

The prosthesis’s embedded system is operated by a single-board computer (model 3B+ from Raspberry Pi) located within the forearm’s structure (see Fig. 2). This model includes a 64-bit quad-core processor running at 1.4 GHz and 1 GB of Random Access Memory (RAM), which offer more than enough computing power to ensure that the control program can operate reliably at frequencies up to 100 Hz.

The choice of a commercial single-board computer, which exceeds the platform’s needs in terms of computing power, was motivated by several concerns. Firstly, such a device provides the flexibility that comes with using a desktop Operating System (OS) as the embedded system (see Section "Operating system" for more details). Moreover, employing an off-the-shelf product for the main board notably shortened the platform’s development process, compared to designing a custom board with similar features. Finally, this model provides out of the box all the connectivity options (wired and wireless) required for the platform’s various components to connect to it (see Fig. 3).

Indeed, this board comes with multiple ports for communication with peripheral components, starting with four Universal Serial Bus (USB) ports. One of them receives a Bluetooth dongle with which the main board connects wirelessly to the prosthetic hand. Besides, the board exposes pins dedicated to its Universal Asynchronous Receiver-Transmitter (UART), communication buses for Serial Peripheral Interface (SPI) and Inter-Integrated Circuit (I2C) protocols, and General Purpose Input-Output (GPIO).

Expansion board

The main board is connected to a custom-designed expansion board which includes several electronic components underlying the interactions between the embedded system and the prosthesis’s sensors and actuators. To begin with, four 16-bit Analog-to-Digital Converters (ADC, model ADS1115 from Texas Instruments) are mounted on this board to receive analog signals from sensors such as electrodes (see next section). Besides, two 12-bit Digital-to-Analog Converters (DAC, model MCP4728 from Microchip) are mounted on the expansion board to send low-voltage (up to 3.3 V) analog signals to the prosthetic hand (see Fig. 3).

Finally, the latest iteration of the expansion board includes an Inertial Measurement Unit (IMU, model BNO055 from Bosch) providing input signals from 3-axis accelerometers, gyroscopes and magnetometers. In addition to these nine signals, the IMU also performs sensor fusion on these signals to provide 3-axis relative orientation measurements, expressed as quaternions. On previous iterations, similar signals were provided by a USB-connected external IMU, such as the x-IMU or NGIMU boards from x-IO Technologies, as visible in Fig. 2.

These three types of active components mounted on the expansion board (ADC, DAC & IMU) communicate with the embedded system through I2C. Additionally, the expansion board acts as a hub for connecting other peripheral components to the main board (see Fig. 3). On its top surface, multi-contact connectors give easy access to the main board’s pins to which are connected the LED strip (SPI), the buttons on the control panel (GPIO) and the actuator controller (UART).

Electrodes

In its current iteration, the Smart ArM prosthesis includes eight commercially available electrodes (model ELEC50 from Steeper) mounted on the socket. Rather than outputting raw signals, these sensors are designed to perform signal processing and filtering in order to provide EMG measurements in the form of a rectified envelope signal. Each electrode is wired to one of the sixteen available analog input channels on the expansion board, in a way that each ADC receives signals from two electrodes. Thanks to the upstream processing performed by these sensors, the signals converted by the ADCs are directly usable by the control program.

The electrode placement was defined after a systematic exploration of available muscle sites on the residual limb of our team’s pilot, a person with right forearm agenesia. This exploration led to the mapping of eight sites capable of voluntary and independent contraction while being isolated enough to reduce cross-talk during surface measurements. The electrodes are located in dedicated holes in the socket, adjusted by the occupational therapist during the socket fabrication process, based on the muscle site mapping. Each electrode is maintained in place with elastic straps and housings, ensuring continuous contact with the residual limb’s skin (see Appendix A "Electrode suspension" for more details).

No surgery was performed on our pilot’s residual limb. However, due to the congenital nature of his condition, these muscle sites do not exactly correspond to muscles that are found in a typical human upper arm, with the exception of the biceps and triceps.

Motor controller

The operation of the DC motors actuating the wrist and elbow is managed by a single motor controller (model RoboClaw V5C 2x7A from Basic Micro). This controller supports two channels operating at a maximum voltage of 34 V and can handle continuous currents of 7.5A and peak currents of 15A per channel. It communicates with the embedded system through a serial connection, via the main board’s UART (see Fig. 3). Thanks to the incremental encoders, it can accurately operate each motor in closed-loop mode to perform position or velocity control. In the case of the Smart ArM prosthesis, it is typically used to perform velocity control with pre-defined acceleration and deceleration profiles. The controller also supports user-defined RoM, which adds another safety layer for joint actuation. With this feature, it will automatically stop a motor when the signal from the corresponding encoder crosses the defined threshold, even if the received motor commands order to keep going. For the elbow joint, the controller is set with a RoM slightly narrower than the mechanical RoM resulting from the prosthesis’s outer envelope.

Batteries and power board

The prosthesis’s on-board power source takes the form of two LiPo batteries. The Taska hand is powered directly by a two-cell battery with a nominal voltage of 7.4 V, a capacity of 800 mAh and a maximum continuous discharge current of 20A (discharge rate of 25). All the other components (i.e. elbow and wrist actuators, main board, expansion board, LED strip and actuator controllers) are powered by a three-cell battery with a nominal voltage of 11.1 V, a capacity of 1500 mAh and a maximum continuous discharge current of 37.5A (discharge rate of 25). For testing purposes, this battery can be replaced by an external power supply unit, thanks to the power connector on the forearm’s internal lateral face (two-contact B-series model from Lemo).

Each battery is packaged in a protective plastic casing with a specific shape, so that it cannot be inserted in the other battery’s housing. Electrical connection is achieved using housed connectors, with the female one on the battery side to ensure that live battery terminals are not exposed. When a battery is plugged in, its battery level is constantly displayed by a dedicated 4-LED indicator, visible on the forearm’s top surface through a window slit.

The prosthesis includes a custom-designed power board that ensures safe power distribution from its batteries to its various components. In particular, it delivers the 5 V and 3.3 V tensions required to power the main board and components mounted on the expansion board. This is achieved using two DC-DC switching converters (models TSR 2-2450 and TSR 1-2433 from Traco Power). The power board also includes diodes for reverse current protection, and fuses for over-current protection: 7A for the actuators and hand, 2A and 1A for the converters.

User interface

The prosthesis incorporates multiple user interface elements, that is: elements that allow the wearer to receive information about the system’s state and interact with it by sending commands (apart from myoelectric control). With respect to the former category, prosthesis state feedback is provided through two sensory modalities: auditory and visual. The expansion board includes a piezo-electric buzzer capable of emitting audio cues of various durations and pitches, with which the prosthesis indicates events such as the detection of a button press. At the wrist level, the lamination ring’s outer surface is circled by a strip of ten addressable LEDs (see Fig. 2), whose hue and brightness can be individually defined. These LEDs provide continuous and non-intrusive feedback, and their placement next to the hand makes them easily noticeable by the wearer without taking too much space in the field of view.

Besides, the wearer can interact with the prosthesis through a control panel located around the wrist (see Fig. 2). This panel includes two toggle switches and three push-buttons, with which the wearer can send specific commands to the prosthesis. The switches are typically employed to enable or disable a specific feature, such as the automatic start of the control program when the prosthesis is powered on. Conversely, the push-buttons are typically employed to load pre-defined settings or for debugging purposes.

Hardware

Elbow

The prosthesis’s elbow joint is based on a motorized exo-skeletal solution similar to a motorized orthosis, with the actuator being placed on the lateral external face of the socket (see Fig. 6). The actuation assembly is attached to a square-shaped aluminium bracket encircling the socket. To ensure a tight mechanical coupling, a 3D-printed plastic adapter ring is fitted between the bracket’s inner perimeter and the socket’s outer surface. In this way, the bracket works as a “one-size-fits-all” fastening system, where only the adapter ring needs to be custom-shaped based on the socket’s unique shape.

Fig. 6

A, B Close-up pictures of the elbow actuation unit, before and after attaching the socket and forearm, respectively. C Computer-aided design (CAD) view of the elbow actuation. unit

At upper arm level, the exo-skeletal elbow assembly is 17 cm long and protrudes by 4 cm on the socket’s external side. Including the socket on which the prosthesis is currently fitted, this protrusion results in an upper arm girth of approximately 35 cm. The actuation assembly weighs about 350 g. Considering the physical envelopes of the socket and the forearm, the joint’s Range of Motion (RoM) spans 125$^\circ$ from full extension (forearm aligned with upper arm) to maximum flexion (forearm in contact with upper arm). The actuation is performed by a Direct Current (DC) motor (model DCX22L from Maxon) associated with a planetary gearhead and a worm-and-wheel transmission (see Appendix B "Elbow actuation" for more details). Due to the non-backdriveable nature of this transmission, the elbow’s motor does not draw any power when the joint is not moving, even under load. When the prosthesis is unloaded (i.e. only carrying its own weight), the elbow’s rotation speed can reach 120$^\circ$/s (see Section "Experimental validation" for more results).

The motor and gear transmission are covered by a 3D-printed piece of plastic material (VisiJet M3-X from 3D Systems, Acrylonitrile Butadiene Styrene (ABS) type). The transmission mechanism also includes a slide lock that allows the wearer to easily disengage the joint from the motor. When pressed, a lever located on the side of the elbow assembly moves a stud out of a 12-hole clutch cogwheel, thus uncoupling the actuation unit’s output shaft from the joint’s axle. In this way, the forearm can balance freely when this clutch mechanism remains disengaged. Additionally, the stud being made out of brass acts as a mechanical fuse. In the event of a critical torque being applied on the clutch while the mechanism is engaged (due to malfunction), the stud will break to avoid applying dangerous forces levels to the user and damaging the motor or transmission.

The choice of an exo-skeletal design for this joint was motivated by the wish to accommodate for all sizes of residual limbs over the elbow. Indeed, with the actuation unit and pivot joints located on the side of the socket, this design does not require taking up space below the residual limb to fit the prosthetic joint. As a result, even for residual limbs with a long humerus, it allows the upper arm segment (i.e. residual limb fitted with the prosthetic elbow) to remain symmetrical to the contra-lateral arm. This proved critical for our Cybathlon team’s pilot, whose residual limb exhibits a long humerus but no functional elbow. The exo-skeletal design trades off compactness for versatility and anatomical consistency, given that the actuation unit takes up a notable amount of space on the side of the socket. However, from our experience with our team’s pilot, we reported no major drawbacks related to the prosthesis’s girth at upper arm level. In particular, this design does not seem to prevent him from slipping the prosthetic arm into a sleeve to put on a piece of clothing.

Forearm

The forearm is built around a machined aluminium structure that works as the prosthesis’s chassis (see Fig. 2). The coupling of the elbow’s axle with the forearm is performed by an aluminium bracket fixed to this chassis with four screws. Inside the forearm’s chassis are located the electronic components and batteries, in dedicated isolated plastic compartments.

On the outside, the aluminium structure is covered with a shell made of plastic material (ABS-type VisiJet M3-X from 3D Systems). The shell is made of two halves (bottom and top). The top half includes windows slits so that the Light-Emitting Diodes (LEDs) of the two battery level indicators are visible from the outside. The bottom half includes two rectangular openings, in which the battery casings are inserted. It is also designed with housings for the emergency stop button (on the external lateral face), as well as a self-latching power connector and toggle switch for power source selection (on the internal lateral face). With respect to dimensions, the prosthesis’s forearm is 30 cm long from elbow to wrist. Its girth ranges from 23 cm at wrist level, to 31 cm near the elbow.

Admittedly, the current design of the prosthesis is not appropriate for a direct adaptation to a trans-radial configuration. Indeed, the prosthesis’s batteries and electronic components take up most of the space inside the forearm. As a result, shortening the prosthesis’s forearm structure to fit it to a trans-radial socket would require to either relocate these elements to the socket’s periphery, or design a larger forearm structure to increase its capacity.

Wrist Rotator

The wrist joint is based on a conventional electric wrist rotator (model 10S17 from OttoBock, see Fig. 7) mounted on a wrist lamination ring (model QDALR=50 from Steeper). The original actuator was replaced by a 22 mm-wide 6 V DC motor (model 22N28-216E-286 from Portescap) and drives the planetary gear transmission included in the rotator. In order to allow for servoing, the motor is also fitted with an incremental encoder. The wrist’s unloaded rotation speed can reach 120$^\circ$/s (see Section "Experimental validation" for more results) and the joint is capable of continuous rotation about its axis (360$^\circ$ range of motion).

Fig. 7

A Close-up picture of the wrist assembly, including the control panel and LED strip described in Section "User interface". B CAD view of the wrist assembly, without the control panel

In order to make maintenance operations easier, the wrist assembly is mounted within a 6 cm-wide nylon cylinder and fixed to the metal chassis with four screws. It weighs 250 g and is only 8 cm long (8.5 cm with an encoder fitted on the actuator). When mounted on the chassis, its weight together with the forearm is 1.05 kg. This is equivalent to the mean weight of a human forearm for a 70 kg individual (approx. 1.5% of total body mass) [30].

The actuation unit is coupled to a Quick Disconnect Wrist (QDW) assembly, allowing any prosthetic or robotic hand compatible with the QDW standard to be plugged to the prosthesis. This component consists of a notched wheel for mechanical coupling, and a four-ring coaxial plug for electric signal transmission to the hand. Two of these rings supply power (ground and positive voltage) while the other two are dedicated to communication. Although communication protocols can vary among models of prosthetic hands, the most common method uses these two rings as two independent analog signal channels, where the information is carried by their respective voltages. In this setup, the communication through the coaxial plug is one-way: the hand can receive commands but cannot send signals back through the connector.

Prosthetic hand

Thanks to the QDW assembly, the Smart ArM prosthesis can be combined with a wide range of prosthetic and robotic hands compatible with this standard, from poly-digital hands such as the i-Limb Quantum (from Össur) and Taska hand (from Taska Prosthetics), to single-DoF devices such as the SensorHand Speed (from OttoBock) (see Appendix D "Compatible prosthetic hands" for more details).

In its current iteration, the prosthesis’s end-effector is a Taska hand [31] (first-generation model, large size), which comprises six motorized DoFs actuating each finger’s flexion as well as the thumb’s opposition. It also includes a passive (i.e. not motorized) wrist flexor that allows the hand to be tilted and locked into one of three orientations (slight flexion, slight extension or neutral). Alternatively, this wrist flexion joint can be unlocked, in which case it behaves in a spring-like fashion, pushing toward wrist extension.

With this device, the method to communicate through the coaxial plug is the previously described analog two-channel method: one triggers the opening of the hand whereas the other triggers the closing. The closing or opening speed is controlled through the voltage of the corresponding channel: the higher the voltage, the higher the finger speed. In the Smart ArM platform, this is achieved with the DACs on the expansion board, which set the analog channels at voltages corresponding to the desired speed.

In order to operate all six DoFs without requiring as many input signals, the hand is typically controlled through pre-saved grip postures. A grip posture is a high-level setting that defines a specific RoM for each DoF and synchronizes their actuation across these ranges. This dimension reduction allows the hand to be controlled along a single dimension, representing the whole range of states from fully opened to fully closed. Then, the hand’s low-level components translate this one-dimensional goal into motor commands for each finger, so that they simultaneously reach the extreme values of their RoM, as defined by the posture.

Only one grip posture can be active at a time, among twenty-three different postures that can be stored into the hand’s embedded system (including five that are user-defined). The hand includes three buttons on its back, that can be programmed to change the hand’s current posture. Additionally, the hand offers Bluetooth connectivity, which allows it to be paired with the prosthesis’s main board thanks to a dedicated USB dongle. The motor commands sent through this Bluetooth connection include changes of grip posture as well as individual finger control.

So far, the Smart ArM platform’s has always been equipped with a commercial device as its end-effector. Admittedly, this makes this part of the prosthetic arm less “open” compared to a robotic hand made in-house, like the rest of the platform. However, this choice is motivated by our focus on elbow and wrist control rather than hand and finger control. In practice, the grip posture features make the hand readily usable and offer enough control options to perform a variety of manual tasks (see Section "Application scenarios").

Software

Operating system

The main board runs the Raspberry Pi OS (previously Raspbian), a lightweight Debian-based Linux distribution for embedded systems (see Appendix C "Operating system" for more details). This OS offers as much flexibility and features as a full-fledged OS for desktop computers, including support of keyboards and monitors, networking services and a large set of built-in Unix tools. Within this system, the program managing the prosthesis control takes the form of a single executable binary file, deployed by copying it from the development computer to the system’s internal memory.

No specific middleware is required as an interface between the embedded system and other computers, allowing researchers and developers to easily interact with the prosthesis, for example when deploying the control program, testing its behavior or monitoring its operation. Additionally, as the embedded system can have up to 512 GB of internal storage space, the main board’s computing power allows the control program to record logs of many variables at high speed for extended periods of time. This is useful when researchers need to investigate the prosthesis’s behavior in contexts where it cannot be monitored externally in real time.

Software stack

The prosthesis control program is structured as a four-layer stack (see Fig. 8).

Fig. 8

Four-layer structure of the embedded system’s software stack. The Web browser interface, shown in dotted lines, is described in Section "Wireless communication"

At the lowest level, the program includes the drivers through which it interacts with the prosthesis’s electronic devices, e.g. ADCs, IMU, addressable LEDs, motor controller. These drivers typically make use of existing Application Programming Interfaces (API) provided by manufacturers, and adapt their features to the prosthesis’s specific needs. This layer also incorporates all software dependencies that are not already included in the OS’s common files.

The second layer takes the form of a components manager, centralizing access to all the components of the prosthesis. This layer carries out the initialization of each component when the program is starting up, and groups them in three categories: sensors, joints and user interface elements.

The third layer consists of sub-programs implementing control laws for the prosthesis. All control sub-programs are built from a generic template, consisting of a control loop, a setup step and a clean-up step. These two steps are routines that are automatically called before the control loop starts and after it stops, respectively. The implementation of a control sub-program also includes a list of required components. For obvious safety reasons, only one control sub-program can be active at once. The active control loop runs in a dedicated thread, at a frequency that is specific to each sub-program. The various components of the prosthesis are accessed through the components manager, whereas connections with external devices can be created and managed from this layer.

The fourth layer, which is optional, consists in a menu-based command-line interface, through which users can activate control sub-programs, change settings on certain components, run tests and perform debugging operations (see Appendix E "Menu interface" for more details).

Following this structure, the control program is at the core of all interactions between the prosthesis’s components. In particular, instead of each actuation unit incorporating its own interface with the prosthesis’s sensors, the program is in charge of receiving and processing all relevant input signals, then computing the corresponding motor commands for each actuator. Thanks to the program acting as intermediary agent, researchers can customize any settings involved in myoelectric control laws, such as filtering methods, thresholds, proportional gains and contraction-based triggers.

Additionally, this software structure was designed to offer developers high levels of modularity and versatility, with the aim of deploying the same program on prostheses equipped with different components. To achieve this, the first layer includes drivers for all the devices compatible with the program, even though they cannot all be available at once (e.g. multiple prosthetic hands). Using these drivers, the components manager carries out automatic device detection at runtime, during the program’s initialization phase. Then, the sub-programs that depend on missing hardware are filtered out, thus restricting the available control laws to those compatible with the current state of the prosthesis. As a result, the combination of the electronic and software components described above forms a base architecture that can be easily adapted to operate devices with various amounts and types of actuators. To this day, this architecture was successfully deployed on six experimental devices, operating one to four DoFs from shoulder to wrist, and compatible with five different models of prosthetic hands.

Wireless communication

After booting up, the embedded system is configured to automatically generate its own password-protected Wireless Local Area Network (WLAN), to which external devices can connect to. Then, wireless communication can be achieved using several methods. Firstly, developers can log into the prosthesis’s OS from a remote computer using the Secure Shell (SSH) protocol. This method makes testing and debugging easier, in that it gives access to the embedded system without disassembling the prosthesis’s shell. In particular, when the control program is called from a terminal window through SSH, the menu interface is displayed on the remote computer. In this way, developers can remotely navigate through menu items, activate sub-programs and monitor their operation with their textual outputs. Additionally, this method can be employed to retrieve log files stored on the memory card, or modify the embedded system’s configuration e.g. permissions, booting sequence, network settings.

A second method relies on the User Datagram Protocol (UDP) to transfer data over the wireless network, from an external device to the prosthesis control program. This one-way communication method is typically used to transmit measurement data from external sensors that cannot be directly connected to the prosthesis’s embedded system, such as an OptiTrack motion tracking setup (from Natural Point). For two-way communication, the program incorporates a messaging service based on the MQTT² protocol, which allows connections between many clients over the wireless network (see Appendix F "Wireless communication" for more details).

Based on this messaging service, a remote dashboard was created to provide real-time information about the prosthesis’s state and operation of the control program. This dashboard is based on a HyperText Markup Language (HTML) interface developed in JavaScript and capable of displaying dynamic elements on a Web browser. Notably, this interface includes a dynamic plot showing EMG measurements in real time from the electrodes (see Fig. 9), allowing researchers and developers to efficiently fine-tune myoelectric control and troubleshoot related issues. The dashboard can also display system-related data such as processor temperature, load and frequency.

Fig. 9

Screen capture of the dynamic HTML interface in operation

Using these means of communication, the embedded system can also interact with remote computers in order to “outsource” computing-intensive tasks. In such a setup, external devices equipped with high-performance hardware can take charge of calculations that would be too demanding for the prosthesis’s main board. As a result, at each cycle of a control loop, the computing load on the embedded system is limited to sending current sensor data, receiving corresponding output data from the remote computer and updating motor commands accordingly.

Interfaces with external devices

This section provides several examples of external devices with which the prosthesis has been interfaced so far, through either wired or wireless connections. A first category of such devices corresponds to measuring tools providing data about the wearer’s natural motion. For instance, the control program can receive motion tracking measurements generated by an OptiTrack setup (from Natural Point), using the data streaming tool NatNet to broadcast the measurements over the wireless network. Besides, several body-worm IMUs (models x-IMU and NGIMU from x-IO Technologies) can be interfaced with the prosthesis, using cables to connect them directly to the main board through its USB ports (see Fig. 2). Thanks to these interfaces, in addition to motion analysis, researchers can explore and test motion-based control laws i.e. laws that drive the prosthesis depending on its wearer’s motion. Indeed, signals measuring the wearer’s natural motion, such as limb orientation or joint rotation speed, can be employed to detect motion intentions and identify how the prosthesis should move in accordance with these intentions.

Additionally, the prosthesis’s control program can interact with a HoloLens headset (second-generation model, from Microsoft) in order to offer a richer user interface to the prosthesis’s wearer. In particular, the head-mounted display can be employed to provide continuous feedback regarding the prosthesis’s current state, such as joint orientations, grip postures or battery level. In this way, the displayed information is available even when the wearer is not able to retrieve it visually e.g. in case of visual occlusion or when the prosthesis is not in the field of view. Moreover, thanks to its head and hand tracking features, the HoloLens can act as a command device, for example by detecting specific hand motion e.g. pressing a virtual button. Then, it can send the corresponding command signals to the prosthesis, to trigger specific actions or behaviors.

Available control modes

In its current iteration, designed primarily for Cybathlon, the Smart ArM prosthesis provides two control modes to its wearer. First, it implements a conventional myoelectric control using the EMG measurements provided by the surface electrodes (see Section "Electrodes"). It involves six out of the eight available muscle sites, as independent sources of command signals. Each of these EMG channels is associated with a threshold, applied to the signal’s amplitude to trigger six different motor commands: elbow flexion and extension, wrist pronation and supination, hand closing and opening. A simple priority rule ensures that the wrist and elbow are not put in motion simultaneously: if two signals overcome their respective thresholds, the elbow is put in motion while the wrist remains inactive.

In parallel, the hand closing-opening is controlled by the corresponding signals, so that the end-effector can move while the elbow or wrist is active. Besides, these two signals are continuously fed to a state machine capable of detecting bursts of two or three short contractions with the same muscle site. When detected, these specific contraction patterns are interpreted as commands to change the hand’s grip posture (see Section "Prosthetic hand"). In this way, the wearer can trigger such actions without using the opposite valid hand to press a button.

Alternatively, the prosthesis provides a movement-based control called Compensation Cancellation Control (CCC) [32], in a single-DoF variant driving the wrist joint. This control law defines the wrist’s rotation speed as proportional to the forearm’s angular deviation about the joint’s axis, with respect to a reference posture. As a result, the prosthesis’s end-effector rotates in the same direction as the current compensatory motion performed by the wearer, without requiring targeted muscular contractions. In this context, CCC is employed in combination with conventional myoelectric control: body movements drive the wrist while EMG signals drive the elbow and hand. In addition to reducing the cognitive load, this approach also lightens the physical efforts required from the wearer to drive the prosthesis.

Experimental validation

This section reports an assessment of the Smart ArM platform’s performance in its current iteration, with respect to joint kinematics and dynamics (Table 1), as well as electric power consumption (Table 2).

Table 1 Experimentally measured maximum torques, rotation speeds and current consumption under various loads. For the wrist, the load was placed at a 20 cm distance from the rotator’s axis

Table 2 Breakdown of electric consumption between components (electronics and actuators)

In its current state, the Smart ArM prosthesis is not suitable for daily use outside of the laboratory, mostly due to the limited battery life. In spite of this limitation, the device’s motor performance is sufficient for achieving a wide range of ADLs. For instance, tests carried out with a bottle crate showed that the wearer can lift and move a 5 kg load with the prosthesis. Additionally, thanks to the elbow’s non-backdriveable transmission, the prosthesis can be employed to carry objects with both arms, without requiring any power consumption from the motors. Table 3 compares its main mechanical characteristics with those of existing trans-humeral prostheses.

Table 3 Comparison with existing trans-humeral prosthetic devices

Application scenarios

This section describes several of the applications scenarios in which the Smart ArM prosthesis was successfully employed by researchers and differently-limbed persons as an experimental platform. Unlike the previous section, the purpose of this section is not to report a quantitative assessment of the platform in terms of performance. Rather, it serves to illustrate how the modularity and versatility of the platform’s base architecture can be of use for researchers.

Cybathlon

The application for which the Smart ArM prosthesis was primarily developed is Cybathlon, an international competition where impaired persons race against each other using powered prostheses and active assistive devices. This event is organized by the Federal Institute of Technology in Zürich (ETH Zürich, Switzerland), and aimed at putting prosthetic devices to the test in the framework of tasks inspired by ADLs (see Fig. 10). Several categories are represented in this event, each corresponding to a different type of prosthetic device. Since 2020, our team has been competing in the “Arm Prosthesis” (ARM) race, a category dedicated to persons with an arm impairment above the wrist. In this category, competitors are required to use their prosthesis to perform various tasks involving both strength (e.g. carry a kitchen pan, use a hammer) and dexterity (e.g. open a bottle, insert a key in a hole).

Fig. 10

Examples of tasks in Cybathlon’s ARM race. A Carrying bottles (weights ranging from 100 g to 1.6 kg). B Stacking cups (photograph ©Pierre Kitmacher, Sorbonne Université). C “Clean Sweep” i.e. small object manipulation

During the competition, our team’s pilot, a person with right forearm agenesia, wears the Smart ArM platform as a prototype prosthesis and employs it to perform tasks along the race’s course. After its creation in 2019, our team competed for the first time in the 2020 edition of Cybathlon, on a race including six different tasks. Our team then took part in three Cybathlon Challenges (2022, 2023 and 2024), which are smaller-scale events where the race’s course includes only a subset of all the tasks planned for the 2024 edition. Videos of our team’s runs during previous competitions are available at [40].

Although built around everyday activities rather than experimental tasks, these races put the Smart ArM prosthesis to the test in terms of motor abilities. For example, the bottle carrying task demonstrates that the elbow can bear loads of up to 5 kg, whereas the clothing task demonstrates the elbow’s wide RoM from full extension to maximum flexion when pulling a sweater’s zipper up. In a broader sense, these races involve the same challenges that an arm prosthesis wearer person would face in everyday life. In this respect, the Smart ArM platform compares to commercial prosthetic arms, given that it allows our pilot to perform these tasks at a speed similar to that of other participants equipped with commercial devices. For instance, during the Cybathlon Challenge 2022, our pilot won the race against the second-place contestant equipped with an OttoBock prosthesis (DynamicArm elbow 12K100, electric wrist rotator 10S17 and SensorHand Speed) on a small object manipulation task.

Since the team’s formation in 2019, the prosthesis has received many improvements with respect to motor performance, myoelectric commands and hand control. These improvements are driven by the feedback from our pilot during our training sessions, where we test the prosthesis’s features and explore strategies to achieve the race’s tasks more efficiently. In this context, we take advantage of the prosthesis’s numerous fine-tuning options to customize its features and behavior for our pilot and his specific needs.

Besides, rather than focusing exclusively on pure performance (i.e. race completion time), one of our primary goals is to improve the prosthesis so that it enables elegant and human-like movements. We put this commitment into practice in the strategies we elaborate to perform tasks during the race, by favoring as much as possible arm gestures and body postures similar to what an unimpaired person would do. In particular, this means employing strategies that focus on prosthesis use to the greatest possible extent, instead of relying extensively on compensatory motion to complete tasks faster. This is most notable on trunk and shoulder motion, when for instance certain pilots change their prosthetic hand’s orientation by tilting their torso laterally by 30$^\circ$ while performing lateral arm rotation (see Fig. 5C in [41] and video clip starting at 1:44:47 in [42]). Even though such strategies may pay off in a competitive setting, excessive compensatory motion is known to lead to musculo-skeletal disorders [43, 44]. Additionally, it seems to us that resorting extensively to compensatory motion instead of prosthetic joint motion somewhat defeats the purpose of a competition aimed at putting prosthetic devices to the test.

With this aim, the prosthesis incorporates control features based on innovations and experimental works published by our research group, such as the CCC described in Section "Available control modes". During Cybathlon events, our pilot can use the control panel on the wrist to switch between myoelectric and motion-based control to drive the prosthesis’s wrist. Meanwhile, the elbow and hand constantly remain operated through myoelectric control.

With respect to fully myoelectric control, this hybrid approach greatly reduces the physical and cognitive load induced by the operation of the prosthesis. It also plays a key role in our strategies to make up for our pilot’s higher level of impairment, compared to most pilots in the ARM race. Indeed, the inclusion criterion in this category allows pilots with a functional elbow to compete against trans-humeral prosthesis wearers. As a consequence, most teams participate with a 2-DoF trans-radial prosthesis, some pilots going as far as using their opposite, functional hand to rotate the prosthetic wrist or thumb faster.

On the Smart ArM prosthesis, CCC and conventional myoelectric control are employed in a complementary fashion to efficiently drive the three DoFs without excessively burdening the pilot. In this context, our participation to Cybathlon is an opportunity to showcase how our latest progress in prosthesis control can benefit differently-limbed persons by providing intuitive control and restoring movements closer to natural arm motion.

Experimental works

In addition to its use as our team’s prototype device for Cybathlon, the Smart ArM prosthesis serves as an experimental platform for research studies carried out in our lab. Indeed, thanks to the reprogrammable nature of the embedded system, researchers working on motor or sensory features for arm prostheses can deploy and test them on the Smart ArM platform, an actual wearable device. This section provides six examples of such studies, addressing three different research themes, where the Smart ArM prosthesis was employed as part of the experimental apparatus. As this section focuses on the platform’s ability to interface with other systems within an experimental setup, the results from these studies are not reported here and the corresponding protocols are not detailed.

Evaluating and comparing prosthesis control schemes

The Smart ArM platform was employed as an experimental test-bed in three studies introducing and evaluating novel control schemes for arm prostheses. In a first study [45], an early prototype of the Smart ArM prosthesis, built on the same architecture, was interfaced with an external 24-channel EMG acquisition system, different from the association of ADCs and socket-mounted electrodes employed on the platform’s current iteration. Instead of being worn, the prosthesis was fixed to a stand and placed next to the participant’s affected side (see Fig. 11). The acquisition system retrieved signals from 12 pairs of surface electrodes placed on the residual limb of two participants with upper limb loss at trans-humeral level. From these signals, a Linear Discriminant Analysis (LDA) classifier was trained to predict voluntary phantom limb motion, that is to say: the participant’s intention to move their missing limb as if it was still present. Due to remaining cerebral networks and neural pathways previously involved in the missing limb’s control, voluntary phantom limb motion induces contractions of residual muscles [46], even though no actual movement is performed.

Fig. 11

Early prototype of the Smart Arm prosthesis driven with phantom limb motion [45]

For each participant, a data acquisition phase was carried out, where the residual limb’s EMG activity was recorded while the participant “performed” eight different arm and hand movements with their phantom limb. The classifier was trained on these data to predict the intended phantom limb motion from EMG signals. Then, during a control phase, the prosthetic arm was put in motion according to the LDA classifier’s live prediction based on the residual limb’s muscular activity. Under this control scheme, the participant drove the prosthesis to perform a grasp-and-release task with three different objects: a ball, a cylinder and a clothespin.

More recently, the Smart ArM platform played the role of test-bed for experimental comparisons of movement-based control schemes with conventional myoelectric control. One such experiment [47] focused on elbow control exclusively, and introduced a control scheme where the prosthetic elbow’s angular velocity is determined from the angular velocities of the residual limb, following a model of inter-joint coordinations. The baseline condition corresponded to a conventional myoelectric control where the elbow was operated with two EMG signals from the biceps and triceps. The experimental task consisted in reaching targets with the prosthesis’s end-effector, by moving the prosthetic elbow as well as the residual limb and the trunk. Six persons with upper limb loss at trans-humeral level (three with osseo-integrated implant) performed the experiment with the early Smart ArM prototype (see Fig. 12).

Fig. 12

Early prototype of the Smart Arm prosthesis driven with a movement-based control while fitted to a conventional trans-humeral socket (A) or an osseo-integrated implant (B) [47]

Another study [32] compared CCC with conventional myoelectric control in the framework of a clothespin placement task. Three DoFs were involved: the two revolute joints actuating elbow flexion-extension and wrist prono-supination, and the prosthetic hand operating finger-thumb pinching to grab the clothespins (the other fingers remained closed). The CCC scheme allowed for simultaneous control of both the wrist and elbow joints based on trunk orientation, whereas the myoelectric scheme relied on sequential (joint-by-joint) control based on biceps and triceps contractions. With this scheme, toggling from one joint to the other required participants to perform a co-contraction with both muscles. For both control schemes, the hand opening and closing motions were controlled using two push-buttons with the contra-lateral hand. Four differently-limbed persons performed the Refined Rolyan Clothespin Test, with each of the two control schemes. One of the participants performed the experiment with the current iteration of the Smart ArM prosthesis fitted to his right side (see Fig. 13) whereas the others used the platform’s early prototype.

Fig. 13

Smart Arm prosthesis fitted with an i-Limb Quantum hand and driven with 2-DoF CCC (wrist and elbow) [32]

Evaluating artificial sensory feedback devices

The Smart ArM prosthesis was also involved in two experiments evaluating how haptic stimuli can convey proprioceptive feedback to arm prosthesis wearers. In the first experiment [48], twenty-three unimpaired participants controlled a prosthetic wrist with contractions of their right arm’s wrist flexor and extensor muscles. In this setup, the apparatus acting as the prosthetic arm was one of the experimental devices based on the same architecture as the Smart ArM prosthesis. Additionally, participants wore on their right forearm a bracelet fitted with six vibrators, evenly spaced by a 30$^\circ$ angle (see Fig. 14). They were instructed to use myoelectric control to bring the prosthetic wrist to one of six target orientations, spaced similarly as the vibrators across the joint’s RoM: from − 90$^\circ$ (pronation) to 90$^\circ$ (supination). While the task was performed, a stimulus was generated by the vibrator corresponding to the angle closest to the wrist’s current orientation, thus providing proprioceptive feedback.

Fig. 14

Experimental setup for testing vibro-tactile proprioceptive feedback with an early version of the Smart Arm platform. [48]

The second experiment [49] focused on proprioceptive feedback regarding a prosthetic hand instead of a prosthetic wrist. For this purpose, a left Taska hand was modified to add force sensors on the tips of the index and thumb, as well as a rotary encoder measuring the angular position of the index about its metacarpo-phalangeal rotation axis (see Fig. 15). Additionally, hand movements were restricted to opening and closing motions with the index and thumb, while other fingers remained closed. The haptic device consisted in an armband fitted with three vibrators and a skin-stretch apparatus, made of a spherical silicone tactor put in motion linearly. This device performed sensory substitution with respect to two types of sensory information. Firstly, the vibrators provided contact feedback by producing vibro-tactile stimuli based on the signals from the fingertips’ force sensors. Secondly, the skin-stretch apparatus provided proprioceptive feedback by displacing the tactor in correlation with the index’s angular position.

Fig. 15

Taska hand equipped with force sensors and rotary encoder [49]

One of the participants was a differently-limbed person, fitted with the Smart ArM prosthesis to his right side and wearing the haptic device on his left forearm. This participant was already familiar enough with myoelectric control to operate the hand opening and closing motions with biceps and triceps contractions. The other eight participants were unimpaired persons, wearing the haptic device on their right forearm. In lieu of the prosthesis, they controlled a custom hand-held device, based on the same architecture as the Smart ArM prosthesis and equipped with the Taska Hand at its end. Two push-buttons allowed them to operate the prosthetic hand opening and closing. The experimental protocol relied on two tasks. The first was a recognition task, where participants were blindfolded and instructed to distinguish different spheres in terms of size and stiffness, based on the feedback provided by the haptic device when the prosthetic hand closes its fingers on the objects. The second task was a pick-and-place task, where participants used the prosthetic hand to displace marbles from a support to a target container.

Providing visual feedback during learning of motor control

Lastly, the Smart ArM platform was employed to provide visual feedback to unimpaired participants learning to operate artificial joints with a myoelectric control scheme based on pattern recognition. In this experiment [50], instead of being worn, the prosthesis was placed on a support to the right of the participants, within their field of view. Myoelectric signals were recorded using a Myo armband worn on the participants’ right forearm (see Fig. 16). With this setup, a classifier was trained to recognize seven movements from the recorded EMG activity: wrist pronation and supination, hand closing and opening, index-and-thumb pinch closing and opening, and index pointing. A remote computer took charge of signal processing and pattern recognition, then sent the corresponding motor commands to the prosthesis through its wireless network.

Fig. 16

Experimental setup for evaluating a pattern-recognition-based control of a prosthetic hand and wrist [50]

Two different devices acted as the prosthetic arm in this experiment, with no difference with respect to the protocol, except for practical concerns. The first device was the current iteration of the Smart ArM prosthesis, and the second was a demonstration device including only a wrist rotator and the Taska hand with respect to actuators. In both cases, the device was not involved in any physical interaction with participants nor objects. Its role was to convey realistic visual feedback of the prosthesis motion triggered by the participants’ myoelectric commands, instead of the classifier’s prediction being illustrated with a video clip or picture on a computer monitor.

Discussion

Main advantages

The main novelty of the Smart ArM platform is the considerable freedom it offers in terms of customization and fine-tuning of prosthesis behavior. In particular, given that the control program acts as sole intermediary agent between the sensors and actuators, control laws are not restricted to pre-defined algorithms or settings. As a result, researchers can implement any formula or algorithm to translate the sensor measurements, such as EMG signals, into motor commands to the prosthesis’s actuators.

Additionally, thanks to its modular architecture and connectivity options, it can be interfaced with a wide variety of external devices, including measuring instruments. In this way, the Smart ArM platform can be employed in experimental setups involving multiple devices communicating with each other. In particular, it allows researchers to explore how alternative command signals, such as motion tracking data and inertial measurements, may be employed to improve prosthesis control (see [51,52,53] for examples of such approaches). Besides, the architecture underlying the Smart ArM platform is flexible and modular in order to make it easier to adapt it to the specific needs of its application cases.

Through the wireless network generated by the embedded board, developers can update the control program remotely, without disassembling the prosthesis or plugging a cable. Besides, the embedded system gives total access to the prosthesis’s internal state and processes, making it easier to monitor its operation and troubleshoot issues.

With respect to participant inclusion in experimental studies, the platform’s elbow actuation assembly was designed so that it can be fitted to any size of residual limb ending above the elbow. In particular, this includes the case of our team’s pilot, a person with right forearm agenesia whose residual limb comprises a long humerus. Section "Experimental works" reports research studies where the Smart ArM platform was adapted in order to be worn by different participants with upper limb impairment.

With respect to its motor performance, even though it may not be as powerful as commercial prosthetic arms, the Smart ArM platform proves to be capable enough for performing a variety of everyday tasks. In this regard, our team’s pilot employs it during Cybathlon’s ARM race to perform tasks requiring strength, such as carrying a frying pan or lifting a bottle crate.

Limitations and future developments

One of the main limitations of the Smart ArM prosthesis is the need for a trained developer or experimenter to be present when the device is operated. Indeed, even though the wearer can autonomously place the batteries, switch the prosthesis on and start the control program, the system’s operation needs to be monitored in order to prevent errors from causing injury or damage. In case of a failure in the device, trained personnel can take the appropriate actions to limit the damage done to the prosthesis (e.g. motors overloading, excessive strain on finger joints) and bring it back to a safe state before restarting it. In this way, the wearer is not exposed to risks related to taking off or restarting the prosthesis from an unsafe state. Such risks include excessive strain on the residual limb due to the prosthesis being stuck in an uncomfortable posture, as well as the unexpected release of an object held in the hand. Additionally, thanks to the wireless monitoring tools described in Section "Wireless communication", the need for a trained operator to be present does not prevent the platform from being employed outside of laboratory settings.

Moreover, the platform is notably limited by its battery life (approx. 2 h of continuous use). To improve this aspect on the next iteration, batteries will be relocated from the forearm to the socket’s suspension harness. By removing the need for these components to fit inside the forearm structure, this relocation will allow the use of larger-capacity models (i.e. over 2000 mAh), thus extending the prosthesis’s battery life by several hours.

Our experience as a team participating in Cybathlon events revealed two notable limitations affecting our pilot’s ability to improve his performance. Firstly, our pilot reported multiple times that the platform’s weight distribution increases his muscular fatigue over the course of a training session. Indeed, the prosthesis is denser near its distal end than its proximal end, due to the weight of the Taska hand itself, as well as the large amount of electronic components located in the distal half of the forearm. In this regard, relocating the batteries will help improve the weight distribution of the prosthesis. Indeed, in its current iteration, the space inside the platform’s forearm is completely occupied by electronic components (see Fig. 2). The space freed up by the battery relocation will make it possible to move the electronics closer to the proximal end of the forearm.

The second limitation that arose when preparing for Cybathlon is our pilot’s inability to perform certain tasks of the race course that require fine control of the end-effector’s orientation. For example, one of the tasks involves screwing a lightbulb inside of a cupboard with the prosthetic hand. In the confined space of the cupboard, the pilot cannot place the prosthesis’s forearm so that the hand is aligned with the lightbulb socket’s axis. This constraint makes it impossible to engage the lightbulb’s thread inside the socket. To overcome this limitation, future developments include the integration of a custom-made wrist with 2 active DoFs, replacing the current wrist rotator. The additional DoF will operate wrist flexion-extension, expanding the ability to change the orientation of the end-effector through prosthetic joint motion. This improvement will allow the pilot to correctly position the lightbulb in the socket’s axis. In a broader way, this will also decrease the need for trunk and shoulder compensatory motion when the wearer performs object picking tasks, thanks to the richer wrist mobility.

Conclusion

As a conclusion, the Smart ArM platform is a control-agnostic robotic arm prosthesis that can act as a versatile robotic test-bed for researchers to address the control issues specific to trans-humeral prostheses. Its reprogrammable embedded system allows researchers to design and test prosthesis control schemes that would be difficult to implement on a commercial prosthetic device. Researchers can also employ it to evaluate their innovations on a physical, wearable device in ecological environments (home or clinic). Its standard wrist connector makes it compatible with a variety of models of commercial prosthetic hands. Lastly, thanks to its multiple connectivity options, it can be interfaced with external devices in an experimental setup.

Appendices

Appendix A: Electrode suspension

Several types of elastic elements are employed to maintain the electrodes in position and in contact with the residual limb’s skin. Firstly, suspension legs (provided by the electrodes’ manufacturer, Steeper) are installed to keep the electrodes in place in their respective brackets. These legs are small pieces of stiff rubber that extend longitudinally on two sides of each electrode. When inserting the electrode into the bracket, the legs are fitted inside dedicated holes in the socket, thus securing it in place. For two electrodes, rectangular rubber housings (provided by the manufacturer) replace suspension legs as the flexible interface between the bracket and the electrode. Finally, elastic straps (custom-made from bicycle tire tube) were slipped over the socket and the electrodes’ rear faces.

The socket is fabricated so that the suspension legs or rubber housings allow electrodes to travel by a few millimeters radially. In this setup, the elastic parts push the electrodes inside, whereas the residual limb pushes them outside when it is inserted into the socket. As a result, the electrodes stay in contact with the residual limb’s skin regardless of the arm posture.

Appendix B: Elbow actuation

The elbow’s actuator is a 12 V brushed Direct Current (DC) motor (model DCX22L from Maxon) equipped with a 1024-impulsion incremental encoder (model ENX16 from Maxon) and a two-stage planetary gearhead (model GPX22 from Maxon) achieving a 1:21 reduction ratio. Its drive shaft is attached to a worm-and-wheel gear reduction unit, achieving a 1:25 reduction ratio. These multiple reduction stages provide a theoretical nominal torque of 11.61Nm at elbow level. The joint’s rotation is guided by a pair of linear annular joints, made of a ball bearing and an IGUS linear bearing that act as pivot joints located on each side of the socket.

Appendix C: Operating system

The prosthesis’s main board runs the Raspberry Pi OS, in association with a Linux kernel tailored for Raspberry Pi boards and configured to allow real-time execution of the prosthesis’s control program. To host the Linux filesystem, a microSD (Secure Digital) card is flashed with a disk image generated by Buildroot, a toolkit for building embedded Linux systems. The main board supports most high-capacity (up to 512 GB) memory cards following the SDHC and SDXC (Secure Digital High Capacity and Extreme Capacity, respectively) standards, which offer considerable storage space with respect to the requirements of the OS and control program.

The control program is developed in C++ and compiled specifically for the 64-bit architecture of the main board’s processor (model BCM2837B0 from Broadcom). The cross-compilation process relies on the toolchain provided by Buildroot, and Meson as the build system. Deploying the program after compilation is achieved by simply copying the corresponding executable binary file from the development computer to the prosthesis’s filesystem.

Appendix D: Compatible prosthetic hands

The Smart ArM platform can be combined with any commercial prosthetic hands compatible with the Quick Disconnect Wrist connector and the two-channel analog control of the opening-closing motion. Commercial devices that have been successfully interfaced with the Smart ArM platform include so far:

• i-Limb Quantum (from Össur)

• Taska hand (from Taska Prosthetics)

• SensorHand Speed (from OttoBock)

The Smart ArM platform is also compatible with the Robo-limb (from Touch Bionics, discontinued academic version of the i-Limb models from Össur) and the IH2 Azzurra (from Prensilia Robotics), two robotic hands controlled using a serial connection following the RS-232 standard. In this setup, the two communication rings on the coaxial plug work as the data transmission and reception channels. To achieve this, these rings are rewired to the corresponding pins on the main board’s serial port, instead of the DACs’ outputs.

Appendix E: Menu interface

The control program provides a menu-based command-line interface designed for terminal window display. The menus are organized following a tree structure, where the root is the main menu and the branches are menu items, which can be either sub-menus or commands. Dedicated sub-menus are automatically added for each control sub-program and each actuator. This layer is optional, in the sense that the program can run even if it is not bound to a terminal window managing the textual inputs and outputs underlying the menu interface. In particular, a sub-program can be activated and a control loop can be started while the program runs in the background.

Appendix F: Wireless communication

Under the UDP protocol, control sub-programs on the prosthesis side can create sockets bound to specific addresses, and continuously listen on them. Then, applications running on external devices connected to the prosthesis’s network can send data to these addresses, in the form of datagrams. This lightweight method requires no additional authentication (devices authenticated on the network are trusted by default) and is appropriate for transmitting signals from measurement devices, as well as short messages. For the sake of compactness, such messages can follow an application-specific code, and be decoded after reception by the corresponding control sub-program. Besides, it is possible for multiple devices to send data to the same address using this method.

Under the MQTT protocol, messages are exchanged following a publish-subscribe system. A client emits a message by publishing it under a specific category called “topic”. Then, every client that is subscribed to this topic receives the message. Given that each topic can support multiple publisher clients, this systems allows many-to-many message distribution without the number of connections growing exponentially. Additionally, the topics follow a tree structure, providing an easy way to hierarchically and thematically organize the transmitted data.