Bionic Arm & Hand Control Systems

Some believe that the most important parts of a bionic arm/hand are its electromechanical components. This is not true. The most important component is the user control system.

Overview

How users control a bionic arm/hand is not a simple subject. It involves technology components, some of which are optional, as well as the basic question of how a patient controls both his residual limb and the socket interface, which we call “implicit control”. Finally, it must ultimately include some form of sensory feedback.

Technology Control Systems

Current technology control options include the following:

• Myoelectric Direct Control
• Myoelectric Pattern Recognition
• Pattern Recognition Using Surgically Implanted Myoelectric Sensors
• Magnetomicrometry (MM)
• Mechanomyogram (MMG) Control
• Neural Interfaces

Myoelectric Direct Control

Most modern bionic hands use myoelectric direct control systems.

This means that either one or two myoelectric sensors are built into an arm socket so that they lie against the skin of the residual limb. Their job is to detect muscle movements intended to:

• open the bionic hand;
• close the bionic hand;
• where two sensors are used, they can also detect a co-contraction of muscles in both areas.

Source: https://link.springer.com/article/10.1007/s40137-013-0044-8/figures/2

Bionic hands usually offer multiple grip patterns, i.e. different combinations of fingers and the thumb open and close depending on the selected grip. The co-contraction signal (or possibly a double-open or double-close signal) is often used to rotate between the available grips.

This type of system is described as “direct control” because the user must explicitly contract specific muscles to control grip selection and the open & close actions. It is not ideal for controlling above-the-elbow bionic arms because it does not generate enough distinctive signals to manage the hand, wrist, elbow, and shoulder joints, if applicable.

• the user has to demonstrate specific muscle control;
• the shape and skin condition of the residual limb must be conducive to detecting myoelectric signals;
• training, patience, and discipline;

• less expensive than more advanced solutions;

• rotating through grips can be slow and cumbersome;
• having to exercise such explicit control over one’s hand can be tiring, especially when every action must be guided visually;
• the size and shape of the residual limb may change as the temperature changes, causing the myoelectric sensors to shift or lose contact with the skin; sweat can also interfere with the sensors; in either case, the hand will become more difficult to control;
• the system doesn’t generate enough distinct signals to efficiently control a bionic wrist, elbow, or shoulder in addition to a hand.

Sometimes, a patient doesn’t have sufficient musculature or physical space on a residual limb to accommodate the required myoelectric sensors. In this case, nerves can be surgically reassigned to alternate muscles using Targeted Muscle Reinnervation (TMR).

If there is sufficient space/musculature, but the muscles are too weak to generate reliable myoelectric signals, this situation can be improved with a surgical procedure called Regenerative Peripheral Nerve Interface (RPNI), though this is primarily intended to reduce limb pain.

• see Finding the Right Myoelectric Control System.

Myoelectric Pattern Recognition

Newer versions of bionic arms/hands now offer myoelectric pattern recognition as an alternative to direct control. In this model, multiple sensors are used, typically between three and eight (or sometimes even more).

BrainRobotics Myoelectric Pattern Recognition Sensors

With so many sensors, patterns of muscle movement can be detected instead of just single muscle movements. Each pattern can be mapped to a specific action. For example, one pattern can be mapped to use Grip 1, another to Grip 2, another to move an elbow joint, and so on.

Using a bionic hand as an example, the mapping process occurs as follows:

• the user is asked to perform an action with his phantom hand, such as picking up a pencil lying flat on a desk;
• this action would normally require a tripod grip involving the thumb, forefinger, and middle finger;
• the user’s attempt to do this with his phantom hand triggers a pattern of muscle movements in the residual limb;
• this pattern is mapped to the required actions in the bionic hand.

The resulting control is far more intuitive because the user doesn’t have to explicitly flex individual muscles. He just thinks about moving his phantom arm/hand exactly as if it were still present and the bionic version will mirror his intent.

It doesn’t even matter which patterns of muscle movement are mapped to specific actions as long as the patterns can be differentiated from each other and are reasonably consistent. We say “reasonably” because artificial intelligence (AI) is used to identify patterns and can adjust for minor variations.

This same mapping process also applies to movements of bionic wrists, elbows, and shoulders, if present. Some systems can even move multiple components simultaneously or be used to control individual digits.

• the user must have sufficient physical space on the residual limb to accommodate more sensors (in the case of shoulder disarticulation, upper torso sites are used instead);
• the shape and skin condition at the sensor location(s) must be conducive to detecting myoelectric signals;
• users must be comfortable enough with technology to interact with software training programs;
• training, patience, and discipline;

• allows more intuitive control over a bionic hand and other bionic arm joints;
• makes using the bionic arm/hand more efficient and less tiring;

• more expensive than direct control myoelectric solutions;
• where sensors are placed on a residual limb, the size and shape of that limb may change as the temperature changes, causing the sensors to shift or lose contact with the skin; sweat can also interfere with the sensors; in either case, the arm/hand will become more difficult to control;

As with Myoelectric Direct Control, the two surgical procedures that can be used to enhance Myoelectric Pattern Recognition are Targeted Muscle Reinnervation (TMR) and Regenerative Peripheral Nerve Interface (RPNI), though the latter is primarily intended to reduce limb pain.

• see Myoelectric Pattern Recognition for Bionic Arms & Hands.

Pattern Recognition Using Surgically Implanted Myoelectric Sensors

Given the problems with maintaining optimal contact between the skin’s surface and myoelectric sensors, it was only a matter of time before someone decided to implant the sensors directly into muscle tissue:

DARPA Implanted Myoelectric Sensors

The number of sensors implanted and the basic control system mirrors that of a skin-surface pattern recognition system with two important differences. Because the sensors are embedded in the muscle tissue, they can pick up deeper muscle movements in addition to those near the skin’s surface. Also, the signals are clearer and more reliable. This makes it easier to detect the simultaneous movements of multiple joints, which can then be used to relay equivalent commands to bionic components. Put another way, implanted myoelectric sensors are better able to support more advanced control systems, though improvements in surface sensors embedded in flexible materials may one day challenge this claim.

• the user must have sufficient musculature on the residual limb to accommodate implanted sensors (in the case of shoulder disarticulation, upper torso sites are used instead);
• users must be able to tolerate surgery;
• users must be comfortable enough with technology to interact with software training programs;
• training, patience, and discipline;

• allows more intuitive control over a bionic hand and other bionic arm joints;
• makes using the bionic arm/hand more efficient and less tiring;
• eliminates the frustration of periodic contact problems between myoelectric sensors and the skin;

• more expensive than skin-surface pattern recognition systems and a lot more expensive than myoelectric direct control;
• with any surgery, there is always the risk of pain, infection, and scarring.

This approach is often used in combination with Targeted Muscle Reinnervation (TMR). Since the patient is undergoing surgery anyway, it makes sense to embed the sensors.

• see Implantable Myoelectric Sensors.

Magnetomicrometry (MM)

Magnetomicrometry (MM) involves implanting a pair of magnetic beads into each muscle to be monitored:

When the muscle moves, the beads move with it. Magnetic sensors outside the limb track the beads. This information is passed to a control system that determines the length and speed of the muscle movement. The system then issues commands to the bionic limb to mirror this movement. This all happens so quickly that the user does not notice any delay.

The speed and precision of Magnetomicrometry promise a much greater level of control over bionic limbs than anything we’ve seen to date.

• the user must have sufficient musculature on the residual limb to accommodate the implanted beads;

• will allow more intuitive control over a bionic hand and other bionic arm joints than any other system;
• may even allow complete control over individual finger movements;
• eliminates the frustration of periodic contact problems between skin-surface myoelectric sensors and the skin because magnetic fields from the beads can be detected through tissue, clothing, and even sockets;

• currently in animal trials, which means it is still a few years away from human trials and likely five years away from commercialization;

The primary surgical technique that can be used to enhance Magnetomicrometry is to create an Agonist-antagonist Myoneural Interface (AMI).

• see Magnetomicrometry (MM) for Bionic Limb Control.

Mechanomyogram (MMG) Control

Mechanomyogram (MMG) sensors monitor the mechanical signals generated by a muscle when it contracts instead of the electrical signals detected by EMG sensors.

To be honest, we don’t know a great deal about this technology because, to our knowledge, it is only used for the control system of one commercial bionic limb: the Grippy Bionic Hand.

Here is what the CEO of Robo Bionics, the company that makes Grippy, had to say about his reasons for using MMG sensors:

“We wanted Grippy to be used equally by someone who doesn’t have a hand due to traumatic loss as well as someone without their hand from birth (congenital). In the case of the congenital cases, they were not able to achieve fine control of muscles as they had never learned it. The same issue was also faced by people who underwent traumatic amputations and were using body-powered, cosmetic, or no prosthesis for over 5 years after amputation. EMG for all these cases needed extensive training time and during the training sessions, the misfiring of the sensors would also demotivate the users. EMG readings were also affected due to humidity and heat which caused a lot of sweating in most places in India. We mitigated all those problems with the use of MMG and it brought our training period down to 15 to 20 minutes, even in trials with above elbow amputation cases.”

Llewellyn D’sa, Co-Founder & CEO, Robo Bionics

Based on our follow-up research, we did not find anything to contradict Mr. D’sa’s claims. So why isn’t MMG used in place of EMG for more devices? It appears as if research on EMG simply advanced more quickly than for MMG, at least for prosthetic control. Thus, there may be more competition between these technologies in the years ahead. If we see another bionic limb adopt MMG, we’ll expand our coverage of it.

Neural Interfaces

Neural interfaces have two potential roles with bionic limbs:

1. They are currently one of the most promising methods of conveying sensory feedback from a bionic device to the nervous system.
2. They have also been used experimentally to intercept signals sent to motor nerves as a more direct method of determining user intent.

Two-Way Neural Interface

Since this article is on control systems, we’re going to focus on the second use. Because we are not research scientists or doctors, we cannot categorically state that the idea of determining a user’s intended movements by intercepting motor signals is dead. In fact, research in this area likely continues. But we can say that every bionic arm/hand device that we’ve examined uses some form of a myoelectric or mechanomyogram sensor system, not a neural interface, to determine user intent. And that’s not just current devices — that’s everything coming to market in the foreseeable future!

So, we’re not going to cover this as a serious user control option at this time.

If you would like to know more about neural interfaces, please see Neural Interface Surgery and Neural Interfaces for Bionic Arms & Hands. Note, however, that these articles pertain to the sensory feedback aspects of neural interfaces, not motor control.

Implicit Control

The most obvious form of implicit control over bionic arms/hands is the socket interface between the residual limb and the device. We’re not going to cover this subject here, as that deserves an entire website on its own. However, we are going to cover a socket replacement — Osseointegration — plus a relatively new surgical procedure that may revolutionize key aspects of user control for all bionic limbs.

Osseointegration involves inserting a metal rod into an arm or leg bone. The bone then fuses with the rod in a manner similar to fracture healing:

Bionic devices are then attached to the metal rod. This method of attachment significantly improves user control over the attached device because it makes the device a direct extension of the user’s skeletal frame. This not only increases the range of motion for the bionic limb; it also provides a form of sensory feedback through vibrations passed from the bionic device to the bone.

A more recent version of this technology — Integrum’s e-Opra Implant System — integrates embedded myoelectric sensors and a neural interface (i.e. for sensory feedback) with the implant, making it capable of supporting the most advanced user control systems on the planet:

• patients must be in suitable health, which typically means patients who do not smoke, have diabetes or other vascular issues, and are not obese;
• patients must have reached full skeletal maturity, meaning they are at least 18 years of age but no more than 70;
• there are also some restrictions related to the use of certain drugs, such as chemotherapy;

• increased comfort, especially with the elimination of socket-related skin problems;
• easier to attach and detach a prosthesis;
• greater strength/stability;
• more intuitive use of the prosthesis including a broader range of motion;
• improved osseoperception;
• overall improved quality of life;

• it’s major surgery with a long recovery time;
• it involves significant costs, which may not be covered by insurance;
• the implant presents a serious risk of deep infection (up to 41 %);
• bone fractures occur in up to 9 % of the cases;
• implant parts break in up to 31 % of cases;
• the implants end up being removed in up to 20 % of the cases.

• see Osseointegration for Bionic Limbs.

AMI is a technology that revolves primarily around surgically restoring the agonist-antagonist muscle pairings that control our joints. Basically, severed muscles in the residual limb are reconnected to form such a pairing:

This restores the brain’s natural sense of position and movement for the missing limb. This is called “proprioception”.

A bionic joint is then calibrated so that its movements correspond to the movements of the restored muscle pair, and, voila, the brain can better control the bionic limb with thoughts and also sense its position in space.

In terms of controlling a bionic limb, a myoelectric system is still used at this time, albeit one with a much better connection between the brain and remaining muscles in the residual limb. Also, there is less muscle atrophy when an agonist-antagonist pairing is restored, so muscle movements are stronger and therefore generate stronger myoelectric signals.

However, it should be noted Magnetmicrometry (MM) is likely to be the control system of choice for AMI patients once MM is commercialized. That is expected to occur sometime around 2027.

• the patient must be able to withstand surgery;
• AMI must still be approved for use with upper limbs, though we have received word that at least one upper-limb patient has successfully undergone the procedure;

• will restore the brain’s proprioceptive sense of the position and movement of a bionic arm or hand;
• will significantly improve user control over larger movements, though control over finer hand movements may be more difficult to achieve solely through AMI without sensory feedback;
• will strengthen the muscles used in pairings;

• involves surgery with all its inherent costs and risks.

• see Agonist-antagonist Myoneural Interface (AMI)

Sensory Feedback

Technically, sensory feedback is not part of a bionic limb’s control system in that it does not directly trigger bionic actions. It is used to inform the user of actions that have taken place, which may then necessitate additional actions.

Yet, sensory feedback may be more crucial to effective user control over bionic hands than any other single factor.

Currently, the following forms of sensory feedback are either available or soon will be, going from the least sophisticated to the most:

Osseoperception

Vibrations pass from a bionic hand into the bone of a residual limb through an osseointegrated implant. This is known as osseoperception and can be surprisingly informative.

For more information, see Osseointegration for Bionic Limbs.

Simple Haptic Feedback

Sensors built into bionic hands pass feedback to vibrators built into the prosthetic socket, which use different levels and frequencies of vibration to convey information. Examples of this include the use of pressure sensors to convey contact with an object (a single vibration) or grip force (repeated vibrations, possibly with increased intensity).

Transcutaneous Electrical Nerve Stimulation (TENS)

TENS feedback systems attempt to stimulate nerves with an electrical charge through the skin and intervening tissue. This can cause the brain to experience certain sensations in the missing hand that abstractly equate to sensory feedback from the bionic hand, such as a tingling sensation in the missing hand when the bionic hand makes contact with an object. This can give the user a greater sense of embodiment than the use of simple haptic feedback, but the information that can be communicated in this manner is limited.

Targeted Sensory Reinnervation (TSR)

TSR is a method where the skin over a target muscle is denervated, then reinnervated with nerve fibers from an amputated limb. When this piece of skin is touched, it provides the amputee with the sensation that the missing limb is being touched.

For example, a patch of skin on the forearm can be reinnervated with nerves from an amputated hand such that stimulating that patch of skin will feel as if the hand is being touched. Because the reinnervated skin is receptive to a near-normal range of stimuli, information can be conveyed from the bionic hand to the skin via any number of stimulation methods, including pressure, temperature, etc.

This type of system can communicate a significant amount of information, including which digit is being touched.

For more information, see Targeted Sensory Reinnervation (TSR).

Neural Interface

We are describing the sensory feedback version of neural interfaces here, not interaction with motor nerves.

Electrodes are surgically implanted to wrap around or transect sensory nerves. The electrodes then stimulate the nerves with an electrical charge that tricks the brain into experiencing whatever the sensors in a bionic hand are experiencing (or some loose translation thereof).

Because this is a direct interface with the nerve, it has the most potential for communicating sophisticated information. It also has several drawbacks, including the risk of infection, scarring, etc.

For more information on this type of neural interface, see Neural Interface Surgery and Neural Interfaces for Bionic Arms & Hands.

For more information on sensory feedback in general, see Sensory Feedback for Bionic Hands and also Understanding Bionic Touch.

For a comprehensive description of all current upper-limb technologies, devices, and research, see our complete guide.