Bionic Arm & Hand Control Systems

Bionic Arm Control System

When looking at a bionic arm, many people believe that the hand and the arm joints are the primary technology components. This is not true. The most important component is the user control system.

Overview

How users control a bionic arm/hand is not a simple subject. It involves technology components, some of which are optional, as well as the basic question of how a patient controls his residual limb (i.e. implicit control), which may or may not augmented by technology.

In this article, we break this subject down into the following sections:

Technology Control Systems

Current technology control options include the following:

Myoelectric Direct Control

Most of the bionic hands in use today utilize myoelectric direct control systems.

This means that either one or two myoelectric sensors are built into an arm socket so that they lie against the skin of the residual limb. Their job is to detect muscle movements intended to:

  • open the bionic hand;
  • close the bionic hand;
  • where two sensors are used, they can also detect a co-contraction of muscles in both areas.
Bionic Arm Myoelectric Direct Control System

Source: https://link.springer.com/article/10.1007/s40137-013-0044-8/figures/2

Bionic hands usually offer multiple grip patterns, i.e. different combinations of fingers and the thumb open and close depending on the selected grip. The co-contraction signal (or possibly a double-open or double-close signal) is often used to rotate between the available grips.

This type of system is described as “direct control” because the user must explicitly contract specific muscles to directly control grip selection and the open & close actions. It is not ideal for controlling above-the-elbow bionic arms because it does not generate enough distinctive signals to manage the hand, wrist, elbow, and shoulder joints, if applicable.

Prerequisites

  • the user has to demonstrate specific muscle control;
  • the shape and skin condition of the residual limb must be conducive to detecting myoelectric signals;
  • training, patience, and discipline;

Advantages

  • simplicity;
  • less expensive than more advanced solutions;

Disadvantages

  • rotating through grips can be slow and cumbersome;
  • having to exercise such explicit control over one’s hand can be tiring, especially when every action must be guided visually;
  • the size and shape of the residual limb may change as the temperature changes, causing the myoelectric sensors to shift or lose contact with the skin; sweat can also interfere with the sensors; in either case, the hand will become more difficult to control;
  • the system doesn’t generate enough distinct signals to efficiently control a bionic wrist, elbow, or shoulder in addition to a hand.

Surgical Techniques That Can Enhance This Technology

Sometimes, a patient doesn’t have sufficient musculature or physical space on a residual limb to accommodate the required myoelectric sensors. In this case, nerves can be surgically reassigned to alternate muscles using Targeted Muscle Reinnervation (TMR).

If there is sufficient space/musculature, but the muscles are too weak to generate reliable myoelectric signals, this situation can be improved with a surgical procedure called Regenerative Peripheral Nerve Interface (RPNI), though this is primarily intended to reduce limb pain.

Additional Information

Myoelectric Pattern Recognition

Newer versions of bionic arms/hands now offer myoelectric pattern recognition as an alternative to direct control. In this model, multiple sensors are used, typically between three and eight, and sometimes even more than this.

Bionic Arm Pattern Recognition

BrainRobotics Myoelectric Pattern Recognition Sensors

With so many sensors, multiple patterns of muscle movement can be detected instead of just single movements. Each pattern can be mapped to a specific action. For example, one pattern can be mapped to use Grip 1, another to Grip 2, another to move an elbow joint, and so on.

Using a bionic hand as an example, the mapping process occurs as follows:

  • the user is asked to perform an action with his phantom hand, such as picking up a pencil lying flat on a desk;
  • this action would normally require a tripod grip involving the thumb, forefinger, and middle finger;
  • the user’s attempt to do this with his phantom hand triggers a pattern of muscle movements in the residual limb;
  • this pattern is mapped to the required actions in the bionic hand.

The resulting control is far more intuitive because the user doesn’t have to explicitly flex individual muscles. He just thinks about moving his phantom arm/hand exactly as if it were still present and the bionic version will mirror his intent.

It doesn’t even matter which patterns of muscle movement are mapped to specific actions as long as the patterns can be differentiated from each other and are reasonably consistent. We say “reasonably” because artificial intelligence (AI) is used to identify patterns and can adjust for minor variations.

This same mapping process also applies to movements of bionic wrists, elbows, and shoulders, if present. Some systems can even move multiple components simultaneously or be used to control individual digits.

Prerequisites

  • the user must have sufficient physical space on the residual limb to accommodate more sensors (in the case of shoulder disarticulation, upper torso sites are used instead);
  • the shape and skin condition at the sensor location(s) must be conducive to detecting myoelectric signals;
  • users must be comfortable enough with technology to interact with software training programs;
  • training, patience, and discipline;

Advantages

  • allows more intuitive control over a bionic hand and other bionic arm joints;
  • makes using the bionic arm/hand more efficient and less tiring;

Disadvantages

  • more expensive than direct control myoelectric solutions;
  • where sensors are placed on a residual limb, the size and shape of that limb may change as the temperature changes, causing the sensors to shift or lose contact with the skin; sweat can also interfere with the sensors; in either case, the arm/hand will become more difficult to control;

Surgical Techniques That Can Enhance This Technology

As with Myoelectric Direct Control, the two surgical procedures that can be used to enhance Myoelectric Pattern Recognition are Targeted Muscle Reinnervation (TMR) and Regenerative Peripheral Nerve Interface (RPNI), though the latter is primarily intended to reduce limb pain.

Additional Information

Pattern Recognition Using Surgically Implanted Myoelectric Sensors

Given the problems with maintaining optimal contact between the skin’s surface and myoelectric sensors, it was only a matter of time before someone decided to implant the sensors directly into muscle tissue:

DARPA Implanted Myoelectric Sensors

DARPA Implanted Myoelectric Sensors

This approach is often used in combination with Targeted Muscle Reinnervation (TMR). Since the patient is undergoing surgery anyway, it makes sense to embed the sensors.

The number of sensors implanted and the basic control system mirrors that of a skin-surface pattern recognition system with two important differences. Because the sensors are embedded in the muscle tissue, they can pick up deeper muscle movements in addition to those near the skin surface. Also, the signals are clearer and more reliable. This makes it easier to detect the simultaneous movements of multiple joints and to relay the equivalent commands to bionic components. Put another way, implanted myoelectric sensors are better able to support more advanced control systems, though improvements in surface sensors embedded in flexible materials may soon challenge this claim.

Prerequisites

  • the user must have sufficient musculature on the residual limb to accommodate implanted sensors (in the case of shoulder disarticulation, upper torso sites are used instead);
  • users must be able to tolerate surgery;
  • users must be comfortable enough with technology to interact with software training programs;
  • training, patience, and discipline;

Advantages

  • allows more intuitive control over a bionic hand and other bionic arm joints;
  • makes using the bionic arm/hand more efficient and less tiring;
  • eliminates the frustration of periodic contact problems between skin-surface myoelectric sensors and the skin;

Disadvantages

  • more expensive than skin-surface pattern recognition systems and a lot more expensive than myoelectric direct control;
  • with any surgery, there is always the risk of pain, infection, and scarring.

Surgical Techniques That Can Enhance This Technology

The primary surgical technique that can be used to enhance this approach is Targeted Muscle Reinnervation (TMR). This is typically used where the patient has insufficient musculature to accommodate the required myoelectric sensors.

Additional Information

Neural Interfaces

Neural interfaces have two potential roles with bionic limbs:

  1. They are currently one of the most promising methods of conveying sensory feedback from a bionic device to the nervous system.
  2. They have also been used experimentally to intercept signals sent to motor nerves as a more direct method of determining user intent.
DARPA Two-Way Neural Interface

Two-Way Neural Interface

Since this article is on control systems, we’re going to focus on the second use. Because we are not research scientists or doctors, we cannot categorically state that the idea of determining a user’s intended movements by intercepting motor signals is dead. In fact, we’re certain that research in this area continues. But we can say that every bionic arm/hand device that we’ve examined uses some form of myoelectric sensor system, not a neural interface, to determine user intent. And that’s not just current devices — that’s every one coming to market in the foreseeable future!

So, we’re not going to cover this as a serious user control option at this time.

Additional Information

If you would like to know more about neural interfaces, please see Neural Interface Surgery and Neural Interfaces for Bionic Arms & Hands. Note, however, that these articles pertain to the sensory feedback aspects of neural interfaces, not motor control.

Implicit Control

The most obvious form of implicit control over bionic arms/hands is the socket interface between the residual limb and the device. We’re not going to cover this subject here, as that deserves an entire website on its own. However, we are going to cover a socket replacement — Osseointegration — plus a relatively new surgical procedure that may revolutionize key aspects of user control for all bionic limbs.

Osseointegration

Osseointegration involves inserting a metal rod into an arm or leg bone. The bone then fuses with the rod in a manner similar to fracture healing:

Osseointegration implant

Bionic devices are then attached to the metal rod. This method of attachment significantly improves user control over the attached device because it makes the device a direct extension of the user’s skeletal frame. This not only increases the range of motion for the bionic limb; it also provides a form of sensory feedback through vibrations passed from the bionic device to the bone.

A more recent version of this technology — Integrum’s e-Opra Implant System — integrates embedded myoelectric sensors and a neural interface (i.e. for sensory feedback) with the implant, making it capable of supporting the most advanced user control systems on the planet:

Integrum e-Opra Implant System

Prerequisites

  • patients must be in suitable health, which typically means patients who do not smoke, have diabetes or other vascular issues, and are not obese;
  • patients must have reached full skeletal maturity, meaning they are at least 18 years of age but no more than 70;
  • there are also some restrictions related to the use of certain drugs, such as chemotherapy;

Advantages

  • increased comfort, especially with the elimination of socket-related skin problems;
  • easier to attach and detach a prosthesis;
  • greater strength/stability;
  • more intuitive use of the prosthesis including a broader range of motion;
  • improved osseoperception;
  • overall improved quality of life;

Disadvantages

  • it’s major surgery with a long recovery time;
  • it involves significant costs, which may not be covered by insurance;
  • the implant presents a serious risk of deep infection (up to 41 %);
  • bone fractures occur in up to 9 % of the cases;
  • implant parts break in up to 31 % of cases;
  • the implants end up being removed in up to 20 % of the cases.

Additional Information

Agonist-antagonist Myoneural Interface (AMI)

AMI is a technology that revolves primarily around surgically restoring the agonist-antagonist muscle pairings that control our joints. Basically, severed muscles in the residual limb are reconnected to form such a pairing:

AMI Recreating Agonist-Antagonist Muscle Pairings

This restores the brain’s natural sense of position and movement for the missing limb. This is called “proprioception”.

A bionic joint is then calibrated so that its movements correspond to the movements of the restored muscle pair, and, voila, the brain can better control the bionic limb with thoughts and also sense its position in space.

In terms of controlling a bionic limb, this is still a myoelectric system. It’s just one with a much better connection between the brain and remaining muscles in the residual limb. Also, there is less muscle atrophy when an agonist-antagonist pairing is restored, so muscle movements are stronger and therefore generate stronger myoelectric signals.

Prerequisites

  • the patient must be able to withstand surgery;
  • AMI must still be approved for use with upper limbs, though we have received word that at least one upper-limb patient has successfully undergone the procedure;

Advantages

  • will restore the brain’s proprioceptive sense of the position and movement of a bionic arm or hand;
  • will significantly improve user control over larger movements, though control over finer hand movements may be more difficult to achieve solely through AMI without sensory feedback;
  • will strengthen the muscles used in pairings;

Disadvantages

  • involves surgery with all its inherent costs and risks.

Additional Information

Sensory Feedback

Technically, sensory feedback is not part of a bionic limb’s control system in that it does not directly trigger bionic actions. It is used to inform the user of actions that have taken place, which may then necessitate additional actions.

And, yet, sensory feedback may be more crucial to effective user control than any other single factor.

Currently, the following forms of sensory feedback are either available or soon will be, going from the least sophisticated to the most:

Osseoperception

Vibrations pass from a bionic hand into the bone of a residual limb through an osseointegrated implant. This is known as osseoperception and can be surprisingly informative.

For more information, see Osseointegration for Bionic Limbs.

Simple Haptic Feedback

Sensors built into bionic hands pass feedback to vibrators built into the prosthetic socket, which use different levels and frequencies of vibration to convey information. One example of this is the use of pressure sensors to convey contact with an object (a single vibration) or grip force (repeated vibrations, possibly with increased intensity).

Transcutaneous Electrical Nerve Stimulation (TENS)

TENS feedback systems attempt to stimulate nerves with an electrical charge through the skin and intervening tissue. This can cause the brain to experience certain sensations in the missing hand that abstractly equate to sensory feedback from the bionic hand, such as a tingling sensation in the missing hand when the bionic hand makes contact with an object. This can give the user a greater sense of embodiment than the use of simple haptic feedback, but the information that can be communicated in this manner is limited.

Targeted Sensory Reinnervation (TSR)

TSR is a method where the skin over a target muscle is denervated, then reinnervated with nerve fibers from an amputated limb. When this piece of skin is touched, it provides the amputee with the sensation that the missing limb is being touched.

For example, a patch of skin on the forearm can be reinnervated with nerves from an amputated hand such that stimulating that patch of skin will feel as if the hand is being touched. Because the reinnervated skin is receptive to a near-normal range of stimuli, information can be conveyed from the bionic hand to the skin via any number of stimulation methods, including pressure, temperature, etc.

This type of system can communicate a significant amount of information, including which digit is being touched.

For more information, see Targeted Sensory Reinnervation (TSR).

Neural Interface

We are describing the sensory feedback version of neural interfaces here, not interaction with motor nerves.

Electrodes are surgically implanted to wrap around or transect sensory nerves. The electrodes then stimulate the nerves with an electrical charge that tricks the brain into experiencing whatever the sensors in a bionic hand are experiencing (or some loose translation thereof).

Because this is a direct interface with the nerve, it has the most potential for communicating sophisticated information. It also has several drawbacks, including the risk of infection, scarring, etc.

For more information on this type of neural interface, see Neural Interface Surgery and Neural Interfaces for Bionic Arms & Hands.

For more information on sensory feedback in general, see Sensory Feedback for Bionic Hands and also Understanding Bionic Touch.

Related Information

For a comprehensive description of all current upper-limb technologies, devices, and research, see our complete guide.

Follow Us