When looking at a bionic arm, many people believe that the hand and the arm joints are the primary technology components. This is not true. It is the user control system that holds the key to restoring near-natural capabilities.
Understanding Your Bionic Arm & Hand Control Options
In this article, we’re going to cover five types of bionic arm/hand control systems:
- Myoelectric Direct Control
- Myoelectric Pattern Recognition
- Pattern Recognition with Surgically Implanted Myoelectric Sensors
- Agonist-antagonist Myoneural Interface (AMI)
- Neural Interfaces
Myoelectric Direct Control
Most of the bionic hands in use today utilize myoelectric direct control systems.
This means that either one or two myoelectric sensors are built into an arm socket so that they lie against the skin of the residual limb. Their job is to detect muscle movements intended to:
- open the bionic hand;
- close the bionic hand;
- where two sensors are used, they can also detect a co-contraction of muscles in both areas.
Bionic hands usually offer multiple grip patterns, i.e. different combinations of fingers and the thumb open and close depending on the selected grip. The co-contraction signal (or possibly a double-open or double-close signal) is often used to rotate between the available grips.
This type of system is described as “direct control” because the user must explicitly contract specific muscles to directly control grip selection and the open & close actions. It is not ideal for controlling above-the-elbow bionic arms because it does not generate enough distinctive signals to manage the hand, wrist, elbow, and shoulder joints, if applicable.
Sensory feedback in this type of system is also limited. Typically, it involves placing pressure sensors in the bionic fingers and triggering small vibrations in the socket to indicate contact and/or the degree of resistance. However, it may also be possible to indicate slippage, temperature, or the sharpness of an object by stimulating the residual limb’s skin in other ways.
- the user has to demonstrate specific muscle control
- the shape and skin condition of the residual limb must be conducive to detecting myoelectric signals
- training, patience, and discipline
- less expensive than more advanced solutions
- rotating through grips can be slow and cumbersome
- having to exercise such explicit control over one’s hand can be tiring, especially when every action must be guided visually
- the size and shape of the residual limb may change as the temperature changes, causing the myoelectric sensors to shift or lose contact with the skin; sweat can also interfere with the sensors; in either case, the hand will become more difficult to control
- the system doesn’t generate enough distinct signals to efficiently control a bionic wrist, elbow, or shoulder in addition to a hand
- the potential for sensory feedback is limited
Like What You’re Reading? Follow Us to Stay Informed:
Myoelectric Pattern Recognition
Newer versions of bionic arms/hands now offer myoelectric pattern recognition as an alternative to direct control. In this model, multiple sensors are used, typically from 8 to 16.
BrainRobotics Myoelectric Pattern Recognition Sensors
With so many sensors, multiple patterns of muscle movement can be detected instead of just single movements. Each pattern can be mapped to a specific action. For example, one pattern can be mapped to use Grip 1, another to Grip 2, another to move an elbow joint, and so on. In more advanced systems, patterns can even be mapped to individual finger movements.
Using a bionic hand as an example, the mapping process occurs as follows:
- the user is asked to perform an action with his/her phantom hand, such as picking up a pencil lying flat on a desk;
- this action would normally require a tripod grip involving the thumb, forefinger, and middle finger;
- the user’s attempt to do this with his/her phantom hand triggers a pattern of muscle movements in the residual limb;
- this pattern is mapped to the required actions in the bionic hand.
The resulting control is far more intuitive because the user doesn’t have to explicitly control the hand. He just thinks about moving his phantom arm/hand exactly as if it were still present and the bionic version will mirror his intent.
It doesn’t matter which patterns of muscle movement are mapped to specific actions as long as the patterns can be differentiated from each other and are reasonably consistent. We say “reasonably” because artificial intelligence (AI) is used to identify patterns and can adjust for minor variations.
This same mapping process also applies to movements of bionic wrists, elbows, and shoulders, if present. The movements of different components can even occur simultaneously, as they do with natural arms/hands.
In terms of sensory feedback, myoelectric pattern recognition systems suffer the same limitations as their direct control counterparts.
- the user must have sufficient physical space on the residual limb to accommodate more sensors (in the case of shoulder disarticulation, upper torso sites are used instead)
- the shape and skin condition at the sensor location(s) must be conducive to detecting myoelectric signals
- users must be comfortable enough with technology to interact with software training programs
- training, patience, and discipline
- allows more intuitive control over a bionic hand and other bionic arm joints
- makes using the bionic arm/hand more efficient and less tiring
- more expensive than direct control myoelectric solutions
- where sensors are placed on a residual limb, the size and shape of that limb may change as the temperature changes, causing the sensors to shift or lose contact with the skin; sweat can also interfere with the sensors; in either case, the arm/hand will become more difficult to control
- the potential for sensory feedback is still limited
Pattern Recognition With Surgically Implanted Myoelectric Sensors
We’re not going to cover this option in great detail — we just want to make readers aware of it.
To avoid the problems with maintaining proper contact between sensors and the skin, one option is to surgically implant the sensors into the target muscles of the residual limb.
DARPA Implanted Myoelectric Sensors
This approach is often used in combination with a process called “Targeted Muscle Reinnervation”, where nerves that once controlled muscles subsequently lost to amputation are surgically reassigned to new muscles. Since the patient is undergoing surgery anyway, it makes sense to embed the sensors.
However, other than the physical placement of the sensors, there is no difference in the nature of the control system, i.e. it is still a myoelectric pattern recognition system, which is why we’re not covering it in more detail.
Agonist-antagonist Myoneural Interface (AMI)
AMI is a technology that revolves primarily around surgically restoring the agonist-antagonist muscle pairings that control our limbs. Basically, severed muscles in the residual limb are reconnected to form such a pairing:
This restores the brain’s natural sense of position and movement for the missing limb. This is called “proprioception”.
A bionic joint is then calibrated so that its movements correspond to the movements of the restored muscle pair, and, voila, the brain can better control the bionic limb with thoughts and also sense its position in space.
All of that is a gross over-simplification, but you can get the full story by reading our complete article on AMI.
Once a bionic limb is set up so that its movements correspond to the movements of the muscle pair, visual cues then reinforce this model, i.e. the user attempts to raise the front of his foot toward his shin, the muscle pair responds accordingly, the bionic foot mirrors this movement, and the user’s eyes confirm it. This creates a closed-loop control system.
Note that we are referencing a lower-limb implementation of AMI in the preceding paragraph. This is not by accident. Although scientists believe that AMI will work for upper limbs, this has not yet been tested.
- AMI must be approved for use with upper limbs
- the patient must undergo surgery to implement AMI
- a sufficiently advanced control system must be used to specifically map each AMI muscle pair to a bionic joint
- will restore the brain’s proprioceptive sense of the position and movement of a bionic arm or hand
- will significantly improve user control over larger movements, though control over finer hand movements may be more difficult to achieve solely through AMI due to a lack of true sensory feedback
- involves surgery with all its inherent cost and risks
- does not address the need for true sensory feedback, i.e. sense of touch
Until the development of AMI, we would have said that the ultimate method of connecting a user’s mind to a bionic arm/hand is through a neural interface. We still hold this view when it comes to sensory feedback, but we now believe that some combination of AMI and pattern recognition may be sufficient for user control, especially if combined with sensory feedback.
Nonetheless, we’ll describe neural interfaces in this section as if AMI doesn’t exist.
The main goal of a full neural interface is to facilitate two-way communication between the user and his bionic arm/hand. By two-way, we mean a system supporting both user control and advanced sensory feedback.
There are a few ways of doing this. In the simplest model, control is still achieved by using implanted myoelectric sensors, while sensory feedback is transmitted from sensors on bionic fingers to implanted electrodes attached to sensory nerves. The electrodes then stimulate the nerves to “trick” the brain into feeling the correct sensation(s).
A more advanced model uses implanted sensors/electrodes to interact directly with both motor and sensory nerves. As far as we know, this type of system is currently confined to the laboratory.
Two-Way Neural Interface
Finally, companies like Integrum are offering two-way neural interfaces as part of an osseointegrated implant:
Integrum e-Opra Implant System
These are all very promising models. However, it should be noted that surgical implants of any kind do pose risks like infection and scarring, not to mention significant added cost.
Also, just because implanted sensors eliminate most of the problems with maintaining skin-surface contact doesn’t mean that they are perfect. Scar tissue can form around the embedded sensors and reduce their ability to detect signals.
The real gain here is with the sensory feedback part of the system. Not only can users experience basic feedback like contact and pressure. Work is also underway to reproduce other facets of touch, such as shape, slippage, and even temperature.
Sensory feedback can also dramatically improve user control for finer hand movements because it allows for small, interactive adjustments between the brain and the bionic hand based on that feedback, which would reduce the user’s reliance on visual guidance.
For example, think about what happens when you pick up a pen to write with a natural hand. You don’t worry about how you pick up the pen. Instead, you grab it however you can. You then use a combination of sensory feedback and finger manipulations to move the pen into the right position.
This same brain/hand adjustment mechanism can be created via a neural interface, and that’s not all. By creating a sensory feedback loop between the bionic arm/hand and the brain, early research indicates that this may help eliminate phantom pain, effectively making the user feel whole again.
That’s the full promise of a neural interface, but, as mentioned above, some of this functionality may be provided by AMI instead.
- the patient must be able to tolerate surgery
- sufficient working nerves must be present in the residual limb or upper torso to interact with the neural interface
- training, patience, and discipline
- allows more intuitive control over a bionic arm/hand
- supports sensory feedback
- makes using the bionic hand/arm more efficient and less tiring
- may cure phantom pain
- the added cost of surgery
- the typical surgical risks of pain, infection, and scarring
- scarring may interfere with the neural interface’s ability to communicate with nerves
For a comprehensive description of all current upper-limb technologies, devices, and research, see our complete guide.