If you want to avoid surgery, combining myoelectric skin sensors with pattern recognition is currently your best option.
The Basics
Most users of bionic arms & hands do not want to explicitly flex specific muscles to control their device. They especially don’t want to do this if it requires multiple steps for each action (e.g. position the wrist or thumb, select a grip, close the hand). This is slow, cumbersome, and tiring.
What users want is to control their bionic limb the same way that they control their natural limbs: to simply attempt an action and have their bionic components automatically carry out the necessary tasks.
How can this happen? When we humans move our limbs, our muscles generate electrical signals. These signals can be detected by myoelectric sensors placed against the skin, then translated into commands sent to a bionic joint or hand.
In a myoelectric direct control system, only two sensors are used and they are typically placed over two antagonistic groups of muscles. Flexing one group represents an open action, while flexing the other group represents a close action.
Source: https://essay.utwente.nl/80234/1/Mateo_Mesonero_MA_ET-BME.pdf
Unfortunately, two sensors only allow for three different signals (open, close, and a co-contraction of both muscle groups simultaneously), and you can’t do much with three signals. You certainly can’t directly select specific grips, move individual fingers, or attempt to control shoulder, elbow, or wrist joints in addition to controlling a hand.
Pattern recognition systems are different because they use many more sensors. More importantly, they don’t require you to flex specific muscles. They just expect you think/move as if you still had your natural arm/hand. This triggers a pattern of movements in the residual muscles that can be translated into commands for bionic components.
The following video provides a quick overview of how Ottobock’s Myo Plus pattern recognition system works, but the same principles apply to all myoelectric pattern recognition systems:
It doesn’t even matter which patterns of muscles are triggered so long as they can a) be differentiated from each other, and b) are reasonably consistent. We say “reasonably” because artificial intelligence (AI) can identify patterns and adjust for minor variations.
Okay, those are the basics. Now let’s get into the details.
Suitability for Using a Myoelectric Pattern Recognition System
The first prerequisite for using a pattern recognition system is that you must have enough space on your residual limb or another target site to accommodate a sufficient number of sensors (typically from 8 to 16).
Second, you must be able to generate enough distinctive patterns of muscle movement. If you had an amputation at or above the elbow and have lost the muscles you need to generate the requisite patterns, you may have to undergo a procedure called Targeted Muscle Reinnervation (TMR). In this procedure, nerves are surgically reassigned to alternative muscles to create the required muscle activity.
The third requirement — as is the case for all myoelectric sensors — is that the skin at the sensor sites must be conducive to detecting myoelectric signals.
Finally, you must be willing to devote significant time and effort to training.
To determine suitability, the provider of the pattern recognition system should a) supply some kind of evaluation system, and b) train prosthetists in the evaluation procedures. The companies we list at the bottom of this article — Coapt Engineering, Infinite Biomedical Technologies (IBT), and Ottobock — all offer this kind of support.
Socket Design and Fit
We’ve created this section mainly to emphasize that pattern recognition systems do not alleviate the need for a great socket fit. Quite the contrary. Any prosthesis that relies on sensor contact with the skin’s surface is entirely dependent on a proper socket fit for maintaining that contact.
As emphasized in our article on myoelectric direct control, this doesn’t only mean a great fit when your arm is lying flat on the prosthetist’s desk in an air-conditioned office. It means a great fit in all arm positions and for all arm movements.
Additionally, you should test your socket in environments that range from cold and dry to warm and humid. If you encounter problems in different environments with either the socket or its embedded sensors, ask your prosthetist to help you mitigate these problems even if your pattern recognition system allows for quick recalibration (see below).
Calibration
Calibration means mapping patterns of muscle movement to the desired actions in the bionic limb. For example, one pattern can be mapped to using Grip 1, another to using Grip 2, another to moving an elbow joint, and so on. In more advanced systems, patterns can even be mapped to individual finger movements.
The following is a rudimentary example of the process used to calibrate a pattern recognition system:
- The user is asked to perform an action like opening the hand or closing the hand; this action can be with the user’s phantom hand or with an actual bionic device.
- The attempted action will trigger a pattern of muscle movements in the residual limb.
- This pattern is then mapped to the required actions in the bionic hand.
This video provides a more detailed description of the calibration process used for IBT’s Sense system:
Sense’s main competitors — Coapt’s Gen2 and Ottobock’s Myo Plus — offer similar calibration tools.
User Training
Despite the use of advanced sensors, processors, and artificial intelligence, user training remains essential to the success of any myoelectric pattern recognition system.
The main issue here has to do with the consistency of muscle movements. Using feedback from software-based training tools, users must learn to perform the same actions in the same manner on every attempt.
One of the best tools we’ve seen for this is IBT’s game-based, virtual learning/training environment for its Sense pattern recognition system:
Similarly, Coapt Engineering provides its Actuate environment to help users practice their control either with the actual device or an on-screen equivalent:
Coapt Gen2 Control Practice
Recalibration
You will likely be forced to use your bionic arm/hand in unfavorable conditions. Even if you have worked closely with your prosthetist to mitigate these circumstances, your muscle movement patterns or the detection of those patterns will likely change in these conditions.
When this happens, you must be able to quickly recalibrate your pattern recognition settings. One of our favorite solutions for doing this is Coapt Engineering’s push-button approach to recalibrating its Gen2 system:
IBT and Ottobock also provide excellent recalibration tools but their systems require a software interface.
What Does the Science Say?
If you have read our article on finding the right myoelectric control system, you know that scientific studies on user satisfaction with direct control myoelectric arms & hands are not favorable.
Pattern recognition systems have also experienced problems, especially due to limb position and contraction intensity in real-world environments1.
However, some recent studies have reported measurable advantages for pattern recognition systems over direct control systems in the areas of increased grip usage2, user preference and dexterity3, and general prosthesis control4.
With improved artificial intelligence, training tools, and the ability to recalibrate control systems “on the fly”, we expect the performance gap between direct control versus pattern recognition systems to broaden in the coming years.
Related Information
For a look at the leaders in the pattern recognition market, see Coapt Engineering’s Complete Control System Gen2, IBT’s Sense Pattern Recognition System, and Ottobock’s Myo Plus.
If you’re looking for a control system that also provides advanced sensory feedback, see our article on neural interfaces for bionic hands.
For a good overview of control systems in general, see bionic arm/hand control systems.
For a complete description of all current upper-limb technologies, devices, and research, see our comprehensive guide.
References
[1] Campbell, Evan & Phinyomark, Angkoon & Scheme, Erik. (2020). Current Trends and Confounding Factors in Myoelectric Control: Limb Position and Contraction Intensity. 10.20944/preprints202002.0415.v1.
[2] Simon, Ann & Turner, Kristi & Miller, Laura & Hargrove, Levi & Kuiken, Todd. (2019). Pattern recognition and direct control home use of a multi-articulating hand prosthesis. IEEE … International Conference on Rehabilitation Robotics : [proceedings]. 2019. 386-391. 10.1109/ICORR.2019.8779539.
[3] Hargrove, Levi & Miller, Laura & Turner, Kristi & Kuiken, Todd. (2017). Myoelectric Pattern Recognition Outperforms Direct Control for Transhumeral Amputees with Targeted Muscle Reinnervation: A Randomized Clinical Trial. Scientific Reports. 7. 10.1038/s41598-017-14386-w.
[4] Uellendahl, J., Tyler, J., Hung, K. (2016). A Case Series Study of Pattern Recognition for Upper-limb Prosthesis Control. Proceedings of 42nd Academy Annual Meeting & Scientific Symposium, March 2016.