Myoelectric Pattern Recognition for Bionic Arms & Hands

Myoelectric Pattern Recognition

If you want to avoid surgery, combining myoelectric skin sensors with pattern recognition is likely the future for bionic arm and hand control systems. Not only is it more intuitive than a direct control system and easier to train — it’s also more adaptable to changing circumstances.

The Basics

Most users of bionic arms & hands do not want to have to explicitly flex specific muscles to control their device. They especially don’t want to do this if it requires multiple steps for each action (e.g. position the wrist or thumb, select a grip, close the hand). This is slow, cumbersome, and tiring.

What users want is to control their bionic limb the same way that they control their natural limbs: to simply attempt an action and have their bionic components automatically carry out the necessary tasks.

How can this happen? When we humans move our muscles, they generate electrical signals. These signals can be detected by myoelectric sensors placed against the skin, then translated into commands sent to an arm joint or bionic hand.

In a myoelectric direct control system, only two sensors are used and they are typically placed over an antagonistic pair of muscles. Flexing the muscles one way represents an open action, while the other way represents a close action.

Myoelectric Direct Control Signals

Source: https://essay.utwente.nl/80234/1/Mateo_Mesonero_MA_ET-BME.pdf

Unfortunately, two sensors only allow for three different signals (open, close, and a co-contraction of both muscles simultaneously), and you can’t do much with three signals. You certainly can’t directly select specific grips or move individual fingers or attempt to control shoulder, elbow, or wrist joints in addition to controlling a hand.

Pattern recognition systems are different because they use many more sensors (typically 8 or 16). More importantly, they don’t expect you to flex specific muscles. They just expect you think/move as if you still had your natural arm/hand. This triggers a pattern of movements in the remaining muscles that can be translated into commands for bionic components.

Myoelectric Pattern Recognition Chart

It doesn’t even matter which patterns of muscles are triggered so long as they can a) be differentiated from each other, and b) are reasonably consistent. We say “reasonably” because artificial intelligence (AI) can be used to identify patterns and adjust for minor variations.

Okay, those are the basics. Now let’s get into the details.

Suitability for Using a Myoelectric Pattern Recognition System

The first prerequisite for using a pattern recognition system is that you must have enough space on your residual limb or another target site to accommodate a sufficient number of sensors (typically from 8 to 16).

Second, you must be able to generate enough distinctive patterns of muscle movement. If you had an amputation at or above the elbow and have lost the muscles you need to generate the requisite patterns, you may require a procedure called “Targeted Muscle Reinnervation”. In this procedure, nerves are surgically reassigned to alternative muscles to create the required muscle activity.

Targeted Muscle Reinnervation

The third requirement — as is the case for all myoelectric sensors — is that the skin at the sensor sites must be conducive to detecting myoelectric signals.

Finally, you must be willing to devote significant time and effort to training.

To determine suitability, the provider of the pattern recognition system should a) supply some kind of evaluation system, and b) train prosthetists in the evaluation procedures. The company we list at the bottom of this article — Infinite Biomedical Technologies (IBT) — offers a model example of this kind of system.

Socket Design and Fit

We’ve created this subsection mainly to emphasize one central point: using pattern recognition for skin-surface myoelectric control does not alleviate the need for a great socket fit. Quite the contrary. Any prosthesis that relies on sensor contact with the skin’s surface is entirely dependent on a proper socket fit for maintaining that contact.

Custom Prosthetic Arm Socket

Source: http://www.product.lth.se/news/article/article/perfect-fit-using-3d-technologies-to-manufacture-custom-prosthetic-arm-sockets/

As emphasized in our article on myoelectric direct control, this doesn’t only mean a great fit when your arm is lying flat on the prosthetist’s desk in an air-conditioned office. It means a great fit in all arm positions and for all arm movements.

Additionally, you should test your socket fit in environments that range from cold and dry to warm and humid. If you encounter problems in different environments with either the socket or its embedded sensors, ask your prosthetist to help you mitigate these problems even if the system you are buying allows for quick recalibration (see below).

Training

Training is fundamentally a matter of mapping patterns of muscle movement to the desired actions in the bionic limb. For example, one pattern can be mapped to use Grip 1, another to Grip 2, another to moving an elbow joint, and so on. In more advanced systems, patterns can even be mapped to individual finger movements.

The following is a rudimentary example of the process used to train a bionic hand:

  • the user is asked to perform an action with his/her phantom hand, such as picking up a pencil lying flat on a desk;
  • this action would normally require a tripod grip involving the thumb, forefinger, and middle finger;
  • the user’s attempt to do this with his/her phantom hand triggers a pattern of muscle movements in the residual limb;
  • this pattern is mapped to the required actions in the bionic hand;

However, in practice, training is more sophisticated and must support consistent repetition with good user feedback. For example, IBT offers a game-based, virtual learning/training environment where users can train as much as they wish:

IBT Sense Training Software

As usual, there is no substitute for effort and persistence in this process. If you can identify and repeat consistent muscle movement patterns, you will be much happier with your level of control.

Recalibration

You will likely be forced to use your bionic arm/hand in unfavorable conditions. Even if you have worked closely with your prosthetist to mitigate these circumstances, your muscle movement patterns or the detection of those patterns will likely change.

When this happens, you must be able to quickly recalibrate your pattern recognition settings. For example, IBT allows users to do this via their mobile Sense application.

IBT Sense Mobile Application

This ability to adjust one’s user control system so quickly and easily is a vital step toward increasing user satisfaction with bionic arms & hands.

What Does the Science Say?

If you have read our article on finding the right myoelectric control system, you know that studies on user satisfaction with direct control myoelectric arms & hands are not favorable.

Pattern recognition systems have also had their challenges and this is reflected in the scientific literature. Some recent scientific studies have reported measurable advantages for pattern recognition systems over direct control systems in the areas of increased grip usage1, user preference and dexterity2, and general prosthesis control3.

Other studies have found that pattern recognition systems still suffer significant accuracy problems due to limb position and contraction intensity in real-world environments4.

However, we believe that recent improvements in training software, artificial intelligence, and the ability to recalibrate control systems “on the fly” are directly addressing these problems.

Related Information

For a look at what we believe is the best myoelectric pattern recognition system currently on the market, see IBT’s Sense Pattern Recognition System.

If you’re looking for a control system that also provides advanced sensory feedback, see our article on neural interfaces for bionic hands.

For a good overview of control systems in general, see bionic arm/hand control systems.

For a complete description of all current upper-limb technologies, devices, and research, see our comprehensive guide.

References

[1] Simon, Ann & Turner, Kristi & Miller, Laura & Hargrove, Levi & Kuiken, Todd. (2019). Pattern recognition and direct control home use of a multi-articulating hand prosthesis. IEEE … International Conference on Rehabilitation Robotics : [proceedings]. 2019. 386-391. 10.1109/ICORR.2019.8779539.

[2] Hargrove, Levi & Miller, Laura & Turner, Kristi & Kuiken, Todd. (2017). Myoelectric Pattern Recognition Outperforms Direct Control for Transhumeral Amputees with Targeted Muscle Reinnervation: A Randomized Clinical Trial. Scientific Reports. 7. 10.1038/s41598-017-14386-w.

[3] Uellendahl, J., Tyler, J., Hung, K. (2016). A Case Series Study of Pattern Recognition for Upper-limb Prosthesis Control. Proceedings of 42nd Academy Annual Meeting & Scientific Symposium, March 2016.

[4] Campbell, Evan & Phinyomark, Angkoon & Scheme, Erik. (2020). Current Trends and Confounding Factors in Myoelectric Control: Limb Position and Contraction Intensity. 10.20944/preprints202002.0415.v1.