Sensory Feedback for Bionic Hands

Bionic Hands Sensory Feedback

The ultimate goal of bionic hands is to replicate natural abilities. For this to happen, bionic hands will need to provide advanced sensory feedback. This is not only a matter of reconnecting users to their surroundings through touch. It is also critical for perfecting user control.

A Daunting Task

The human sense of touch is remarkable. There are more than 3,000 touch receptors on a human fingertip. We can feel not only the size and shape of an object but also its texture, pliability, temperature, and more.

Even blindfolded, most of us can identify common objects through touch alone.

Yet, it goes deeper than this. Your mind retains a sophisticated spatial map of your body. It can feel the current location of every body part relative to you as a whole and also to your surroundings.

Ever reach for a bedside glass of water in the dark? Ever touch your finger to your nose with your eyes closed? This innate sense of self is so strong that the brain can even feel a lost limb after amputation — the “phantom limb” effect.

sensory and motor cortex areas involved in bionic touch

The brain’s sensory cortex and motor cortex (responsible for movement) share an intimate relationship. If you reach for an object but don’t quite grasp it correctly, your mind and natural hand will quickly exchange sensory and movement information to make the required adjustments.

Without these capabilities, users of bionic hands are forced to visually guide every nuance of hand movement and use — a process that is often slow, frustrating, and tiring.

Unfortunately, it’s going to take us a while to make bionic hands that provide the same type of sensory feedback as their natural counterparts. First, we’ll need to take some baby steps.

The Use of Pressure Sensors in Commercial Bionic Hands

In September 2019, Psyonic entered the bionic hand market with its Ability Hand.

They took the same sensors found in modern cell phones and repurposed them as highly sensitive touch sensors, which they placed in the Ability Hand’s fingertips.

When the fingers touch an object, they send a signal that triggers vibrations against the skin on the residual limb, informing the user of both contact and pressure.

This makes the Ability Hand far more intuitive to use, as demonstrated in this short video of a user picking up an eggshell while blindfolded:

This doesn’t replicate natural touch by any means. But it’s an important leap forward in practical functionality. And at $10 for the sensors, no one can argue with its cost-effectiveness.

We believe that the Vincent Evolution offers similar capabilities but have been unable to confirm this due to the lack of documents and videos available in English.

Others will follow suit or are already doing so.

E-Dermis Electronic Skin

John Hopkins University (JHU) has created an electronic skin called “e-dermis”. This video from freethink.com provides an excellent overview:

This represents a giant technological leap over the use of simple pressure sensors. By embedding sensors in multiple layers, e-dermis has the potential to detect not only pressure but also pain. Eventually, it may even be able to sense temperature.

E-dermis also offers significant commercial advantages over other methods of sensory feedback. First, it conveys its sensory information by stimulating nerves in the user’s residual limb through the skin. This non-invasive technique, known as transcutaneous electrical nerve stimulation (TENS), is more advanced than simple vibrations yet still a lot less expensive and risky than surgically implanting electrodes.

Another advantage is that e-dermis is not tied to a specific prosthetic device. It can be layered on top of any prosthesis, making it a kind of development platform that can be used by scientific teams around the world for many purposes. This serves to pool resources and accelerate advancements.

E-dermis is not the only option in this category. To read about others, see our article on electronic skin.

However, even if this type of electronic skin technology gives bionic hands a near-natural ability to detect different sensations, trying to convey this information via vibration or TENS may not be enough. Fully re-establishing the intimate relationship between the brain’s sensory and motor cortices will likely require true two-way communication with living nerve bundles.

The organization leading the charge on this initiative is the Defense Advanced Research Projects Agency, otherwise known as DARPA.

DARPA’s HAPTIX Initiative

In 2006, DARPA launched its Revolutionizing Prosthetics program. This resulted in two advanced bionic arms: the LUKE Arm and the Modular Prosthetic Limb (MPL). You can read more about them in our A Complete Guide to Bionic Arms & Hands.

While advanced in many ways, both arms lacked a sense of touch. To address this, DARPA launched its Hand Proprioception and Touch Interfaces (HAPTIX) program in 2015. The goal of this program is to enable precision control of a bionic hand and sensory feedback via bi-directional nerve implants. In other words, true mind control and bionic touch.

One of the program participants is a biomechanical engineering team at the University of Utah. It’s objective was to upgrade the LUKE arm with these new capabilities:

The advantages of this approach are many. First, it helps amputees identify and manipulate objects much faster than they could without sensory feedback. It also seems to reduce phantom pain from the missing limb — a matter of great importance to amputees.

There are, however, drawbacks. One of these is the cost. Implanting electrodes in a residual limb is expensive. And at $100,000, the LUKE arm remains beyond the budget of most amputees. Most people will have to wait for this technology to trickle down into lower-cost prostheses.

Another drawback is portability. Until recently, the hand’s sensory feedback had to be channeled through a laptop, translated into the language of the brain, and then communicated to the target nerve bundle via the implanted electrodes. Scientists have come up with a portable device to handle the translation but it is still in field trials.

Finally, there are potential complications with implanting electrodes in the human body beyond just the basic medical risks of invasive surgery, such as infection. Scar tissue can form around electrodes or the electrodes can shift position — either of which may interfere with the reception or transmission of signals.

Integrum’s e-Opra System

As experimental techniques prove themselves and become commercialized, they tend to be incorporated into broader solutions for added convenience and reduced costs.

This seems to be the case with Integrum’s e-Opra system, which combines osseointegration with myoelectric control and sensory feedback into one implant solution:

Integrum e-Opra Implant System

We haven’t yet been able to find a patient who is willing to talk about their experiences with this new version of Integrum’s implant (the first patient in the U.S. isn’t scheduled for the procedure until spring of 2021). But we are thrilled to see this kind of combination of technologies because it suggests more routine use of them going forward. More use ultimately means more advancement.

We’ll return to update this section as soon as we get more information on patient outcomes.

Related Information

For more information on these types of embedded systems, see Advanced Neural Interfaces for Bionic Hands.

Sensory feedback is an important aspect of achieving advanced user control systems. For more information on control systems in general, see Bionic Hand & Control Systems.

For a comprehensive description of all current upper-limb technologies, devices, and research, see our complete guide.