The ultimate goal of bionic hands is to replicate natural abilities. For this to happen, bionic hands need to provide sensory feedback to end-users. This is not only a matter of connecting users to their surroundings. It is also critical for perfecting user control.
Sorting Out the Different Types of Feedback
Not all of the feedback available to bionic hand users matches the traditional definition of sensory input. Osseointegration and the Agonist-Antagonist Myoneural Interface (AMI) provide feedback that may be just as important.
However, in this article, we’re going to focus mainly on the traditional forms of feedback so that users who have not undergone osseointegration or AMI may clearly understand their sensory options.
A Daunting Task
The human sense of touch is remarkable. There are more than 3,000 touch receptors on a human fingertip. We can feel not only the size and shape of an object but also its texture, pliability, temperature, and more. Even blindfolded, most of us can identify common objects through touch alone.
We do not just gather this information solely to satisfy our curiosity. When it comes to using our hands, sensory feedback plays a crucial role in guiding our movements. For example, it helps us manipulate an object into the ideal position for a specific task. This is possible because the brain’s sensory cortex and motor cortex (responsible for movement) share an intimate relationship:
Without these capabilities, users of bionic hands are forced to visually guide every nuance of hand movement — a process that is often slow, frustrating, and tiring.
Unfortunately, it’s going to take a while for bionic hands to provide the same level of sensory feedback as their natural counterparts. First, we’ll need to take some baby steps.
The Use of Simple Haptic Feedback in Commercial Bionic Hands
In September 2019, Psyonic entered the bionic hand market with its Ability Hand. They took the same sensors found in modern cell phones and repurposed them as highly sensitive touch sensors, which they placed in the Ability Hand’s fingertips.
When the fingers touch an object, they send a signal that triggers vibrations in the socket on the residual limb, informing the user of both contact and pressure. This makes the use of the Ability Hand more intuitive, as demonstrated in this short video of a user picking up an eggshell, while blindfolded, without crushing it:
This doesn’t replicate natural touch but it’s an important leap forward in practical functionality. And at $10 for the sensors, no one can argue with its cost-effectiveness!
The Vincent Evolution offers similar capabilities and others are sure to follow. For example, Robo Bionics has incorporated sensory feedback into its Grippy bionic hand, which is already being sold through clinics in India. We believe that this is again just contact and pressure feedback relayed through vibration but will confirm when we write our main article on Grippy in a few weeks.
The C-Hand, from Mexico’s Human Assistive Technologies (HAT), will provide sensory feedback for contact, pressure, and temperature. We have not yet confirmed how this feedback will be conveyed to patients, but we suspect that it will again use vibration for at least the contact/pressure sensations and will not require any kind of surgery. Already in trials, we expect the C-Hand to enter the commercial market in the next year or two.
The one element common to these examples is that the sensory feedback coming from the bionic hands is relatively simple and must be consciously interpreted by the brain. That is, when the prosthetic socket vibrates, it is up to the brain to equate that vibration to making contact with an object, or to interpret the strength of the vibration as the amount of force being applied to the object.
The user doesn’t actually feel the object; they just feel the vibration.
More Advanced Uses of Sensors in Commercial Bionic Hands
Work on the Modular Prosthetic Limb (MPL) — DARPA’s lead project for adding sensory feedback to bionic hands — began in 2009. The first commercial version of the MPL, the Atom Touch, will hit the market in 2024. With more than 200 sensors, it will provide users with four types of sensory data: contact, force, position, and velocity. To perceive this data, users will currently have to undergo Targeted Sensory Reinnervation (TSR) surgery. However, the company is also working on the ability to convey contact, force, and heat through the skin without surgery by using transcutaneous electrical nerve stimulation (TENS).
What is the difference between TENS versus simple haptic feedback? A TENS system uses an electrical charge delivered through the skin to directly stimulate a nerve and trick the brain into thinking it has felt something. There is no conscious interpretation.
The challenge with TENS is that the feedback system is attempting to stimulate the nerve through the skin and other intervening tissue. The easiest way to overcome this challenge is to directly interact with the nerve through what is called a neural interface.
Neural Interface
In a neural interface, electrodes are surgically implanted in the residual limb to either wrap around a nerve bundle or to intersect it. When sensors on a bionic hand detect sensory input, this input is communicated to a control system, which in turn uses an electrical charge to directly stimulate the nerve, again to trick the brain into feeling the appropriate sensation. For example, if a bionic finger encounters a sharp object, the brain may be tricked into feeling pain. Please see our article on Advanced Neural Interfaces for Bionic Hands for a more detailed look at this technology.
The part we want you to see is the net effect of using a neural interface with a bionic hand, as shown in this video:
As you can see, this is getting pretty close to what most of us envision when we think of sensory feedback.
The problem with neural interfaces is that they involve surgery, which is not only expensive but also risks infection and scarring. The latter is particularly problematic because scar tissue may degrade communication between the electrode and the nerve over time.
But what if you were going to have surgery anyway?
Integrum’s e-Opra System
This section does involve osseointegration but it’s a special version that includes the creation of a neural interface:
The label “Cuff e.” near the bottom center of the diagram shows an electrode cuff attached to the nerve. As with other neural interfaces, an electrical charge is applied to the nerve through the cuff to communicate with the brain.
The appeal of this approach is that, since the patient is undergoing major surgery anyway, adding the neural interface does not incur all of the drawbacks and risks of a separate surgery. This changes the cost/benefit analysis of this form of sensory feedback.
Note, the e-Opra Implant System is not yet available commercially.
E-Dermis Electronic Skin
Whether we perfect TENS or neural interfaces as the ideal way of stimulating a nerve, sooner or later we are going to need the ability to obtain more sophisticated sensory input so that we can eventually sense things like shape and even texture. This will require electronic skin.
John Hopkins University (JHU) has created an electronic skin called “e-dermis”. This video from freethink.com provides an excellent overview:
This obviously represents a giant technological leap over the use of simple pressure sensors, and e-dermis is not alone. New electronic skins are popping up all over the world, some with more advanced capabilities than e-dermis. To read about these, see our article on electronic skin.
Conclusion
As you can see from the preceding walkthrough of simple haptic feedback to TENS to neural interfaces to electronic skin, we are inching ever closer to restoring natural sensory capabilities to bionic hand users. All the pieces of the puzzle are now present. We just need to find the best way to integrate them.
Related Information
Sensory feedback is only one aspect of a deeper connection between a bionic limb and its user. To get the whole picture, see Understanding Bionic Touch.
For more information on upper-limb bionic control systems in general, see Bionic Arm & Hand Control Systems.
For a comprehensive description of all current upper-limb technologies, devices, and research, see our complete guide.