APL Engineers Deploy "Guitar Hero" to Help Patients Test Prosthetic Arm
"Wii-habilitation"—using Nintendo's Wii video game system in rehabilitation therapy—continues to be popular with health care workers looking to help patients get through what some see as the pain and torture of physical therapy. Now two engineers in APL's National Security Technology Department have cranked that concept up a notch. Bobby Armiger and Jacob Vogelstein rewired Nintendo's "Guitar Hero III: Legends of Rock" game to allow amputees to rock out with the game and get valuable training with prosthetic prototypes at the same time.
Their gaming is part of the APL-led Revolutionizing Prosthetics 2009 (RP 2009) effort, funded by the Defense Advanced Research Projects Agency, to develop a prosthetic arm that will be controlled, feel, look, and perform like a natural limb. So far the project has produced two prototypes. It has also leveraged a surgical technique developed at the Rehabilitation Institute of Chicago by Dr. Todd Kuiken that reroutes the nerves that once controlled an amputee's arm to remaining muscles. These "reinnervated" muscles naturally amplify the nerve signals so that electrodes placed against the skin can detect activity and control the prosthetic arm.
The surgery has enabled patients to control the first prototype's 7 degrees of freedom and will eventually be used to control individual fingers of the final prosthetic design. But for the prosthetic arm to operate correctly using these rerouted nerves, the system must learn to interpret the patient's muscle signals. (The process is similar to training voice-recognition software where, for example, you are prompted to read the Gettysburg address, except that instead of analyzing tones, this system records and classifies muscle twitches.)
This training takes place in a Virtual Integration Environment (VIE), another RP 2009 innovation. "In the VIE, an animated on-screen arm mimics the patient's intended movements in real time, based on inputs from the electrodes attached to the user's residual muscles," Vogelstein explains. "For the training, a patient sits in front of a computer and an on-screen prompt tells them to ‘flex your wrist,' ‘extend your wrist,' ‘close your fist,' etc."
Calibrating the mechanical limbs to correctly recognize and respond properly to electrical signals in an amputee's residual muscles is an exhausting and draining process, says Armiger, who has been on the front lines of training patients. "There is no real interaction and no feedback," he said. "And there is no encouragement for the patient to do more or do it better."
The idea to adapt the game for amputees came to Armiger while he was playing Guitar Hero at a party. He and Vogelstein borrowed a colleague's copy of Guitar Hero and modified the controller with a soldering iron to allow it to be controlled by the VIE. Then button clicks were substituted with muscle contraction signals as picked up by the electrodes.
Last fall, Armiger and Vogelstein traveled to RP 2009 partner Duke University to test the rigged system on Iraq War veteran Jonathan Kuniholm, who lost his right hand to shrapnel in 2005. With electrodes attached to what was left of his arm, Kuniholm was able to operate the frets using signals from his muscles.
The researchers found that playing a game is a far more intuitive way to speed up the tedious calibration process and make it more fun for volunteers. "It allows for large numbers of rapid, dynamic movements that are more natural and not stereotyped," Vogelstein says. "Ultimately, we are going to get more input and longer training cycles out of patients—and that will translate to better and more natural limb control."