The first human-sized, five-fingered robotic hand also has a brain: it has the cognitive power to learn to grasp and manipulate a range of delicate and oddly shaped objects just as humans do. A time will come when robots will be as ubiquitous in homes as personal computers, Bill Gates said in his January 2007 article in Scientific American. Camera and infrared sensors on new video game consoles hint at a future when we will be able to control devices just by speaking or hand gestures, and self-propelled vacuum cleaners already navigate around homes. But one challenge the robotics community struggles to solve is a robot’s ability to manipulate. "We are not talking about grippers," insists professor Bruno Siciliano of the University of Naples, Italy. "Industry robots are pretty good at picking things up and putting them down. If robots are to really transform life and work alongside humans, then they must be able to handle items just as humans can with their hands. Manipulation is a complex task that robots must learn to master." As coordinator of the Dexmart project (Dexterous and autonomous dual-arm/hand robotic manipulation with smart sensory-motor skills: A bridge from natural to artificial cognition), Siciliano led a collaborative team of robotics researchers from Saarland University and the University of Bologna in Italy.The anthropomorphic five-fingered robotic hand they built can handle eggs, pick up and turn around a credit card or take a pen from another person. Their creation, dubbed the Dexmart Hand, involves optoelectronic sensors, polymer strings, high-speed motors and image-processing algorithms. Working together, the actuator system allows the hand to touch diverse objects, grasp and lift them and place them gently in a new position. The hand was able to lift an 11-lb load by over an inch within a split second using the strings. A robotic hand gently holds an egg. © Markus Breig Each finger is controlled by polymer "tendons" that mimic human tendons. “The capability of the robotic hand is so near to that of humans that the vision of robots as personal assistants in the household, in the operating room as well as in industrial settings, is becoming more realistic,” said Chris May, a scientist at Saarland University’s Laboratory of Actuation Technology. In addition to manipulation, robots must be able to behave autonomously, Siciliano said, responding to situations in appropriate ways that are not preprogrammed. The cognitive power was supplied by scientists from Karlsruhe University, Second University of Naples and UK technology firm OMG. By sticking visual trackers on a special sensorized glove and performing manipulations that were captured on video, they used advanced image processing technology to study the fine details of human hand movements. The image processing algorithms created rules on how the robotic hand should handle and manipulate similar objects. Coordinating the movements of five fingers, and their respective joints, was a challenge. “If you look at each joint and what each motor would have to do individually, you are dealing with 20 degrees of freedom and that is too complex,” Siciliano said. His group took its inspiration from biology to simplify the problem. “Research into human hand control and coordination has shown that we don’t control each joint separately; our brain controls all the joints at the same time in a coordinated fashion,” said Fanny Ficuciello of the Naples team. “Neuroscientists have demonstrated that human hand positions and movements can actually be simplified to just three so-called ‘postural synergies.’ These three synergies can describe about 80 percent of all possible grasping actions and positions.” The Naples team's control system takes input from optoelectronic sensors on the hand (which measure the grasp pressure), working out the synergies and actuating the finger movements. The grip precision that these three create permits extremely refined manipulations. The optoelectronic sensors are another innovation: several measure joint angles, tendon forces and tactile interactions with objects. Within the tactile sensor, advanced computational analysis of the captured light intensity inside a number of sensitive elements makes it possible to calculate the forces exerted on the object by the fingers and also whether the object is slipping out of grasp. Its developers have filed for a European patent. A prototype hand and two fully sensorized fingers ran without error for four days last December at RobotVille, a European exhibit of 20 robots at the Science Museum in London, and successfully took a range of objects out of visitors’ hands. The consortium is exploring the commercial viability of producing fully functional hands initially for the academic community. For more information, visit: www.uni-saarland.de