Avatars in Space
For those who haven't seen the world box-office-record-holding film, it is the tale of a wounded war veteran, Jake, who travels to a mining colony on the distant moon Pandora. Jake's goal is to aid in negotiations with the native race of humanoid Na'vi who call Pandora home. The air on Pandora is toxic to humans, and most of the Earthlings on Pandora are forced to live in habitats or carry their oxygen with them through the jungles that thrive on the small moon. Jake, however, is part of the Avatar Program, where people are able to mentally link with genetically engineered human-Na'vi bodies. By doing so, Jake and his fellow teammates 'inhabit' the artificial bodies and are able to explore and interact with the environment and people of Pandora as if they themselves were exploring.
The story of Avatar is a fantastic portrayal of science fiction fantasy, but the concepts behind exploring environments through avatars is not a foreign concept to real-life scientists. NASA has long examined technologies like virtual reality and telerobotics to allow human beings to remotely explore from the safety of Earth, space stations or habitats. As early as 1961, NASA provided research grants to fund work at the Massachusetts Institute of Technology (MIT) exploring remote manipulation of robots for exploration purposes. The official NASA Telerobotics Program was closed in 1997, yet the research is still being carried out through more specific mission-related research at NASA and other institutions around the world.
From robotic to virtual exploration
The Mars Exploration Rovers (MER) are a prime example of how remote-controlled robots can expand our ability to explore beyond the Earth without putting humans at risk. The actions of the MER rovers are decided and controlled by a team of scientists on Earth. They are able to see the martian environment through the eyes of the rovers, determine the best routes for travel and objects for scientific investigations, and then send their commands back to the rovers. Spaced-based telescopes, such as Hubble, are also controlled from Earth through telerobotics. Even the International Space Station has a telerobotic, two-armed manipulator named Dextre. These robots, however, are a far cry from true avatars that allow the human user to truly 'experience' the environment. This is where virtual reality technologies come into play.
Ideas for digitally displaying information are as old as computers themselves. But it wasn't until computer technology matured that 'virtual reality' became a concept that scientists could experiment with.
The potential benefits of using technologies like VIEW to safely interact with a virtual world were immediately recognized by NASA. Beyond the comforts of Earth's atmosphere, human explorers face dangerous and unpredictable situations. Humans survive on a delicate balance of environmental conditions, and providing things like oxygen and food mean that many of the resources used for human missions are spent simply keeping astronauts alive. The men and women that brave orbit are under constant threat from unpredictable factors like exposure to radiation, or debris and meteorites damaging vital life-support equipment. The dangers are greater when traveling to distant locations like Mars to explore the surface on foot. The farther martian explorers travel from their habitat, the more dangerous their exploration efforts become. This is why humans are typically preceded by robots, which are less of a liability if things go wrong.
Today, NASA uses 3D and virtual technologies for a number of public outreach and education projects. The technology can also be used for training purposes, allowing an astronaut to practice, say, walking on the Moon. However, the technologies are not yet used to fully connect the user with a virtual representation of a true-life environment in real time. NASA is still progressing toward this goal.
Real time interactions
The ultimate idea would be to allow a human explorer based on Earth, or in the relative safety of a space station or habitat, to actually experience exploration of a distant location without leaving the comfort of home. If the technology can be tied to robotic 'avatars' on a planetary surface in real-time, the user would not simply experience a simulation of the world – but could directly participate in exploration and science as if they were there.
In 2001, researchers at NASA Ames demonstrated that a Boeing 747 passenger jet could be controlled using human muscle-nerve signals sent to a computer. In a simulation, the human pilot was able to land a jet at San Francisco International Airport by simply gesturing with his hands. Aspects such as touch-sensitive controls that allow the user to experience the 'feeling' of handling the airplane or a robot is essential to the success of this kind of work.
Closer to the exploration front, similar technologies are also being used in NASA's most avatar-like experiment of all – the Robonaut. According to researchers on the project, "Robonaut systems are the first humanoids specifically designed for space."
In 2004, the second generation of Robonaut gained mobility when engineers attached its body to a Segway Robotic Mobility Platform (RMP) commissioned by DARPA. Using virtual reality instruments, a human operator was immersed in the Robonaut's actual environment and was able to perform remote operations.
When reporting the results of this experiment, the research team commented that, "It is surprising how easily Robonaut is teleoperated. It was initially thought that the teleoperator might suffer from control overload, but this has not been the case."
Human operators successfully used the robot to navigate hallways and work with tools designed for humans to perform tasks like cutting wires.
The mobility and dexterity of robots still has a long way to go in terms of truly simulating the tasks that a human can perform. Additionally, operating a robot that can move like you doesn't provide a completely 'immersive' experience. In order to bring the human operator's 'senses' along for the ride, additional methods must be used.
According to researchers on Robonaut, "As the project matures with increased feedback to the human operator, the Robonaut system will approach the handling and manipulation capabilities of a suited astronaut."
This is where inventions like 'haptic technology' come into play. 'Haptic' refers to our sense of touch, and 'haptic technology' uses sensory feedback to recreate the sense of touch for users. A simple example is modern videogames where the controller vibrates in response to actions on the screen. With more advanced technology, a user might wear gloves that allow them to 'feel' objects in a virtual world. You could examine the texture and weight of rocks, or even experience the crunch of martian sand between your toes.
The technologies being developed by NASA and other institutions are not only useful for human beings exploring dangerous and distant environments. There are plenty of ways in which avatar-like virtual experiences can be used to benefit and even save lives here on Earth. Teleoperated robots could one day be exploring disaster sites to search for survivors of earthquakes. Such robots could also aid in mining operations in locations that are risky for humans to enter – or on asteroids and other planets where humans would need special equipment to survive.
This technology has also made the transition into the commercial realm with huge benefits for medicine. Companies like Intuitive Surgical are using robotic surgical equipment that interfaces with human surgeons through 'virtual' methods like 3D animation and haptics. With this technology, surgeries that would normally require cutting open a patient's body to provide access for a surgeon's hands can now be done through tiny incisions. The surgeon can 'virtually' experience the interior regions of the body and manipulate small surgical instruments with the aid of robotics.
Real-time interaction through technology can also help bring doctors together with patients and students all over the world. Based at Stanford University, the HavNet project (Haptic Audiovisual Network for Education and Training) is networking students and educators to aid in health education and medical research. HavNet projects like the Clinical Skills Testbed use haptic technology to help teach surgical skills. Multiple students can control models of surgical tools and literally 'feel' what it's like to perform an operation. The project is also developing a 'remote tacticle sensor', which can be mounted on a robotic arm. The device is able to distinguish between different skin types and textures, transmitting this information to a remote user.
The concepts presented in James Cameron's Avatar go well beyond the limits of modern technologies. We're not going to be growing any biological avatars for human explorers in the lab – but modern robotics are getting close to providing a 'human' experience through increased dexterity and mobility. With the right technology, robotic avatars could allow humans to fully experience the environment of other worlds. Astronauts on Mars could watch the sunrise over the rusty, red horizon and feel the wind brush across their cheek – without having to experience suffocation, the icy death of -200°C on their skin or the sting of microscopic dust in their eyes.
These technologies are already providing benefits in orbit and for doctors and patients on Earth. Nobody knows for sure when humans will travel to locations like Mars, but as telerobotic technologies develop, robotic explorers may begin to bring us 'virtually' closer to these worlds before we actually set foot on them ourselves.
Even though NASA and others have come a long way in developing avatars, the technology still has a long way to go before we're having adventures on Pandora-like planets. Perhaps more advanced civilizations on distant worlds have developed avatars just as good as those in the movie.
Ambrose, R.O., Aldridge, H., Askew, R.S., Burridge, R.R., Bluethmann, W., Diftler, M., Lovchik, C., Magruder, D., Rehnmark, F. 2000. Robonaut: NASA's Space Humanoid. Intelligent Systems, 15(4), 57-63.
Goza, S.M., Ambrose, R.O., Diftler, M.A. 2004. Telepresence Control of the NASA/DARPA Robonaut on a Mobility Platform, CHI, 6(1), 623-629.
Diftler, M.A., Ambrose, R.O., Goza, S.M., Tyree, K.S., Huber, E.L. 2005. Robonaut Mobile Autonomy: Initial Experiments, Proceedings of the IEEE International Conference on Robotics and Automation.