An actor robot runs on a playpen trying to catch the visible green food, while...
An actor robot runs on a playpen trying to catch the visible green food, while an observer machine learns to predict the actor robot's behavior purely through visual observations. Although the observer can always see the green foods, the actor, from its own perspective, cannot due to occlusions.
Source: Creative Machines Lab/Columbia Engineering

Robot displays a glimmer of empathy to partner robot

Columbia engineers have created a robot that learns to visually predict how its partner robot will behave, displaying a glimmer of empathy. This “Robot Theory of Mind” could help robots get along with other robots—and humans—more intuitively.

Like a longtime couple who can predict each other’s every move, a Columbia Engineering robot has learned to predict its partner robot’s future actions and goals based on just a few initial video frames.

When two primates are cooped up together for a long time, we quickly learn to predict the near-term actions of our roommates, co-workers or family members. Our ability to anticipate the actions of others makes it easier for us to successfully live and work together. In contrast, even the most intelligent and advanced robots have remained notoriously inept at this sort of social communication. This may be about to change.

The study, conducted at Columbia Engineering’s Creative Machines Lab led by Mechanical Engineering Professor Hod Lipson, is part of a broader effort to endow robots with the ability to understand and anticipate the goals of other robots, purely from visual observations.

The researchers first built a robot and placed it in a playpen roughly 3x2 feet in size. They programmed the robot to seek and move towards any green circle it could see. But there was a catch: Sometimes the robot could see a green circle in its camera and move directly towards it. But other times, the green circle would be occluded by a tall red carboard box, in which case the robot would move towards a different green circle, or not at all.

After observing its partner puttering around for two hours, the observing robot began to anticipate its partner’s goal and path. The observing robot was eventually able to predict its partner’s goal and path 98 out of 100 times, across varying situations—without being told explicitly about the partner’s visibility handicap.

“Our initial results are very exciting,” says Boyuan Chen, lead author of the study, which was conducted in collaboration with Carl Vondrick, assistant professor of computer science. “Our findings begin to demonstrate how robots can see the world from another robot’s perspective. The ability of the observer to put itself in its partner’s shoes, so to speak, and understand, without being guided, whether its partner could or could not see the green circle from its vantage point, is perhaps a primitive form of empathy.”

When they designed the experiment, the researchers expected that the Observer Robot would learn to make predictions about the Subject Robot’s near-term actions. What the researchers didn’t expect, however, was how accurately the Observer Robot could foresee its colleague’s future “moves” with only a few seconds of video as a cue.

The researchers acknowledge that the behaviors exhibited by the robot in this study are far simpler than the behaviors and goals of humans. They believe, however, that this may be the beginning of endowing robots with what cognitive scientists call “Theory of Mind” (ToM). At about age three, children begin to understand that others may have different goals, needs and perspectives than they do. This can lead to playful activities such as hide and seek, as well as more sophisticated manipulations like lying. More broadly, ToM is recognized as a key distinguishing hallmark of human and primate cognition, and a factor that is essential for complex and adaptive social interactions such as cooperation, competition, empathy, and deception.

In addition, humans are still better than robots at describing their predictions using verbal language. The researchers had the observing robot make its predictions in the form of images, rather than words, in order to avoid becoming entangled in the thorny challenges of human language. Yet, Lipson speculates, the ability of a robot to predict the future actions visually is not unique: “We humans also think visually sometimes. We frequently imagine the future in our mind’s eyes, not in words.”

Lipson acknowledges that there are many ethical questions. The technology will make robots more resilient and useful, but when robots can anticipate how humans think, they may also learn to manipulate those thoughts.

“We recognize that robots aren’t going to remain passive instruction-following machines for long,” Lipson says. “Like other forms of advanced AI, we hope that policymakers can help keep this kind of technology in check, so that we can all benefit.”

The research was published Nature Scientific Reports.

Subscribe to our newsletter

Related articles

A robot scientist is ready for drug discovery

A robot scientist is ready for drug discovery

The robot scientist Eve has been assembled and is now operating at Chalmers University of Technology. Eve’s f​irst mission is to identify and test drugs against Covid-19.​

Using drones to elicit emotional responses

Using drones to elicit emotional responses

Researchers have conducted the first studies examining how people respond to various emotional facial expressions depicted on a drone.

Robot touch makes you feel better

Robot touch makes you feel better

People who were touched by a humanoid robot while conversing with it subsequently reported a better emotional state and were more likely to comply with a request from the robot.

Xenobots 2.0: The next generation of living robots

Xenobots 2.0: The next generation of living robots

Researchers have created life forms that self-assemble a body from single cells and do not require muscle cells to move. They're faster, live longer, and can now record information.

Robots can draw out reluctant participants in groups

Robots can draw out reluctant participants in groups

Can a robot draw a response simply by making “eye” contact, even with people who are less inclined to speak up. A recent study suggests that it can.

Most patients wouldn't mind a 'robotic doc'

Most patients wouldn't mind a 'robotic doc'

A study finds patients are receptive to interacting with robots designed to evaluate symptoms in a contact-free way.

An ear-bot 'hears' through the ear of a locust

An ear-bot 'hears' through the ear of a locust

For the first time, the ear of a dead locust was connected to a robot that receives the ear’s electrical signals and responds accordingly.

How do people react to appearance of robots?

How do people react to appearance of robots?

'Moralities of Intelligent Machines' is a project that investigates people’s attitudes towards moral choices made by artificial intelligence.

Artificial skin brings robots closer to 'touching' human lives

Artificial skin brings robots closer to 'touching' human lives

Researchers have constructed a 3D vision-guided artificial skin that enables tactile sensing with high performance, opening doors to innumerable applications in medicine.

Popular articles

Photo

The “RoboWig” untangle your hair

Nurses typically spend 18 to 40 percent of their time performing direct patient care tasks, oftentimes for many patients and with little time to spare. Personal care robots that brush your hair could provide substantial help and relief.

Subscribe to Newsletter