New software developed by Carnegie Mellon University helps mobile robots deal efficiently with clutter, whether it is in the back of a refrigerator or on the surface of the moon.New Technologies 

The Human-Robot Virtuous Circle of Creativity


By Norman Rusin @normanrusin

Artificial intelligence (AI) has been applied to many tasks, from fighting poaching to preventing illegal logging to separating an Oreo cookie. Although seemingly more mundane, this last task is leading AI researchers to a key path toward creating robots that can help humans in their environment and daily lives, exploring a characteristic of highly developed animals: creativity. And new Carnegie Mellon University software is helping robots get creative.

Why Do We (and Robots) Need Creativity?

Tina Seelig, who teaches creativity and innovation at the Hasso Plattner Institute of Design at Stanford University, writes in her 2012 best seller inGenius, “Creativity allows you to thrive in an ever-changing world and unlocks a universe of possibilities. With enhanced creativity, instead of problems you see potential, instead of obstacles you see opportunities, and instead of challenges you see a chance to create breakthrough solutions … Without creativity we are not just condemned to a life of repetition, but to a life that slips backward.”

So far, robots have done a great job with “pick and place” (P&P) processes, picking up an object from a specified place and putting it down in another specified place. Siddhartha Srinivasa, associate professor of robotics at Carnegie Mellon University, says that this process has useful applications in places where clutter is not a problem, such as factory production lines. However, that is not what robots encounter when they land on distant planets or when “helpmate” robots eventually land in people’s homes. In fact, when people reach for a milk carton in a refrigerator, they do not necessarily move every other item out of the way. Rather, they might move an item or two while shoving others out of the way as the carton is pulled out. P&P-programmed robots simply do not work well in a world full of clutter. What they need is a “push and shove” (P&S) strategy to move around clutter.

Robots and Clutter: The Challenge and the Virtuous Circle

Clutter is a particular challenge for robots, but the new software helps robots find a balance between the two strategies based on their progress on a task, whether it is beating a path across the Moon or grabbing a milk jug from the back of the refrigerator. The robot is programmed to understand the basic physics of the world, so it has some idea of what things can be pushed, lifted, or stepped on. And it can learn how to pay attention to items that might be valuable or delicate, in case it must extricate a bull from a china shop.

The software not only assists robots in dealing efficiently with clutter, but also surprisingly reveals their creativity in solving problems. “It was exploiting sort of superhuman capabilities,” Srinivasa says of his lab’s two-armed mobile robot, the Home Exploring Robot Butler, or HERB. “The robot’s wrist has a 270-degree range, which led to behaviors we didn’t expect. Sometimes, we’re blinded by our own anthropomorphism.” Human wrist movement, by comparison, ranges from 20 to 90 degrees. In one case, the robot used the crook of its arm to cradle an object to be moved. “We never taught it that,” Srinivasa adds. Therefore, while the robot thinks creatively about its body, thanks to the new human-engineered software, it also allows humans to think about problems in different ways—forming a virtuous circle of creativity.

One limitation of this system is that once the robot has evaluated a situation and developed a plan to move an object, it completely “closes its eyes” to execute the plan. Work is underway to provide tactile and other feedback that can alert the robot to changes and miscalculations and can help it make corrections when necessary.

From the Kitchen to the Moon

This rearrangement planner software was developed in Srinivasa’s lab by Jennifer King, a PhD student in robotics at Carnegie Mellon University, and Marco Cognetti, a PhD student at Sapienza University of Rome, who both spent six months in Srinivasa’s lab. They presented their findings on May 19, 2016, at the IEEE International Conference on Robotics and Automation in Stockholm, Sweden.

In addition to HERB, the software was tested on NASA’s KRex (rover) robot, which is being designed to traverse the lunar surface. While HERB focused on the clutter typical of a home, KRex used the software to find traversable paths across an obstacle-filled landscape while pushing an object.

The Path of a Rising Robot Star

However, being a home helper or a Moon explorer is just not enough for HERB. It wants to be an actor. In fact, on its personal web page, it introduces itself as “CMU’s robotic butler, cookie separator & aspiring actor.” And it has already made its debut on stage, playing Bill in David Ives’s one-act comedy Sure Thing. But HERB is not alone in its creative endeavors. Last March, in fact, two short novels written with the help an artificial intelligence program passed the first screening process of a Japanese literary prize. Although neither of them won the award, they showed that robots were capable of telling stories almost like humans. Aristotle said that historians make us see what has happened, while poets make us see what could happen. The advent of creative robots, then, tremendously enhances human ability not only to find more breakthrough solutions to problems but also to see the world through new eyes.


Seelig, Tina. inGenius: A crash course on creativity. New York: HarperOne, 2012.

About the Author

Norman Rusin is a freelance journalist and copy editor who helps writers produce sound and telling communications. He is completing his PhD in Italian Studies at the University of Pennsylvania. In his research he looks at how science, literature, and art complement each other. Connect with him at [email protected]. translates complex research findings into accessible insights on science, nature, and technology. Help keep GotScience free: Donate or visit our gift shop. For more science news subscribe to our weekly digest.


Recommended for You