For Smarter Robots, Just Add Humans

Autonomous machines are still too clumsy for delicate tasks. But humans can operate mechanical arms from afar, turning physical labor into remote work.

Courtesy of Sanctuary AI

Teleoperating a physical robot could become an important job in future, according to Sanctuary AI, based in Vancouver, Canada. The company also believes that this might provide a way to train robots how to perform tasks that are currently well out of their (mechanical) reach, and imbue machines with a physical sense of the world some argue is needed to unlock human-level artificial intelligence.

Industrial robots are powerful, precise, and mostly stubbornly stupid. They cannot apply the kind of precision and responsiveness needed to perform delicate manipulation tasks. That’s partly why the use of robots in factories is still relatively limited, and still requires an army of human workers to assemble all the fiddly bits into the guts of iPhones. 

But when such work is nothing for humans, why not forgo the complexity of trying to design an algorithm to do the job?

Here’s one of Sanctuary’s robots—the top half of a humanoid—doing a range of sophisticated manipulation tasks. Offscreen, a human wearing a VR headset and sensor-laden gloves is operating the robot remotely.

Sanctuary recently ran what it calls the first “real world” test of one of its robots, by having a humanoid like this one work in a store not far from the startup’s headquarters. The company believes that making it possible to do physical work remotely could help address the labor shortages that many companies are seeing today.

Some robots already get some remote assistance from humans when they get stuck, as I’ve written about. The limits of AI mean that robots working in restaurants, offices, and on the street as delivery mules are flummoxed by unusual situations. The difficulty of pulling off fully autonomous driving, for example, means that some firms are working to put remotely piloted trucks on the roads. 

Sanctuary’s founders, Geordie Rose and Suzanne Gilbert, ran Kindred, another company doing robotic teleoperation that was acquired in 2020 by Ocado, a UK supermarket firm that uses automation extensively. In this video the pair talk about the company’s history and plans for the future.  

The aim is ultimately to use data from humans teleoperating the robots to teach algorithms to do more tasks autonomously. Gilbert, Sanctuary’s CTO, believes that achieving humanlike intelligence in machines will require them to interact with and learn from the physical world. (Sorry, ChatGPT.)

OpenAI, the company behind ChatgGPT, is also taking an interest in teleoperated humanoids. It is leading a $23.5 million investment in 1X, a startup developing a human-like robot. “The OpenAI Startup Fund believes in the approach and impact that 1X can have on the future of work,” Brad Lightcap, OpenAI’s COO and manager of the OpenAI Startup Fund says.

The ALOHA teleoperation system.Courtesy of Tony Zhao/UC Berkeley

For humans to help robots with teleoperation, AI might also need to be developed to ease the collaboration between person and machine. Chelsea Finn, an assistant professor at UC Berkeley, recently shared details of a fascinating research project that involves using machine learning to allow cheap teleoperated robot arms to work smoothly and accurately. The technology may make it easier for humans to operate robots remotely for more situations.

I don’t think I’d much enjoy teleoperating a robot all day—especially if I knew that robot would someday turn around and kick me out the door. But it might make working from home a possibility for more people, and also make certain types of job more widely accessible. Alternatively, we may have just gotten a glimpse of a potentially dystopian future of the workplace.

For Smarter Robots, Just Add Humans

(May require free registration to view)