When we think 28﻿<img src="https://media.npr.org/assets/img/2020/07/19/gettyimages-522033558_custom-08572bc4bdca7ac4ac244181c96265b902d8f1b1-s1100.jpg">

When we think of the <a href="https://en.wikipedia.org/wiki/Skynet_(Terminator)">Skynet </a>scenario, videos of increasingly nimble anthropomorphic machines from Boston Dynamics awe, or terrify, us. Just a couple years ago these humanoids developed an ability to get up when we knocked them down. Then, they began parkour-ing with finesse around our man-made obstacles. Now, they’re elegantly out-performing most humans in gymnastics. It all portrays a deceiving story of the rapid evolution of intelligence in robots. But in reality, these robots are still far from possessing the intelligence to fold our laundry, let alone become our overlords.

Baba

When we think of the Skynet scenario, videos of increasingly nimble anthropomorphic machines from Boston Dynamics awe, or terrify, us. Just a couple years ago these humanoids developed an ability to…

sciencefiction

Science Fiction

technologyandthefuture

Technology & The Future

artificialintelligence

Artificial Intelligence

When we think of the <a href="https://en.wikipedia.org/wiki/Skynet_(Terminator)">Skynet </a>scenario, videos of increasingly nimble anthropomorphic machines from Boston Dynamics awe, or terrify, us. Just a couple years ago these humanoids developed an ability to get up when we knocked them down. Then, they began parkour-ing with finesse around our man-made obstacles. Now, they’re elegantly out-performing most humans in gymnastics. It all portrays a deceiving story of the rapid evolution of intelligence in robots. But in reality, these robots are still far from possessing the intelligence to fold our laundry, let alone become our overlords.

There’s a lot of hype playing into the robot takeover narrative. The purpose of this blog post is to present some exciting breakthroughs in robotics research while debunking fact from fiction. Robots have been around and widely used in manufacturing since the 1960’s. While we’ve been calling them robots for all this time, a more fitting name would be ‘reprogrammable motion machines’. They are explicitly programmed to repeat trajectories exactly the same way, every time. They lack the intelligence to self-adapt if their environment or task changes even the slightest.

Now fast forward to today — almost nothing has changed. Nearly all deployed robot arms are still not intelligent and are confined to highly structured manufacturing environments. Yet, if we look beyond the walls of factories, the world is full of monotonous tasks across logistics, delivery, farming, construction, and transportation which are prime for automation. The reason many of these labor intensive jobs are not yet automated is because they inherently bear an enormous amount of variability — the achilles heel of robots.

One example of a repetitive yet enormously varying task is ‘picking and packing’ in eCommerce fulfillment centers. This job requires proper handling of millions of different products — all with varying sizes, shapes, weights, colors, textures, stiffnesses and fragilities. There isn’t a one-grasp-fits-all solution to handle any object. As humans, we take our innate ability to grasp, assemble, disassemble, reorient, fold, pack and generally manipulate any object for granted. For robots, this is very hard.

In addition to lacking general intelligence, robots are still very expensive. The mainstream arms from UR, Kuka, Franka, Yaskawa, Fanuc, and ABB start at $20k and can easily cost over $100k. The inability to handle variability along with a high price tag makes it difficult to justify the economics of most robotic applications — this is the reason many robotic startups fail. If you replace a burger flipper in a fast-food restaurant with a robot, you are not replacing one employee. You’re replacing a lot less. In one minute a person can be flipping burgers. When they’re not flipping burgers they can be making fries, wiping tables, cleaning bathrooms or taking orders. Replacing a small fraction of a minimum wage employee is not financially compelling, especially given the cost and practical complexity of implementing such a piece of technology. A lot of robot applications suffer from this dilemma.

If your goal is to build a valuable robot application and deploy it successfully in the real world today, then my recommendation is to consider the following: 1. The price you charge your customer should be a fraction of the total cost of labor you are replacing over some reasonable period of time (usually no more than 2 years). Alternatively, the demand for the labor your robot can perform should be extremely high, to the point where the pool of available human labor is not willing or able to provide the total labor needed.

2. Robots aren’t people. Retro-fitting an environment designed for humans with robots will ALWAYS be less optimal, in the long-run, than designing the environment around the robot’s capabilities. Robots love structure, so give them as much structure as possible so long as it does not impose unreasonable cost or additional labor on the customer. In the same vein, we should not design robots to exactly mimic things we find in nature. Just because humans do things a certain way doesn’t mean there isn’t a simpler, more optimal solution enabled by modern engineering. See Rubik’s cube example…

3. Build a solution that works really well for one task. In most applications the customer expects a solution, not a piece of the solution they then need to stitch together with other technologies to make an actual solution. In addition, this solution is usually expected to work 99.9% of the time. As good as 95% success (in a simplified lab scenario) might sound in academia, that won’t cut it in industry. Following the standard learning curve and Pareto principle, the last 5% is the hardest to obtain and almost always comes down to engineering edge cases versus fundamental research. Focus on one <a href="https://en.wikipedia.org/wiki/Whole_product">whole product </a>and deliver it super reliably.

4. Choose a task that robots are capable of reliably performing within a few years . If you can’t fully automate the task with 99.9% reliability today, then narrow the scope of the task, add more structure, or use teleoperators to handle edges cases. As long as the teleoperator-to-robot ratio is economical, teleoperators can give robots the dexterity they need today while helping train them to become increasingly intelligent and fully autonomous over time. <a href="https://nimble.ai/">Nimble </a>, <a href="https://phantom.auto/">Phantom Auto </a>, <a href="https://www.kiwicampus.com/technology">Kiwi </a>, and others do this.

Boston Dynamics, the most advanced robotics company in the world, has struggled to successfully commercialize their robots because of their high cost and unclear value. While their videos amaze us all, at the end of the day two metrics, cost and value, determine the adoption of any technology. Robotics — no matter how cool — are no different.

This blog post discusses the progress being made to improve robot’s cost:value ratio (what I’ll call the <a href="https://www.youtube.com/watch?v=1pphyvgd7-k">Jetsons </a>ratio) by orders of magnitude so that we can continue to deploy more robots in the real world in increasingly more challenging and useful tasks. At the heart of the currently terrible Jetsons ratio is a classic chicken and egg problem. Robots are expensive because they aren’t yet mass produced. They aren’t yet mass produced because they don’t yet offer real value to the common consumer. They don’t yet offer value to the common consumer because they aren’t yet intelligent. They aren’t yet intelligent because there isn’t a large-scale dataset on which to train them. There isn’t a large-scale dataset because robots aren’t mass produced. We’re back to the beginning.

However, there is an out to this vicious cycle and it’s through riding the success waves of the commercial drone industry and deep learning research. Recent market factors like the explosion of the consumer drone and scooter industries have justified the mass production of robotics-grade motors and electronics. While not specifically designed for use in articulated robot limbs, these components have catalyzed the emergence of a new generation of low-cost robots.

Research from the legged robot community produced a new low-cost but high-performance actuator called the <a href="https://www.ri.cmu.edu/pub_files/2016/8/kaloucheThesis.pdf">quasi-direct-drive </a>actuator. Robots like <a href="https://www.ri.cmu.edu/pub_files/2016/8/kaloucheThesis.pdf">GOAT </a>, <a href="https://repository.upenn.edu/cgi/viewcontent.cgi?article=1766&context=ese_papers">Minitaur </a>, and<a href="https://dspace.mit.edu/handle/1721.1/118671">MIT’s Mini Cheetah </a>have been designed to balance the inherent trade-offs between force control (required for safe interaction with humans), high torque density (required for interacting with household objects with a reasonably sized robot), mechanical robustness, and low cost. Similarly, the $5,000 <a href="https://spectrum.ieee.org/automaton/robotics/industrial-robots/uc-berkeley-blue-low-cost-force-controlled-robot-arm?utm_source=dlvr.it&utm_medium=twitter">Blue </a>arms utilize the same quasi-direct drive actuation scheme to enable a capable, force controlled manipulator at low cost.

The secret to these super low-cost robots was retro-fitting mass-produced ‘drone’ motors with custom drive electronics, low-cost magnetic encoders, single-stage transmissions and advanced field-oriented control. This combination forged the path for high-performance robotic actuators at a tenth the cost of traditional robotics drives from vendors like Maxon, AMC, Elmo, Harmonic Drive, etc. This innovation will be an inflection point in the coming robot revolution.

Deep learning, while not the answer to all our problems, offers the promise of liberating robots from manufacturing into use-cases with significantly more variability. Instead of programming robots explicitly for any and every scenario, deep learning, while data hungry, leverages experience to learn control strategies that can adapt to new scenarios on the fly without explicit instruction.

Unlike most deep learning applications, which perform visual understanding and reasoning, robots need to be able to act in response to their perceived environment. Doing so requires a precise spatiotemporal understanding of the world. This requires significantly more data than using a neural network to determine if a picture contains a dog, cat, or airplane.

The dearth of a diverse, large-scale robot dataset is at the heart of our chicken and egg problem and there isn’t much of a consensus on how to collect a widely useful one. Unlike videos, audio, pictures, and text which are abundant online and in everyday life, robots aren’t. Collecting data on real robots is time-consuming, potentially dangerous, and expensive.

1. What dataset size and distribution is needed to generalize a learned task to different situations?

With a deployed fleet of low-cost robots, learning is certainly scalable though. Each robot can learn from every other robot’s collective experience so that every new robot deployed in a distributed network need not be retrained — a true robot hive mind. Before deploying a fleet of low-cost robots, we first need to answer 4 questions to ensure that we can properly learn from the collected robot experiences. In the context of creating a personal, in-home robot we would want the same robot to work equally well in my home and yours. But the problem is every home is radically different. Different furniture, different lighting, different flooring and layouts, different door knobs and appliances … different everything. How do we determine the appropriate sample size and diversity of data to be collected so that robots don’t overfit to the set of homes on which they were trained and instead can generalize to open doors or clean up rooms equivalently well in any home?

2. Which suite of sensor data is most conducive to learning?

Do we need visual data from cameras; 3D point cloud data from depth imagers or LIDAR; trajectory and motion data from encoders; tactile, proprioceptive or haptic data from load cells, <a href="https://www.youtube.com/watch?v=6CLTv-CyMH4">soft artificial skin sensors </a>, the <a href="https://www.youtube.com/watch?v=0eQCSrKdt5w">GelSight sensor </a>; or some complex combination thereof? Different tasks most likely require different sensing modalities. Grasping an item may only require cameras but reorienting the object once it’s been grasped may be done more efficiently with some form of haptic information. The same question exists for self-driving cars. On one hand, <a href="https://techcrunch.com/2019/04/22/anyone-relying-on-lidar-is-doomed-elon-musk-says/">Elon Musk claims that fully self-driving vehicles can be achieved without expensive LIDARs </a>. On the other hand, many top AI researchers and other self-driving car companies like Waymo disagree. There isn’t a strong consensus on the minimal set of sensors needed.

3. What’s the best reward signal for our data?

Possibly the most difficult question is determining how to inform robots if their actions during each experience are successful so that they can learn which actions lead to success and which lead to failure. Robots can be engineered in ways to do this automatically via self-supervision or reinforcement learning. Alternatively, humans can manually provide demonstrations or annotations indicating successful versus failed actions. The hard part is determining a generalizable framework for defining rewards such that each task doesn’t require its own list of finely tuned conditions or demonstrations in order to classify behaviors as good or bad.

4. How should we frame the learning problem?

Do we learn end-to-end models, or is it smarter to utilize human intelligence to frame tasks in ways that simplify what the neural network must learn? For example, with grasping we can try to learn end-to-end 6-DoF grasp poses directly from raw pixel inputs. Alternatively, we can frame grasping as an image segmentation problem using fully convolutional networks to classify each pixel as a good or bad area to grasp an object at and, as a post-process, use surface normals computed from depth images to obtain the full 6-DoF grasp pose. The latter method is more data efficient since the problem is simplified and the action space is reduced from 6 dimensions to 2. The classic trade-off to this simplified approach is that the method is no longer generalizable to many different manipulation tasks — each subsequent task will require significant re-engineering and data collection.

Similarly, <a href="https://openai.com/">OpenAI </a>trained a high degree of freedom <a href="https://robots.ieee.org/robots/shadow/?utm_source=spectrum">Shadow Hand </a>, to solve a Rubik’s cube. However, instead of training the neural network to ‘just solve the cube’ in and end-to-end fashion, they broke the problem down into many sub-problems using a pipelined approach. To actually solve the cube’s puzzle they use a conventional cube solving algorithm — Kociemba’s algorithm. Instead of using a camera to determine the state and orientation of the cube through visual perception — like humans — they instead retro-fitted the cube with a variety of internal sensors. Instead of learning to generally manipulate any face of the cube, they use human intuition about which face the hand was best at rotating (the top face), and constrain the solution moves to only rotate the top face. In this way they took a complex problem like solving a Rubik’s cube and significantly simplified the learning task by narrowing the scope to simply learning 1) how to rotate just the top face of the cube and 2) how to reorient any face to be the top face of the cube. They left the rest to conventional engineering and non-learned algorithms.

While manual engineering subgoals makes the learned skills less generalizable, it makes the task at hand easier to learn and the performance more reliable. We’ll talk more about the end-to-end vs pipelined approaches in <a href="https://medium.com/@simon.kalouche/the-evolution-of-intelligence-in-robots-part-2-43aea6985b8f?sk=e89b12b4fa65043baa3b5fe3767c05c4">Part 2 </a>. There are many system architectures whose design choices have inherent trade-offs and implications on how much data you need and how well your eventual model can generalize to different robots, different tasks and different environments. Knowing what to have the neural network learn and what kind of data it should learn from can significantly reduce the amount of data needed.

<a href="https://medium.com/@simon.kalouche/the-evolution-of-intelligence-in-robots-part-2-43aea6985b8f?sk=e89b12b4fa65043baa3b5fe3767c05c4">Part 2 </a>of this post discusses clever, state-of-art research directions for ways in which we can make progress in addressing each of the 4 questions outlined in this post! I’m a robotics PhD student in <a href="http://svl.stanford.edu/people/">Stanford’s Vision and Learning Lab </a>and the Founder of <a href="https://nimble.ai/">Nimble </a>, a startup building the robotic hive mind to automate complex tasks in warehouses. Thanks to my awesome friends and colleagues <a href="https://huyenchip.com/">Chip Huyen </a>, <a href="https://spectrum.ieee.org/author/ackerman-evan-">Evan Ackerman </a>, <a href="https://www.andreykurenkov.com/">Andrey Kurenkov </a>, and Jordan Dawson for their feedback and suggestions on this post!

The evolution of intelligence in robots: Part 1

GSF

Baba

Adi Puti's ideas are part of this journey:

Related collections

Similar ideas