From eight to 56 percent: The household robot clears the toughest hurdle

Until recently it was taken for granted that a robot must learn each home individually. Figure published numbers on September 17 that put that to rest. The humanoid from the California manufacturer tidied away toys, made beds, and folded cloths in 30 unfamiliar homes in the Bay Area that the model had never seen and without a minute of on-site training. 237 of 420 attempts went completely through. That's 56 percent. The same architecture without the new pre-training achieves eight.

The jump came years earlier than most in the industry had expected. What's striking is where it comes from: not from better mechanics, but from 16 million smartphone videos that 44,000 paid people uploaded from their own kitchens and bedrooms.

How good the number really is is shown only by the test setup, and it's tougher than what manufacturers usually calculate. The same frozen model version ran in all 30 homes. Only fully completed tasks were counted, and if a person intervened for safety reasons, the attempt was considered failed. When making beds, the robot achieved 67 percent, when folding 62, when tidying up thirteen to fifteen scattered objects still 40. Anyone who has ever seen a gripper fail on a wrinkled terry cloth towel knows what these numbers are worth. One limitation remains central, however: the robot had learned the three activities beforehand. What was new were the homes and the things in them, not the task.

The lever behind this is called Index and is a little social history in itself. Since August, an app has been available in the stores with which anyone can film their household and get paid for it. 264,000 downloads in 108 countries, nearly 15 million dollars already paid out. In doing so, Figure captures around 35 minutes of human experience per second, equivalent to 50,000 hours of material per day or almost six human-years, day in and day out. Over a billion dollars is said to flow into data over the next twelve months, plus 3.5 billion into computing power. It is the most expensive attempt yet to teach a machine everyday life by observation rather than instruction.

For the first time, this allows for reasonably serious calculations of when such a device can manage without supervision. Figure trained four models on staggered amounts of data across a range of factor eight and reports that the error decreases with each doubling so reliably that the value of the largest run could be predicted in advance to four decimal places. The company does not publish the exponent of this curve, and everything depends on it. Using the values common in the literature, it takes between 77 times and 50,000 times the amount of data for a 95 percent success rate. The range looks absurd, but it's the honest answer: the difference between a favorable and an unfavorable exponent decides over years.

Placed alongside the collection rate, it becomes concrete. If Index continues to grow as quickly as before, the data mountain roughly doubles every six months, and the necessary six to eleven doublings would be achieved in three to five years. If the rate stalls where it is today, the same distance takes decades, because each additional doubling takes twice as long as the previous one. For the last few percent, something different applies anyway. To achieve 99 percent by video alone requires between two thousand and one hundred million times today's data, depending on the exponent. That won't happen. This leg must be covered differently, through error detection, through second attempts, through a robot that notices the cloth is lying wrong and simply tries again.

That it works differently is demonstrated by Sunday Robotics from Mountain View. The company reported 99.1 percent for laundry folding in August, 778 of 785 attempts in 25 unfamiliar homes. A single task, a mobile robot, but nearly perfect in return. Figure goes the opposite way, broad instead of deep, and accepts that every other task still fails. Which path arrives first in the living room is open. What is certain is that the harder part is done: a model that actually understands an unfamiliar home was a research subject two years ago and is a product being tested today.

For Switzerland, that means two things. Such a device cannot be sold yet, and until it can be, not only the success rate must be clarified but also who is liable if the arm hits the neighbor's vase, and how a robot in residential space obtains its declaration of conformity. That is not a formality and cannot be done in three years. The other side is more immediate. The Index app downloads here too, for whoever wants it. Whoever films their kitchen is working on a system that someday will stand in that kitchen. Payment is by the minute, not by shares.