Human-like robots for the household on the rise

Have you always hated ironing, find tidying up annoying and have an inexplicable weakness when it comes to the dishwasher? Then you might soon be helped. Humanoid robots are no longer just part of science fiction series, but are increasingly conquering everyday areas of life – a text on the current state of research by Prof. Dr. René Peinl, Head of the Institute for Information Systems at Hof University of Applied Sciences (iisys).

We have long been familiar with the images from factories and production halls: until now, robots have mainly been used in industry to perform exactly the same tasks. This usually involves very exact timing or precision in the millimeter or sub-millimeter range. They are often separated from the rest of the factory by barriers so that people are not injured – after all, this type of robot does not care whether someone is in the way or not.

Previous household helpers still without AI

So far, vacuum cleaning and mopping robots on the one hand and lawn mowing robots on the other have established themselves in the household, which, despite slightly different sensor technology, are nevertheless based on similar and predominantly quite primitive basic mechanisms. Roughly speaking, they drive straight ahead until they encounter an obstacle, then turn at a randomly selected angle and continue on their way. They do this until, statistically speaking, every spot in the room or every square meter of lawn has been covered. When they are finished or the battery level is low, they return to the charging station. There is (still) no sign of artificial intelligence (AI).

Deep learning mechanisms are being integrated

Recently, however, household robots in practice and industrial robots, at least in research laboratories, have been increasingly equipped with more “intelligence”. This enables them to deal with different situations more flexibly, act in a more targeted manner and therefore reach their goal more quickly. This is achieved by integrating deep learning mechanisms – the very technology behind the current AI hype. This ranges from interference-insensitive recognition of the environment and speech recognition to interaction with people or even planning units based on large language models (LLMs), i.e. the type of AI that drives the well-known “ChatGPT”. For example, robotic vacuum cleaners can now recognize their position in the room via a pattern projected onto the ceiling and thus clean the floor in a more targeted manner. Lawn mowers, on the other hand, use cameras to learn to distinguish lawns from paths and flowers and will soon be able to do without boundary wires.

Humanoid robots as a direct human replacement

In the course of these developments, humanoid robots, i.e. robots modeled on humans, are suddenly becoming interesting again. For a long time, they were considered far too inefficient and unnecessarily complex, as locomotion with wheels or on four legs instead of two is much easier to master and, in the case of wheels, even more energy-efficient. However, humanoids have a decisive advantage: they can be used directly as a replacement for human workers without the working environment having to be specially adapted to them. The “Boston Dynamics Atlas” robot was the flagship humanoid for a long time – at least after the Honda company stopped developing its “Asimo” after 2014 – and showed impressive performance in demonstrations in terms of walking speed, immunity to interference and, later, precision even in athletic-looking movements. Running a course, somersaulting backwards, flipping over obstacles – none of this was a problem for “Atlas”. And this despite the fact that – to put it mildly – it was built rather clumsily.

Hardly any autonomous actions so far

Ultimately, however, there were only demo videos and there were never any reports of productive use in industry, so it can be assumed that the videos were meticulously prepared and scripted down to the smallest detail. In fact, the autonomy and spontaneity of action suggested in the videos were probably just a big bluff – something that is apparently not uncommon throughout the industry. Tesla also received a lot of criticism for a video of the Tesla robot “Optimus”, which supposedly folds laundry on its own. Elon Musk only admitted that the action was completely remote-controlled after other users had already loudly pointed this out on social media.

Nevertheless, humanoids have been on the rise in the last 2-3 years. Companies with highly developed prototypes or even operational models are currently springing up like mushrooms. “Figure 01”, “Agility Robotics Digit”, “Tesla Optimus” and, last but not least, the completely redesigned electric “Atlas” from Boston Dynamics are examples of this. Whereas in the past, walking and maintaining balance was often a challenge in itself, speeds of 5-8 km/h are now standard and attention is shifting to interaction with a dynamic environment.

Human-like hands are the primary object of research

The hands are particularly crucial for this. Here too, the trend is towards imitating humans with five-fingered hands. Although these are very expensive due to the large number of motors and are therefore certainly more error-prone and fragile than two-fingered grippers or vacuum-based suction devices, they allow significantly better fine motor skills. This is particularly helpful for applications where force is less important than feel, e.g. in the household. For example, some manufacturers allow their products to peel cucumbers or crack raw eggs. However, the difficult folding of T-shirts or shirts is a real test of their suitability for everyday use. On the one hand, this requires a certain dynamism of movement, but on the other, precise timing and good visual recognition. As with AI in general, you can learn an incredible amount about people’s abilities by trying to imitate their activities with robots.

Speed still insufficient despite progress

Less force is also positive when it comes to direct interaction with humans. If the robot cannot exert the force required to break an arm or finger in the first place, then there is less to fear. Cobot is the keyword for this category of robots. This often includes padding for the extremities to further reduce the risk of injury. Ideally, however, the sensor system prevents unwanted physical contact from occurring in the first place. Even though a lot of research is being carried out into the interaction between humans and cobots, the question arises as to whether robots should still be working on their own so as not to put too much strain on people’s patience. Despite all the acceleration in recent years, the speed of execution is usually still well below that of “normal” humans, not to mention trained personnel with lots of practice and routine. Many videos are therefore also recorded in fast motion, which reputable companies also write about. However, you should always be skeptical if no time indication (2x, 1x, …) is shown in the videos.

The autonomous completion of tasks is relevant not only in the household but also in logistics. Amazon already employs 750,000 robots worldwide, although most of them still belong to the “old school”. The potential for automation in this sector is enormous. Robots will soon be able to do everything from “picking goods from the shelf” and “packing parcels” to “loading boxes into trucks”. It is not for nothing that investors are pumping several hundred million USD into promising companies every year. Elon Musk is somewhat optimistic about a market of 1 billion humanoid robots per year and wants to earn 1 trillion dollars with them. But even realistically, the potential of humanoid robots is very large, as can be seen from the keen interest shown by many car manufacturers, for example.

Research is progressing

Commercial manufacturers are naturally keeping a low profile when it comes to the use of special AI models and even the sensor technology used. However, if you look at the research results, it becomes clear what ingredients are needed to build advanced autonomous robots: A central role is played by “seeing”. The previously common object detection method (see Figure 1, case b), in which the area in which a certain object was recognized is only roughly marked with a rectangle, has now been replaced by more precise methods. Semantic segmentation can trace the outlines of the objects (figure, case c), instance segmentation can distinguish between different instances, e.g. person 1, person 2 and person 3 (case d).

Panoptic segmentation is the most advanced variant that combines all possibilities (see Figure 2). A distinction is made between “objects”, moving things and people, and “stuff”, immovable things such as roads, the sky, houses and trees. Currently, the most advanced models can not only distinguish the predefined object classes such as bicycle, dog, human and car with which they have been trained, but can also infer objects that have never been trained before by combining them with text models that determine similarities in the content of objects. This is called “open vocabulary object detection”. If, for example, horse and zebra stripes were present in the images that were trained, but not the zebra itself, the model can still recognize the zebra in the image via the “knowledge” acquired from the text or “assume” that it must be a zebra, similar to a human.

Object Tracking can also recognize the identity of individual objects across several consecutive images and thus “understand” that the object has moved and not disappeared and another has appeared in a new place. The main challenge with 2D RGB videos is estimating distances. This is why some robot manufacturers install additional LIDAR sensors with laser distance measurement, or use RGB-D cameras that provide distance information in addition to the color information by calculating how long a signal takes to be reflected back to the source (time of flight).

Conclusion

Overall, there are already good solutions for the most important challenges in robotics. They “only” need to be combined in one device, although the computing power of the onboard hardware is of course also a limiting factor. Outsourcing AI to a server is also difficult with 5G data transmission due to the amount of data. The lack of standardization is another stumbling block because AI models cannot be easily transferred from one piece of hardware to the next. Nevertheless, models are also being discussed for the Robotics Foundation and Jim Fan, a leading AI researcher at Nvidia, sees robotics’ ChatGPT moment coming in the next 2-3 years.

It has just been announced that the first autonomous humanoid robot has been given a permanent position in a logistics company. This is certainly just the beginning of a major change in society, in which it will become increasingly normal for robots to take over many tasks that were previously carried out by humans. Initially, this will alleviate the shortage of skilled workers, but it is foreseeable that in the not too distant future there will also be a displacement of human workers, especially if politicians cannot bring themselves to tax machine work.

Still thirsty for knowledge? Then read more on the topic in “c’t” issue 23/2024 – it will be in stores from October 18, but is already available digitally here.