How far are we really from fully autonomous earth moving machinery?

November 12, 2019

This article was first published on September 6, 2019 as a LinkedIn article by our Business Develoment Manager, Miika Kostamo.

Kobelco working on Tampere Tramway project with BIM ready machine control system.

After visiting the Finnish earth moving expo, Maxpo 2019, this week, I started to think, how far away we really are from fully autonomous earth moving machinery. Lot of things enabling it has already being developed, but there is still a lot of things to develop, before the technology and common requirements to allow this to happen on a wider scale..

What we have already

In the Maxpo exhibition by looking in the the several booths and all the new machinery visible it was clear that 2D and 3D machine guidance systems have already reached the market in Finland and from every major manufactures booth you could find machines ready to be shipped to the customers after the expo and with GNSS antennas attached to the back of the machines.

This was not a surprise as Finland has been one of the leading countries in the world using models and BIM in infrastructure construction. Back in 2015 Finland was the first country in the world to publish common infrastructure BIM requirements, infraBIM, for the country and since use of BIM has become a common practice on all major construction sites. Second generation of these common infrastructure BIM requirement were already launched this year and is free for anybody to read as were the previous ones.

Even it is not the reality in the whole world, at least in Finland there is already a common openBIM way of modelling infrastructure projects. Common requirements (InfraBIM), common classification system (InfraBIM classification), and common data transfer format (Inframodel, extended LandXML) are basic requirements for autonomous earth moving machinery. With those it is possible to make the plans that all machine control systems supporting openBIM format can understand in the same way, kind of a traffic rules for infrastructure construction.

From the mechanical point of view several machine are already equipped with semi-autonomous functions (control of specific movements) and this can be extended to full automation once the other aspects still under development come clear. This technology is not limited to new machines, as last year was seen on MOBA booth in bauma China, where 30 year old Chinese excavator was equipped with semi-automation system.

RTEC semi-automation of excavator movements

What is still under development

On international level the “common traffic rules” are still missing, but there are already hundreds of people working on it. Within buildingSMART International there are people from most developed countries taking part in the common BIM standards development for buildings and infrastructure. The first area of this development, BIM for buildings (IFC = Industry Foundation Class), has already reached the version IFC4 few years ago with a possibility to store name of the coordinate system used in the models. In the next release will start to have also functions that support global positioning and local transformations (requirement for autonomous vehicles that are using global satellite positioning systems (GNSS) and need to know their position in the world as well as on site with some local coordinate definition).

Also extensions of the current openBIM standards there is new development coming on the infrastructure IFC and even the final direction is still a bit open, it is expected to be ready in next 2-4 years. After this we would have a common format that would be enforced through ISO and CEN as a global way of defining BIM models for infrastructure projects.

Another aspect of autonomous earth moving machinery is that in a constantly changing environment machines have to be able to exchange information of what has been done and how it has changed the topography in the area of working. This is also a topic that is under development in ISO TC127 and as many of the global earth moving machinery manufacturers are involved in the process it is very likely to be agreed by the end of the project couple of years.

One aspect of fully autonomous earth moving machinery is also safety and capability to navigate in ever changing environment. What is we have in autonomous vehicles today is not yet ready for work in hail, mud and ever changing terrain, where one down pour can change the earlier dry dirt track to a flooding mud bath.

Object recognition and AI is not there either, as we can see from these AI presentations in TED.com, teddy bear can be recognized, but how to recognize a teddy bear buried in 3 feet of mud has not been solved yet.

What still needs to be developed

Major technological breakthroughs have been made in last 5 years on technology, but what we still lack is

1. Very fast speed internet connection that allows exchange of heavy construction models and ever changing environmental information real-time.
2. Advanced algorithms capable of multi-sensor and technology modelling of changing construction environment all the time.
3. IP classed multisensor measuring system that would cost less than an excavator and still be able to work in extreme weathers from sand storms to mudslides (realities of earth moving work sites).
4. Common traffic rules on global scale (IFC Infra and data transfer)
5. Way to do underground maps in detail and in real-time (ground penetrating radar with accurate 3D mapping capabilities and adaptive estimation to define possible danger areas underground).
6. Advanced safety system that would allow autonomous earth moving machinery to work in populated areas, where most construction sites exist.

Best guess of time line

For me personally digital transformation started in 1995.

In that year I got my first computer, but also got introduced a pretty advanced technologies back then; like digital pocket camera, electron microscopy, molecular analysis of soil samples and gene transfers on plans, during my secondary school student practice in the Helsinki University. The first one became main stream in just few years after that with IT bubble, open internet access and lowering costs of personal computers. Pocket digital cameras broke through few years later.

For the other the cost of sensors and equipment as well as involvement of radioactive materials in the process has limited the entry to main stream shopping centers.
Reflecting to the past it used to take almost a decade for any new breakthrough to become mainstreams. Even with the ever faster development speed the construction industry is still made mostly of people and retraining of millions of people working the sector will take time. My guess is that in next few years we will see the first fully automatic proof-of-concept (works in limited area), but it will probably take until 2030 until anything starts to happen on a global scale.

By then we should also have other aspects of the fully autonomous and electric earth moving equipment available. Autonomous service trucks, charging stations for high power batteries and so on.

The first 15 years in surveying and construction field have been definitely interesting once and I am waiting a lot, what the next 15 years will bring. Change from hammering sticks to autonomous equipment will be a major one, if we consider that the manual paint and sticks marking method for infrastructure was already invented in the time, when first pyramids were built in Egypt.

Miika Kostamo,
Business Develoment Manager