LIBORG 2.0: A ROBOT FOR ON-THE- FLY 3D MAPPING OF ENVIRONMENTS - IMEC

Edition | October 2018

     Smart Industries

     LiBorg 2.0: a robot for on-the-
     fly 3D mapping of
     environments

     Today, the 3D reconstruction of, for
     example, tunnels or industrial buildings is
     a time-consuming and expensive process.
     To simplify this process, IPI – an imec
     research group at Ghent University –
     developed the mapping platform LiBorg.
     This lidar-based platform allows detailed
     3D models of various environments to be
     built on the fly. PhD student Michiel
     Vlaminck and project manager Hiep Luong
     explain how LiBorg works and what the
     benefits are.

How it started...
The foundations of the 3D mapping platform LiBorg were laid about four years ago,
during the imec.icon project GIPA. In the frame of this project, industrial partner
Sweco (the former engineering firm Grontmij) was looking for a system that could
efficiently monitor large-scale building sites or constructions, to verify, for example,
if a construction is proceeding according to plan. Michiel Vlaminck: “Today, static
laser scanners are used for 3D reconstruction. They need to be set up manually at
various spots within the environment. This requires considerable expertise, and is
often a very expensive and time-consuming process. Sweco was therefore looking

                                                                                       1/6

for a more efficient and cost-effective system, able to reconstruct 3D models on the
fly.”

LiBorg 2.0
In the course of that project, LiBorg was born – a 3D mapping platform that was
continuously refined after the GIPA project was ended. In its current implementation
– LiBorg 2.0 – the mapping platform consists of a mobile robot, sensors including a
lidar scanner, and reconstruction software allowing to generate in real time a 3D
model of nearly any environment. Hiep Luong: “Our off-the-shelf lidar scanner is
equipped with 16 laser beams that are continuously spinning at 10Hz. By measuring
the time that each laser beam needs to move back and forth from scanner to object,
the distance to the object can be calculated. By putting these distances in a
coordinate system, a three-dimensional point cloud of the environment can be
created. The system is also equipped with an RGB camera. A synchronization module
allows data from the color camera to be synchronized with the lidar, providing a
means to add realistic colors to the point cloud.”

LiBorg 2.0: a robot with lidar scanner, reconstruction software and RGB camera.

                                                                                  2/6

A unique localization system
The lidar scanner is also used to localize the robot, without using other techniques
such as gps. And that makes the mapping platform unique. Michiel Vlaminck: “To
generate a 3D model of an environment, the location of the robot within the
environment must be known at any time. In a first step, this is realized by relating
the newly incoming sensor data to the sensor data that have been captured before.
This technique is called SLAM (simultaneous localization and mapping): while the
map of the environment is being built, the robot puts itself into the map. In
subsequent steps, the localization is being refined. For example, in a second step,
data are related to the complete model that was created so far. In a final step, the
robot moves in a closed loop – ending at its (well-known) departure point. This allows
correcting the localization errors that have accumulated during the entire trajectory.”

The benefits at a glance...
This localization technique makes the platform independent of external localization
systems such as gps or wheel odometry (allowing to calculate distances based on
the number of wheel rotations). Each of these techniques has its own limitations. For
applications that require a high accuracy, satellite navigation can currently only be
used outdoor, and can as such not be deployed for mapping tunnels, mines or
buildings. Hiep Luong: “In addition, we are currently investigating the integration of
our LiBorg system in a drone. Here, of course, we cannot make use of, for example,
wheel odometry. A platform that is independent of an external localization system
has fewer constraints and can be used for all applications.”

Integrating the LiBorg system on a drone (picture) broadens the range of
applications.

                                                                                    3/6

But the fully automatic operation of the system and its ability to generate 3D models
on the fly – anywhere, anytime – are without doubt the biggest advantages of the
system compared to existing mapping solutions. Michiel Vlaminck: “This technology
allows to quickly and efficiently map nearly any environment, without human
intervention. During movement of the robot, a 3D model of the environment is
created – automatically and in real time. Currently, we steer the robot by means of a
controller, but we are also investigating whether it can move fully automatically. And,
in contrast to other existing systems, LiBorg can reach and map even the smallest
corners of the environment.”

3D map of streets, reconstructed with LiBorg 2.0

Trade-off between speed and accuracy
LiBorg allows 3D models to be generated with high accuracy, but for each
application, a trade-off must be made between accuracy and speed. Hiep Luong:
“There are different ways to express the system’s accuracy. One way is by
comparing the points from the calculated point cloud with the real environment. For
an environment of 4000m2, we typically reach an accuracy of 1cm for two arbitrary
points of the cloud. This is comparable to or even better than state-of-the-art.
Another way is to assess the accuracy of the estimated position of the robot in the
map. For a trajectory of 12 meter, the technique is accurate up to 5mm.”
The 3D model is being created on the fly, meaning that the time needed to create a
model is determined by the time that the robot needs (or has) to move around. Hiep
Luong: “LiBorg 2.0 can accurately generate 3D models on the fly. But if an

                                                                                      4/6

application requires a more accurate result, we can also do more complex
calculations ‘off board’. This of course requires more time.”

Data intensive
The on-the-fly processing of data that are being generated by the lidar scanner is
executed by an ARM processor that runs on a Jetson TX2 board – a platform for
embedded devices that is integrated within LiBorg. Michiel Vlaminck: “The lidar
scanner generates about 300,000 points per second – a lot of data to process in real
time. Therefore, a large part of my research activities focused on the fast processing
of data, for example by making the processing algorithm more memory efficient.
Therefore, we first translated the point cloud into a more efficient data structure.
This way, we are able to filter the 3D point cloud and remove redundant points. If
you don’t do that, and data keep on coming in at a rate of 300,000 points per
second, the memory gets fully occupied, slowing down the operation of the system.”

From train station to building information model for
architects
Thanks to these properties, LiBorg is suitable to map all types of environments.
Michiel Vlaminck: “In the frame of the GIPA project, we were asked to map a
chemical site. But besides that, our system can also be used for inspecting public
buildings such as train stations or shopping centers, and for mapping more critical
infrastructure such as tunnels, bridges, sewers or mines. LiBorg can also be deployed
to monitor escape routes or to help planning events.”
The mobile mapping platform is especially suited for monitoring construction sites on
a regular basis, and as such detect changes automatically. This brings us to another
application: creating BIM models (building information models). Hiep Luong: “We are
currently investigating if we can translate the scan data into a BIM model. A BIM
model is a simplified CAD model allowing architects to check if the construction of a
building proceeds according to plan, and to digitize the building’s life cycle. Our
robot can repeatedly monitor a site, even at night. It can also be deployed after the
construction works have ended, to regularly check changes made to the building.
That’s why we are currently investigating if we can integrate change detection in the
algorithm that we developed for LiBorg. That will be the focus of further research and
development.”

Want to know more?
• Read more about the imec.icon project GIPA
• LiBorg 2.0 was among the demos that were shown at the 2018 Imec
  Technology Forum in Antwerp, Belgium. Watch the movie (Kanaal-Z, in Dutch
  only) to see the lidar based robot (and other imec technologies) at work.
• In this work, an off-the-shelf radar was integrated in the LiBorg system. Imec
  is also working on the next-generation lidar systems. More info on
  the website.

                                                                                   5/6

Biography Michiel Vlaminck
                                               Michiel Vlaminck received his MSc degree
                                               in Computer Science Engineering from
                                               Ghent University in 2013. Since January
                                               2014, he is working as a PhD researcher at
                                               Image processing and Interpretation (IPI),
                                               an imec research group at Ghent
                                               University. He is currently working on the
                                               topic of 3D scene reconstruction using
                                               active depth sensors. His research focuses
                                               on applications in the domains of
                                               augmented reality, autonomous robots and
                                               UAV.

Biography Hiep Luong
Hiep Luong received his PhD degree in
Computer Science Engineering from Ghent
University in 2009. Currently he is working
as a researcher and project manager in
Image Processing and Interpretation (IPI),
an imec research group at Ghent
University. His research and expertise focus
on image and real-time video processing
for diverse fields such as HDR imaging,
(bio-)medical imaging, depth and multi-
view processing, and multi-sensor fusion
for UAV and AR applications.

                                                                                      6/6