In this paper, we address the problem of creating an objective benchmark for evaluating SLAM approaches. We propose a framework
for analyzing the results of a SLAM approach based on a metric for measuring the error of the corrected trajectory. This metric
uses only relative relations between poses and does not rely on a global reference frame. This overcomes serious shortcomings
of approaches using a global reference frame to compute the error. Our method furthermore allows us to compare SLAM approaches
that use different estimation techniques or different sensor modalities since all computations are made based on the corrected
trajectory of the robot.
We provide sets of relative relations needed to compute our metric for an extensive set of datasets frequently used in the
robotics community. The relations have been obtained by manually matching laser-range observations to avoid the errors caused
by matching algorithms. Our benchmark framework allows the user to easily analyze and objectively compare different SLAM approaches.
Keywords SLAM - Mapping accuracy - Benchmarking