Deep Fusion of Camera and LIDAR

Abstract

Fusing camera and LIDAR data in autonomous driving poses challenges such as accurate calibration, differing data representations, and extensive training data requirements. This dissertation addresses these by three contributions: a deep neural network for LIDAR-to-camera calibration, two depth completion approaches for processing sparse depth measurements in the image space, and a large-scale dataset of 93k RGB and depth images for training and evaluating deep networks.