Towards HD Map Updates with Crosswalk Change Detection from Vehicle-Mounted Cameras

Abstract

Many autonomous vehicles rely on high-definition maps that contain road layout and road semantics as priors for perception, planning and prediction. However, these maps can become stale over time as the road environment changes. This thesis develops a road monitoring framework that allows for automatic change detection of crosswalks with a cost-effective sensor suite of vehicle-mounted cameras and GPS data. Furthermore, this thesis explores using an edge computer on a commercial bus to receive and analyze live data captured in Pittsburgh.

Contributions of this thesis include evaluating object detectors trained from different types of datasets, representing crosswalks in the bird's-eye-view for more robust change detection, and finally incorporating the system on an actively running bus. The first contribution of this thesis is an evaluation of the CARLA simulator as an effective tool to provide automatic annotations for custom street-view objects on a simulated vehicle-mounted camera. Despite the sim-to-real domain gap, models trained on CARLA-generated annotations for two custom objects, fire hydrants and crosswalks, are shown to perform as well as those trained on 200 real-world images and can be used to augment existing datasets. The second contribution of this thesis is a method that maps detections from 2D images onto a ground plane by using multi-view geometry and 3D reconstructions of the scene. With this method, detections from multiple frames can be accumulated in the bird's-eye-view to better represent an intersection, and consistency checks can be performed to remove false detections. Lastly, this thesis explores using the crosswalk change detector in an edge-computing enabled commuter bus that has active cameras. With GPS locations of seventeen existing crosswalk intersections, the bus can send relevant images for the crosswalk change detector to analyze. Change detection results show robustness in high-traffic scenes where vehicles often occlude the road and robustness to pose differences between current and reference images.