3D Computer Vision Helps Autonomous Vehicles Locate Themselves on Maps

Self-driving cars need onboard artificial intelligence (AI) technology able to link them to highly detailed maps that reflect every change in the status of lanes, hazards, obstacles, and speed-limits in real time.

NYU Tandon researchers are developing an artificial intelligence system for autonomous vehicles that links them to HERE Live Map cloud-based mapping system. Courtesy of NYU Tandon.

Researchers at the NYU Tandon School of Engineering are working hard to make this machine-to-machine interaction possible. Yi Fang, a research assistant professor in the Department of Electrical and Computer Engineering and a faculty member at NYU Abu Dhabi, and Edward Wong, an associate professor in the NYU Tandon Department of Computer Science and Engineering, are developing a deep learning system that will allow self-driving cars to navigate, maneuver, and respond to changing road conditions. To do this, they are combining data from onboard sensors to information on the HERE HD Live Map — a cloud-based service for automated driving.

High-definition maps meant for machine-to-machine communication must be accurate to within 10 to 20 centimeters. Self-driving vehicles need to continuously update and register their location on these maps with an equally high degree of accuracy. Fang said the goal of the research is to enhance car-to-map precision to within 10 centimeters.

“Essentially, we want to be able to precisely match what the car sees with what's in the cloud database,” said Fang. “An incredibly precise ruler isn't of much use if your vision is blurry.”

Wong said their work involves using computer vision techniques to refine the vehicle’s ability to located itself with respect to HERE’s cloud-based service. “That requires real-time images of the street and surrounding objects derived from cameras, lidar, and other on-board sensors,” he said.

Automobiles connected to HERE's HD Live Map service will deliver data to the cloud on road conditions, traffic, weather, obstacles, speed limits and other variables, allowing the service to upgrade nearly in real-time to reflect changing conditions.

“3D computer vision and Deep Neural Network are the technologies driving the development of high- definition live maps for self-driving cars,” said Xin Chen, HERE senior engineering manager and research scientist.

The HERE mapping project joins a number of recent initiatives at NYU Tandon addressing safer and smarter transportation.