High-Tech Lidar: The Future Looks Fly

Advanced lidar technology is poised to be at the center of a revolution in skyward autonomous mobility involving robotic passenger drones, personal helicopters, and flying vehicles.

VALERIE C. COFFEY, SCIENCE WRITER

Over 50 years ago, light detection and ranging technology (lidar) found its earliest applications in military targeting and meteorology research, and famously, in mapping the surface of the moon in 1971. But the development of lidar for widespread commercial use is relatively new. Only in the past decade has commercial lidar become widespread beyond one-off custom devices for military and research applications. Commercial volume production has reduced the cost of lidar sensors, which can return 3D point clouds containing distance and spectral data that cameras cannot. Combined with battery-operated drones, lidar sensors have found wide acceptance in aerial 3D mapping, surveying, and remote sensing for agriculture, geology, topology, and mining, among other applications.

Courtesy of Vahana/Airbus Urban Mobility.

Courtesy of Vahana/Airbus Urban Mobility.

The A³ Vahana electric self-piloting passenger aircraft conducted testing of its vertical takeoff, horizontal flight, and landing capabilities this year. Courtesy of Vahana/Airbus Urban Mobility.

In addition, dozens of vehicle manufacturers are now racing to be the first to introduce lidar systems into the upcoming generation of autonomous cars, trains, taxis, and trucks — many to be launched in 2021. And why stop there? Advancements in drones (unmanned aerial vehicles, or UAVs), machine learning, computing, and small low-cost lidar sensors are helping to shorten the runway in another up-and-coming market — electric vertical takeoff and landing vehicles (eVTOLs), often called flying cars.

A common thread in many eVTOL designs is the flying taxi, which would pick up cargo and passengers and deliver them in automated pilotless pods to and from urban helipads.

Flying cars may sound like science fiction, but scores of companies around the world, from Airbus to Uber, are funding development efforts, vying to be the first to launch a vehicle that can fly over traffic, cut emissions, and replace ground-based cars, taxis, and shuttles. The goal is to establish urban air mobility (UAM): on-demand, sustainable autonomous air transportation that shortens commutes, reduces urban congestion, and saves energy.

According to Louay Eldada, CEO of solid-state lidar developer Quanergy Systems Inc., Level 5 autonomy (in which drivers can take a nap) will be seen in flying cars before it becomes available in terrestrial vehicles. “Auto-pilot was originally designed for aircraft,” said Eldada. “Flying cars will likely be guided along air corridors, following a prescribed path, so autonomy in flying vehicles makes perfect sense. Ground vehicles face obstacles like debris, broken down vehicles on the roadway, [and] people and animals crossing the street. In any large, chaotic city, travel in an automated air vehicle would be safer than in one on the ground.”

From an engineering standpoint, lightweight batteries that power electric vehicles have evolved to be reliable. The ability to take off vertically and zoom horizontally is well established in small multirotor drones. The next step, making eVTOL drones big enough to carry heavy cargo and people, is now in the prototype and testing stages. But while the engineering of eVTOLs has been conquered in many ways, safety, regulation, and legal concerns remain. The consequences of an accident in the air are much more severe than in one on the ground. Thus, the aviation industry is more likely to embrace large-scale, automated air transportation if it relies on infallible technology. The first commercial flying vehicles are likely to have pilots, either in the pilot’s seat or assisting flights remotely, and to carry cargo, not people, in short hops. Achieving these steps reliably will pave the way to unpiloted, autonomous flying taxis, shuttles, and cars with passengers.

What is the difference between flying cars and small personal planes? Some flying cars may resemble planes, but are intended to replicate and replace the function of cars in urban mobility. In fact, some flying vehicles will require both pilot’s and driver’s licenses because after landing, the vehicles transform into a street-legal wheeled car. The eVTOL and flying car have recently become synonymous, defined by vertical takeoff and landing, requiring a helipad rather than a runway.

Numerous designs reveal the first flying cars will look more like helicopters or large drones than a Jetson’s flying saucer. A common thread in many eVTOL designs is the flying taxi, which would pick up cargo and passengers and deliver them in automated pilotless pods to and from urban helipads. As in self-driving cars, lidar is poised to be a critical part of the solution, enabling real-time scanning of distances to objects in the environment, even in the dark. Flying cars, whether piloted or not, are likely to use lidar for obstacle detection, especially upon landing.

Flying high

Whether they call the designs eVTOLs, personal planes, or flying cars, several manufacturers plan to launch commercially available personal air vehicles between 2020 and 2025. The following are several eVTOL flying car designs that plan to incorporate lidar. (See sidebar for more companies working on eVTOLs.)

Figure 1. The Peregrine sensing and computing payload on the Vahana includes a mechanically spinning lidar sensor from Velodyne called Puck, which verifies that landing sites are safe from obstructions and provides alternate locations if necessary. Courtesy of Vahana/Airbus Urban Mobility.

Figure 1. The Peregrine sensing and computing payload on the Vahana includes a mechanically spinning lidar sensor from Velodyne called Puck, which verifies that landing sites are safe from obstructions and provides alternate locations if necessary. Courtesy of Vahana/Airbus Urban Mobility.

In February 2019, a San Jose, Calif., spinoff from aerospace manufacturer Airbus, called A³ (pronounced “A-cubed”), achieved successful full-scale model test flights of the Vahana, a self-piloted, all-electronic eVTOL fixed-wing aircraft designed to skip over urban traffic. Self-piloting means no pilot — not only is it a flying vehicle, it drives itself. No Jetsons needed. The Vahana has a sensor suite called Peregrine, which incorporates a Puck lidar system from Velodyne, a compact version of the spinning lidar system mounted on top of Google driverless cars (Figure 1). The Puck lidar uses 16 905-nm lasers that incorporate time-of-fight measurement to create a 3D real-time point cloud image of the landing area out to 100 m (Figure 2). In recent months, the Vahana has continued flight testing that successfully demonstrated vertical takeoff, rotor rotation for forward horizontal flight, and safe landing.

Figure 2. The Puck lidar module from Velodyne can create a 3D image of a landing environment, even in the dark. This image looks down on a city scene of San Francisco’s Union Square. Courtesy of Velodyne/SLAM Data by Emescent.

Figure 2. The Puck lidar module from Velodyne can create a 3D image of a landing environment, even in the dark. This image looks down on a city scene of San Francisco’s Union Square. Courtesy of Velodyne/SLAM Data by Emescent.

Airbus has also collaborated with German automaker Audi and Italian design firm Italdesign on another flying car concept designed to solve urban traffic problems. Pop.Up Next, a self-driving car and passenger drone connected by an ultralight two-seat modular pod, debuted at the Geneva Motor Show in March 2017 (Figure 3). The passenger cabin of the Pop.Up Next attaches interchangeably to either a car module with wheels or a flight module with eight tilt-wing rotors. The cabin features a 49-in. display with speech and face recognition, eye tracking, and a touch function. The first flight of the scale model occurred in 2018. A prototype of the Pop.Up Next, which reportedly will incorporate proprietary lidar, is scheduled for testing in 2020.

Figure 3. The Pop.Up Next flying car, scheduled for testing in 2020, is a modular two-passenger pod that latches interchangeably to an Audi wheelbase and a drone-like set of rotors for urban air mobility. Courtesy of Italdesign/Airbus/Audi.

Figure 3. The Pop.Up Next flying car, scheduled for testing in 2020, is a modular two-passenger pod that latches interchangeably to an Audi wheelbase and a drone-like set of rotors for urban air mobility. Courtesy of Italdesign/Airbus/Audi.

In January, Aurora Flight Sciences of Manassas, Va. (an independent subsidiary of Boeing since November 2017) conducted the first controlled takeoff, hover, and landing of Boeing’s unpiloted Passenger Air Vehicle (PAV) flying taxi prototype (Figure 4). Future flights will test the transition of the rotors between vertical and horizontal flight modes for takeoff and landing. Aurora is evaluating commercial opportunities for PAVs with Boeing NeXt, a division of Boeing that is exploring UAM. Boeing NeXt is also working on novel swarm coordination and planning tools that would help integrate autonomous and piloted air traffic around the world.

Figure 4. The Passenger Air Vehicle (PAV) flying taxi prototype by Aurora Flight Sciences demonstrated its vertical takeoff and landing capabilities in January in Virginia, with a dummy pilot in the cockpit. Courtesy of Boeing.

Figure 4. The Passenger Air Vehicle (PAV) flying taxi prototype by Aurora Flight Sciences demonstrated its vertical takeoff and landing capabilities in January in Virginia, with a dummy pilot in the cockpit. Courtesy of Boeing.

Aurora is a partner in Uber’s Elevate program, which aims to provide commercial on-demand air taxi service by 2023. Ride-sharing giant Uber is working with partners — including aircraft contractor Bell in Fort Worth, Texas; EmbraerX of Brazil; and Karem Aircraft in Orange County, Calif., among others — to begin testing of eVTOL fleets in Los Angeles, the Dallas-Forth Worth area, and Melbourne, Australia, in 2020. The first air taxis will likely require licensed pilots, and they could cost as much as a Lamborghini ($400,000 to $600,000), but for now they are designed to be ride-sharing vehicles, not owned individually.

Boeing is also developing safe, economical, and reliable unmanned cargo delivery via large autonomous drones using lidar. The company successfully conducted test flights of an eVTOL autonomous cargo aerial vehicle (CAV) prototype in 2017, which could carry up to 500 lbs (227 kg). The CAV model includes an experimental lidar system with a range of 10 miles, an industry first, capable of detecting clear air turbulence with a 60-s warning to ensure a smooth ride. Boeing has confirmed that lidar technology is likely to be part of the design of future autonomous aircraft for detection of air turbulence.

Sabrewing Aircraft Company Inc. in Camarillo, Calif., designed an unmanned hybrid-electric VTOL named the Rhaegal (after a dragon in “Game of Thrones”) to carry up to 1000 lbs (454 kg) of cargo over 1150 miles (1850 km) without a pilot to remote destinations in all weather (Figure 5). The company hopes to deliver cargo such as medical supplies and fresh produce to the remote Aleut Community of St. Paul Island in the Aleutian Islands of Alaska. The 30-ft (9-m) wingspan drone, scheduled to begin testing in 2020 with a remote pilot, will synthesize data from a GPS navigation system, radar, cameras, infrared detectors, and a lidar system with a 350-m range to avoid obstacles and ensure safe landing in adverse weather, even at night.

Figure 5. The Rhaegal eVTOL cargo-only pilotless drone is designed to carry a payload of up to 1000 lbs over 1150 miles over the Bering Sea in icy conditions or heavy crosswinds. The lidar system will have a range of 350 m. Courtesy of Sabrewing.

Figure 5. The Rhaegal eVTOL cargo-only pilotless drone is designed to carry a payload of up to 1000 lbs over 1150 miles over the Bering Sea in icy conditions or heavy crosswinds. The lidar system will have a range of 350 m. Courtesy of Sabrewing.

In 2015, a group of volunteers at Toyota-backed startup Cartivator in Tokyo raised ¥302.6 million ($2.68 million) to develop an eVTOL passenger-carrying quadcopter using Quanergy’s lidar sensors. The electric flying vehicle, named SkyDrive, conducted its first test flight of a scaled model in December 2018. The compact flying car is designed for easy maneuverability. The group hopes to use the SkyDrive without a human being to carry the Olympic torch in Tokyo’s 2020 Summer Olympics.

Next-gen approaches

Many research and development efforts are focused on decreasing the size, weight, and cost of advanced lidar systems without losing the distance range that lidar can detect. Mechanically scanning lidar is the most well-established technology and can offer a wide field of view on the order of 360° × 40° out to hundreds of meters, but it is expensive at thousands of dollars per unit. The moving parts, size, and weight of spinning lidar is a disadvantage in vehicles, whether in the air or not. Next-generation lidar will have to overcome these limitations.

Several companies are pursuing solid-state lidar, which can be fabricated on a tiny chip, enabling fully integrated lidar imaging that can eventually be mass-producible to reduce costs by more than 10× that of existing systems. In April, Voyant Photonics, a spinout of Columbia University in New York, raised $4.3 million to develop its solid-state lidar technology based on an optical phased array and a frequency-modulated continuous wave (FMCW) lidar engine (Figure 6). The FMCW approach uses a bidirectional chirped 1550-nm CW laser with a frequency that changes over time¹. The frequency difference in the returning beam imparts the distance to objects out to hundreds of meters, as well as the velocity of moving objects — data that would certainly come in handy in autonomous vehicles. An optical phased array inside the tiny lidar engine steers the beam using a record-low power consumption of less than 1.8 W. The approach also results in higher resolution sensing than traditional pulsed lidar and prevents interference from other sources. While the approach is complex, the fingertip-size lidar on a chip would vastly reduce size and weight, creating endless possibilities in numerous applications, such as in cellphones and centimeter-precision 3D mapping via self-flying drones.

Figure 6. Solid-state lidar will integrate lidar onto a photonic integrated circuit that fits on a fingertip. Voyant Photonics is pursuing such lidar on a chip using an optical phased array to steer an FMCW lidar beam in two dimensions, enabling distance and velocity sensing of objects tens to hundreds of meters away. Courtesy of Voyant.

BAE Systems Sensor Solutions - Fairchild - FS Sensor Solutions 4/24 MR

Figure 6. Solid-state lidar will integrate lidar onto a photonic integrated circuit that fits on a fingertip. Voyant Photonics is pursuing such lidar on a chip using an optical phased array to steer an FMCW lidar beam in two dimensions, enabling distance and velocity sensing of objects tens to hundreds of meters away. Courtesy of Voyant.

According to Jason Eichenholz, chief technology officer and co-founder of lidar company Luminar Technologies Inc. in Palo Alto, Calif., “It’s been 100 years since the Model T. It’s high time for flying cars. The lidar systems, drone technology, and factory automated electric vehicles are ready. The biggest barrier is really a matter of developing regulations and flying rules.”

The aviation industry consensus is that flying cars will be launching in the next few years and are sure to become common by 2030.

Reference

1. S. Miller et al. (2018). 512-element actively steered silicon phased array for low-power LIDAR. Conf. on Lasers and Electro-Optics, OSA, paper JTh5C2.

Flying Vehicles in Development

As of this year, more than 100 companies are developing flying vehicles, from electric bikes to hexacopters. Below is a list of 36 such companies — showing headquarters, models, and flight mechanisms. Of this list, eight (to our knowledge) are using lidar, and these are highlighted in dark blue.

COMPANY (Headquarters)	MODEL	FLIGHT MECHANISM
A3/Airbus (Calif.)	Vahana	Autonomous 8-propeller tilt-wing eVTOL
AeroMobil 5.0 (Slovakia)		Folding-wing VTOL
Airbus/Audi/Italdesign (Germany/Italy)	Pop.Up Next	Modular pod docking with autonomous car or 8-rotor eVTOL quadcopter
Alaka’I (Mass.)	Skai	Hydrogen fuel cell eVTOL flying taxi
Aston Martin (England)	Volante Vision
Aufeer Design (Czech Republic)	Flying Taxi
Aurora Flight Sciences/Boeing (Va.)	Passenger Air Vehicle (PAV)	Unmanned fixed-wing eVTOL air taxi
Aviaereo (England)	Aereo-bee
Bartini (Russia)	Flying car
Bay Zoltán Nonprofit (Hungary)	Flike	Three-rotor personal manned electric flying bike
Bell Aerospace (Texas)	Nexus	Piloted VTOL hybrid-electric 4-passenger hexacopter with tilting rotors
Boeing (Illinois)	Cargo Aerial Vehicle	Unmanned 8-propeller eVTOL
Cartivator (Japan)		Drone
CollaborativeBee (France)	Mini-Bee
DeLorean Aerospace (Calif.)	DR-7
EHang (China)	E-184	Electric fan quadcopter
Electric Visionary Aircrafts (France)	X01	eVTOL with 26 propellers, folding wing
EmbraerX (Brazil)	eVTOL	Octocopter with rear propellers
Gizio (Italy)	CellCraft G450
HopFlyt (Md.)	Venturi
Hoversurf (Calif.)	Formula (no wing)
Joby Aviation (Calif.)		Fixed-wing VTOL
Karem Aircraft (Calif.)	Butterfly	Fixed-wing quad tilt-rotor eVTOL air taxi
Kitty Hawk (Calif.)	Flyer	Over-water recreational ultralight
Lilium Aviation (Germany)	Jet	Pilotless fixed-wing 36-fan eVTOL taxi
NASA (Va.)	Greased Lightning
NEO Aeronautics (Singapore)	Crimson S8	Two-passenger, 4-propeller hybrid eVTOL ultralight
Neva Aerospace (England)	AirQuadOne	8-fan quadcopter
Opener (Calif.)	BlackFly	Fixed-wing VTOL ultralight
PAL-V (Netherlands)	Liberty	Retractible single-rotor gyrocopter
Sabrewing Aircraft (Calif.)	Rhaegal	VTOL 4-fan hybrid electric turbine quad cargo plane
Samad Aerospace (England)	Starling Jet
Sukhoi Civil Aircraft JSC (Russia)	Begaero	VTOL air taxi
Terrafugia (Mass.)	Transition	Piloted folding-wing STOL hybrid-electric roadable plane
Volocopter (Germany)	Volocopter 2X	Two-passenger, 18-fan aerial electric copter
Waters Trust (U.S.)	Vision VTOL	Electric long-range VTOL single-seat prototype

Compiled by: Valerie C. Coffey, September 2019. Sources: Technologyreview.com, Evtol.news, Wired.com.

Abbreviations: eVTOL (electric vertical takeoff and landing); STOL (short takeoff and landing). First flights: Between 2014 and 2019.

Range: 25 to 400 miles (40 to 643 km). Note: Some cells intentionally left blank where data was not available.