From The Editor | June 13, 2023

A Look At The Future Of 4D Imaging Radar In Autonomous Driving

John Headshot cropped  500 px wide

By John Oncea, Editor

GettyImages-1383567972 Autonomous Vehicles

4D imaging radar, used in vehicles to detect, track, and analyze the surrounding environment in real time, provides accurate and detailed information about objects in 3D space including their positions, velocities, trajectories, and time dimensions.

Its primary function is to detect and track objects around the vehicle, such as vehicles, pedestrians, cyclists, and obstacles. By continuously updating the position, velocity, and trajectory of these objects, the radar system helps drivers avoid collisions and enables advanced driver assistance features (ADAS) like adaptive cruise control.

According to IEEE, “4D image radar has shown great potential of being either a stand-alone or a complementary sensor in level-x autonomous driving (AD) system while contributing its unique measurement – velocity to enhance the accuracy of perception under a dynamic traffic environment.”

Because of these benefits, 4D imaging radar – particularly the 4D millimeter-wave radar – is expected to play a key role in the development of AD due to its ”robustness in extreme environments and outstanding velocity and elevation measurement capabilities,” according to research posted to arXiv. “However, despite the rapid development of research related to its sensing theory and application, there is a notable lack of surveys on the topic of 4D mmWave radar.”

To address this gap, the authors – Zeyu Han, Jiahao Wang, Zikun Xu, Shuocheng Yang, Lei He, Shaobing Xu, and Jianqiang Wang – administered a comprehensive survey on the use of 4D mmWave radar in AD. They looked at signal processing flow, resolution improvement ways, extrinsic calibration processes, and point cloud generation methods before predicting future trends in the field of 4D mmWave radar.

The Future Of 4D mmWave Radar And Autonomous Vehicles

Though far from mature, 4D mmWave radar, according to this study, has the potential to bring about profound changes to autonomous vehicles. The authors cite four areas on which they think future trends of 4DmmWave may rely:

  1. Point cloud enhancement: 4D mmWave radar point clouds, the most used data format, have lower quality due to multi-path effects. Information loss during signal processing can be reduced by replacing CFAR with learning-based methods. DOA estimation also can benefit from exploring learning-based methods for super-resolution angle estimation instead of DBF methods.
  2. Application algorithms redesign: Improving 4D mmWave radar algorithms is important. Some current ones are adapted from LiDAR algorithms. Future research should explore the unique abilities of 4D mmWave radars. Multi-modal fusion is the future for perception tasks, but it's unclear if integrating other sensors will weaken 4D radar's robustness in extreme weather. There's potential for discovery in fusing 4D mmWave radar with LiDARs and cameras for localization and mapping.
  3. Pre-point cloud data utilizing: Utilizing unique 4D mmWave radar data formats for perception, localization, and mapping is an untapped research area. Learning-based models with real-time performance could be a hot topic.
  4. Dataset enriching: There is a need for more 4D mmWave radar datasets as they are rare and there is room for expansion in data formats and scenario richness.

Maybe The Future Here Now

NXP Semiconductors recently announced that Chinese electric vehicle (EV) automaker NIO will be deploying NXP’s radar technology, including its 4D imaging radar solution. “These technologies promise significant advantages – accurate and detailed object detection and classification, sophisticated sensor resolution, and extended detection range – over traditional radars,” writes Frost & Sullivan. “Such enhanced capabilities are set to strengthen safety and convenience features while accelerating the development of next-level ADAS and AD.”

The acquisition, along with ZF’s launch of its 4D, high-resolution imaging radar technology for Chinese automaker SAIC Motor Corporation’s R-Series vehicles and MobilEye’s collaboration with Wistron to produce imaging radars within the next two years, continue to signal the impending importing and growth of 4D imaging radar.

“Such radars could replace traditional perception radars, although this will depend on cost considerations and OEMs’ sensor suite strategy,” Frost & Sullivan writes. “A key advantage of 4D imaging sensors over traditional radar systems is their ability to determine the height of objects on the vertical axis and accordingly, classify them.

“At the same time, the combination of camera and imaging radar could become a common occurrence in mass market vehicles, with the expensive LiDAR sensors limited to premium vehicles, and vehicles with L3 and higher autonomous capabilities. Imaging radar sensors are not expected to face competition from LiDAR sensors till they are economical enough to be used in mass market vehicles, which is expected to happen beyond 2025.”

According to Frost & Sullivan, the majority of OEMs are expected to implement 4D imaging radars in their ADAS/AD sensor suite, along with cameras, LiDARs, and other perception and imaging radars. This is due to the rising popularity of L2 and L2+ ADAS-equipped vehicles in developed markets, which requires a cost-effective sensor that can provide accurate point cloud data with minimal false positives.

They also predict that imaging radar sensors will excel in this area. The improvement in price-performance ratio, scalability, and technology advancements – from multiple chipsets and sensors to single-chip solutions with simultaneous short, medium, and long-range detection capabilities – will increase the adoption rates of imaging sensors. This also will lead to the potential of reduced sensor suite costs for ADAS.

Frost & Sullivan further expects the market for passenger vehicle imaging radar sensors in North America and Europe to reach approximately 3.7 million units by 2030. As vehicles become more advanced, they may require between one to five imaging radars, depending on the level of autonomy. These radars have the potential to replace traditional short and long-range radars.