Hundreds of Experts on the Internet of Vehicles | How to build a smart and high-speed Leivision fusion perception base?

Editor’s note: In order to promote the development of the Internet of Vehicles industry, experts and scholars in the industry are invited to jointly offer advice and suggestions, and launch a series of “Hundreds of Talks on the Internet of Vehicles”. Whether the roadside perception system can provide reliable and accurate data will become one of the key points to achieve the near- and long-term goals of smart highway construction. Mr. Yang Daning’s article analyzes the realization of all-weather global all-element target detection, holographic perception of global identity and feature tracking, and event detection with lane-level precision through the Leivision fusion perception base, which has strong reference value for industry partners.

Introduction

In the past six months, many self-driving truck companies have received hundreds of millions of dollars in financing. The self-driving truck industry is considered to be the first self-driving track with a profit model and commercial landing, and has been favored by capital. On the other hand, with the implementation of my country’s strategy of strengthening the country in transportation, the development of expressways has entered a new stage of vehicle-road coordination. How to establish a data-driven as the core, through the full-time full coverage of all-element perception, to realize the intelligent highway above I3 level, and to solve the current industry pain points, will become the key to a new round of highway construction.

01. New infrastructure leads a new wave of smart high-speed

01. With the support of national policies, giant companies are rushing in

In recent years, the relevant state departments have issued a number of industrial policies to support the development of intelligent transportation, such as the “Outline for Building a Strong Transportation Country”, “Outline for National Comprehensive Three-dimensional Transportation Network Planning”, “The Fourteenth Five-Year Plan for National Economic and Social Development and 2035”. The Outline of Annual Vision and Goals, etc., all pointed out that it is necessary to deploy a traffic perception system in an all-round way, digitize infrastructure, integrate road transportation, vehicle-road coordination, comprehensive application of Beidou high-precision positioning, comprehensive management of road network based on big data, “Internet +” Road network integrated services and a new generation of national traffic control network will be the focus to develop autonomous driving and vehicle-road coordinated travel services.

With the support of policies, many companies have also poured into the smart high-speed market. Emerging Internet companies, large-scale transportation informatization construction integrators, and well-known suppliers of technology products in the highway industry have all carried out smart high-speed industrial layouts, focusing on high-speed cloud. control platform, V2X management and control platform, AI intelligent analysis platform and intelligent holographic perception base station (Rayvision fusion, millimeter wave radar, video, RSU, etc.). 02. Smart high-speed pilots are “blooming everywhere”, and the industrial construction goals are becoming clearer and more intelligent high-speed pilots are increasing. The latest released data include Guizhou, Shandong, Hebei Xiong’an, Shenzhen, Guangxi, Zhejiang, Jiangsu, Jilin, Guangdong, Jiangxi, Anhui, Shanxi , Sichuan, Shaanxi, Yunnan, etc. The current smart high-speed pilot has achieved phased results, but there are still many key technologies that need to be overcome, especially how the fusion perception technology can provide basic data support for autonomous driving and vehicle-road collaborative interaction. With the continuous development of smart high-speed pilot projects, its construction goals have gradually become clear.

The short-term goal is to achieve safety improvement, efficiency improvement and service improvement through all-factor perception, all-round service, whole-process control, all-digital operation, and vehicle-road collaboration. The rate of road traffic accidents in severe weather and complex environments has dropped significantly. The traffic efficiency of key nodes and road sections has been greatly improved. Realize the digitization and intelligence of the whole life cycle of highway construction, management, maintenance and operation. The long-term goal is to realize all-factor perception, all-round service, all-process control, all-digital operation, etc. within the entire road network, realize the intelligent network connection and efficient collaboration of “people, vehicles, and roads”, and realize a fully intelligent expressway. Business management, to achieve vehicle formation and L3-level automatic driving. Whether the roadside perception system can provide reliable and accurate data will become the key to realizing the near- and long-term goals of smart expressway construction.

02. The advanced roadside perception system is the basis for the construction of a new generation of intelligent expressways

At present, the main pain points of my country’s expressway transportation management are: congestion: how to improve traffic efficiency, and use intelligent means to monitor, model, optimize, and divert. Accidents: how to find, deal with, analyze and prevent accidents in the first place. At this stage, the highway information collection system mainly relies on high-definition video surveillance equipment.

Although it can be processed by the background video AI algorithm to realize functions such as license plate recognition, traffic flow statistics, and abnormal event recognition, there are still many problems, such as insufficient coverage, detection accuracy is greatly affected by weather, and delay is too long. In the new round of high-speed intelligent investment and construction, it is not only necessary to consider providing better monitoring methods for existing operation and management, but also to provide effective roadside infrastructure support for autonomous driving and vehicle-road coordination in the future. As we all know, the cost of self-driving vehicles depends largely on the perception system and computing system.

If roadside equipment can provide effective perception redundancy for vehicles running at high speeds, it will greatly reduce the manufacturing cost of autonomous vehicles themselves and improve the safety of vehicle operation. The transportation and management of expressways can also be charged for services, forming a return on investment in the construction of smart expressways. The advanced roadside perception system is an important foundation and base for smart highways, and it is also an important symbol to distinguish the intelligentization of the new generation of highways. The advanced roadside perception system should be data-driven as the core and based on holographic all-element perception, and build a roadside perception system with all-weather, full-time, full-featured, full-coverage, and high-integration.

According to the characteristics of the regional differences of highways in my country, it can accurately realize various traffic incident information collection and accident prevention functions, facilitate linkage with the traffic Electronic police system, conduct illegal evidence collection, standardize driving behavior, and create a green, safe and smooth traffic environment. At the same time, the accurate all-element target data can support the landing requirements of future vehicle-road collaboration scenarios, and provide over-the-horizon, all-weather, and full-coverage perception redundancy for autonomous vehicles.

03. Leivision Integrates to build a smart and high-speed perception base

01. System architecture In order to meet the current high-standard construction requirements for the rapid development of wisdom, it supports the integrated application of the existing equipment and the new infrastructure vehicle-road coordination system. Lei Sen launched the Lei Shi fusion roadside perception system to build a smart high-speed perception base.

The following figure shows the deployment diagram of the Raivision fusion roadside perception system. With the help of traffic poles or street light poles at the side of the highway, a dual-domain directional traffic millimeter-wave radar is installed every 1 km to form a front and rear coverage of 500 meters, and to accurately identify and locate traffic targets in the lane. The output data information includes position, speed, heading, traffic flow, average speed, etc. In addition, a Raivision integrated machine is installed on the highway gantry, which can collect information such as license plates (recognition rate over 99%), 23 types, 250 vehicle logos and colors of past targets, and at the same time, it can identify the accidental entry into the highway. of pedestrians and non-motorized vehicles. The data collected by the traffic millimeter-wave radar and the Raivision fusion machine are aggregated into the target fusion edge server for fusion, realizing the synchronous restoration of traffic flow and vehicle characteristics in the same coordinate system.

With the help of high-precision maps, lane-level high-definition real-time road condition information is displayed, and based on all-digital simulation technology and traffic algorithm models, refined management and control of expressways is completed.

Directional traffic millimeter-wave radar can cover a wide range of lanes, recognize many types of targets, and can detect various events. The Raivision integrated machine can perform traffic target identification, vehicle model identification and feature identification. Two-way transmission of the same target information between adjacent radars and radars and Raivision, including vehicle dynamic information, feature information and unique ID number identification information in the entire radar perception system, the same target between adjacent radars and Raivision The success rate of vehicle information transfer to each other is over 98%. At the same time, the radar and Raivision are linked with the existing video probes to implement secondary snapshot verification.

Through the ARM+NPU architecture, the target fusion edge server has AI computing capabilities and data storage capabilities. It can integrate data collected by multiple traffic radars, Raivision and RSU, and perform efficient target fusion through track processing algorithms to form a stable and reliable low-time The extended global target data source, combined with the high-precision map, shows the intelligent speed of holographic perception.

02. System application The Thunder Vision fusion roadside perception system can provide high-precision, high-reliability and low-latency global target data information for traffic control systems and vehicle-road collaboration scenarios, and build a smart high-speed data fusion perception base.

Flow control and accident prevention

Through traffic radar and Raivision roadside sensors, multi-target information such as traffic flow on highway main lines and ramps, and emergency event detection is collected, and the surrounding environment information collected by on-board sensors is fused. To the information processing and analysis system, the latest road traffic travel situation is provided to traffic participants in real time through roadside broadcasting, public account platforms, navigation platforms, guidance screens and on-board equipment, so that they can choose reasonable driving routes and travel modes. Flow control can be achieved by closing and adjusting the ramps according to traffic flow or emergencies. For hidden traffic accidents, based on the historical trajectories of vehicles involved in previous accidents, we can establish a mathematical model to analyze their characteristics, compare and analyze with the existing vehicle trajectory data, and formulate corresponding control strategies to avoid similar accidents from happening again.

Global real-time incident response

The entrances and exits of expressway service areas, ramps and junctions, emergency lanes and tunnels are all prone to traffic accidents, including illegal lane changes, reverse traffic at expressway intersections, overspeed and underspeed, and other accident causes. The Raivision Fusion Roadside Perception System detects vehicle speed, position, traffic and other information in real time through radar and Raivision global all-weather, tracks all target trajectories in real time, and releases information through the roadside unit RSU. The target fusion edge server will link the existing video probes for secondary capture and forensics, and upload specific traffic events to the highway accident early warning system in real time, supporting relevant personnel to make research and decision-making and event analysis, implement risk prevention and accident handling, and improve highway safety. capacity.

Vehicle-Road Collaboration Scenario Service

Whether it is the current stage of intelligent transportation development, or the future vehicle-road collaboration scenario service. The global total factor target is the data foundation supporting all traffic incidents. The Raivision fusion roadside perception system can realize real-time and full-element structured processing of all targets on the road. At the same time, it supports the requirements of I3-level conditional autonomous driving/highly networked infrastructure, such as supporting the construction of all DAYI/DAYII standard scenarios, including perceptible and transparent transmission of traffic events such as forward congestion reminders, abnormal vehicle reminders, and pedestrian intrusions. It can identify the identity of the bound vehicle and detect and report various traffic events. Provide rich data support for future vehicle-road collaboration scenarios.

03. System Features

All-weather global all-element object detection

The detection accuracy of the integrated traffic radar and Raivision is not affected by weather and environmental factors, and it can globally detect and track 512 targets within a range of 10 horizontal lanes and 500 meters vertically.

Holographic perception for global tracking of identity and features

By installing a dual-domain directional radar on the side of the light pole every 1 kilometer or so on the highway, covering 500 meters in the front and the rear, combined with the target fusion edge server, the “holographic restoration” of the entire road section is realized, and the Leivision fusion machine is linked. , realize the binding of traffic target identity and feature, and track the running status of each target in real time. Provides a high-reliability, high-integration, and low-latency data perception base for smart high-speed vehicles.

Lane-level accuracy, rich events, and convenient engineering layout

The Raivision fusion roadside perception system can track the trajectory in real time at 20Hz globally, and the vehicle speed can be accurate to 1km/h. Support 10 kinds of traffic event detection, locate accidents, include ramp split and merge guidance, special vehicle/toll evasion vehicle tracking, traffic accident prediction/discovery/playback, foggy driving guidance, control the main line traffic flow, support I3 level and above automatic Driving, holographic restoration of tunnel traffic conditions can be performed. The transportation millimeter-wave radar and Raivision integrated machine are easy to deploy, and the calibration of radar, video and GPS coordinate systems can be completed within 15-20 minutes using proprietary tools.

04. Conclusion

Compared with the traditional video network perception system, the Raivision fusion roadside perception system can realize all-weather, full-time, full-section, full-element, full-function real-time target detection and traffic parameters in the face of highways with large traffic, complex road conditions and rich scenes. Statistics and traffic incident reporting. The Raivision fusion roadside perception system not only realizes the holographic perception of the road section, but also can be integrated with the original video network equipment to collect evidence and capture road events. In the future, Raivision’s fusion roadside perception system can support high-speed applications of vehicle-road coordination and autonomous driving, and realize a commercial closed-loop. The perception base based on Rayvision fusion is the foundation of future high-speed intelligence, which will help improve highway safety, traffic efficiency, and management and operation capabilities.