This use case describes a scenario where a vehicle equipped with an advanced driver assistance system (ADAS) encounters a truck with cargo, such as lumber, jutting out of its bed into the lane. This situation poses a significant safety risk, as the protruding cargo can be difficult for conventional perception systems to detect, especially under adverse lighting or weather conditions.
Detecting cargo protruding from a vehicle is challenging because conventional perception systems (cameras, radar, and fixed-pattern LiDAR) struggle with small, stationary, or non-reflective objects, especially in poor lighting or weather. Cameras may misinterpret the cargo due to 2D limitations and lighting issues, while radar often ignores stationary or non-reflective objects. Conventional LiDAR may not collect enough data points early enough to recognize the threat.
AEye's iDAR (Intelligent Detection and Ranging) system dynamically adapts its LiDAR scanning patterns to focus on the detected cargo. Upon a single detection, iDAR schedules additional LiDAR shots to interrogate the area, increasing spatial and temporal sampling density. This ensures accurate detection and classification of the cargo, allowing the domain controller to plan the safest response, such as braking or swerving to avoid a collision.
Cameras are limited by their 2D perspective, making it difficult to interpret the 3D position of cargo. Lighting conditions, such as glare or low light, can create blind spots or insufficient dynamic range, further hindering detection. Cameras also require extensive training to recognize the vast variety of cargo shapes and scenarios.
Radar systems typically disregard small, stationary, or non-reflective objects to avoid overwhelming the system with irrelevant data. In the case of protruding cargo, radar may not detect the object at all, especially if it is narrow or surrounded by reflective surfaces like a truck bed.
Conventional LiDAR systems use fixed scan patterns and may not collect enough data points on small or narrow objects like protruding cargo. Many systems require multiple detections to register an object, so they may fail to recognize the threat in time to avoid a collision.
A Dynamic Region of Interest (ROI) is an area that AEye's iDAR system focuses on after detecting a potential threat. The system increases the density of LiDAR shots in this region, gathering more detailed data to accurately classify and locate the object, such as protruding cargo.
iDAR employs feedback loops between the camera and LiDAR sensors. When more data is needed, the camera can cue the LiDAR to focus on specific areas, and vice versa. This iterative process ensures comprehensive data collection for accurate object detection and classification.
Computer vision in iDAR combines 2D camera pixels with 3D LiDAR voxels to create Dynamic Vixels. This fusion allows the system's AI to refine the LiDAR point cloud, focusing on relevant data and eliminating noise, which is critical for accurately detecting and classifying protruding cargo.
Upon a single LiDAR detection, iDAR immediately flags the cargo as a potential threat and schedules a rapid series of additional LiDAR shots to interrogate the area. This dynamic adjustment ensures the system gathers enough data to accurately assess the situation and inform the vehicle's domain controller.
AI-embedded LiDAR sensors, like AEye's iDAR, can intelligently adapt their scanning patterns and data processing in real time. This enables faster and more accurate detection of complex or rare scenarios, such as cargo protruding from vehicles, improving overall safety and reducing false negatives compared to passive data collection systems.
You can download the technical PDF for the 'Cargo Protruding from Vehicle' edge case directly from this link.
By dynamically focusing its sensors and increasing data collection on the detected cargo, iDAR provides the vehicle's domain controller with accurate information about the object's position and distance. This enables the system to plan and execute the safest response, such as braking or swerving, to avoid a collision.
If the camera cannot provide sufficient data due to poor lighting or other factors, iDAR's LiDAR system will automatically increase point density on and around the detected object, ensuring reliable detection and classification even without camera input.
Dynamic Vixels are created by fusing 2D camera pixels with 3D LiDAR voxels. This allows iDAR's AI to refine the point cloud, focusing only on relevant data and improving the accuracy of object detection and classification in complex scenarios like protruding cargo.
The domain controller receives actionable data from iDAR about the detected cargo, including its position and distance. It then determines and executes the safest maneuver, such as braking or swerving, to avoid a collision.
Unlike camera and radar fusion systems, which may miss small or stationary objects due to their inherent limitations, AEye's iDAR system uses dynamic LiDAR scanning and AI-driven data fusion to ensure reliable detection and classification, even in challenging scenarios and environments.
Dynamic scan patterns allow iDAR to focus sensor resources on areas of interest, such as detected cargo, increasing detection reliability and reducing false negatives. This adaptability is crucial for handling edge cases that conventional fixed-pattern systems may miss.
iDAR's LiDAR technology is less affected by lighting and weather conditions compared to cameras and radar. Its dynamic scanning and AI-driven data processing ensure reliable detection of objects like protruding cargo, even in rain, darkness, or glare.
You can explore additional use cases and edge case scenarios for AEye's technology on the AEye Resources Page.
For more information about iDAR and its applications, visit the iDAR technology page or download technical whitepapers from the AEye Resources Page.
AEye's iDAR system offers dynamic scan patterns, real-time adjustment of LiDAR focus, AI-driven data fusion, feedback loops between sensors, and the ability to create Dynamic Regions of Interest. These features enable reliable detection of complex scenarios like cargo protruding from vehicles.
Yes, AEye's software-defined LiDAR technology supports over-the-air updates, ensuring the system remains adaptable and future-proof as new challenges and scenarios emerge. [Source]
AEye's LiDAR systems are engineered to perform reliably in challenging environments, including rain, darkness, and fog, ensuring consistent detection and operational reliability. [Source]
Yes, AEye's compact LiDAR design supports flexible mounting options, including in-cabin, on the roof, or in the grille, allowing OEMs to implement safety features with minimal impact on vehicle design. [Source]
AEye provides specification sheets, white papers, validation reports, and case studies. For example, the Apollo solution spec sheet and 'Lidar Case Studies for ITS Use Cases' are available for download on the AEye Resources Page.
AEye's Apollo sensor is integrated with the NVIDIA DRIVE AGX platform, including NVIDIA AGX DRIVE Thor™, and supports flexible OEM integration options for various vehicle placements. [Source]
Key performance highlights include dynamic scan patterns, ultra-long-range detection (up to one kilometer with Apollo), high resolution, adaptability to challenging environments, future-proof technology with over-the-air updates, and flexible placement options. [Source]
AEye's LiDAR systems use AI-driven data fusion and dynamic scanning to differentiate between real and false obstacles, reducing unnecessary braking or maneuvers and improving operational efficiency. [Source]
Industries benefiting from AEye's LiDAR include automotive, trucking, smart infrastructure, aviation, defense, rail, and logistics. [Source]
Case studies include 'A Pedestrian in Headlights', 'Flatbed Trailer Across Roadway', 'Obstacle Avoidance', 'False Positive', and 'Cargo Protruding from Vehicle'. These demonstrate AEye's effectiveness in challenging scenarios. [Source]
You can access whitepapers, datasheets, case studies, and technical documents on the AEye Resources Page.
Velodyne offers traditional LiDAR systems with fixed scan patterns and focuses on high-resolution imaging. AEye differentiates itself with dynamic scan patterns, software-defined architecture, and over-the-air updates, allowing for greater adaptability and future-proofing. [Source]
Luminar focuses on long-range LiDAR for autonomous vehicles with a primarily hardware-based approach. AEye offers dynamic scan patterns, adaptability to challenging environments, and flexible mounting options, providing more versatility for different applications. [Source]
Innoviz offers solid-state LiDAR with a focus on automotive applications but has limited software-defined customization. AEye's LiDAR is customizable via software, supports over-the-air updates, and offers high performance for demanding scenarios. [Source]
AEye's LiDAR stands out due to its dynamic scan patterns, software-defined architecture, adaptability to challenging environments, future-proof design with over-the-air updates, and flexible placement options. These features provide scalability, efficiency, and long-term value for various industries. [Source]
AEye's products are designed for ease of integration with existing systems, supported by comprehensive technical documentation, validation tools, and direct technical support. This ensures a smooth and efficient onboarding process. [Source]
AEye provides extensive training resources, technical documentation, tutorials, hands-on training sessions, and direct access to technical experts to assist with implementation and troubleshooting. [Source]
You can find case studies about LiDAR use cases from AEye in the Lidar Case Studies for ITS Use Cases PDF.
AEye's technology is used by companies in automotive, intelligent transportation, aviation, defense, rail, and smart infrastructure. Notable partners include Continental, Sanmina Corporation, and NVIDIA. [Source]
Human drivers confront and handle an incredible variety of situations and scenarios—terrain, roadway types, traffic conditions, weather conditions—for which autonomous vehicle technology needs to navigate both safely, and efficiently. These are edge cases, and they occur with surprising frequency. In order to achieve advanced levels of autonomy or breakthrough ADAS features, these edge cases must be addressed. In this series, we explore common, real-world scenarios that are difficult for today’s conventional perception solutions to handle reliably. We’ll then describe how AEye’s software definable iDAR™ (Intelligent Detection and Ranging) successfully perceives and responds to these challenges, improving overall safety.
A vehicle equipped with an advanced driver assistance system (ADAS) is driving down a road at 20 mph. Directly ahead, a large pick-up truck stops abruptly. Its bed is filled with lumber, much of which is jutting out the back and into the lane. If the driver of an ADAS vehicle isn’t paying attention, this is a potentially fatal scenario. As the distance between the two vehicles quickly shrinks, the ADAS vehicle’s domain controller must make a series of critical assessments to identify the object and avoid a collision. However, this is dependent on its perception system’s ability to detect the lumber. Numerous factors can negatively impact whether or not a detection takes place, including adverse lighting, weather, and road conditions.
Today’s advanced driver assistance systems (ADAS) will experience great difficulty recognizing this threat or reacting appropriately. Depending on their sensor configuration and perception training, many will fail to register the cargo before it’s too late.
Camera. In scenarios where depth perception is important, cameras run into challenges. By their nature, camera images are two dimensional. To an untrained camera, cargo sticking out of a truck bed will look like small, elongated rectangles floating above the roadway. In order to interpret this 2D image in 3D, the perception system must be trained—something that is difficult to do given the innumerable permutations of cargo shapes. The scenario becomes even more challenging depending on time of day. In the afternoon, sunlight reflecting off the truck bed or directly into the camera can create blind spots, obscuring the cargo. At night, there may not be enough dynamic range in the camera image for the perception system to successfully analyze the scene. If the vehicle’s headlights are in low beam mode, most of the light will pass underneath the lumber.
Radar. Radar detection is quite limited in scenarios where objects are small and stationary. Typically, radar perception systems disregard stationary objects because otherwise, there would be too many objects for the radar to track. In a scenario featuring narrow, non-reflective objects that are surrounded by reflections from the metal truck bed and parked cars, the radar would have great difficulty detecting the lumber at all.
Camera + Radar. Due to the above explained deficiencies, in most cases, a system that combines radar with a camera would be unable to detect the lumber or react quickly. The perception system would need to be trained on an almost infinite variety of small stationary objects associated with all manner of vehicles in all possible light conditions. For radar, many objects are simply less capable of reflecting radio waves. As a result, radar will likely miss or disregard small, non-reflective stationary objects. In addition, radar would be incapable of compensating for the camera’s lack of depth perception.
LiDAR. Conventional LiDAR doesn’t struggle with depth perception. And its performance isn’t significantly impacted by light conditions, nor by an object’s material and reflectivity. However, conventional LiDAR systems are limited because their scan patterns are fixed, as are their Field-of-View, sampling density, and laser shot schedule. In this scenario, as the LiDAR passively scans the environment, its laser points will only hit the small ends of the lumber a few times. Typically, LiDAR perception systems require a minimum of five detections to register an object. Today’s 4-, 16-, and 32-channel systems would likely not collect enough detections early enough to determine that the object was present and a threat.
Accurately measuring distance is crucial to solving this challenge. A single LiDAR detection will cause iDAR to immediately flag the cargo as a potential threat. At that point, a quick series of LiDAR shots will be scheduled directly targeting the cargo and the area around it. Dynamically changing both LiDAR’s temporal and spatial sampling density, iDAR can comprehensively interrogate the cargo to gain critical information, such as its position in space and distance ahead. Only the most useful and actionable data is sent to the domain controller for planning the safest response.
Computer Vision. iDAR combines 2D camera pixels with 3D LiDAR voxels to create Dynamic Vixels. This data type helps the system’s AI refine the LiDAR point cloud on and around the cargo, effectively eliminating all the irrelevant points and creating information from discrete data.
Cueing. As soon as iDAR registers a single detection of the cargo, the sensor flags the region where cargo appears and cues the camera for deeper real-time analysis about its color, shape, etc. If light conditions are favorable, the camera’s AI reviews the pixels to see if there are distinct differences in that region. If there are, it will send detailed data back to the LiDAR. This will cue the LiDAR to focus a Dynamic Region of Interest (ROI) on the cargo. If the camera lacks data, the LiDAR will cue itself to increase the point density on and around the detected object creating an ROI.
Feedback Loops. A feedback loop is triggered when an algorithm needs additional data from sensors. In this scenario, a feedback loop will be triggered between the camera and the LiDAR. The camera can cue the LiDAR, and the LiDAR can cue additional interrogation points, or a Dynamic Region of Interest, to determine the cargo’s location, size, and true velocity. Once enough data has been gathered, it will be sent to the domain controller so that it can decide whether to apply the brakes or swerve to avoid a collision.
LiDAR sensors embedded with AI for intelligent perception are very different than those that passively collect data. As soon as the perception system registers a single valid LiDAR detection of an object extending into the road, iDAR responds intelligently. The LiDAR instantly modifies its scan pattern, increasing laser shots to cover the cargo in a dense pattern of laser pulses. Camera data is used to refine this information. Once the cargo has been classified, and its position in space and distance ahead determined, the domain controller can understand that the cargo poses a threat. At that point, it plans the safest response.
