Occlusion Detection Without Labels: Dual Monocular Distance Cue Disagreement

Robotic perception systems often need to determine whether a pedestrian is occluded, but occlusion-labeled data is extremely difficult to obtain in real-world deployments. A new study proposes that occlusion can be detected without training and without occlusion labels by simply exploiting the ordinal disagreement between two distance estimates already computed in the monocular camera pipeline—apparent height and monocular metric depth. The method achieves an AUC of 0.909 for detecting severely occluded pedestrians on the KITTI dataset, does not degrade with distance, and offers a new approach for cross-domain deployment.

The Annotation Dilemma in Occlusion Detection and Limitations of Existing Methods

In robotics applications, knowing that a pedestrian is occluded is valuable in itself, even if the detector cannot recover the occluded pixels. The problem is that occlusion estimators typically rely on occlusion annotations for training, and real deployment scenarios rarely have such annotations. For example, a detector trained on German roads, when transferred to a self-annotated street scene in Lagos, Nigeria, saw its accuracy drop by half. This domain shift exposes the fragility of relying on local annotations.

The research team first evaluated how three standard architectures handle occlusion: deep fusion, occlusion simulation augmentation, and attention mechanisms. In the configurations tested, none of the three methods improved detection performance outside the training distribution. This prompted the authors to look for signals that require no local annotations.

Core Idea: Ordinal Disagreement Between Two Distance Cues

The key observation of the method is that apparent height is barely affected by occlusion, because the bounding box still covers the entire person; whereas monocular metric depth is pulled closer by the occluder, because the depth map reports the nearest surface within the box. Thus, occlusion can be inferred simply by comparing the ordinal relationship between two distance estimates—the distance inferred from apparent height and the distance given by the depth map.

Because it relies only on ordinal relationships, the method requires no focal length, height, or depth scale. The premise is that the depth model's error is approximately a pure gain, a condition that can be checked without labels. The authors emphasize that the score is not a proxy for distance: within fixed distance intervals, its AUC remains between 0.894 and 0.925, whereas the AUC using true distance alone is close to random.

Apparent height refers to the pixel height of the pedestrian bounding box in the image. Under the pinhole camera model, the image height of the same object is inversely proportional to distance: the farther away, the smaller the image. Thus, if the typical height of a pedestrian and the camera focal length are known, distance can be inferred from the bounding box height. However, the method does not require these parameters because it only compares relative ordinal relationships between different pedestrians, not absolute distances. When occlusion occurs, the bounding box typically still covers both the visible and invisible parts of the pedestrian, so apparent height changes little, making it an occlusion-insensitive distance cue.

Monocular metric depth refers to estimating the absolute depth of each pixel (usually in meters) from a single RGB image. The depth map reports the nearest distance to the scene surface corresponding to each pixel. When a pedestrian is occluded, the bounding box contains occluders (such as vehicles or railings) that are usually closer to the camera than the pedestrian, causing the statistical value (e.g., median or mean) of the depth map within the box to skew toward closer distances, systematically underestimating the pedestrian's distance. This bias is precisely the source of the ordinal disagreement.

Experimental Validation and Cross-Domain Performance

On the KITTI dataset, the method achieves an AUC of 0.909 for detecting severely occluded pedestrians; when including all annotated pedestrians (including those missed by the detector), the AUC is 0.888. In synthetic CARLA scenes with measured visibility, the method reaches 0.819, compared to 0.644 for distance alone. These results are consistent across evaluations from multiple sources, indicating that the ordinal disagreement signal is robust.

You may not have noticed that the method requires no training and no occlusion labels, making it particularly suitable for deployment environments where annotations are scarce. The authors point out that in scenarios not represented by benchmarks, signals that require no local annotations are more valuable.

AUC (Area Under the ROC Curve) is a common metric for measuring the performance of a binary classifier, with 0.5 indicating random guessing and 1.0 indicating perfect discrimination. In the occlusion detection task, AUC measures the probability that, given a randomly selected occluded pedestrian and a non-occluded pedestrian, the model assigns a higher score to the occluded one. An AUC of 0.909 means that about 91% of sample pairs can be correctly ranked. AUC is insensitive to class imbalance, making it suitable for cases with few occlusion samples.

The KITTI dataset is a classic benchmark dataset in the autonomous driving field, containing synchronized camera images, LiDAR point clouds, and GPS/IMU data from urban, rural, and highway scenes, along with bounding box annotations for pedestrians and other objects. Its pedestrian annotations include occlusion levels (e.g., 0, 1, 2, 3), but this study does not use these occlusion labels for training—only for evaluation. KITTI's diversity makes it a common platform for testing cross-domain generalization.

Implications for Robotic Perception

This study suggests that leveraging complementary cues already computed in the existing pipeline may be more effective than introducing complex architectures. The limitations of deep fusion, data augmentation, and attention mechanisms in out-of-distribution scenarios highlight the value of a simple ordinal signal. For robotic systems that need to work in diverse environments, this annotation-free occlusion detection method provides a low-cost, transferable path.

However, the method relies on the assumption that the depth model's error is approximately a pure gain, which may be limited in scenarios with complex depth estimation biases. Future work could explore combinations of more distance cues and validation on more diverse datasets.

One-line Summary

Exploiting ordinal disagreement between apparent height and monocular depth to detect pedestrian occlusion without training or occlusion labels, achieving an AUC of 0.909 on KITTI.

Paragraph Summary

Occlusion annotations are hard to obtain in robotic perception, and existing occlusion estimators rely on local annotations, suffering severe performance drops in cross-domain deployment. A new study proposes an occlusion detection method that requires no training or occlusion labels: it compares the ordinal disagreement between the distance inferred from apparent height and the monocular metric depth already computed in the monocular camera pipeline. Apparent height is barely affected by occlusion, while the depth map is pulled closer by occluders; the ordinal disagreement between the two indicates occlusion. On KITTI, the AUC for severely occluded pedestrian detection reaches 0.909, with AUC stable at 0.894–0.925 within fixed distance intervals; in synthetic CARLA scenes, the AUC is 0.819, significantly better than 0.644 using distance alone. Deep fusion, occlusion augmentation, and attention mechanisms did not improve performance in out-of-distribution scenarios, highlighting the value of a simple ordinal signal. The method relies on the assumption that the depth model's error is approximately a pure gain, providing a low-cost, transferable path for robotic deployments with scarce annotations.