LiDAR and 3D Point Cloud Annotation: The Annotation Stack Most Vendors Get Wrong

Most annotation platforms were built for 2D images, then bolted on 3D support when self-driving and robotics customers started asking. The result is a category of tools that technically support point cloud labeling but make the workflow miserable. If you've spent a week trying to label cuboids on point clouds in a 2D-first tool, you know what miserable looks like.

What makes 3D fundamentally different

Three properties of point cloud data that 2D-first tools don't handle well. Sparsity — a LiDAR scan is mostly empty space; objects of interest are surrounded by null voxels and need 3D context to be visible. Multisensor fusion — most AV stacks include LiDAR plus camera plus radar; annotations need to align across modalities. Temporal continuity — objects move across frames at known velocities; annotation tools need to respect this physics.

Each of these is non-trivial. Together they require a workflow that thinks in 3D + time, not in still images.

The features that actually matter

Cuboid annotation with axis-locked editing — a 3D box with separate controls for length, width, height, yaw, position. 6 degrees of freedom. Most tools expose this awkwardly.

Side-view + top-view + camera-view side-by-side. Annotators need to verify a cuboid is correct from multiple angles. Single-view 3D labeling produces unusable annotations.

Cross-sensor projection — when the annotator places a cuboid in LiDAR, the same box appears projected onto the camera image. Misalignment becomes visible immediately.

Multi-frame tracking — once an object is labeled in frame N, the tool propagates the cuboid forward, the annotator corrects rather than re-creates. SAM-2-style tracking adapted to 3D.

Semantic segmentation of point clouds — for road / sidewalk / building classes, polygon-equivalent labeling that handles thousands of points at a time.

The taxonomy problem

AV taxonomies are large — 50-200 classes covering vehicle types, pedestrian states, road infrastructure, traffic controls. Building this taxonomy once is hard; evolving it is harder. Tools need first-class versioncontrolled taxonomies that can update mid-project without re-labeling history.

Quality measurement

IoU on 3D cuboids is a different beast than 2D IoU. Inter-annotator agreement on 3D segmentation is lower than 2D because subtle boundary disagreements are easier in 3D space. Plan for 60-75% agreement on the first pass, with calibration tasks pulling that up over time.

Sensor fusion is the differentiator

The annotation tool's value compounds when it natively handles camera + LiDAR fusion. Annotators label a vehicle in the LiDAR view, the tool draws the projected 2D bounding box on the camera image, the annotator confirms or adjusts. This bidirectional editing produces cleaner data than labeling each modality separately.

Where Intellabel sits

Intellabel's Geospatial / Sovereign vertical pack includes native LiDAR support, cross-sensor projection, multi-frame tracking, and 3D segmentation. The Enterprise tier adds on-prem deployment, which is essential for AV data that often cannot leave customer infrastructure for regulatory reasons. The pack is optional — only relevant for teams shipping 3D or geospatial AI — but for those teams it's the difference between an annotation tool that works and one that's a daily friction.

If you're labeling in a 2D-first tool, you know

Teams currently labeling 3D in tools that were not built for it produce data slower, with more errors, and with worse inter-modality alignment than they'd accept on 2D projects. The fix isn't workarounds. It's a tool that was built for the modality.

From Labeling to Structured AI Data Pipelines