Point Cloud
Encord Computer Vision Glossary
What Is a Point Cloud?
A point cloud is a collection of data points in three-dimensional space, where each point represents a location on a surface measured by a sensor. In physical AI, point clouds are primarily generated by LiDAR sensors, which emit laser pulses and record the 3D coordinates of whatever they reflect off.
Each point typically carries at a minimum its (x, y, z) position. In most automotive LiDAR systems, points also carry intensity, which indicates how strongly the laser pulse reflected, and sometimes additional attributes like return number or timestamp. The result is a structured representation of physical geometry that software can reason over.
How Point Cloud Data is Used
Point clouds serve as the input to 3D perception tasks: object detection, segmentation, and tracking. For object detection, annotators place 3D bounding box cuboids around objects of interest — vehicles, pedestrians, cyclists, so models can learn to detect and localise them. For segmentation, individual points are labelled with semantic categories, road surface, vegetation, building, and moving objects.
Beyond detection, point clouds are used to estimate free space (where can the vehicle safely drive?), build HD maps of environments, and fuse with camera data to produce complete scene representations.
The Challenges of Point Cloud Data
Point clouds are fundamentally different from images, and that difference shapes everything about how they're processed and annotated. They're unstructured, unlike images, where pixels are arranged in a regular grid, points are distributed irregularly across 3D space. They're sparse at distance — a pedestrian 80 metres away might be represented by only 5–10 points. And they change in density across the scene based on the object's distance and surface properties.
These properties make point cloud annotation harder than image annotation. Fitting a precise cuboid around a sparse cluster of points requires annotators to infer the full object shape from limited evidence, with no colour or texture cues to help.
Encord for Point Cloud Annotation
Encord's 3D workspace handles point cloud annotation natively, supporting cuboid labeling, semantic segmentation, and multi-frame object tracking on LiDAR data, with simultaneous camera projection for cross-sensor verification. Tools for pre-labeling seed initial annotations automatically, and timeline tracking maintains consistent object identities across sequences. The platform handles the high data volumes that LiDAR pipelines generate without requiring teams to stitch together separate tools for each modality.
→ Explore Encord for Physical AI
→ Explore Annotation & Labeling
Related Resources
Informational Guides:
Technical Documentations:
Webinars and video content:
Frequently Asked Questions
Q1: What's the difference between a point cloud and a 3D mesh?
A point cloud is a raw set of measured 3D points, no connectivity, no surfaces, just locations. A 3D mesh connects those points into triangles or polygons to form a surface. Point clouds are the direct output of LiDAR sensors; meshes are typically a post-processing step for applications that need a continuous surface representation.
Q2: How dense does a point cloud need to be for reliable annotation?
It depends on the object and the task. Close-range objects typically have hundreds or thousands of points and can be annotated precisely. Distant objects may have only a handful of points, annotation at that range requires experienced annotators and clear guidelines on how to handle sparse returns. Most annotation programs set distance thresholds below which objects don't need to be labeled.
Q3: Can point clouds be annotated automatically?
Pre-labeling models can propose initial cuboid placements based on learned object shapes, significantly reducing manual workload. But human review remains essential, particularly for occluded objects, sparse distant objects, and edge cases where the model's confidence is low.
