Label MCAP Recordings Natively in Encord (Beta)

We are excited to announce that we now support MCAP natively, in beta. You can upload .mcap recordings exactly as they were captured, label sensor streams on a single timeline, and export labels on the recording’s own timestamps. No conversion step, no offset to fix, on one timeline.

We built this for robotics and physical AI teams who:

• record in MCAP, or in ROS bags (which convert to MCAP easily)
• capture several sensors at once, such as LiDAR, cameras and IMU
• label 3D objects or time series, like tracking a pallet through a warehouse or marking a period of hard braking on a velocity signal

If that sounds like your data, here’s what you need to know.

What is MCAP?

MCAP is an open file format for robot recordings. A single file holds everything a robot logged during a run: camera images, point clouds, IMU readings, joint states and transforms. Messages are grouped by topic, and every one carries its own timestamp, so one file can hold a dozen sensors all logging at different rates.

Why frame-based labeling breaks down for multi-sensor data?

Most labeling tools think in frames. A multi-sensor sequence goes in as a list of frames, each with one image per camera and one point cloud per LiDAR.

The problem is that robot recordings don’t share a clock. Each sensor logs at its own rate, and the moment you force everything onto a single frame grid, you have to give something up.

• Frame counts grow fast. Take a 60-second recording with a LiDAR at 10 Hz, a camera at 30 Hz and an IMU at 100 Hz. That’s 600 frames at the LiDAR rate, 6,000 at the IMU rate, and up to 8,400 if you keep every message.
• Short events fall between frames. Lowering the frame rate keeps the count manageable, but a quick grasp or a car cutting into your lane can start and finish between two frames.
• Labels drift from the data. A frame’s timestamp often doesn’t match any message a sensor actually recorded. Before labels can be lined up with joint states or actions for training, someone has to nudge them back into place.

None of this is easy to fix downstream. In practice, getting it wrong usually means more time on relabeling.

How MCAP works in Encord

Here’s the workflow from upload to export.

1. Upload the MCAP as it is

Add .mcap files in the Encord app, through the SDK, or register them straight from your own cloud storage, like S3 or Google Cloud Storage. Each recording becomes one Scene. Large files aren’t loaded in full, since Encord reads only the parts of the recording it needs.

2. See key streams on one timeline

Camera views, point clouds and numeric signals play back together, each at its real timestamp. There’s no frame grid. The timeline runs from the first message in the file to the last, so sensors that start late or log at different rates still line up exactly as they were recorded.

3. Label 3D objects as events

Draw a cuboid when an object appears, adjust it as it moves, and mark when it leaves. Cuboids, polylines, keypoints and point cloud segmentation are all supported.

4. Label time series

Mark time ranges directly on numeric signals, e.g. IMU or velocity plots.

5. Choose how the playhead moves

Snap from one LiDAR sweep to the next, step through every camera image, or turn snapping off and move freely with full precision.

6. Export on the recording’s timestamps

Read and write labels with the Encord SDK (v0.1.206 or later). Every label is tied to the recording’s timestamps in nanoseconds, so there are no offsets to correct before the data goes into training.

Try labeling MCAP recordings in Encord

Native support for MCAP is in beta and available to all Encord platform customers, starting with 3D and time-series annotation. We’re building it closely with our beta customers, and what they run into is directly shaping what comes next.