Announcing the Launch of SAM 3 in Encord

In April 2023, we introduced the SAM model to our platform a mere few days after its initial release. Similarly, when SAM 2 was released a year ago, Encord built an integration just one day after its official release. 

We are excited to say Encord has done it again. Just one day after Meta released the SAM 3 model, it is now available in the Encord platform.

SAM 3 (Segment Anything Model 3) is Meta’s latest vision foundation model that can detect, segment, and track all instances of a concept in images and videos using open-vocabulary text prompts (“yellow school bus”) or visual exemplars. Its architecture decouples “what” (recognition) from “where” (localization), enabling far more flexible and fine-grained segmentation than previous SAM versions. It’s trained on a massive dataset of millions of labeled concepts (SA-Co) and supports real-time inference for hundreds of objects per frame.

What this means in practice is a massive leap in productivity. Instead of manually finding and labeling dozens of similar objects one by one, SAM 3 can identify and label all of them at once. For example, clicking a single button and instantly capturing all 50 instances of an object in a frame.

Here’s what you need to know about how it’s integrated with Encord:

SAM 3 is now available in Encord and we’ve started rolling it out to our clients. Once enabled, users can detect, track, and create labels with one click. The power of SAM 3 lies in both the ability to detect multiple objects automatically and then track it across frames. It also supports text prompts to auto-detect and segment multiple objects at once.

The enhanced magic wand icon in the editor will have a toggle, which when turned on will give users the latest and most powerful tool for annotation tasks. 

Refining objects within this modal is the same for those customers who already have used the SAM 2 integration in the Encord platform, so you’ll get the same reliable fine-tuning experience you’ve come to expect.

How SAM 3 Works in Encord

SAM 3 brings smarter, faster, and more intuitive workflows for labeling objects in your images.

Encord enables SAM 3 in two key ways:

1. One-Click Detection 

A sparkles button next to each class name in the left-hand panel uses the class name as a text prompt and labels all detected instances with one click. This can also be adjusted for greater accuracy. Ctrl+Z can be used to undo the action instantly.

2. Updated SAM Modal 

When using the SAM modal (via Shift+A or the Magic Wand button), users will now see a toggle allowing them to switch between:

• the classic SAM 2 single-object refinement workflow
• the new SAM 3 text-prompt mode for multi-object detection which can be added or removed if need be

Refinement behavior remains identical for teams already familiar with SAM 2, ensuring a smooth transition.

Try the Encord SAM 3 Integration

We are eager for our customers to try SAM 3 and experience the productivity boost firsthand. This integration significantly enhances the speed and accuracy of data annotation, and we’re excited to continue pushing the boundaries of what’s possible in computer vision tooling.

We invite all users to send their feedback to product@encord.com. Your insights are invaluable as we continue to push the boundaries of what’s possible in machine learning annotation and evaluation.

Thank you for being part of this journey. We look forward to continuing to deliver world-leading technology at a rapid pace for our innovative customers.