Back to Blogs

Automating Foundation Models with Segment Anything Model (SAM) Using Encord Annotate

April 11, 2023
8 mins
blog image

At Encord, our mission is to accelerate the development and democratization of quality AI and computer vision applications by providing tools which enable actionable insights across your data, labels and models. Today, we’re bringing that one step further announcing our product launch integrating Meta’s Segment Anything Model (SAM) into the Encord Annotate platform.

Watch the video below to learn more about SAM and its integration with Encord.

SAM, or the Segment Anything Model, is Meta’s new zero-shot foundation model in computer vision, a cornerstone of their Segment Anything project. As a zero-shot foundation model, and as its name suggest, SAM is immediately capable of "segmenting anything" including image data it hasn't seen before, from a simple combination of keypoints and, if you wish, a delimiting bounding box.

light-callout-cta For all the details of the inner workings and greater significance of SAM, check out our SAM explainer.

The release last week set the internet ablaze with possibilities, those both obvious and those yet to come. We’re here to tell you about the possibilities available now. Integrating SAM with Encord Annotate pairs the power of SAM to segment anything with Encord’s powerful ontologies, interactive editor, and comprehensive media support. Encord supports using SAM to annotate images and videos, as well as speciality data types such as satellite and DICOM data. DICOM support includes X-ray, CT, and MRI among others — with no additional effort from you.

Image displaying manually produced vs. SAM-produced segmentation masks on a brain MRI scan

Our powerful labeling tool gives you an interactive editor experience allowing you to define regions to include and exclude, producing both bounding boxes and segmentations to your exact specification. Of course, integrating with Encord means you can take advantage of our annotation workflows as well — ensuring you get all the benefits of a collaborative annotation and review platform powered by AI-assisted labeling and our annotator training module.

Image displaying the Segment Anything Model (SAM) used in Encord Annotate

We’re very excited to bring SAM to Encord to support your AI initiatives - get started here. You can also check out our tutorial on how to fine-tune Segment Anything here.

Supercharge Your Annotations with the
Segment Anything Model
medical banner

cta banner

Build better ML models with Encord

Get started today
Written by

Justin Sharps

View more posts
Frequently asked questions
  • SAM in Encord enables label creation for distinct features across supported file formats, supporting Polygon, Bounding box, or Bitmask annotation types. It allows segmenting frames/images, creating labels for existing instances, and including/excluding areas from selection.

  • To use SAM in Encord, navigate to the frame, click - Instantiate object - or press the instance's hotkey, then press Shift + A to enable SAM. Click to segment areas, confirm labels, and include/exclude areas as needed.

  • Auto-segmentation with SAM in Encord involves clicking on the area to segment. The pop-up indicates auto-segmentation is running. Once highlighted, apply the label or press Enter to confirm. Exclude or include areas by clicking or right-clicking accordingly.

  • SAM in Encord streamlines auto-labeling by enabling precise segmentations of distinct features across various file formats. Its support for multiple annotation types and intuitive interface enhances efficiency and accuracy in labeling tasks.

  • Auto-labeling with SAM involves segmenting areas on images/frames by clicking or dragging the cursor. Apply the label once highlighted, ensuring accurate delineation of features. Adjust selections by including/excluding areas as necessary, facilitating efficient auto-labeling workflows.