Encord Blog
Automating Foundation Models with Segment Anything Model (SAM) Using Encord Annotate
At Encord, our mission is to accelerate the development and democratization of quality AI and computer vision applications by providing tools which enable actionable insights across your data, labels and models. Today, we’re bringing that one step further announcing our product launch integrating Meta’s Segment Anything Model (SAM) into the Encord Annotate platform.
Watch the video below to learn more about SAM and its integration with Encord.
SAM, or the Segment Anything Model, is Meta’s new zero-shot foundation model in computer vision, a cornerstone of their Segment Anything project. As a zero-shot foundation model, and as its name suggest, SAM is immediately capable of "segmenting anything" including image data it hasn't seen before, from a simple combination of keypoints and, if you wish, a delimiting bounding box.
The release last week set the internet ablaze with possibilities, those both obvious and those yet to come. We’re here to tell you about the possibilities available now. Integrating SAM with Encord Annotate pairs the power of SAM to segment anything with Encord’s powerful ontologies, interactive editor, and comprehensive media support. Encord supports using SAM to annotate images and videos, as well as speciality data types such as satellite and DICOM data. DICOM support includes X-ray, CT, and MRI among others — with no additional effort from you.
Our powerful labeling tool gives you an interactive editor experience allowing you to define regions to include and exclude, producing both bounding boxes and segmentations to your exact specification. Of course, integrating with Encord means you can take advantage of our annotation workflows as well — ensuring you get all the benefits of a collaborative annotation and review platform powered by AI-assisted labeling and our annotator training module.
We’re very excited to bring SAM to Encord to support your AI initiatives - get started here. You can also check out our tutorial on how to fine-tune Segment Anything here.
Explore the platform
Data infrastructure for multimodal AI
Explore product
Explore our products
Index
Manage & curate your data
Understand and manage your visual data, prioritize data for labeling, and initiate active learning pipelines.
Annotate
Supporting your labeling needs
Super charge your data annotation with AI-powered labeling — including automated interpolation, object detection and ML-based quality control.
Active
Find & fix data issues with ease
Monitor, troubleshoot, and evaluate the data and labels impacting model performance.
SAM in Encord enables label creation for distinct features across supported file formats, supporting Polygon, Bounding box, or Bitmask annotation types. It allows segmenting frames/images, creating labels for existing instances, and including/excluding areas from selection.
To use SAM in Encord, navigate to the frame, click - Instantiate object - or press the instance's hotkey, then press Shift + A to enable SAM. Click to segment areas, confirm labels, and include/exclude areas as needed.
Auto-segmentation with SAM in Encord involves clicking on the area to segment. The pop-up indicates auto-segmentation is running. Once highlighted, apply the label or press Enter to confirm. Exclude or include areas by clicking or right-clicking accordingly.
SAM in Encord streamlines auto-labeling by enabling precise segmentations of distinct features across various file formats. Its support for multiple annotation types and intuitive interface enhances efficiency and accuracy in labeling tasks.
Auto-labeling with SAM involves segmenting areas on images/frames by clicking or dragging the cursor. Apply the label once highlighted, ensuring accurate delineation of features. Adjust selections by including/excluding areas as necessary, facilitating efficient auto-labeling workflows.
Encord provides robust tools for data annotation and segmentation, allowing users to effectively label and organize their datasets. These features are designed to streamline the process of preparing data for training machine learning models, ensuring high-quality input for model performance.
Encord addresses challenges with variable segmentations and free text inputs by providing structured tools that facilitate consensus among annotators. This helps ensure that all team members are on the same page, thereby improving the quality and efficiency of the labeling process.
Encord includes advanced annotation tools that support auto-segmentation using off-the-shelf models, allowing for more efficient data processing and annotation. This feature enhances the speed and accuracy of labeling tasks, making it easier for teams to manage complex datasets.
Encord simplifies the evaluation process by automating the generation of evaluation codes and visualizations. This allows teams to quickly analyze predictions without the need for extensive manual coding, resulting in faster project timelines and more efficient workflows.
Yes, Encord is designed to accommodate various data formats from different customers. The platform allows for customization in data ingestion, enabling seamless integration and standardization of diverse data types before they enter the machine learning pipeline.
Encord is designed to manage multi-modal data, allowing for seamless transitions between 2D and 3D labeling. Users can annotate objects that may only be visible in certain data types, ensuring comprehensive tracking and labeling across various sensor inputs.
Yes, Encord can convert incoming data formats on the fly, allowing for a seamless integration with the human in the loop use cases. This flexibility ensures that various payloads, such as JSON with bounding boxes and points, can be efficiently processed.
Encord is capable of managing multi-class annotations by allowing users to lock structures and brush over gaps to fill in missing areas. This method is particularly useful for nested objects, such as distinguishing a tooth within a jaw.
The typical workflow in Encord involves segmenting motion data first and then labeling the individual segments. This two-step approach can improve efficiency and accuracy for human labelers compared to annotating everything in one go.
Yes, Encord can seamlessly integrate with existing data pipelines to enhance predictive maintenance analytics by processing both image and tabular data, making it easier to analyze and predict operational outcomes.