Upgraded workflows, improved bitmask and DICOM editor, a new automated labeling system, and an enhanced interface between Annotate and Active round out November with Encord. Looking forward to the holidays ahead! 💪 🚀 Workflows - the new Task Management System Workflows graduated from Beta and are now the default task management system for all customers. Scaled to handle 500,000 tasks — workflows is your new go to for large-scale annotation projects. In addition to performance, Workflows enjoy a significantly enhanced UX with a new template editor and easy-to-use task queue, now with priorities! Of course — no matter the priority, some tasks can still pose a problem — corrupted data and unclear annotation instructions are among the main reasons annotators might need to skip a task until later. Fortunately, workflows will enhance collaboration by prompting the annotator to clearly explain why the task is being skipped for clear communication with the labeling manager. Redesigned Bitmask Tool We're thrilled to announce upgrades to the threshold brush tool for bitmask labels! In addition to B/W (black and white) say hello to RGB (red/green/blue) and HSV (hue/saturation/values) thresholds - allowing you to target specific color ranges with ease. And to make sure you’re labeling what you think you are, the threshold preview has evolved to keep pace. You can now preview your work in transparent and inverted modes in addition to B/W. Whether you're fine-tuning reds, greens, blues, or playing with hues, saturations, and values, these new options make sure bitmask labeling can fit your data. Combine the above threshold range improvements with our bitmask lock feature, which allows you to designate that certain bitmask labels should not overlap. Our bitmask brush is now the perfect tool to ensure perfect annotation coverage per frame. Encord Active - Supporting Nested Attributes, Custom Metadata, Priorities Encord Active is improving across the board. We have extended support for custom metadata which means you can now filter your images/videos based on all the extra information associated with your data such as location, time of day, or anything else you might have available. Simultaneously we added the option to filter your classes based on all the nested attributes in your label ontology on the Encord platform. Pair these enhanced filtering capabilities with improved synergy between Active <> Annotate and you have a fully featured computer vision data platform. Among the headline improvements in November were the ability to create datasets with only specific frames from videos and image groups and send bulk or individual comments to Annotate. Coming soon, adjust priorities, bulk classification, and much more — contact us to learn how Encord Active can help you label the right data faster. DICOM The world of DICOM imaging is sprawling with various stakeholders and implementations. While it’s great to have so many organisations working together, in practice it means there may be difficulties handling various interpretations or small inconsistencies. To alleviate these pains, we're excited to announce a significant upgrade to our DICOM parsing capabilities! We’re introducing a new parser which extends our compatibility capabilities as well as increasing DICOM performance in the browser! In addition, we are enhancing our Multiplanar Reconstruction (MPR) capabilities — we’re making it possible to create bitmask annotations in the reconstructed views. With MPR Annotations you can now view and annotate from the best angle — giving you the power to annotate accurately and efficiently. Introducing VLM Automated Labeling You may have seen OpenAI’s exciting announcement about GPT 4 Vision, and if you’re anything like us, you were probably thinking something like: ”how can we leverage this to accelerate annotation?” 🤔 After some thinking, we’re happy to introduce our new VLM prediction feature! Currently available for classifications, our VLM prediction tool is a new repertoire in automated labelling which understands the content of the image and automatically creates the appropriate classifications from your ontology. Contact us to discuss if it’s right for your use case, and watch this space as we expand our automated labelling capabilities going forward!

Introduction Expert review workflows are crucial for accurate and successful annotation projects ensuring high data quality, efficient task allocation, and time savings. In this walkthrough, you’ll learn how to customize workflows to facilitate expert review flows and improve collaboration. As the AI and computer vision landscapes evolve, expert review workflows help you maintain data integrity, ensure optimal model performance, and maintain flexibility for future unknown labeling demands.  Understanding Workflows Workflows are systematic processes (or graphs) that define how tasks are organized, assigned, routed, and automated within an annotation project. They provide a structured way of handling various stages of a project, ensuring that each step is completed efficiently and in the correct order while tracking performance at each step. Expert Review With the importance of training data ever-increasing, expert review workflows ensure the highest quality of annotations, in turn leading to improved model performance. The expert review ensures data meets the required standard through subject matter experts thoroughly checking and validating a subset of the annotations created.  Benefits of Expert Review Workflows Expert review workflows offer a range of benefits that contribute to the success of data-centric projects: Improved Data Quality: Expert review ensures that data is accurate and error-free, leading to more reliable models and results. Efficient Task Allocation: Workflows help allocate tasks to the right experts, ensuring that each annotation or review is handled by the most qualified individuals. Error Detection and Correction: Issues can be identified and addressed promptly during the review process, preventing them from propagating further in the project. Time and Resource Savings: Automation within workflows streamline the process, reducing the time and effort required for manual coordination and ensuring experts aren’t wasting their time on menial tasks. Setting up Expert Review Workflows with Encord Create a New Workflow Template First, navigate to the "Workflow Templates" and click on the "+ New workflow template" button. For this walkthrough, we will create a workflow for an object detection model. Configuring the Workflow Template In the center, you will find the edit button and by clicking on it you will find on the right-hand side of the screen, you'll find the workflow library. This library contains components to build your workflow. Let’s look at each of these components as we add them to our insect detection project. Start Stage It's where your project begins, offering a clear overview of the project's foundation and helping team members understand the data they'll be working with. Annotate Stage This stage is the heart of the workflow, where data is annotated. The stage initially includes all annotators by default. To choose specific annotators, click the Annotate component, go to the Selective tab, enter the user's email, and select from the list. Only collaborators added via Project-level Manage collaborators will be available. The optional Webhook feature adds a layer of real-time notifications, enhancing project monitoring. Review Stage Multiple review stages can be included within a project, each with its unique set of reviewers and routing conditions helping to establish a structured process where subject matter experts validate annotations and detect errors. Strict Review With strict review, tasks stay put after label approval or rejection, giving reviewers time for adjustments and the ability to add comments for missing annotations. This provides reviewers with an additional opportunity to evaluate and, potentially, revise their judgments. This added layer of scrutiny helps to maintain accuracy and quality. Router A Router divides the pathway that annotation and review tasks follow within the workflow. You have the choice between two router types to select for your project: Percentage Router Precisely allocates annotations based on defined percentages, which is useful for the precise distribution of tasks, ensuring an equal workload split between different stages or groups of reviewers. Collaborator Router Customize annotation workflows based on collaborators to assign tasks strategically, ensuring alignment with expertise and responsibilities, and providing flexibility for diverse collaborators. For instance, a new annotator, Chris, may have his tasks automatically routed to an expert review queue, assigning pathology annotations to Dr. Smith and radiology annotations to Dr. Johnson. This approach optimizes the workflow, maintains quality through expert review, and allows flexibility for exceptions, enhancing collaboration in diverse teams Now that we've covered each element of the workflow, let's explore an instance of a workflow designed for object detection. Using Workflows in Annotation Projects To understand the integration of workflows in annotation projects, let's create an annotation project for an insect detection model with the following steps: Select and name the annotation project. Add insect dataset. You can create a new dataset here as well. Add the ontology for the annotation project. For quality assurance, opt for a workflow, either by creating a new one or utilizing an existing setup. And you are ready to start annotating! Select your annotation project and open the summary. The Summary provides an overview of your workflow project, displaying the status of tasks in each workflow stage, and offering a high-level visual representation of project progress. Navigate to the Queue for task management and labeling initiation, with options tailored to user permissions. It encompasses the Annotator's Queue, Reviewer's Queue, and Annotator & Reviewer, Admin, and Team Manager Queue. Users can filter, initiate, and assign tasks as needed, and this functionality varies based on the user's role. Admins and Task Managers can assign and release tasks, ensuring efficient task management within the project. Select the Start Labeling button to annotate your dataset. Label your dataset! Once the data has been annotated, reviewers find the labeled data to be reviewed in Queue as well.  The reviewer has the option to exert bulk action on multiple reviews at once. Once the review is complete, any rejected images can again be found in the Annotator’s queue. The reason for rejection can also be specified and the annotator must resolve the issue to submit the re-annotated data. The approved images are found in the expert review queues. Once all the reviews are accepted the annotation is complete! The multiple review stages process in the annotation project contributes to the refinement of the dataset, aligning it with the desired standards and objectives of the project. The flexibility to perform bulk actions on multiple reviews simultaneously streamlines the review workflow and the ability to specify reasons for rejection provides valuable feedback to annotators.  Wrapping Up In conclusion, expert review workflows play a pivotal role in ensuring the accuracy and success of data-centric projects like annotating an insect detection model. These workflows offer benefits such as improved data quality, efficient task allocation, and time savings. As technology advances, the importance of expert review workflows in maintaining data integrity becomes increasingly evident. They are an essential component in the evolving landscape of data-driven projects, ensuring optimal model performance. {{Training_data_CTA:: Optimize your annotation project with expert review workflows}}

The world of computer vision and AI never slows down. OpenAI is pushing new boundaries, adding foundation sight and sound support to ChatGPT. Encord isn’t slowing down either, making it easier than ever to annotate and understand your datasets with improvements to our workflows, benchmark quality assurance algorithms, and active learning platform. Read on for all the details below. Workflow Enhancements: Force Review and Change Review Judgement Reviewing annotations is a critical part of ensuring label quality going into downstream workloads. Depending on the domain or sample difficulty, it can often be difficult even for subject matter experts to make the correct judgment in one shot, or correctly identify labels in every sample. As part of empowering teams with flexible review workflows, Encord is unveiling two improvements to reviews inside our annotation workflows. First, by toggling on Strict Review for any given review stage, you’ll be able to ensure a review task is not over until you explicitly reject or approve it. This means tasks without any labels will be stopped for a review, as well as making it easy to make detailed comments on reviews before sending them back to annotation. We’re also making it possible to change whether a label should be approved or rejected — even if you have already decided once. Simply select the labels under the ‘completed’ section in question and press ‘Return to pending.’ Combine this with the ‘Strict Review’ feature to carefully consider the most difficult quality control workloads and make fully informed judgments. Upgrades to Annotator Training and Benchmark QA: DICOM and Keypoint Support We're excited to announce several improvements to Benchmark QA and training projects! We’re introducing support for DICOM data, as well as for ‘Keypoint’ ontology types in Benchmark QA and training projects — making it even easier to use Encord to upskill annotation workforces and scale your annotation efforts while maintaining consistent quality control. Don’t hesitate to reach out to us for assistance in setting up your Benchmark QA or training project workloads! Single Label Review Settings Navigating and reviewing labels just got even more efficient! We're excited to introduce an update to our Single Label Review - you now have the power to adjust the default zoom level for the label editor, providing greater control and precision when reviewing individual labels. Don’t worry about having to find a setting that works for all labels - you can easily fine-tune the zoom level to focus on the details that matter most to you when a label is being reviewed while having a useful default that accelerates your workflow when reviewing many labels in a single session! Did you know? Set Dynamic Attributes with our Keyframe Editor Objects in temporal media may have different attributes at different points in time. Dynamic attributes are the perfect way to model attributes that change over an object’s lifetime, and our powerful keyframe editing interface is the perfect way to scale annotation workloads with such attributes. Set the frames where the attribute values change, and then propagate those changes along the object's lifecycle with just a few simple clicks! The label editor timeline instantly reflects any changes you make for easy verification and inspection. Dive into our documentation for more details! Encord Active: Search Anything and Look Good Doing It Take your dataset querying to the next level with Search Anything! Building on the success of our natural language search feature, we're excited to introduce a new dimension to your search capabilities. Our latest feature, Search Anything, uses embeddings-based search technology to find images in your dataset that closely resemble an external image you provide. Simply drag and drop your chosen external image into Encord Active whereafter our algorithms will match it against your dataset, quickly and accurately identifying the most similar images for your review. For when you can’t describe an image, let the image describe itself! In addition to putting all that power in your hands, we’re making it easier to wield as well. We are thrilled to announce that we're saying goodbye to Streamlit in our OSS and Deployed versions of Encord Active. Streamlit was an invaluable partner during our 6-month beta phase, aiding in swift prototyping. As we transition, we're focusing on a platform that is aligned with Encord’s core design principles and meets our customers' scalability and performance needs. You'll soon witness a completely refreshed version of Encord Active—stay tuned and feel free to contact us if you want an early test drive and the chance to offer pre-release feedback. Thanks for reading — as always, the inbox at product@encord.com is always open — we’d love to hear your thoughts on the feedback on the above! Talk to you soon!