Best DICOM Labeling Tools [2024 Review]

In 1985, the PUMA 560 surgical robot made history by assisting the team at Memorial Medical Center during a stereotactic brain biopsy, marking one of the earliest recorded instances of robotic-assisted surgery and astonishing the world. Fast forward to today — surgical robotic systems are supporting surgeons across a growing array of medical interventions, assisting surgeries in ways few people imaged a few decades ago. Over the past eight years alone, the Robotically-Assisted Surgical (RAS) Devices market has expanded from $800 million in 2015 to well over $3 billion today. From prominent healthcare organizations to cutting-edge research institutes, from rapidly growing startups to non-profit initiatives, diverse teams are busy developing innovative surgical robotic systems. Their goal is to enhance surgical efficiency, improve precision and, ultimately, deliver better outcomes for patients. The recent leaps in computer vision have also further spurred this growth, as artificial intelligence is rapidly entering the operating room and enabling these systems to better perceive and interpret visual information in real time and aid surgeons on a wider range of tasks. This article explores the landscape of AI applications in surgical video analysis, some of the key innovators in the space and the role of high-quality training data in the development of AI-assisted surgical systems. AI-Assisted Surgical Robotics Companies like Intuitive Surgical, creator of the Da Vinci Surgical System, led the way in the 1990s: Da Vinci was the first robotics system approved by the FDA, initially for visualization and tissue retraction in 1997 and later for general surgery in 2000. With over 6,000 robots installed worldwide and over $6b in annual revenue, Intuitive has dominated the surgical robotics industry for the better part of the last 20 years, transforming the industry and enabling patient outcomes that were previously impossible. Yet 2019 marked the start of some of its patent expiries, and with that, a wave of new entrants and innovators. The use of AI-assisted techniques in robotics now extends from preoperative planning, to intraoperative guidance and postoperative care, and has advanced significantly thanks to the close collaboration of surgeons, programmers, and scientists. Let’s discuss some of the major real-world applications and teams working in this field — starting with preoperative planning. Preoperative planning Preoperative (pre-op) planning includes a range of workstreams — from visualizing the steps of the operation, to forming a plan to tackle navigation or improve precision. Machine learning and computer vision are being leveraged in pre-op planning in many ways: from rapidly analyzing the tabular and visual data of patients (like medical records or scans), to ensuring precise trajectory planning, optimizing incision sites, and gaining more insights into potential complications. Surgical planning begins with processing and fusing various medical imaging modalities, such as CT scans, MRI scans, and ultrasound scans, to generate a comprehensive 3D model of the patient's anatomy. Computer vision algorithms and deep learning models are then employed to quickly analyze this visual data and surface recommendations and risks with pursuing different surgical steps. Algorithms also enable surgeons to identify and segment specific anatomical structures and regions of interest from the imaging data (like organs, blood vessels, abnormalities, and other critical structures within the 3D model). This segmentation is crucial for surgical planning as it provides a clear visualization of the target area. From here, surgeons can explore different surgical approaches and plan the optimal trajectory for instruments and incisions, assessing the risk factors by quantifying the distance or overlap between the planned surgical path and nearby structures. Pre-op data can also be combined with intraoperative data to achieve surgical outcomes not otherwise possible. One of the most innovative end-to-end platforms is Paradigm™ by Proprio Vision, who just a few days announced the successful completion of the world's first light field-enabled spine surgery. Using an array of advanced sensors and cameras, Paradigm captures high-definition multimodal images during surgery and integrates them with preoperative scans to provide surgeons with real-time mapping of the anatomy. In addition to augmenting navigation capabilities during a procedure, Paradigm also collects large amounts of pre-op and intra-op data to inform future surgical decision-making and improve surgical efficiency and accuracy. You can read more about Proprio's announcement on their website here. Another end-to-end robotic system is Senhance, by Asensus Surgical, which in 2021 was cleared by the FDA for general surgeries. Senhance allows surgeons to create simulations for preoperative planning, while also providing real-time data for intraoperative guidance and generating insightful analytics for postoperative performance assessments and care. Intraoperative guidance A recent report by Bain & Company found that over 50% of surgeons surveyed made use of robotic systems in some capacity during general surgeries. During procedures, where even the slightest hand trembling can risk causing significant harm, image-guided surgery is turning into a requirement. Here, computer vision is often employed for instrument tracking and object recognition, which in turn are leveraged to feed video data to AI models that can monitor the procedure and generate guidance and warnings in case of anomalies, such as excessive bleeding or tissue damage. AI-assisted systems allow surgical robots to locate and follow the movement of surgical instruments, ensuring they are precisely positioned and maneuvered. They can also be used to identify critical structures and masses in the video footage, providing augmented guidance to the surgeon in real time. Model-assisted annotations of polyps in the Encord training data platform General and Minimally Invasive Surgery (MIS) Robotic assisted devices are more and more frequent in Minimally Invasive Surgeries (MIS). The primary objective of MIS is to reduce the trauma to the patient's body; the incision surface area is smaller, and often serves as an entry point, or port, for specialized instruments and a camera, known as a laparoscope, to enter the tissues and feed back real-time video data, which allows surgeons to view internal stuctures on a monitor and be guided through the procedure. MIS employs long, thin instruments with articulating tips that can be maneuvered through the small incisions. Systems like Dexter (by Distalmotion) are currently being used for daily gynecology, urology and general surgery procedures in Europe. “Surgeons can choose to operate entire procedures robotically, or they can leverage the ability to easily switch between the robotic and laparoscopic modalities to perform specialized tasks such as stapling with their preferred and trusted instruments,” Distalmotion CEO Michael Friedrich said in a recent press release announcing their upcoming US expansion. Another promising platform is Maestro (built by Moon Surgical), which sits at the intersection of robotic-assisted surgery and conventional surgery: acting as a robotic surgical assistant, it augments the precision and control of laparoscopic surgery, increasing the dexterity of a surgeon's own hand. Just this month, Moon Surgical announced the successful completion of the first 10 laparoscopic surgeries with its Maestro system in France. The procedures — bariatric and abdominal surgery procedures — were performed by laparoscopic surgeons Dr. Benjamin Cadière and Dr. Georges Debs, who said that the platform provided them with stability and precision that are difficult to match with human assistance. Many different procedure types are benefitting from the innovation in surgical assisted devices. A few examples are: Orthopedic Surgery. Orthopedic surgery is primarily used for the treatment of musculoskeletal conditions and disorders, mostly relating to bones and joints. With deep learning and computer vision, surgeons can build a pre-op model to plan the creation of patient-specific implants and the precise alignment of bones and joints, and then leverage a robotic arm to facilitate the optimal placement during the surgery. Stryker, the creators of the MAKO surgical assistant, are one of the pioneers in this space: MAKO turns a CT scan of a patient's joint into a 3D model, measures soft tissue balance, and, during surgery, ensures the placement is optimized to the patient's anatomy. Ganymed Robotics is another innovator in the space of orthopedic robotics. The Paris-based startup's team of computer vision and deep learning imaging experts have built a tool that leverages multimodal sensors to improve hard tissue surgery, starting with total knee arthroplasty (TKA). Robotic Bronchoscopy. Bronchoscopy helps evaluate and diagnose lung conditions, obtain samples of tissue or fluid, and remove foreign bodies. During a robotic bronchoscopy, the doctor uses a controller to operate a robotic arm, which guides a catheter (a thin, flexible, and maneuverable tube equipped with a camera, light, and shape-sensing technology) through the patient’s airways. Noah Medical received FDA clearance earlier this year for its Galaxy System™: a computer vision powered lung navigation system that improves the visualization and access of robotic brochoscopies. Microsurgery. Microsurgery requires the use of high-powered microscopes and precision instruments to perform intricate procedures on tiny structures within the body, such as blood vessel, nerve and tissue repairs. These kinds of surgeries operate hard-to-see anatomical structures that are often invisible to the human eye, and surgeons performing them need to undergo extensive training to develop exceptional hand-eye coordination. A handful of computer vision powered systems are being built to help improve the outcomes of these delicate surgeries, like MUSA-3, the microsurgery robot by Microsure, which allows surgeons to use a joystick to control instrument positioning during lymphatic surgery. The system is optimized for tremor-filtered movements and high-precision, and uses high-definition on-screen displays to enable real-time image analysis during these exceptionally delicate procedures. The Microsure team raised a €38m Series B earlier this month, as they eye FDA clearance in the US and CE-mark in Europe. Postoperative analysis and training Successful patient outcomes are achieved before, during, and after what happens in the operating room. AI surgical systems are valuable in post-operative analysis, as surgeons can review the process to understand improvement areas, identify potential health risks for the patient, and share insights to align expectations. Video data can also help trained newly formed surgeons, and provide education and knowledge share for the academic surgery community. Annotated surgical videos contain information regarding critical procedures, and can help inform students about effective surgical practices or risks involved with specific techniques. AI systems can also assess surgical performance by monitoring live video feeds and comparing a surgeon’s techniques with those used in similar procedures previously. The system can record custom metrics such as an operation’s total duration, patient satisfaction and post-operative complications, establishing benchmarks and shared understanding. A leader in this space is Orsi Academy, a Belgian training and research community that helps train medical professionals in new AI-driven techniques, such as computer vision for analyzing surgical videos, surgical data science for performance evaluation, and 3D printing, to simulation to help surgeons better understand and view specific body parts and surgical sites. Just a few days ago, Orsi Academy announced that their augmented reality tool (developed by Orsi Innotech) had enabled surgeons at Erasmus Medical Center to perform the world's first robot-assisted lobectomy using augmented reality, marking a huge achievement for the AI-assisted world of surgery. During this surgery, virtual overlay of the tumor, blood vessels and airways were projected over the camera image of the patient’s lung and was rendered with real-time AI-assisted robotic instrument detection. This allows surgeons to find their way inside the patient’s body more safely & effectively.  Orsi Academy will be hosting their annual Orsi Event in Belgium, on December 14th and 15th. Details will be available on their website shortly.

Medical image segmentation is used to extract regions of interest (ROIs) from medical images and videos. When training computer vision models for healthcare use cases, you can use image segmentation as a time and cost-effective approach to labeling and annotation to improve accuracy and outputs.  Segmentation in medical imaging is a powerful way of identifying objects, segmenting pixels, grouping them, and using this approach to labeling to train computer vision models.  In this guide, we’ll explore medical image segmentation, its role in healthcare computer vision projects, applications, and how to implement medical image segmentation.  What is Medical Image Segmentation? Computer vision models rely on large training datasets used to train the algorithmic models (CV, AI, ML, etc.) to achieve high-precision medical diagnostics. An integral part of this process is annotating and labeling the images or videos in a dataset. One method for this is image segmentation, which this article will explore in more detail.  Medical image segmentation involves the extraction of regions of interest (ROIs) from medical images, such as DICOM and NIfTI images, CT (Computed Tomography) scans, X-Rays, and Magnetic Resonance Imaging (MRI) files.  There are numerous ways to approach segmentation, from traditional methods that have been around for decades to new deep-learning techniques.  Naturally, everything in the medical profession needs to be implemented with precision, care, and accuracy. Any mistakes in the diagnosis or AI model-building stage could have significant consequences for patients, treatment plans, and healthcare providers. This guide is for medical machine learning (ML), data operations (DataOps), and annotation teams and leaders wanting to learn more about how they can apply image segmentation for their computer vision projects. Read more: Encord’s guide to medical imaging experiments and best practices for machine learning and computer vision. Why is Medical Image Segmentation used In Healthcare Computer Vision Models?  Healthcare organizations, medical data operations, and ML teams can use medical image segmentation for dozens of computer vision use cases, including the following: Radiology Radiology is a medical field that generates an enormous amount of images (X-ray, mammography, CT, PET, and MRI), and healthcare organizations are increasingly turning to AI-based models to provide more accurate diagnoses at scale.  Training those models to spot what medical professionals can sometimes miss, or identify health issues more accurately, involves labeling and annotating vast datasets. Image segmentation is one way to achieve more accurate labels so that models can go into production faster, producing the results that healthcare organizations need.  Gastroenterology We can say the same about gastroenterology (GI) model development. Machine learning and computer vision models can be trained to more accurately identify cancerous polyps, ulcers, IBS, and other conditions at scale. Especially when it comes to outliers and edge cases that even the most skilled doctors and practice specialists can sometimes miss.  Histology Medical image annotation is equally useful for histology, especially when AI models can accurately apply widely-used staining protocols (including hematoxylin and eosin stain (H&E), KI67, and HER2). Image segmentation helps medical ML teams train algorithmic models, implement labeling at scale, and generate more accurate histology diagnoses from image-based datasets.  ‍Ultrasound Image segmentation can help medical professionals more accurately label ultrasound images to identify gallbladder stones, fetal deformation, and other insights.  ‍Cancer Detection When cancerous cells are more difficult to detect, or the results from scans are unclear, computer vision models can play a role in the diagnosis process. Image segmentation techniques can be used to train computer vision models to screen for the most common cancers automatically, medical teams can make improvements in detection and treatment plans.  Looking for a dataset to start training a computer vision model on? Here are the top 10 free, open-source healthcare datasets. Different Ways to Apply Medical Image Segmentation In Practice   In this section, we’ll briefly cover 8 types of segmentation modes you can use for medical imaging. Here we’ll give you more details on the following types of image segmentation methods:  Instance segmentation  Semantic segmentation  Panoptic segmentation  Thresholding  Region-based segmentation  Edge-based segmentation  Clustering segmentation  Foundation Model segmentation For more information, check out our in-depth image segmentation guide for computer vision that also includes a number of deep learning techniques and networks.  Instance segmentation  Similar to object detection, instance segmentation involves detecting, labeling, and segmenting every object in an image. This way, you’re segmenting an object’s boundaries, and whether you’re doing this manually or AI-enabled, overlapping objects can be separated too. It’s a useful approach when individual objects need to be identified and tracked.  Semantic Segmentation  Semantic segmentation is the act of labeling every pixel in an image. This will provide a densely labeled image, and then an AI-assisted labeling tool can take these inputs and generate a segmentation map where pixel values (0,1,...255) are transformed into class labels (0,1,...n). Panoptic Segmentation  Panoptic is a mix of the two approaches outlined above, semantic and instance. Every pixel is applied a class label to identify every object in an image. This method provides an enormous amount of granularity and can be useful in medical imaging for computer vision where attention to detail is mission-critical.  Thresholding Segmentation  Thresholding is a fairly simple image segmentation method whereby pixels are divided into classes using a histogram intensity that’s aligned to a fixed value or threshold. When images are low-noise, threshold values can stay constant. Whereas in noisy images, a dynamic approach for setting the threshold is more effective.  In most cases, a greyscale image is divided into two segments based on their relationship to the threshold value. Two of the most common approaches to thresholding are global and adaptive.  Global thresholding for image segmentation divides images into foreground and background regions, with a threshold value to separate the two.  Adaptive thresholding divides the foreground and background using locally-applied threshold values that are contingent on image characteristics.  Region-based Segmentation  Region-based segmentation divides images into regions with similar criteria, such as color, texture, or intensity, using a method that involves grouping pixels. With this data, regions or clusters are then split or merged until a level of segmentation is achieved.  Annotators and AI-based tools can do this using a common split and merge technique or graph-based segmentation.  Edge-based Segmentation  Edge-based segmentation is used to identify and separate the edges of an image from the background. AI tools can be applied to detect changes in intensity or color values and use this to mark the boundaries of objects in images.  One method is the Canny edge detection approach, whereby a Gaussian filter is applied, applying non-maximum suppression to thin the edges and using hysteresis thresholding to remove weak edges.  Another method, known as Sobel, involves computing the gradient magnitude and direction of an image using a Sobel operator, which is a convolution kernel that extracts horizontal and vertical edge information separately. Clustering Segmentation  Clustering is a popular technique that involves grouping pixels into clusters based on similarities, with each cluster representing a segment. Different methods can be used, such as K-mean clustering, mean-shift clustering, hierarchical clustering, and fuzzy clustering. Visual Foundation Model Segmentation: (SAM) Segment Anything Model Meta’s Visual Foundation Model (VFM), called the Segment Anything Model (SAM), is a powerful open-source VFM with auto-segmentation workflows, and it’s live in Encord!  It’s considered the first foundation model for image segmentation, developed using the largest image segmentation known, with over 1 billion segmentation masks. Medical image annotation teams can train it to respond with a segmentation mask for any prompt.  Prompts can be asking for anything from foreground/background points, a rough box or mask, freeform text, or general information indicating what to segment in an image.  Here’s how to use SAM to Automate Data Labeling in Encord. How to Implement Medical Image Segmentation for Healthcare Computer Vision with Encord  With an AI-powered annotation platform, such as Encord, you can apply medical image segmentation more effectively, ensuring seamless collaboration between annotation teams, medical professionals, and machine learning engineers. At Encord, we have developed our medical imaging dataset annotation software in collaboration with data operations, machine learning, and AI leaders across the medical industry – this has enabled us to build a powerful, automated image annotation suite, allowing for fully auditable data and powerful labeling protocols. A few of the successes achieved by the medical teams we work with:  Stanford Medicine cut experiment duration time from 21 to 4 days while processing 3x the number of images in 1 platform rather than 3  King’s College London achieved a 6.4x average increase in labeling efficiency for GI videos, automating 97% of the labels and allowing their annotators to spend time on value-add tasks Memorial Sloan Kettering Cancer Center built 1000, 100% auditable custom label configurations for its pulmonary thrombosis projects Floy, an AI company that helps radiologists detect critical incidental findings in medical images, reduces CT & MRI Annotation time with AI-assisted labeling RapidAI reduced MRI and CT Annotation time by 70% using Encord for AI-assisted labeling.  Ready to automate and improve the quality, speed, and accuracy of your medical imaging segmentation?  Sign-up for an Encord Free Trial: The Active Learning Platform for Computer Vision, used by the world’s leading computer vision teams.  AI-assisted labeling, model training & diagnostics, find & fix dataset errors and biases, all in one collaborative active learning platform, to get to production AI faster. Try Encord for Free Today.  Want to stay updated? Follow us on Twitter and LinkedIn for more content on computer vision, training data, and active learning. Join our Discord channel to chat and connect.

Machine learning has made waves within the medical community and healthcare industry. Artificial Intelligence (AI) has proven itself useful in numerous uses across a variety of domains, from Radiology and Gastroenterology to Histology and Surgery.  The frontier has now hit Electrocardiography (ECG) as well. With an annotation tool, you can annotate the different waves on your Electrocardiogram diagrams and train machine learning models to recognize patterns in the data. The first open-source frameworks have been developed to build models based on ECG data e.g. Deep-Learning Based ECG Annotation. In this example, the author automated the process of annotating peaks of ECG waveforms using a recurrent neural network in Keras. Even though the model was not 100% performant (it struggles to get the input/output right). It seems to work well on the QT database of PhysioNet. The Authors does mention it fails in some cases that it has never seen. Potential future development of machine learning would be to play with augmenting the ECGs themselves or create synthetic data. The 3 main components of an ECG: the P wave, which represents the depolarization of the atria; the QRS complex represents the depolarization of the ventricles; and the T wave, which represents the repolarization of the ventricles. Source: Wikipedia Another example of how deep learning and machine learning is useful in ECG waveforms can be found in the MathWorks Waveform Segmentation guide. Using a Long Short-Term Memory (LSTM) network, MathWorks achieved impressive results as seen in the confusion matrix below: If you want to get started yourself you can find a lot of open-source ECG datasets, e.g. the QT dataset from PhysioNet. Why are ECG Annotations Important in Medical Research? ECG annotation is an essential aspect of medical research and diagnosis, involving the identification and interpretation of different features in the ECG waveform. It plays a critical role in the accurate diagnosis and treatment of heart conditions and abnormalities, allowing you to detect a wide range of heart conditions, including arrhythmias, ischemia, and hypertrophy. Through the meticulous analysis of the ECG waveform, experts can identify any irregularities in the electrical activity of the heart, accurately determining the underlying cause of a patient's symptoms. The information gleaned from ECG annotation provides vital indicators of heart health, including heart rate, rhythm, and electrical activity. Regular ECG monitoring is invaluable in the management of patients with chronic heart conditions such as atrial fibrillation or heart failure. Here ECG annotation assists experts in identifying changes in heart rhythm or other abnormalities that may indicate a need for treatment adjustment or further diagnostic testing. With regular ECG monitoring and annotation, clinicians can deliver personalized care, tailoring interventions to the unique needs of each patient. How can Machine Learning Support ECG Annotations? Machine learning has significant potential in supporting and automating the analysis of ECG waveforms, providing a powerful tool for clinicians for improving the accuracy and efficiency of ECG interpretation.  By utilizing machine learning algorithms, ECG waveforms can be automatically analyzed and annotated, assisting clinicians in detecting and diagnosing heart conditions and abnormalities faster and at higher accuracy. One of the main benefits of machine learning in ECG analysis is the ability to process vast amounts of patient data. By analyzing large datasets, machine learning algorithms can identify patterns and correlations that may be difficult or impossible for humans to detect. This can assist in the identification of complex arrhythmias or other subtle changes in the ECG waveform that may indicate underlying heart conditions. Additionally, machine learning algorithms can help in the detection of abnormalities or changes in the ECG waveform over time, facilitating the early identification of chronic heart conditions. By comparing ECG waveforms from different time points, machine learning algorithms can detect changes in heart rate, rhythm, or other features that may indicate a need for treatment adjustment or further diagnostic testing. Lastly, machine learning models can be trained to recognize patterns in ECG waveforms that may indicate specific heart conditions or abnormalities. For example, an algorithm could be trained to identify patterns that indicate an increased risk of a heart attack or other acute cardiac event. By analyzing ECG waveforms and alerting clinicians to these patterns, it can help in the early identification and treatment of these conditions, potentially saving lives. The three tools we will be reviewing today are: Encord ECG OHIT ECG Viewer WaveformECG Encord ECG Encord is an automated and collaborative annotation platform for medical companies looking at ECG Annotation, DICOM/NIfTI annotation, video annotation, and dataset management. It's the best option for teams that are: Looking for automated, semi-automated or AI-assisted image and video annotation. Annotating all ontologies. Working with other medical modalities such as DICOM and NIfTI. Wanting one place to easily manage annotators, track performance, and create QA/QC workflows. Benefits & Key features: Use-case-centric annotations — from native DICOM & NIfTI annotations for medical imaging to ECG Annotation tool for ECG Waveforms. Allows for point and time interval annotations. Supports the Bioportal Ontology such as PR and QT intervals. Integrated data labeling services. Integrated MLOps workflow for computer vision and machine learning teams. Easy collaboration, annotator management, and QA workflows — to track annotator performance and increase label quality. Robust security functionality — label audit trails, encryption, FDA, CE Compliance, and HIPAA compliance. Advanced Python SDK and API access (+ easy export into JSON and COCO formats). Best for teams who: Are graduating from an in-house solution or open-source tool and need a robust, secure, and collaborative platform to scale their annotation workflows. Haven't found an annotation platform that can actually support their use case as well as they'd like (such as building complex nested ontologies, or rendering ECG waveforms). Team looking to build artificial neural networks for the healthcare industry. AI-focused cardiology start-ups or mature companies looking to expand their machine-learning practices should consider the Encord tool. Pricing: Free trial model, and simple per-user pricing after that. OHIF ECG Viewer The OHIF ECG Viewer provides can be found from Radical Imaging’s Github. The tool provides a streamlined annotation experience and native image rendering with the ability to perform measurements of all relevant ontologies. It is easy to export annotations or create a report for later investigation. The tool does not support any dataset management or collaboration which might be an issue for more sophisticated and mature teams. For a cardiologist just getting started this is a great tool and provides a baseline for comparing to other tools. Benefits & Key features: Leader in open-source software. Renders ECG waveform natively. Easy (& free) to get started labeling images with. Great for manual ECG annotation. Best for: Teams just getting started. Pricing: Free. WaveformECG The WaveformECG tool is a web-based tool for managing and analyzing ECG data. The tool provides a streamlined annotation experience and native image rendering with the ability to perform measurements of all relevant ontologies. It is easy to export annotations or create a report for later investigation. The tool does not support any dataset management or collaboration which might be an issue for more sophisticated and mature teams. So if you're new to the deep learning approach to ECG annotations the WaveformECG tool might be useful but if you’re looking at more advanced artificial neural networks or deep neural networks it might not be the best place. Benefits & Key features: Allows for point and time interval annotations and citations. Supports the Bioportal Ontology and metrics. Annotations are stored with the waveforms, ready for data analysis. Renders ECG waveform natively. Supports scrolling through each ECG waveform. Best for: Researchers and students. Pricing: Free. Conclusion There you have it! The 3 Best ECG annotation Tools for machine learning in 2023.  We’re super excited to see the frontier being pushed on ECG waveforms in machine learning and proud to be part of the journey with our customers. If you’re looking into augmenting the ECGs themselves or creating synthetic data get in touch and we can provide you input and help with it!