Founded in 2014, Tractable uses computer vision technology to assist insurance providers in performing visual assessments of damaged assets. The company designs customizable machine learning models to serve insurance providers around the globe. As the complexity of its data annotation task grew, Tractable turned to Encord for a user-friendly platform that allowed for complex ontologies, annotator monitoring, and quality assurance.

Customer: Meet Tractable

By analyzing customer images, Tractable quickly facilitates accurate damage appraisal, making the recovery from accidents 10 times faster and enabling people to move through the claims process quickly and efficiently after they’ve had an automobile accident. Tractable also offers products for property assessment, providing the same high-quality experience for incidents that involve property damage and thereby helping people to recover faster after a disaster strikes their home.

Beyond accident recovery, Tractable’s technology can assist people throughout the entire lifecycle of owning a car or property. With Tractable, they can perform visual inspections before selling an asset, determine which parts of an asset to salvage or replace, obtain a condition report on a leased asset, and more.

By using Tractable, insurance providers can free up employee time for high-value tasks, improve customer experiences, accelerate repairs, and increase recycling– all of which has a positive impact on people and the world they live in.

Problem: Performing Quality Assurance When Scaling Up Projects with Complex Ontologies

When building their first models for image classification, Tractable’s team designed and used internal tools for data annotation. However, as Tractable grew their product offerings, the annotation tasks became more complicated, and the team needed a training data platform that supported segmentation. Building one in-house would be costly and time consuming.

Tractable’s team tried a few third-party platforms, but as the projects scaled and grew in sophistication– moving towards more pixel-level annotations with more complex ontologies– they found that many of these platforms had limitations, especially when it came to quality assurance (QA) functionality.

As Tractable began building its property assessment models, the remote annotation team began to grow rapidly and the need for improved data annotation workflows became more urgent. Ensuring the quality of both the data and its labels became increasingly important as annotation tasks grew in complexity and size.

“As our ambition grew, we realized that the functionality of many of the tools we were using was quite limited,” says Camilla Gilchrist, Head of Operations at Tractable. “We had a lot of steps in our annotation workflows, with many different pipelines, and the platforms couldn’t handle that complexity. We also had a bottleneck around quality assurance. With image segmentation, you can’t do a quick agreement rate analysis and doing a manual quality assurance check on each piece of data isn’t feasible. We needed a far more efficient way to assess quality.”

Tractable needed a training data platform that could incorporate the feedback of expert decision makers in the annotation process and provide QA functionality.

Solution: A User-Friendly Training Data Platform with Quality Assurance and Annotator Review Functionality

When annotating data for its computer vision models, Tractable needs to take into account expertise beyond visual observations.

“Think about how motor engineers assess damage on a vehicle,” explains Camilla. “They don’t just look at the car: they open and close the doors, feel the body work, and much more. As much as possible, we incorporate that expertise into our models, which requires determining specific annotation criteria that’s often quite complex. We need an accessible training data platform that allows annotators to break these complex labeling tasks into smaller parts and provides quality assurance and annotation review functionality. Otherwise, annotation tasks quickly become overwhelming, and the risk of mistakes increases.”

The higher the granularity of an annotation task, the greater the amount of detailed features that must be labeled in each piece of data. This level of detail increases the need for annotator monitoring and quality assurance. 

Unlike Tractable’s legacy training data platforms, Encord built out the QA functionality that the company needed, enabling Tractable to unlock the power of using granular annotation techniques at scale. 

At the same time, Encord’s user-friendly platform provided a level of control that allowed Tractable’s operations professionals to build out ontologies quickly. 

The other platforms that Tractable had used allowed for custom mixtures of segmentation and frame-level annotation, but they required users to implement these customizations programmatically. SageMaker, for instance, allows for users to create custom UIs and ontologies, but to do so Tractable’s engineering and research teams had to put in a lot of time and effort into delivering the tooling the company needed.

Results: Improved Quality Control, Data Governance, and Annotator Training

With Encord’s QA and annotation review features, Tractable has been able to train annotators efficiently, monitoring their performance and providing feedback along the way. Their remote annotation team has grown to over 40 people.

“Managing QA workflows efficiently and having an overview of the entire annotation process has been so important for our success. The more manual that QA process is, the more the amount of work explodes as the annotation workforce scales,” explains Camilla. “It’s always about balancing speed and quality. A lot of platforms prioritize speed over quality or quality over speed. Encord speeds up annotation while still allowing for strong quality control.”

Encord’s API also works with S3 servers as a primary integration, so Tractable can keep customer data on its own servers. Because Tractable operates globally, it has to adhere to different data governance rules depending on the region in which a customer is located. The other tools that Tractable tried didn’t prioritize S3 integrations, so when the API broke, Tractable had to wait for and nudge those platforms to fix it– a less-than-ideal situation for a global company.

“Encord’s team has been very hands-on in delivering what we need to succeed,” says Camilla. “A lot of companies promise that level of commitment, but Encord actually delivers it. We receive excellent support. They build out new features quickly based on our feedback. With other platforms, we’ve had to troubleshoot problems on our own– even to the extent that we basically needed to build our own offline tooling to process the data we labeled using them! Encord has worked with us to find a solution for every challenge.”