Saving the Honey Bees with Computer Vision

The Far Out Thinking Company, a UK based environmental R&D company that uses AI to combat pollinator decline, needed an accessible, affordable annotation tool to enable all members of their team to label images at speed. Using Encord’s platform, the team quickly annotated 4500 images of the European Hornet, Asian Hornet, and the Common Wasp and built a computer model to help distinguish between these different species and to confirm and detect the presence of Asian Hornets, an invasive species to the UK.

Customer: Meet Pollenize

In late 2018, Pollenize, a U.K.-based pollinator conservation community interest company, launched its R&D arm The Far Out Thinking Company. The Far Out Thinking Company develops technology and AI solutions that help protect pollinators and preserve biodiversity. 

Pollinators, and in particular bees, play a crucial role in the viability of our plant and animal ecosystems. Bees are responsible for pollinating a large percentage of crop species. According to the Food and Agriculture Organization of the United Nations, bees pollinate around 75% of the world's food crops. 

However, bees are disappearing at a terrifying rate: 40 percent of insect pollinator species are facing extinction, with eight species of bees having been declared endangered.

Working in tandem with Pollenize, The Far Out Thinking Company is developing technology that helps to better understand pollinator populations and their resourcing needs. By devising devices such as AI beehives, hornet identification boxes, and moth detectors, the Far Out Thinking Company helps determine gaps in flora, pinpoint resource-stressed areas, inform public rewilding strategies, identify invasive predators, and more.

Problem: An Invasive Species Threatens the U.K.’s Honey Bee Population

In 2003, the Asian Hornet arrived in Europe. An invasive species, the hornet has now spread throughout Europe, taking up the resources of natural pollinators and decimating the honey bee population. Asian hornets attack honey bees outside of beehives, a behavior known as “hawking,” biting off their heads and abdomens and taking their thoraxes back to their nests to feed their larva.

Killing individual Asian Hornets won’t solve the problem. Local authorities need to destroy the nests, each of which can house from several hundred to several thousand individual hornets.

Asian hornets haven’t yet naturalized in the U.K.; however, every year, they make their way into the country via shipping crates and the caravans of returning holidaymakers. 

Asian Hornets create their nests in April, and in October each nest produces an average of 14 reproductive individuals, so nests must be destroyed before October. Otherwise, the Asian Hornets will multiply rapidly. 

Since 2016, nine nests have been destroyed throughout the U.K.. However, local residents aren’t familiar with the look of this invasive species, and locating and destroying nests without a system for identifying hornets creates many challenges.

Solution: Training Computer Vision Models to Identify Invasive Species and Protect Local Pollinators 

The Far Out Thinking Company developed the Asian Hornet ID Device. It uses computer vision to distinguish between local pollinators and Asian Hornets. 

With a network of these devices, port cities and areas of entry into the U.K. would no longer have to rely on individuals to identify Asian Hornets. Instead, the devices will provide an automated early detection system and prevent the hornets from naturalizing in the U.K.

The Asian Hornet ID Device is situated in a box that contains a small reservoir of hornet  attractant. After an Asian Hornet lands on the attractant, a camera located above the attractant observes it, and the computer vision model uses object detection to determine whether it is a common wasp, Asian Hornet, or European Hornet. 

If the AI system detects an Asian Hornet, it sends an alert to the Far Out Thinking Company, which contacts the authorities who put a radio tag on it. They then track the hornet back to its nest and destroy it.

To build its computer vision model, The Far Out Thinking Company needed access to a data annotation tool that enabled multiple team members to label hundreds of images of pollinators at the same time. 

Only one member of the team has a computer science and computer vision background. The other members of the team found the open-source annotation tools alienating. Without a foundation in computer science or a background in data labeling, they couldn’t easily use the tools to assist in annotation despite having valuable expertise in entomology.

An Encord employee familiar with the company’s work reached out, asking if they’d like to demo the product. 

“The platform was so intuitive. It provides a lot of different levels of control and team management. Everyone on the team could label and review images easily,” says Mathew Elmes, Director of Pollenize and The Far Out Thinking Company. “We could also do computation both on the cloud and locally on our deep learning machine. All those factors made it really advantageous for us, and I saw the opportunity to save a lot of time and costs by using Encord to speed up our model development.”

Although Encord’s platform was initially designed for video annotation, the Encord team helped The Far Out Thinking Company string together a number of still images so that they could use the product to meet their specific annotation needs.

“After they helped us put the images together, we pressed a couple of buttons, and the platform made a model for us,” says Elmes. “It was so easy.”

Results: Accessible Annotating, New Projects, Pollinator Protection

The Far Out Thinking Company’s team culled its training data iNaturalists, a crowdsourced image database. All of the images have been second certified to research grades. Using Encord, two members of the AI Hornet Device team annotated 1500 images of each pollinator class (4500 total) in just three hours. 

“We built separate pipelines for the three sets of images and labeled them accordingly. Then we trained and tested this small model. We got really good results, and it took more time to pull the images off the internet than it did to label them and build the model,” says Elmes.

The Far Out Thinking Company has installed the first two of its Asian Hornet ID Devices on the Isle of Scilly, off the coast of Cornwall. They’re now working with Associated British Ports to build and install a device in Plymouth Port.

The University of Plymouth has also helped them build an image scraper to speed up the process of gathering images, and the company is now building a larger model for its Asian Hornet ID Devices, training it on 1500 images of each pollinator class.

“With Encord, myself and other team members can label the data ourselves. We have numerous projects that we want to achieve this year, and I think having an easy to use platform where we can all work together is going to help us move them forward quickly and successfully,” says Elmes.  

Beyond the Asian Hornet ID Device, The Far Out Thinking Company is using Encord in three other projects – one to determine the plant biodiversity in local fields, one to identify local moth diversity, and one to decode the honey bees’ waggle dances. All of these projects will help local pollinators survive by providing valuable information about the resourcing and health of the environment.