Imagine yourself as a data analyst working to identify objects in aerial imagery. Maybe you’re tracking fishing vessels through the ocean to support ocean sustainability and protect marine ecosystems, or perhaps you’re looking at cities that have been hit by a natural disaster to determine where to allocate your relief efforts. In either case, you would need a dedicated team to comb through thousands of satellite images, manually identifying, tracking, and analyzing relevant objects. Extracting insights from imagery data is an expensive process that could take upwards of days, delaying the decision-making process in mission critical situations.
Inspired to make this process more efficient, the two co-sponsors decided to team up to pitch Garage interns on tackling this problem by leveraging AI. The Garage Internship Program offers teams of 5-6 university students the opportunity to design and build their own project in response to a challenge by sponsoring Microsoft groups. The Vancouver-based group of interns loved the AI for Good pitch. “In the start of May, our sponsors came to us with an idea that immediately sparked interest among the six of us: an opportunity to leverage AI to help with environmental efforts headed by researchers and conservationists,” shared Michelle Chen, Program Manager intern for Earth Lens.
AI for Good
Equipped with a trained machine learning model from the AI for Earth team, the interns set out to build a minimally viable product over the course of their remaining 16-week internship. They approached design with a customer obsessed mindset and interviewed prospective users like to the Red Cross, OceanMind, and FarmBeats to identify useful features and iterate on their prototype. The team quickly realized that, in addition to disaster relief, such a tool might be useful in a variety of social impact and sustainability applications such as agriculture, forestry, fishing, and land conservation.
The team built an iOS app that offers a host of features that humanitarians and environmental researchers can leverage to accelerate their analyses:
• Automatically identify, classify, and label objects in satellite images, encircling like objects in color-coded bounding boxes
• Count and aggregate objects identified by the machine learning model
• Toggle classes on/off to focus in on specific objects
• View images over time alongside a data visualization chart to identify trends and patterns in a Time Series mode
• Use the app remotely, without an internet connection
With the Earth Lens source code and corresponding instructions now available on GitHub, researchers will be able to build their own iOS apps that automatically extract information from a large dataset and scan that analysis in a useful and interactive view. As a result, this open source project can be useful in various industrial or agricultural applications to further Microsoft’s commitment to sustainability. In a disaster relief scenario, another minute could mean another life lost.
An Internship Where You Can Make an Impact
Interested in trying Earth Lens? Check out the source code on GitHub, where you can also find instructions on how to build the iPad application and leverage the machine learning model that powers the project. You can learn more or provide feedback via GitHub.