Deep Learning and Drones

Deep Learning Part 5: Running Pre-trained Deep Neural Networks through Microsoft Cognitive Services APIs on Raspberry Pi 3 & Parrot Drones

by Anusua Trivedi, Microsoft Data Scientist

This blog series has been broken into several parts, in which I describe my experiences and go deep into the reasons behind my choices. In Part 1, I discussed the pros and cons of different symbolic frameworks, and my reasons for choosing Theano (with Lasagne) as my platform of choice. In Part 2, I described Deep Convolutional Neural Networks (DCNN) and how transfer learning and fine-tuning improves the training process for domain-specific images. Part 3 of this blog series is based on my talk at PAPI 2016. In Part 4, I show the reusability of trained DCNN model by combining it with a Long Short-Term Memory (LSTM) Recurrent Neural Network (RNN). We apply the model on ACS fashion images and generate captions for these images.

In this part, we explore AI on the Internet of Things (IOT). In the video below, you can see an intelligent drone, built by two high school students, that can recognize objects and people in real-time using the Microsoft Computer Vision APIs. This post details how we ran pre-trained Deep Neural networks on Raspberry Pi & Parrot Drones to achieve this.

Motivation

Isha Chakraborty and Neelagreev Griddalur are rising juniors at Monta Vista High School and active members of their high school’s FRC Robotics team.  They are smart tinkerers who are both really interested in the world of IOT (Internet of Things). They wanted to expand their knowledge on AI & create some AI for IOTs. They read my blogs and approached me to do a fun AI project using either Raspberry Pi or Drones. We ended up working on both! In this blog post, we describe our experiences working together on this AI project. Here we are applying image and object recognition for Raspberry Pi and Drones. We first began our endeavors with the Raspberry Pi, and after succeeding with that we ventured into working with Drones.

Introduction