Aerial Telepresence™ and Machine Learning

  Object detection from a drone

Object detection from a drone

By: Rick Pasetto

Many of our previous blog posts have focused on use cases for Cape’s Aerial Telepresence™. They include public safety, oil and gas, and even cattle ranching!  Having a birds-eye view of the world from a remote location has many obvious and exciting benefits.  But Cape isn’t stopping there. Having a person fly remotely over the internet is great, but there is much more we can do.  That’s where Machine Learning comes in.

What is Machine Learning (ML)?  

In addition to being an exciting new field of computing that is revolutionizing everything from self-driving cars to voice-operated devices and searching the web, ML is a technology that is ripe for use in drone operation.  Almost all of the use cases previously mentioned would benefit from ML (We’ll get to that in a moment). But at its core, it is just an algorithm. What is special about it is that it is a learning algorithm.  That means the technology can adapt to various drone use cases, depending on the need.

Let’s take one common use case: object detection and classification.  ML algorithms can be “trained” to identify specific areas of an image where objects exist (for example, locations of vehicles).  Not only that, but the same process can classify the objects -- say, cars versus trucks. The algorithm needs data -- lots of data sometimes -- for that training process.  What is important is that it needs relevant data; in this case, data specifically associated with flying a drone.  The exciting thing about what Cape is doing is: we already have that drone data, and we’re collecting more all the time.

Let me demystify this a little.  ML models don’t really “learn” like a person does; we only use this terminology in order to draw analogies with common experience. One type of ML, Neural Networks, was inspired by neurophysiology -- it is built up of virtual “neurons” similar to how we think the brain might work.  But again, this is really just a model. At its core, an ML model is just a mathematical function (sure, a very complex, multidimensional function, but just a function). Actually, it is a statistical function, in the sense that all it produces is the probability that something is true.  The training process improves the likelihood that the model will make an accurate prediction (for instance, that an object is a car or a truck).


So what does this have to do with what Cape is doing?

As I mentioned above, Cape has gathered data for use in training our own ML models, and the more data we gather, the more we can improve our models.  

But what can we do with that drone data?  Here’s an example where we’ve trained a model to identify (and locate) cars in an image, and then we “taught” the drone to follow the car with the camera.  This is just a proof-of-concept, but it gives you an idea of what is possible with ML, and the potential for even more models to train.

We have other ideas too: using ML to identify cracks in oil pipelines, to recognize dangerous swimming conditions at beaches (look out for sharks!), or even to count cows on a big ranch. And that’s just using ML for object detection and classification. There are numerous other types of ML models to explore. We’re just getting started.

To learn more about our software visit or fly a drone at today!