When AI has a Person-al problem!

I was in need of a flexible and easy to configure Security Camera Systems for my home and after some research, I had decided to try out Arlo’s home security camera system. My main objective is to see how Arlo’s Artificial Intelligence (AI) recognition features perform with the real-time image/video data.

Installation was quick and I was really impressed with the Wi-Fi connectivity, 2K resolution video feed, local storage (this is my favorite feature), clod-based features and off course Artificial Intelligence capabilities.

One of the feature I was looking forward to test is built-in (cloud based?) AI algorithm on how it recognizes People, Vehicles, Animals and detect Motion. I don’t have the complete insight into how and where this detection is performed. It could be at the camera level, base station or in the cloud.

  • Camera Level – Could be quicker
    • Base Station – Camera relays data to the Base Station and receive the feedback from Base Station
      • Cloud – Same as Base Station scenario but Base Station uploads data to Cloud

So, I went ahead and turned on the alerts for all the possible combinations and within few minutes I started receiving notifications on my mobile app. That was pretty quick and made me trust the system more.

Most of the notifications are accurate but for Person detection, 80% of the notifications are false alarms.

Arlo has a very helpful built-in feature and this feature actually captures a few seconds of video whenever a Person is detected.

Majority of the time Person detection is a result of a motion in the front yard like a moving vehicle or motion caused by wind. But after reviewing footage for Person Detected alerts, 80% of the time I did not see an Person in the video clips. This made me think that I had to configure AI detection features in the system more accurately but found no way to do the same.

At this point I thought I need to trust the AI and assume that Arlo did a good job of building this Person Detection Machine Leaning Model. Arlo might have used a significantly big training dataset (Images), tested against big dataset (Images) and also validated against some real life scenarios. Some scenarios like multiple people in an image (ex: a family walking on side way in from of your home).

Meanwhile mystery continues and at one point I thought of reaching out to Arlo, which I will do in future any way, and explain the problem.

This is one the main challenge we would face while developing or implementing AI capabilities or solutions.

As part of building a ML Model, for example, detecting a Person in an image, we often rely on Supervised Leaning approach, using massive datasets of images for training, testing (fit, score and predict) and pick the model with a higher R Squared value aka Coefficient of Determination Factor.

Experience with my security system made me think that it’s not just about building a model and leave the model in a static state. When I say static what I mean is that model is not designed to take new information, fine tune and come up with more accurate predictions.

This may lead us into adopting Unsupervised Learning approach but still we still need a Data Science team to classify/refine and validate predictions.

In my opinion every Data Science project or while building AI capabilities we need to think like we are building a Product rather than simply creating a Solution. From a Product perspective, Product architecture should allow configuration, automation and a Strategy to keep AI capabilities or ML Models to up-to-date. This is very important otherwise your ML Models will become stale and so are the predictions.

Back to my original story, after studying the Arlo’s mobile app (where I receive notifications) I found out that when ever Arlo’s AI detects a Person, Arlo’s system starts capturing a quick video for future reference and in the actual notification it displays a snapshot of the image (still image) where actual Person, which AI thinks is highlighted.

This provides an instant view of the entire front yard, highlighting the area where AI thinks that there is a Person.

Finally mystery is solved!

The way our mailbox, my neighbor’s mailbox, and the mailbox of the home (opposite to our home), the angle of the camera, and most importantly the way Arlo’s Security Camera sees and interprets all these 3 mailboxes as a single image. Of course from the angle, all these 3 mailboxes definitely looks like a Person 😊

So, whenever there is motion in front of yard, Camera takes a snapshot and performs “predict” to identify, Person, Vehicle and Animal and ultimately predicts that all three mail boxes resemble a Person.

Until Arlo’s ML Model is updated with the latest dataset, I will end up receiving these alerts for the rest of my life or rest of the Security System’s life. I will try to reach out to Arlo’s team and hopefully they will provide a solution.

Meanwhile other solutions I can think of are

  • Change the Camera Spot
  • Paint my mailbox with white color
  • Disable Alerts
  • Look for a new spot for my mailbox

It’s very important to have a strategy and plan to keep every component of the AI or ML solution/product up-to-date. If not, your AI solution will have a “Person“al problem like my security system.

Leave a Reply

Your email address will not be published.

Close Bitnami banner