The Demos
You might not get to see the videos in the highest resolution here. To check them out in a higher resolution click into the video and see it on Odysee.com.
I am demonstrating how the Computer Vision, object detection model works to generate the "detections". I also share which models I am using to generate the detections.
Moving forward I will implement features to record, save, and share the video with detections. Integrations with other applications will be seamless.
The Benefits
Off the bat, I can think of two major benefits of using an on-device AI system.
1) Privacy
Sending users live video data to a local Computer Vision, object detection service affords users the ultimate in privacy by mitigating a number of risks. A third party can not store and retrieve a user's video data for future uses like user profiling, reviews, or model training.
Also, Man In The Middle (MITM) attack risks are effectively eliminated by removing any reliance on external API calls to third party services over Wide Area Networks (WANs).
This brings me to the next benefit.
2) Availability
According to various tech industry studies, approximately 75-80% of mobile apps require an internet connection to function fully or partially.
In the event that users lose network or internet access, the local, on-device AI app will remain operational. As internet access is not required for it to be functional like many other applications are.
In a scenario where the internet is down and a user still needs information, the on-device AI app will remain fully functional and always available.
Resources
In order to get started you will need to get familiar with Kotlin, Android's native programming language. Head over to Develop Android apps with Kotlin.
Then, head over to MediaPipe and check out their documentation. Read through it to understand how the framework works.
From there, read the Object detection task guide. It will describe which features are available, how the model works, which models are available and how to use them.
Next you'll want to read the Object detection guide for Android. It will show you how to get it up and running on your device.
If you're new to Android app architecture I highly recommend you read the Guide to app architecture. It will give you a deeper understanding of what is going on in your app and how it works.
If you want to go even deeper and REALLY get into the weeds check out the Android Open Source Project. Here's where you can dive DEEP into the operating system and learn how to optimize your app to run super smooth on target devices.