The Role of tinyML in Enabling Computer Vision at the Edge – Thought Leaders


By: Davis Sawyer, Co-Founder and Chief Product Officer, deeplite

Computer vision has great potential to improve our daily lives – and there are many applications and uses for it. Here are some examples :

  • Smart doorbells for home security, help prevent “porch hackers” and burglaries. According to research by IHS Markit (published in SecurityInfoWatch) the number of global surveillance cameras in the world is expected to reach one billion in 2021. In the United States alone, the number of cameras is expected to reach 85 million;
  • In parking spacesAI-enabled cameras automate tracking of available and occupied parking spaces to let consumers know where open spaces are;
  • Dash cameras on trucks now read speed limit signs and dynamically reduce truck speed to improve safety;
  • And drones with connected cameras monitor remote and hard-to-reach areas, and they can process images and make decisions in real time.

All of these apps use intelligent video analytics, driven by AI and machine learning (ML), to watch videos, use intelligence to make decisions, and then take action.

Computer vision needs more resources at the edge

However, like many AI-based applications, computer vision needs computing power, memory, and energy to perform its complex analysis and make decisions. While this is fine in a data center with a lot of computing power, it can prevent AI from moving to the edge. Specifically, small devices located far from corporate data centers and running on small batteries need a new generation of AI that’s smaller, faster, and “lighter” than traditional approaches. And existing devices will need to be upgraded with the new AI+ML (computer vision) functionality to remain viable and competitive.

New advances boost deep neural networks

Today, new advances in AI are making deep neural networks (DNNs) faster, smaller, and more energy efficient — and helping to move AI from the cloud and data centers to edge devices and battery-powered sensors. When it comes to training AI models, the staggering carbon footprint has been documented and discussed (i.e. training an AI language model emits as much CO2 as 5 cars over their lifetime). However, we need to understand what is the environmental impact of AI Inference Model is and how to reduce that footprint. This is where model optimization can have huge benefits by reducing the economic and environmental cost of DNNs.

TinyML enables AI on small devices

One of these advancements is lowercaseML, a powerful new trend enabling small battery-powered devices to use advanced ML to deliver computer vision and other perceptual tasks. It facilitates ML inference on small, resource-constrained devices, typically at the edge of the cloud, and helps bring edge applications closer to the user.

For example, a server GPU like an NVIDIA A100 has over 40 GB of available memory, which is suitable for running complex AIs like computer vision and natural language processing. However, when we talk about edge devices and tinyML, a common microcontroller (MCU) may only have 256 KB of on-chip memory, which is over 100,000 times less memory than the cloud! Also, unlike data centers and the cloud, the hardware of edge devices cannot be easily updated in the field. This means that we have to “fit” our AI to the available hardware, which can take developers months or even years of trial and error, if at all. This is where tinyML, specifically Automated Machine Learning (also known as AutoML), can play a major role in breaking down barriers to real-world AI adoption.

And tinyML’s influence is growing. With over 10,000 members, the tinyML Foundation develops the ecosystem to support the development and deployment of ultra-low power machine learning solutions at the edge. The Foundation brings together a global community of hardware, software, machine learning, data scientists, systems engineers, designers, products and business people.

A world of opportunities

In total, there are billions of small connected devices everywhere that can benefit from advanced intelligence. The challenge is that they have very limited resources, so how can we add intelligence to them? tinyML can play a key role in bringing AI and ML to more real-world computer vision-based applications at the edge of small devices. And it can unlock a world of benefits for people and businesses across a range of products, services and industries, helping us push new frontiers for AI.