设置
快速交货
免运费
国际贸易术语
支付类型
市场产品
Contents
Top
Edge Compute
Edge AI and Machine Learning
Machine Learning training methods
Embedded ML
ML Software
TensorFlow
TensorFlow Lite
PyTorch and ExecuTorch
Edge AI and ML Hardware
Requirements
NPUs
TPUs
Edge Impulse
Sensing with AI
Vision sensors with machine learning
Motion sensors with machine learning
Edge AI
Edge computing is just as it sounds; computing near the source of data, as opposed to cloud computing (which lives off-site in a datacenter). Things like a smart home assistants (Alexa or Siri) have Edge AI in them. When you say "Hey Alexa" the Edge AI recognizes that phrase and wakes up the smart home assistant up.
Edge Compute
As the need for computing services grows daily, the move away from large scale centralized data centers may seem surprising. However, computing closer to the source of data can lower privacy concerns. The network edge, previously the domain of peripherals and end terminals, is now where we can find advanced processing ability.
Many real-time activities are driving the need for Edge compute. These applications may already be happening around you.
- Smart home assistants (like Alexa, Google, and Siri) use "wake word" detection as a form of Edge computing. Edge compute is necessary in this case because the devices need to respond quickly and maintain privacy for the users.
- Advanced driver assistance systems (ADAS) which focuses on features like notifying a driver when they are departing from their lane of traffic, cannot wait for the round-trip processing to the cloud or risk a delayed network response.
- Patient monitoring (like Glucose monitors) operate near the patient. May only send data to the patient's smart phone (or a local device), and by doing so reduces any chance of exposing protected health data to the internet.
- Predictive maintenance models run anomaly detection on the amount of vibration for industrial motors to detect when motors will burn out. An alert may be triggered (and subsequently sent to the cloud when an anomaly is detected) but the rest of the time the software is analyzing data on the edge (which reduces network congestion).
What is the difference between Edge computing and Cloud computing? View Answer
Cloud compute often refers to a set of services living in a data center (usually far from where the data being generated is processed). Cloud computing workload is processed on servers that run Central Processing Units (CPUs), or Graphic Processing Units (GPUs).
Edge compute refers to services that are located near to the data being generated. Edge computing workload is often processed on Microcontrollers, or Single Board Computers (SBCs).
Use Single-Board Computers to Create an IIoT Edge Computing Platform
Cloud-based data centers are important resources for the Industrial Internet of Things (IIoT), providing scalable mass storage, processing, analytics, regular attention to security, and continual functionality updates; but relying on the cloud for everything has several disadvantages.
Install Home Assistant and Set Up a Cost-Effective Smart Home Hub
Let’s take a look at the installation and setup process of Home Assistant on a cost and power-effective x86 single-board computer (SBC)
Edge AI and Machine Learning
Artificial intelligence (AI) and Machine Learning (ML), in the broadest definition, is the use of computer algorithms and statistical models that improve task performance without direct explicit instructions. This ability to allow developers to not define every action is a change to traditional programming that are explicitly defined, allowing for self-improvement and adaptability that is finding new uses in nearly everything from internet content recommendations and speech recognition to medical science and autonomous vehicles.
As more and more systems in everyday life now have some level of ML interaction, understanding this world becomes vital for the Engineers and Developers to understand the future of user interactions.
Artificial Intelligence (AI) is a wide field, in particular Machine Learning (ML) is the field we see the most opportunity at the Edge. Machine Learning is a process that does pattern matching based on a statistical algorithm.
Usage of neural networks is a common training method in Machine Learning. The picture below denotes the nodes and weights in such a model.
Source:
Source:
What is a Machine Learning model? View Answer
A machine learning model is a software program that has been set to recognize and sort input data based on patterns found in the training data. By using training data ML models can extract patterns found in data and then use those patterns to predict future results and refine more precise models.
Machine Learning training methods
To get a functional ML model, it must be trained. There are several different learning methods that can be used to train the model.
- Supervised - is a type of training based on labeled and tagged sample data, where the output is a known value that can be checked for correctness, like having a tutor to correct work along the way. This type of training is typically used in applications such as classification work or data regression. Supervised training can be very useful and highly accurate, but it depends greatly on the tagged dataset and may be unable to handle new inputs.
- Unsupervised - instead of labeled training data with defined outputs, unsupervised training uses learning algorithms to study, analyze and find clusters of data out of unlabeled datasets. Typically, unsupervised training is used in applications where large datasets need to be studied and relationships between datapoints need to be found.
- Semi-supervised - is a mixture of both Supervised and Unsupervised training. The training dataset contains both tagged and unlabeled data. While Semi-supervised training can handle a wider range of input data then other methods, it can be more complex than either Supervised or Unsupervised training and the quality of the unlabeled data may affect the accuracy of the final model.
- Reinforcement learning from human feedback (RLHF) - is a type of training that requires the use of clearly definable actions it can take along with, performance metrics and an outcome that can be graded and improved on. By defining the ruleset and actions that can be taken, the Reinforcement training can then iterate continuously to evaluate different courses of action to achieve the goal conditions.
Embedded ML
Embedded ML is a subset of Machine Learning focused on running models that are:
- low latency (not waiting for servers or dealing with the network latency to get to the servers)
- low bandwidth (not returning high-def data over the network)
- low power (mW, instead of W)
Instead of relying on large data processing hardware (like servers or personal computers), Embedded ML models can be deployed to small, low power devices (like microcontrollers, FPGAs, and DSPs).
Training a model for an Embedded ML application may occur on a server or computer. This is a process where all the data is fed in, and a resulting model is generated.
Once an Embedded ML model that is generated it can be found running on an embedded system. Common applications within Embedded ML include recognizing and activating on a wake word, identifying people or objects, or finding anomalies based on a stream of data from a sensor.
What are the benefits of Edge AI (or Embedded ML)? View Answer
- Low latency: not needing to wait for remote servers or network latency
- Low data bandwidth: not sending or returning large amounts of high-definition data over networks
- Privacy: making decisions at the edge devices along with prefiltered sensor outputs, data does need to be propagated over the internet
- Low power consumption: Embedded ML uses power in terms of milliwatts, instead of Watts, making it perfect for battery powered mobile devices
- Small data footprint: Embedded ML is targeted to fit on small amounts of flash memory, on the order of 10's of kilobytes
- Economical and lower cost devices: Embedded ML can be computed locally on a 32-bit MCU, with the entire system often falling below $50
What is Edge AI? Machine Learning + IoT
The term “Edge AI” might be the new buzzword, much like “Internet of Things” was in 2016/2017. To understand this growing new trend, we need to provide a solid definition of what constitutes “Artificial Intelligence on the Edge.”
TinyML: Getting Started with TensorFlow Lite for Microcontrollers
TensorFlow is a popular open source software library (developed by Google) for performing machine learning tasks. A subset of this library is TensorFlow Lite for Microcontrollers, which allows us to run inference on microcontrollers
ML Software
TensorFlow
Sourced from: https://www.tensorflow.org/
TensorFlow is a free and open-source software library for building, training, and deploying machine learning models. Originally developed by Google for the Google Brain project in 2011, it was released to the public in 2015 with the updated and current version TensorFlow 2.0 released in 2019.
TensorFlow is the most popular and widely used framework for machine learning training and inference of deep neural networks. Many developers interact with TensorFlow via its Python API library, but TensorFlow is also compatible with Java, JavaScript, and C++ programming languages. Third party packages increase the options allowing for nearly everything from MATLAB, R, Haskell, Rust, and others.
TensorFlow Lite
Sourced from: https://www.tensorflow.org/lite/guide
TensorFlow Lite is a subset of TensorFlow designed for working on small scale. This includes hardware limited and power restricted devices such as embedded systems, mobile devices, and edge computing devices. A developer can train, create, or modify an existing ML model in TensorFlow and then convert it to a smaller and more efficient software package using TensorFlow Lite, that allows it to run on mobile devices.
Sourced from: Once deployed to the microcontroller platform however, there is typically no further training for the ML model on the device as that requires too much computation power and time to perform locally. This means that the model is typically fully trained in offline with no new sources of data. This limited approach makes it perfect for applications that perform a single task. But need to sort a large range of data.
Sourced from: What is the difference between TensorFlow and TensorFlow lite? View Answer
TensorFlow requires larger computing hardware, along with a full General Purpose Operating System (GPOS).
TensorFlow Lite is optimized for running ML models on embedded systems, mobile devices, and edge computing devices.
Single Board Computers (SBCs) such as the Raspberry Pi 4 that operate GPOSs can run the full TensorFlow on an appropriately sized SBC but do be aware that the power consumption is typically much higher than an MCU or DSP, usually running into 10 ~ 20 Watts.
Generally, the training of the models will be done on TensorFlow, and for the model to run on a lightweight embedded system it will be converted to run on TensorFlow Lite.
PyTorch and ExecuTorch
Sourced from: https://github.com/pytorch/pytorch
To meet the growing need of ML models to run on localized hardware platforms, PyTorch has developed ExecuTorch, a full end to end software tool to run machine learning models on smartphone wearables and embedded platforms. Able to operate the same toolchains and SDK as regular PyTorch from modeling to conversion and deployment ExecuTorch is also compatible with a wide range of CPU, NPU and DSP platforms.
ExecuTorch is built with PyTorch 2.0 making it user friendly and supported by a wide range of devices including Android support for smartphone applications.
What is PyTorch Edge vs ExecuTorch vs PyTorch Mobile? View Answer
PyTorch Edge is the concept of running an ML model at the edge. PyTorch Mobile is the legacy toolset that achieved this, and ExecuTorch is the modern toolset for running a PyTorch model at the edge.
PyTorch Mobile was an earlier attempt at transferring PyTorch to mobile devices, particularly iOS, Android and Linux based but was limited by requiring a static memory footprint at the start of the application. Whereas ExecuTorch has dynamic memory which means it only allocates the memory it needs when it needs it, a critical step in memory constrained environments.
Can I run my TensorFlow model on PyTorch (and vice versa)? View Answer
While TensorFlow and PyTorch are similar software environments, they are not directly cross compatible with each other currently. Due to the different training methods and model file outputs each system uses, it is tricky to use a model from one ecosystem to another.
Luckily, there is another option that makes conversion between the two easy. ONNX or Open Neural Network Exchange is an open-source ML software system that has a tool for the conversion of training model files between ML ecosystems.
With ONNX acting as an intermediate step between ML ecosystems it allows developers to work with many different ML training methods and optimize models across a range of methods.
TensorFlow Lite Tutorial Part 1: Wake Word Feature Extraction
Machine Learning (ML) is gaining increasing traction in the tech community, and TensorFlow is one of the most popular framework. Using embedded devices for ML is a subset of TensorFlow, called “TensorFlow Lite” and has been released to run inferences of models on smaller, low power devices like single board computers.
How to Perform Object Detection with TensorFlow Lite on Raspberry Pi
In this tutorial, I’ll walk you through the process of installing TensorFlow Lite on a Raspberry Pi and using it to perform object detection with a pre-trained Single Shot MultiBox Detector (SSD) model.
TensorFlow Lite for Microcontrollers Kit Quickstart
Machine learning has come to the 'edge' - small microcontrollers that can run a very miniature version of TensorFlow Lite to do ML computations. But you don't need super complex hardware to start developing your own TensorFlow models! We've curated a simple kit to dip your toes into machine learning waters.
Edge AI and ML Hardware
TensorFlow Lite for Microcontrollers is a Machine Learning software platform designed for microcontroller scale applications. C or C++ Libraries or dynamic memory applications make TensorFlow Lite for Microcontrollers small but significantly more powerful than standard programming options.
Why do Edge AI on a microcontroller? View Answer
Microcontrollers are small, flexible, low power, and low-cost options found in billions of devices worldwide. Microcontrollers are advantageous when a full computer system is not required. Microcontrollers are of particular interest to the field of ML, the ability to make decisions and process incoming data locally. An additional benefit to processing data locally is that end user privacy can be protected. For example, you may want to know if someone is approaching your front door without having to have video data from your doorbell up in the cloud.
Requirements
TensorFlow Lite for microcontrollers is developed in the C++17 programming language and requires a 32 Bit Microcontroller platform to operate.
The software core operation of TensorFlow for Microcontrollers can run on ARM Cortex M platform with as little of 16 KB of memory, it has also been ported to the popular EPS32 platform was well. There is even an Arduino software library for the TensorFlow Lite for Microcontrollers framework if you prefer working with a suitable Arduino platform.
What are the limitations of the TensorFlow Lite for Microcontrollers? View Answer
While TensorFlow Lite for Microcontrollers is a powerful Machine Learning platform that can operate in the smallest hardware requirements there are number of limitations that can make development challenging:
- Official support for a small number of devices, typically high powered 32-Bit platforms
- Like TensorFlow Lite, on device training for microcontrollers is not supported.
- A small subset of Basic TensorFlow operations is supported.
- A Low-level C++ Application programming interface (API) may be required for memory management.
Other Embedded Systems
Deploying ML models on more powerful hardware systems, that can run Embedded Linux or an SBC like the Raspberry Pi platform, the TensorFlow lite platform can run as well.
Hardware accelerators
At a chip level, there's more dedicated arithmetic logic being added to certain processors to help aid in the math for Machine Learning and the neural networks used.
NPUs
Neural processing units (NPUs) are specialized IC that are designed to accelerate the processing of Machine Learning and Artificial intelligence applications based on neural networks.
Neural networks are structures based on the human brain with many interconnected layers and nodes called neurons that process and pass along information.
TPUs
Tensor processing units (TPUs) are specialized ICs developed by Google in 2015 that are designed to help processing of neural networks-based systems but unlike NPU this is done not with a neuron-based system architecture but by quickly processing matrix multiplications and convolution operations.
TPUs are highly optimized for the processing of mathematical matrix operations at low energy usage making them ideal for training ML models that use such operations like TensorFlow and ExecuTorch.
The first generations of TPU hardware are typically found only in datacenter applications due to their power supply and cooling requirements. But the newest generation of TPUs are designed for edge-based applications.
Why aren’t CPUs and GPUs used more in Edge AI? View Answer
Traditional central processing units (CPUs) are great for general purpose operations but are not optimized to efficiently crank through all the arithmetic of neural networks. Graphics processing units (GPUs) have also become very useful for machine learning and artificial intelligence applications however the large overhead in energy usage and required hardware makes them limited in their usage at the edge.
Entry-level products
SparkFun Edge Development Board - Apollo3 Blue
Apollo3 Blue - ARM® Cortex®-M4F MCU 32-Bit Embedded Evaluation Board
Adafruit EdgeBadge
ATSAMD51 EdgeBadge SAM D ARM® Cortex®-M4 MCU 32-Bit Embedded Evaluation Board
Arduino® Nano 33 BLE Sense Rev2
NINA-B306, nRF52840 Arduino Nano 33 BLE Sense Rev2 with Headers - ARM® Cortex®-M4F MCU 32-Bit Embedded Evaluation Board
Intermediate products
Wio Terminal: ATSAMD51 Core
ATSAMD51 Wio ARM® Cortex®-M4F MCU 32-Bit Embedded Evaluation Board
XIAO ESP32-S3 Senes
OV2640 Transceiver; 802.11 b/g/n (Wi-Fi, WiFi, WLAN), Bluetooth® 5.x 2.4GHz Evaluation Board
Advanced-level products
STM32F746G-DISCO
STM32 F7 Discovery STM32F7 ARM® Cortex®-M7 MCU 32-Bit Embedded Evaluation Board
Expert-level products
Jetson Nano Dev Kit
Maxwell NVIDIA Jetson Nano Developer Kit - ARM® Cortex®-A57 MPU Embedded Evaluation Board
Himax WE-I Plus EVB Endpoint AI Development Board
HX6537-A Himax - ARC-EM9D DSP Embedded Evaluation Board
Additional products
Edge Impulse
Edge impulse is a cloud based, integrated development environment (IDE) that allows software developers to collect and import real world data, build, train, and test ML models which then can run efficiently on edge compute devices.
The difference between Edge Impulse and TensorFlow machine learning software. With Edge Impulse the model development process is abstracted away from the user while TensorFlow development is a much more involved process.
What Edge Impulse is good at
Edge Impulse framework can work with everything from audio (from microphones) to images (from cameras) to sensor data (from vibrational sensors). Some additional examples include:
- Object recognition and image detection (for things like detecting people)
- "Wake words" via audio detection (for smart home assistants)
- Anomaly detection (for preventative maintenance)
- Activity/Pattern recognition (for streamlining process)
Why should I use Edge Impulse? View Answer
Edge Impulse can lighten the load on the amount of work needing to be done with ML training software (which is largely TensorFlow at this time). Edge Impulse provides a web-based interface where data can be tagged and uploaded to train the model.
Edge impulse was designed for first time users or non-ML developers that have never used ML training software before, but it is also fully extendable to expert level users as it compatible with TensorFlow python framework. By removing much of the coding barrier in creating Edge AI models, one can focus more on providing the right data / training the right model and less on which python libraries should be included to run the code.
While there is a free usage tier for most developers and users, it is limited in the number of projects and processing power each project can use. For professional developers there is a paid tier that you can contact Edge impulse to negotiate rates with.
Lego Brick Finder with OpenMV and Edge Impulse
In this tutorial, Shawn shows you how OpenMV and Edge Impulse trained for image recognition can be used to create Lego® brick finder.
Getting Started with the OpenMV Cam: Machine Vision with MicroPython
In this tutorial, Shawn shows you how to get started with the OpenMV H7 camera module, which uses MicroPython to perform a variety of machine vision tasks
Car Parking Detection System Using Edge Impulse
In this example project, we have built an object detection model to detect the number of cars in a parking lot and deploy it on a Raspberry Pi 4.
Nordic Semiconductor 的 Nordic Thingy:53 利用集成的运动、声音、光线和环境传感器来帮助构建概念验证和原型开发。
A full listing of compatible Edge Impulse boards can be found here:
https://docs.edgeimpulse.com/docs/edge-ai-targets/fully-supported-development-boards
Sensing with AI
Although running a Machine Learning model will require an embedded system, there will be more products that continue to be released as AI enabled electronic components. This includes AI enabled sensors, also known as machine learning sensors.
What machine learning sensors can do
While adding a machine learning model to most sensors will not make them more efficient at the application there are a few types of sensors which ML training can enable to perform in significantly more efficient ways.
- Camera sensors where machine learning models can be developed to track objects and people in frame.
- IMU, accelerometer and Motions sensors to detect activity profiles.
Vision sensors with machine learning
By using neural nets to provide computational algorithms, it is possible to detect and track objects and people as they move into the field of view of the camera sensor.
AI Vision Modules - Electronics with Becky Stern
AI vision boards allow you to incorporate features like face, person, and object recognition into your projects.
Getting Started with the OpenMV Cam: Machine Vision with MicroPython
In this tutorial, Shawn shows you how to get started with the OpenMV H7 camera module, which uses MicroPython to perform a variety of machine vision tasks.
Introducing the Arduino Nicla Vision Board for Professional Machine-Vision Applications
This article focuses on the main features and specifications of the Arduino Nicla Vision, a compact industrial-grade board that has the potential to drive professional machine-vision applications.
AI vision sensors
Person Sensor by Useful Sensors
The Person Sensor from Useful Sensors is a small, low-cost hardware module that detects nearby peoples’ faces, and returns information about how many there are, where they are relative to the device, and performs facial recognition.
Motion sensors with machine learning
Motion sensor with integrated machine learning platforms are able to track and process data in real time in the same package as the sensor reducing power consumption and processing time. Most sensors of this type can either have a small computational logic-based decision tree or the ability to predefine filter or trigging thresholds. As in when the input values reach this level, perform this action. Other motion sensors have a complete DSP unit built into the package allowing for multiple ML algorithms to run without the primary CPU saving power and time.
Cat.Ai – Potentially Genius | DigiKey
n this episode, Tomorrow Lab collaborates Cornel Pizara from Analog Devices to create a product using the MAX78000— a new breed of AI Micro built to enable neural networks to execute at ultra-low power.
Use a Smart Sensor’s Built-In Machine Learning Core to Optimize “Always-On” Motion Tracking
The emergence of low-power sensors with built-in motion detection capabilities helps developers eliminate this compromise in their designs.
Do You Really Need All That Raw Sensor Data? No! There’s a Better Way
Sensor systems can capture macro events that are deconstructed to an extended sequence of measurements with astounding levels of detail. Even the most ardent data junkie will nevertheless admit that massive amounts of raw data by themselves don’t advance the objectives of the application.
AI motion sensors
STEVAL-MKI119V1
L3GD20, LSM303DLHC iNEMO Accelerometer, Gyroscope, Magnetometer, 3 Axis Sensor Evaluation Board
