Kwabena Agyeman discusses optimization, cameras, machine learning, and vision systems. OpenMV utilizes MicroPython for easy development. They focus on optimization, performance benchmarks, and future speed gains. The podcast explores image processing, machine learning with TensorFlow Lite, wasp identification, and tracking small creatures. Balancing ML and physics in problem-solving, transitioning to full-time work at Open MV, and reflections on the embedded systems journey are also covered.
Event cameras provide rapid object tracking capabilities by creating movement-based images for high-speed motion scenarios.
Custom convolution algorithms on ARM Cortex-M4 processors enhance tracking accuracy for identifying specific features or objects.
Prioritizing image quality and object size is crucial for effective machine learning projects, optimizing models for real-world applications.
Deep dives
Challenges of High-Quality Image Capture for Wasp Identification
Having high-resolution images of wasps is crucial for accurate identification. Considerations such as the proximity of the wasps to the camera, lighting conditions, and camera resolution play a significant role. Options like thermal cameras for day and night visibility, event cameras for fast object tracking, and global shutter imagers to reduce motion blur are discussed.
Utilizing Event Cameras for Object Tracking and Detection
Event cameras offer rapid object tracking capabilities by providing shape outlines at high frame rates. They detect changes in charge at each pixel to create movement-based images. This technology is effective in scenarios where traditional cameras struggle with high-speed motion, such as identifying objects like wasps in flight.
Leveraging Frame Differencing and Convolution for Precision Tracking
Frame differencing provides a simpler but effective method for detecting moving objects by subtracting consecutive frames. Custom convolution algorithms can enhance tracking accuracy, with faster implementations on ARM Cortex-M4 processors facilitating real-time processing. These techniques are beneficial for identifying specific features or objects.
Addressing Physics Challenges in Machine Learning Applications
Prioritizing the physical aspects of a problem, such as image quality and object size, is essential in machine learning projects. Ensuring high-quality data captures is key before delving into complex ML algorithms. Appropriate problem setup, including physics considerations, helps optimize the effectiveness of machine learning models in real-world applications.
Evolution and Growth of Open MV's Product Line
Open MV has evolved its product line by expanding beyond just the STM32 processors, with a recent introduction of the Open MV cam RT 1060 that supports NXP's IMXRT. This diversification has enabled the company to offer features like built-in Wi-Fi, Bluetooth, Ethernet support, and low power consumption, catering to customers' needs for various applications.
Advancements in Embedded Processors and Machine Learning
The discussions touch on the advancements in microcontroller processors, especially with the upcoming Cortex M55 processors. These new processors are set to offer enhanced processing power, integrated hardware modules, and significant compute capabilities for machine learning applications. The focus shifts towards optimizing algorithms for these processors to harness their potential, enhancing performance and efficiency in embedded systems.
Kwabena Agyeman joined Chris and Elecia to talk about optimization, cameras, machine learning, and vision systems.
Kwabena is the head of OpenMV (openmv.io), an open source and open hardware system that runs machine learning algorithms on vision data. It uses MicroPython as a development environment so getting started is easy.
Their github repositories are under github.com/openmv. You can find some of the SIMD details we talked about on the show:
Kwabena has been creating a spreadsheet of different algorithms in camera frames per second (FPS) for Arm processors: Performance Benchmarks - Google Sheets. As time moves on, it will grow. Note: this is a link on the OpenMV website under About. When M55 stuff hits the market expect 4-8x speed gains.
The OpenMV YouTube channel is also a good place to get more information about the system (and vision algorithms).
Elecia is giving a free talk for O'Reilly to advertise her Making Embedded Systems, 2nd Edition book. The talk will be an introduction to embedded systems, geared towards software engineers who are suddenly holding a device and want to program it. The talk is May 23, 2024 at 9:00 AM PDT. Sign up here. A video will be available afterward for folks who sign up.
Get the Snipd podcast app
Unlock the knowledge in podcasts with the podcast player of the future.
AI-powered podcast player
Listen to all your favourite podcasts with AI-powered features
Discover highlights
Listen to the best highlights from the podcasts you love and dive into the full episode
Save any moment
Hear something you like? Tap your headphones to save it with AI-generated key takeaways
Share & Export
Send highlights to Twitter, WhatsApp or export them to Notion, Readwise & more
AI-powered podcast player
Listen to all your favourite podcasts with AI-powered features
Discover highlights
Listen to the best highlights from the podcasts you love and dive into the full episode