TINYML-BASED MODEL OPTIMIZATION FOR RESOURCE-CONSTRAINED IOT SYSTEMS: EXPERIMENTAL STUDY AND DEPLOYMENT ANALYSIS

Abstract

The prevalence of low-power microcontroller units (MCUs) deployed in applications related to industrial processes, environmental monitoring, and consumer products led to the emergence of a new challenge associated with implementing machine learning inference at the absolute edge of computation. The existing solutions based on deep learning models face issues connected with the excessive memory and computational complexity requirements which significantly exceed the RAM and flash capacities provided in MCU-class hardware. The current thesis is devoted to investigating TinyML as a practical approach to overcoming the aforementioned obstacle through examining the influence of post-training processing methods, including quantization, magnitude-based pruning, and Huffman weights encoding on the accuracy memory consumption trade-off landscape. Canonical benchmarks of interest include image classification based on the MNIST dataset of handwritten digits and HAR relying on the corresponding UCI database gathered using mobile phones. Training and model conversion to a flat-buffer file in TensorFlow Lite have been carried out within Google Colab notebooks. Eight-bit integer quantization has been applied, resulting in the decrease of memory overhead by a factor of four compared to baseline counterparts while remaining below one percent accuracy loss. The application of magnitude- based pruning of neural networks in the sparse representation range from twenty to sixty percent leads to additional optimization opportunities. The nonlinear problem-dependent character of the accuracy dependency on pruning makes its evaluation necessary. The results prove that the combination of quantization and pruning makes it possible to obtain models with less than one megabyte memory overhead on both problems, matching the flash memory capacity of ARM Cortex-M devices. Simulation through deployment profiling with TensorFlow Lite Micro provides estimations of the required inference time and RAM. Performance measurements confirm that optimized models comply with the performance budget of both Cortex-M4 and Cortex-M7 devices while being applicable for MNIST tasks and requiring further research in HAR cases.