COMPARATIVE ANALYSIS OF TinyML MODELS FOR AUTOMOTIVE ECU FAULT DETECTION USING OBD-II DATA

Abstract

Modern cars are filled with electronic components called Electronic Control Units (ECUs) that play a crucial role in the management of most vehicle functionalities by processing huge amounts of sensor data required for the operation of functions such as fuel injection, timing of the combustion, and emissions control. Traditionally, any malfunction in ECUs was diagnosed via sending this data to cloud servers (that introduces unacceptable delay) or bringing the car to a shop equipped with expensive diagnostic equipment - a far cry from the real-world budget constraints of embedded automotive systems. TinyML can help here by efficiently compressing neural networks, thus allowing running inference on low-cost microcontrollers onboard the car, even those with as little as 256 KB memory. We perform a detailed comparison between four neural architectures designed for onboard ECU fault detection using OBD-II signals: Multi-Layer Perceptron (MLP), a one-dimensional Convolutional Neural Network (CNN-1D), a Long Short-Term Memory network (LSTM), and a hybrid CNN-LSTM network combining both convolution and sequence modeling.The five classes (Normal, Overheating, Misfire, Air Leak, Throttle Fault) are embedded in the KIT Automotive OBD-II Dataset and the models are fine-tuned using these data, with class labels assigned by our custom rule-based fault labelling approach which was validated by automotive engineers. After fine-tuning, all models are quantized to INT8 precision using TensorFlow Lite, and the changes in fault detection performance, memory consumption, and inference speed caused by the quantization are analyzed. In all three metrics, CNN-LSTM performs best with 96.1% fault detection accuracy (F1 = 0.960), followed by CNN-1D with 94.2%, LSTM with 93.7%, and MLP with 91.4%. The main reason behind this performance is the ability of hybrid architecture to leverage convolution operations for detecting local transient features of faults while using LSTM layers for understanding their long-term dependencies. Quantized to INT8, the hybrid model needs 164 KB of flash storage and takes 31 ms for inference, thus becoming suitable for high-end automotive processors like STM32H7 and NXP i.MX RT series. For applications constrained by hardware, MLP can become an alternative providing 91.4% accuracy with 38 KB of flash consumed and 12 ms of inference speed The major contributions of this paper are as follows: (1) a reliable labeling protocol for unlabelled OBD-II dataset; (2) a consistent benchmarking procedure for automotive TinyML models including fault detection accuracy, memory consumption, and inference speed; (3) the first analysis of performance of a hybrid CNN-LSTM architecture under INT8 quantization constraints.