EFFICIENT MULTI-CLASS TOXICITY DETECTION ON EDGE DEVICES USING QUANTIZED LARGE LANGUAGE MODELS

Abstract

The growth of online platforms has increased the need for automated systems that can identify harmful comments at scale. Toxic comment classification is a challenging natural language processing problem because harmful language varies in tone, context and intensity. Transformer-based language models provide strong contextual representations, but their size and computational requirements can limit deployment in resource-constrained environments. This dissertation presents a resource-efficient toxic comment classification pipeline using fine-tuned and INT-8 dynamically quantized transformer models. The study uses the Jigsaw toxicity dataset and converts its continuous toxicity annotations into a three-class classification task: Safe, Toxic and Obscene. The original mapped dataset is highly imbalanced, containing 1,656,908 Safe comments, 138,360 Toxic comments and 9,603 Obscene comments. To reduce class imbalance, the dataset is down-sampled to 9,603 examples per class, producing a balanced dataset of 28,809 comments. The balanced dataset is split into 20,166 training samples, 4,321 validation samples and 4,322 test samples. Three transformer models are evaluated: DistilBERT, Toxic-RoBERTa and RoBERTa. Each model is tested in three stages: base model with an untrained classification head, fine-tuned model and INT-8 dynamically quantized model. The models are compared using accuracy, macro F1-score, macro precision, macro recall, latency and model size. Toxic-RoBERTa achieves the best fine-tuned result with accuracy of 0.8987 and macro F1-score of 0.8985. After INT-8 dynamic quantization, Toxic-RoBERTa retains nearly the same performance with accuracy of 0.8984 and macro F1-score of 0.8982 while reducing size from 475.5 MB to 148.9 MB. The results show that fine-tuning is essential for the three-class toxicity task and that dynamic quantization can significantly reduce model size with small performance loss. The thesis recommends quantized Toxic-RoBERTa as the best overall choice when accuracy and storage efficiency are both important. It also highlights that latency comparison should be performed under matched hardware conditions because the quantized models in this experiment run on CPU while the fine-tuned models were evaluated on GPU.