VLLM FEATURE READINESS ON TEXT GENERTAED LLAMA 3.1-8B MODEL

Abstract

The recent developments in Artificial Intelligence, Deep Learning, and Natural Language Processing have revolutionized the areas of speech synthesis and language generation. Contemporary neural Text-to-Speech (TTS) models are now able to synthesize highly human-sounding speech that is characterized by improved intelligibility, pronunciations, and prosody. At the same time, there has been great success in using Large Language Models (LLMs) for tasks like conversations, text generation, summarization, and reasoning. The current research work comprises a detailed comparative analysis of sophisticated Neural TTS systems alongside inference optimization methods for Large Language Models. This analysis is concentrated on three contemporary TTS models, namely Tacotron, FastSpeech 2, and MatchaTTS. These models are comparatively studied regarding their performance parameters such as synthesizer quality, training difficulty, inference speed, computation, and real-world implementation feasibility. In addition to speech synthesis, this study also considers advanced inference acceleration and optimization methods for autoregressive Large Language Models (LLMs). The considered methods include speculative decoding, LoRA adapter, and Multi-LoRA adapter, which were implemented on an Intel AI accelerator with Intel Gaudi B70 accelerator and the Llama 3.1 8B model. The following three speculative decoding strategies have been considered in order to speed up the inference process based on generation of several token candidates through draft/retrieval procedures and subsequent validation by the target LLM: EAGLE-3, N-gram prompt lookup decoding, and suffix array retrieval decoding. Moreover, the LoRA adapter and Multi-LoRA adapter methods have been considered for efficient parameter tuning and multitask fine tuning, respectively. The effectiveness of the presented inference strategies has been assessed by using the key inference parameters: time to first token (TTFT), time per output token (TPOT), and throughput.