Improving Assamese Speech Recognition Performance

Summary & Methodology Analysis

The researchers addressed the scarcity of annotated training data for the Assamese language by performing controlled fine-tuning, which is the process of adjusting a pre-trained model on a specific downstream dataset, of the Whisper-Small architecture. To prepare the audio inputs, the team resampled all data to 16 kHz and converted it to single-channel mono. They also implemented a custom normalization pipeline using NFKC Unicode standards to ensure Graphemic Standardization, which helps align written text characters with their spoken representations. The model was trained using mixed-precision (FP16) training to reduce memory overhead and utilized linear learning rate warmup paired with weight decay to stabilize convergence during the learning process. Inference was conducted using beam search, an algorithm that explores multiple potential text sequences during decoding to find the most likely transcription, with a beam width of 5. Performance metrics recorded include a Word Information Loss of 64.81%, a Match Error Rate of 43%, and BLEU and METEOR scores of 30.81 and 0.5262, respectively. The development was physically limited by the available GPU VRAM, forcing the team to focus on the Whisper-Small model rather than the larger Medium or Large variants. Furthermore, the reliance on the Mozilla Common Voice 24.0-Assamese corpus means the model is optimized for read speech in controlled environments. As a result, the system lacks robust representation for regional dialectal variations and potentially spontaneous, noisy real-world speech inputs.