BELLA: Bilevel Efficient Learning with LoRA for Multilingual ASR

Multilingual automatic speech recognition (ASR) faces challenges due to diverse linguistic structures and resource disparities. We propose an efficient BilEvel Learning with LoRA for Multilingual Speech Recognition (BELLA), a framework that couples a pre-trained ASR encoder with a large language model (LLM) decoder via a trainable bridge to align speech representations with the LLM token space. The decoder incorporates mixture-of-experts LoRA (MoE-LoRA) modules for language specialization, guided by a router. Training is formulated as a bilevel program: the upper level optimizes router and expert LoRA modules for task performance, while the lower level updates the bridge and shared adapter for alignment. We develop an efficient single-loop, value-function-free penalty solver with practical updates. BELLA is parameter-efficient, scalable, and reduces cross-language interference. Experiments on five CoVoST 2 languages show consistent gains over strong multilingual ASR baselines.