Speech & audio papers
What AI researchers are reading: the papers the Hugging Face community picks each day, and new arXiv submissions in the directions we follow. Titles and abstracts are arXiv’s; every paper links to arXiv.
Earlybird: Predictive Streaming Speech Recognition in 29M Parameters
Sahil Mahendrakar
Streaming speech recognizers output a word only after the speaker has finished it, typically 90-400 ms later. For a voice agent every millisecond counts: the sooner the words are outputted, the sooner the agent can start a search or plan its reply. We present Earlybird, a 29M-parameter streaming recognizer (16.8 MB…
Exploring Post-Hoc Disentanglement of Audio Foundation Models for Music Similarity
Lucas de Souza Brandão, Gabriel da Costa Merlin, Diego Furtado Silva
Audio foundation models like CLAP, MuQ, and MERT excel at capturing general musical similarity. On the other hand, their entangled latent spaces limit interpretability and fine-grained attribute control. Conditional Similarity Networks (CSN) address this by pairing a backbone representation model with learned masks…
Dyship: Leveraging Acoustic Features For Speaker Relationship Classification From Dyadic Conversations
Kieran Sturges, Thomas Thebaud, Yaohan Guan et al.
We introduce DyShip, a framework for classifying speaker familiarity and kinship from dyadic conversations in the Seamless Interaction corpus. We compare five speech representations: spectrograms, MFCCs, self-supervised WavLM representations, x-vectors, and estimated articulatory movements. Spectral features…
WaveGen: Truly End-to-End Waveform Generation via Internal Spectral Trajectory Learning
Sang-Hoon Lee, Ha-Yeong Choi
This work proposes WaveGen, a fully end-to-end diffusion-based text-to-waveform framework based on Spectral Forcing, which jointly learns an internal spectral trajectory and the target waveform trajectory within a single model, enabling direct end-to-end text-to-waveform generation. Unlike recent TTS pipelines,…