SpectralBridge: How We Taught an Audio Model to Hear Higher

SpectralBridge, a method from Nebius R&D, lets an audio-language model hear the 8–16 kHz band by training just a small adapter on top of a frozen model. Many audio AI models resample sound to 16 kHz mono, so everything above 8 kHz is lost before the model hears it. That works well for speech. But cymbals, insect sounds, hiss, and the “air” in a musical mix all contain important information above 8 kHz. SpectralBridge uses heterodyning to shift the 8–16 kHz band down into the 0–8 kHz range and adds it as an additional input. Both bands pass through the same frozen audio encoder, while a small adapter connects them at each time step. Only the adapter and rank-8 LoRA modules added to the frozen model are trained. Accuracy rises from 26.2 to 55.4 after fine-tuning on a 10-class insect detection task. SpectralBridge extends the model’s hearing by one octave, from 8 to 16 kHz. Sound above 16 kHz remains out of reach, and the method has so far been tested only on the AF-Next audio model. Read more on the Nebius Science blog: https://nebius.science/stories/nebius-audio-model #AudioAI #MachineLearning #Nebius