Machine learning's open frontier.

Music and audio are still underexplored in machine learning, especially next to language and vision. There is far more research happening than most people expect, yet the field is nowhere near saturated. For a student, that gap is the opportunity: the fundamental questions are still open, and newcomers can reach the frontier fast.

Working on music means re-learning the fundamentals by adapting them to a new domain. Contrastive embeddings become audio-text models in the spirit of CLAP. Tokenization becomes neural audio codecs built on residual vector quantization. Autoencoders, diffusion models, and preference tuning all reappear, and each one has to be rethought for sound. At the same time, the state of the art stack is fully present: latent diffusion transformers, flow matching, RLHF, and beyond.

As a signal, music is closer to video than to text: continuous, high rate, and perceptual. One second of audio is 44,100 samples, and a three minute song is roughly eight million. Structure lives at every timescale simultaneously, from the milliseconds of timbre to the minutes of song form, and a model must stay coherent at all of them at once. Long-range structure is a recognized open problem in music generation, and solving it would advance sequence modeling far beyond audio. In practice, audio ML has more in common with video ML than with NLP: dense temporal perceptual signals, temporal coherence as the shared battle, and techniques that travel between the two fields constantly.

That is why efficiency is one of our main themes: subquadratic architectures such as state space models, Mamba-style networks, TRMs, and hybrids like diffusion Mamba. It is also why we design new architectures from scratch. One ongoing project explores wavelets and related multi-resolution function families: music is hierarchical by nature, and a transform that zooms may fit it the way convolutions fit images.

Music is a natural multimodal playground: audio, text, MIDI, video, images, and even brain responses, with recent models predicting how a brain reacts to sound. And generation is only part of the picture. Transcription, clustering, deduplication, neural watermarking, and retrieval are all live problems.

At Munich Music Labs, R&D is broader than machine learning alone. Teams work on classical DSP and build their own hardware, guitar pedals included. Chip design is on the table too, and we are already talking with major companies in that space. Our interests stretch from circuits and signal processing through neuroscience to state of the art deep learning. Projects run in small teams: some are started by members, others come from collaborations with companies and university chairs, and our work is supported by our partners. Outcomes range from research papers to artistic projects, and anyone can propose a direction.