Audio Transition Recommendation System

A deep learning system that recommends songs creating high-quality audio transitions — not just similar songs.

Architecture

A_end audio → CLAP (frozen) → z1
B_start audio → CLAP (frozen) → z2

features = [z1, z2, z2-z1, |z2-z1|, bpm_diff, energy_diff, key_compat]
score = MLP(features) → transition quality

Project Structure

src/audio_transitions/
├── config.py              # Central configuration
├── data/
│   ├── segment_extractor.py   # Extract start/end segments from tracks
│   └── dataset.py             # Training pair dataset (balanced sampling)
├── features/
│   └── extractor.py           # CLAP embeddings + music features + Camelot wheel
├── model/
│   └── scorer.py              # TransitionScorerV1 (Phase 1 MLP)
├── training/
│   └── trainer.py             # Training loop (BCE + ranking loss)
├── retrieval/
│   └── retriever.py           # Two-stage retrieval (FAISS + model re-ranking)
├── evaluation/
│   └── metrics.py             # Hit@K, Contrast Recall@K, Mode Balance Ratio
└── api/
    └── server.py              # FastAPI endpoint

Quick Start

1. Install

pip install -e ".[dev]"

2. Prepare Data

from audio_transitions.data.segment_extractor import build_segment_catalog
from audio_transitions.features.extractor import build_embedding_catalog
from pathlib import Path

# Extract segments
catalog = build_segment_catalog(Path("data/raw/fma_small"))

# Compute embeddings + features
full_catalog = build_embedding_catalog(catalog)

3. Train

from audio_transitions.model import TransitionScorerV1
from audio_transitions.data.dataset import TransitionPairDataset
from audio_transitions.training.trainer import Trainer
from torch.utils.data import DataLoader

model = TransitionScorerV1()
dataset = TransitionPairDataset(full_catalog)
loader = DataLoader(dataset, batch_size=64, shuffle=True)

trainer = Trainer(model, loader)
trainer.train(num_epochs=50)

4. Serve

uvicorn audio_transitions.api.server:app --reload

5. Query

curl -X POST http://localhost:8000/transitions \
  -H "Content-Type: application/json" \
  -d '{"track_id": "track_001", "k": 10}'

Key Design Decisions

• Two transition modes: Continuity (smooth flow) + Controlled contrast (intentional shift)
• 50/50 retrieval: Similarity candidates + diversity candidates to prevent contrast suppression
• Contrast Recall@K: Dedicated metric to detect similarity collapse
• Phased complexity: Phase 1 = MLP, Phase 2 = Cross-Attention Transformer (when data justifies it)