Building a Sample Manager: the stack, the why, and where things stand

If you produce music, you’ll know the problem already: thousands of samples scattered across drives, half-remembered filenames, and no good way to find “that one weird vocal chop from three years ago” without scrolling through folders for twenty minutes. Worse, most of that library only exists on one drive, which is one dead SSD away from disappearing entirely. That’s the itch this project is scratching. A self-hostable sample library manager that brings everything into one place, backed up to the cloud, with organisation, playback, and eventually similarity-based recommendations on top, so you can actually find what you’re looking for instead of just hoping you remember where you put it, and stop crossing your fingers every time a drive makes a weird noise.

The stack

The frontend is SvelteKit, which is my preferred stack these days for exactly this kind of project, fast, minimal ceremony, and it gets out of the way when you just want to build. Data lives in PostgreSQL, managed through Drizzle ORM, which I moved to from an initial SQLite setup once it was clear this needed to scale past a single-user toy. Samples themselves are backed by S3-compatible storage, so the actual audio files live in the cloud rather than depending on any one machine’s local disk.

The more interesting decision was splitting out a separate FastAPI service in Python purely for audio analysis. SvelteKit and TypeScript handle everything else, but the audio analysis work, BPM detection, key detection, MFCC extraction, leans on librosa, which is a genuinely excellent Python library with no real equivalent in the JS ecosystem. Rather than fight that, I split it into its own microservice and let each language do what it’s actually good at. The two services share a Docker volume for file access, and the whole thing runs via Docker Compose so it’s one command to spin up locally.

It meant properly learning Python from a TypeScript background, which has been a nice change of pace, routing, tooling, and project structure (pyproject.toml, hatchling, venv) all work differently enough to keep it interesting without being a totally alien language.

Where things stand

Auth is handled by Better Auth, a framework-agnostic TypeScript library that takes care of session management, rate limiting, and the general plumbing you don’t want to hand-roll yourself. For a project like this it meant proper email/password auth without reinventing session handling or security basics from scratch, which is exactly the kind of solved problem I’d rather not solve again badly.

The FastAPI service is currently handling the audio analysis pipeline, BPM, key, and MFCC extraction via librosa, with uploads flowing through an async “pending” status pattern so the frontend isn’t blocked waiting on analysis to finish.

Waveform generation is working end to end: the Web Audio API decodes the audio client-side, peak data gets extracted and drawn to a canvas, and the resulting peak data is stored as JSON in the database so it doesn’t need recalculating every time you load a sample.

Next up is rounding out the UI and getting proper category tagging in place, being able to actually organise and browse a growing library matters more right now than clever recommendations. The pgvector-powered similarity search is still coming, using feature embeddings to surface “samples like this one”, but that’s a step for once the basics of finding and organising things are solid.