Storage & media plane

The media plane is how every generated asset — a song master, album art, a stem, a video — gets stored once, deduplicated, transcoded for seekable playback, and served back to the browser without ever leaking a durable blob URL. Two services split the job: core-storage (Python) owns the bytes — it hashes, dedups, transcodes, and mints SAS URLs — while core-database owns the rows (storage.blobs, storage.blob_variants, storage.files). core-storage never talks to Postgres directly; it calls the StorageDataService on core-database over the internal channel, so the data door stays singular.

Deploy honesty

The lossless-master + CMAF/HLS pipeline and the cross-song playlist-dedup fix are recent and land here as the current design. The dedup fix additionally needs a one-time storage.blob_variants cleanup for songs rendered before it shipped (see the caution below). Treat the streaming path as In code, recently rolled and verify against the live revision before assuming a given song is self-healed.

Content-addressed dedup

Every upload is hashed by its raw bytes, then the storage-facing identity folds in the container it lands in:

app/grpc_server/servicers/storage.py

raw_file_hash = sha256(bytes)
db_blob_hash  = sha256(f"{raw_file_hash}:{container_name}")

db_blob_hash is the dedup key: two uploads of the same bytes into the same container map to one blob row and one set of stored bytes (_process_and_save_blob logs a “Deduplication hit … reusing existing blob”). Folding the container in is deliberate — the same bytes uploaded to the public container and the private container must not collide, because visibility (and therefore who may mint a URL for it) differs. The container is chosen by visibility: azure_public_container for public assets, azure_private_container otherwise, plus a separate azure_live_container for ephemeral live previews.

• Directoryprivate/ (or public/)

  • Directorygeneral/

    • db_blob_hash.flac the lossless master blob
  • Directorystreaming/

    • Directorymaster_hash/

      • Directorystreaming_aac/

        • init.mp4 fMP4 initialization segment
        • seg-00001.m4s … immutable CMAF segments
        • playlist.m3u8 the VOD playlist (this is the only blob_variant row)
        • signed.m3u8 per-playback signed playlist (overwritten each serve)

Masters are lossless FLAC

The song-generation worker (core-tool-sansaarm) prefers the provider’s lossless flac_url for the stored master and only falls back to the lossy MP3 when no FLAC exists — it never re-encodes lossy → FLAC (bigger file, no quality gain). The key non-obvious mechanic lives in soundverse-py: LiveAudioIngest.finalize(final_mime_type=…) decouples the master mime from the live-preview mime, so an MP3 live preview stream can finalize into an audio/flac master. See _master_source() in core-tool-sansaarm/app/tools/music_gen.py.

Lossless masters exist so reference-guided re-generation and the studio’s decodeAudioData path get a clean source — the master is not what the player streams.

The FLAC → CMAF/HLS streaming pipeline

Browsers can’t seek a raw FLAC efficiently, so core-storage renders a seekable VOD rendition in CMAF/HLS (fragmented MP4) from the master, not from the original upload. FfmpegTranscoder.to_hls emits an fMP4 init segment, .m4s segments (hls_segment_seconds = 4s target, streaming_audio_bitrate = 256k), and a VOD playlist.

The render is backgrounded. It used to run synchronously inside the upload/finalize RPC, which blocked the worker’s terminal “done” event and made the whole ~40–80s transcode (for a ~5-minute master) user-visible latency. Now _maybe_generate_streaming_variant fires _spawn_streaming_variant (a tracked asyncio task bounded by a semaphore) and returns immediately. Playback resolves the rendition lazily and self-heals:

1. Upload commits the master and returns; the worker reports “done” without waiting on the transcode.

2. A background task renders the HLS rendition by re-fetching the just-committed master by hash, single-flighted per master via a lock + a GetBlobVariant dedup check (mirrors the lazy download_m4a path).

3. First plays fall back to the master. The chat player keys off the master blob_hash and asks for CreateDownloadLink(role="streaming_aac"). Until the rendition lands, that RPC aborts NotFound, which the BFF turns into a directly-playable master fallback — so playback works either way.

4. CreateDownloadLink self-heals. On NotFound it also re-kicks the background render (idempotent), so a pod restart that dropped an in-flight task still recovers on the next play.

Only the playlist is a blob_variant row (anchored via streaming_variant_role = streaming_aac). The init + segments upload straight to streaming/{master_hash}/{role}/ and are not blob rows, so they’re never deduped. There is no proto change — a streaming variant is still exactly one blob (the playlist).

The playlist that streamed the wrong song

An HLS playlist carries only relative segment filenames and per-segment #EXTINF durations — not the audio. Two different masters with the same segment layout render byte-identical playlists. The variant was originally content-addressed on the playlist bytes alone, so distinct songs deduped to one playlist blob; the second song’s playlist row pointed at the first song’s directory and served the wrong audio. The fix folds the master hash into the variant identity:

streaming_variant_db_hash = sha256(f"{raw_playlist_hash}:{container_name}:{master_db_hash}")

This keeps same-master re-renders idempotent while never colliding across songs. Ops follow-up manual step: the code fix only helps new renders. Already-rendered songs keep their wrong rows — clear them with DELETE FROM storage.blob_variants WHERE role='streaming_aac';. Next play then returns NotFound → master fallback + a fresh master-keyed re-render. Masters and segments are intact; only the playlist pointers were wrong.

Serving: short-lived SAS

core-storage authorizes reads by minting a short-lived Azure SAS URL per request, never by exposing a durable link. Default lifetime is download_link_default_ttl = 3600s (caller may override via expiry_seconds).

A single blob SAS can only sign one blob, but an HLS rendition is a playlist plus sibling segments. So CreateDownloadLink detects the playlist by mime (PLAYLIST_MIME) and routes to _sign_hls_playlist, which signs the whole rendition at serve time: read the stored playlist, sign the #EXT-X-MAP init and every .m4s segment under the playlist’s prefix, rewrite them to absolute signed URLs, write a per-playback signed.m3u8, and return its SAS. hls.js then fetches the segments directly. Roles resolve like so:

Role	What it is	How it’s served
master (or "")	The lossless FLAC master blob	Direct single-blob SAS; audio masters are paywalled
streaming_aac	The CMAF/HLS VOD rendition	Serve-time-signed playlist (_sign_hls_playlist)
download_m4a	Lazy single-file faststart M4A	Rendered from the master on first download, then cached

Same-origin stream proxy (frontend)

The browser’s global player routes <audio> through a Web-Audio graph (createMediaElementSource → analyser → destination) for the visualizer. A cross-origin Azure blob with no CORS grant taints that graph and the player outputs zeroes (silent, but position keeps advancing — the tell). To dodge that and to keep the durable SAS off the wire, audio is streamed through a same-origin proxy in soundverse-saas-2.0:

1. lib/stream-token.ts seals { sasUrl, exp } (AES-256-GCM) keyed by the STREAM_PROXY_SECRET env var — names only; it’s a server-only secret, never NEXT_PUBLIC.

2. app/api/library/stream-url/route.ts mints the SAS inline (no download_filename), seals it into a token, and returns /api/library/stream/<token> instead of the raw SAS.

3. app/api/library/stream/[token]/route.ts (Node runtime) opens the token, checks an Origin/Referer allowlist, fetches the SAS forwarding Range/If-Range, and streams back 200/206 inline and same-origin — so the Web-Audio graph gets real samples.

Images and video take the direct SAS path

Only audio uses the proxy. Images and videos are minted as a direct blob SAS (they have no lossy rendition), which is why the account-wide CORS rule (soundverse-saas-2.0/scripts/setup-storage-cors.sh) still matters — collab-studio and image/video fetch+decode hit the blob cross-origin. ALLOWED_ORIGINS in the proxy must stay in sync with the origins in that script; Azure CORS can’t do subdomain wildcards.

The master paywall is audio-only

The full-quality lossless audio master is a paywalled entitlement, gated in the gateway by MintDownloadUrl → _can_download_master (which fails closed on any resolver uncertainty, checking LICENSE_MASTER on the caller’s subscription entitlement). Crucially the gate is audio-only: the BFF mints images and videos with role="" (they have no lossy rendition), and an empty role is not a lossless-audio master. An earlier version paywalled role in ("", "master") unconditionally, so every album-art cover and video 403’d for free-tier callers — who then hammered stream-url on every re-render because the client couldn’t cache a failed mint. The fix resolves the blob’s mime and only paywalls when it startswith("audio/"). See core-gateway-consumer/app/grpc_server/servicers/library.py.

The pipeline, end to end

flowchart TD
    W["Song-gen worker<br/>music_gen.py (prefers lossless flac_url)"]:::worker
    ST["core-storage · StorageService<br/>hash · dedup · transcode · SAS"]:::data
    DB["core-database · StorageDataService<br/>storage.blobs / blob_variants"]:::data
    BLOB[("Azure Blob<br/>FLAC master")]:::external
    SEG[("Azure Blob<br/>init + .m4s segments + playlist")]:::external
    GW["core-gateway-consumer<br/>MintDownloadUrl · master paywall"]:::gateway
    PROXY["saas-2.0 BFF<br/>same-origin /api/library/stream proxy"]:::frontend
    PLAYER["Browser player<br/>hls.js + Web-Audio graph"]:::frontend

    W -->|upload master| ST
    ST -->|"db_blob_hash = sha256(raw:container)"| BLOB
    ST -->|insert blob / variant rows| DB
    ST -.->|"background asyncio task"| HLS["to_hls · CMAF/HLS fMP4"]:::data
    HLS -->|"streaming/{master}/{role}/"| SEG
    PLAYER -->|"stream-url(role=streaming_aac)"| GW
    GW -->|CreateDownloadLink| ST
    ST -->|"serve-time signed playlist (NotFound → master fallback)"| GW
    GW --> PROXY
    PROXY -->|"same-origin 206, hides SAS"| PLAYER

    classDef frontend fill:#6366f1,color:#fff,stroke:#4338ca
    classDef gateway  fill:#0ea5e9,color:#fff,stroke:#0369a1
    classDef data     fill:#8b5cf6,color:#fff,stroke:#6d28d9
    classDef worker   fill:#10b981,color:#fff,stroke:#047857
    classDef external fill:#f59e0b,color:#111,stroke:#b45309

Related

• core-storage — the service field-guide (RUN / FILES / ENV / TESTS)
• core-tool-sansaarm — where lossless masters come from
• Frontend architecture — the SPA + BFF that hosts the stream proxy
• Data model overview — storage.blobs / blob_variants / files
• core-billing — the entitlement behind the master paywall
• Deploy a service — the CORS-script + STREAM_PROXY_SECRET deploy obligations