Skip to content
SoundverseSoundverse 2.0 — Engineering Docs

core-tool-agent

core-tool-agent is the LLM agent packaged as a Soundverse tool worker. It runs a tool-calling loop over an LLM and drives other Soundverse tools by consuming core-mcp as a client — so it is both a worker on the queue and a consumer of the tool registry. It streams its reasoning and each sub-tool’s progress live, persists the whole transcript to chat, and reports token usage for billing.

The billing for an agent run — and for every sub-tool the agent calls — is settled by a second pipeline inside core-mcp, not by this worker. The worker only calls ctx.report_usage(total_tokens). See The agent + MCP path for the second billing pipeline and how nested tasks are queued.

The three tiers

Section titled “The three tiers”

The agent registers as three separate MCP tools sharing one implementation (AgentTool in app/tools/agent_tool.py). They differ only in tier (which model to drive) and model (the stable Soundverse id used for pricing / credential lookup):

MCP tool name model id tier default LLM string provider route reasoning
agent_flash agent-flash flash openai/Kimi-K2.6 OpenAI-compatible Azure AI Foundry off
agent_standard agent-standard standard anthropic/claude-sonnet-4-6 Anthropic direct off
agent_pro agent-pro pro anthropic/claude-opus-4-8 Anthropic direct on

Three tools rather than one tool with a model parameter because core-mcp resolves pricing, credentials, and rate-limits per (environment, model, operation) before the worker runs. Three tools let ops price/limit each tier independently and give the calling agent three selectable names. All three share operation="agent", cost_unit="tokens", and are priced free today (cost_base=0 under LICENSE_ROYALTY_FREE).

How a tier picks its provider

Section titled “How a tier picks its provider”

litellm routes by the model-string prefix. resolve_model_config in app/llm/provider.py maps each prefix to its endpoint + key:

  • anthropic/… → Anthropic’s native Messages API (AnthropicSettings; ANTHROPIC_API_KEY, base normally unset → api.anthropic.com). Native Messages API means real prompt caching and “thinking”, unlike an OpenAI-compatible shim.
  • openai/… → an OpenAI-compatible endpoint — the Foundry /openai/v1 route where Kimi-K2 is hosted (OpenAISettings; OPENAI_API_BASE / OPENAI_API_KEY, no api-version).
  • azure/… → classic Azure OpenAI (same creds plus an api-version).
  • anything else (litellm_proxy/…) → the Soundverse litellm proxy (LITELLM_BASE_URL / LITELLM_API_KEY) — the fallback path for a tier that hasn’t been repointed.

Repoint a single tier per environment via AGENT_FLASH_MODEL / AGENT_STANDARD_MODEL / AGENT_PRO_MODEL with no code change.

Per-tier reasoning (AGENT_FLASH_REASONING / _STANDARD_REASONING / _PRO_REASONING, defaults off/off/on) exists because the tiers now span providers: the reasoning path omits sampling params a reasoning model rejects — Anthropic Opus 4.8 returns 400 on temperature — and gives the run the larger reasoning_max_tokens budget. Kimi-K2 and Claude Sonnet 4.6 accept temperature, so their tiers leave it off.

What this worker hosts

Section titled “What this worker hosts”

The worker registers seven tools in app/tools/__init__.py:ALL_TOOLS — the three agent tiers plus four co-hosted leaf tools the agent orchestrates:

tool model id operation role
agent_flash / agent_standard / agent_pro agent-* agent the LLM orchestrator tiers
lyric_writer lyric-writer write_lyrics single-shot lyrics (reuses the standard tier → Claude)
song_namer song-namer name_song single-shot title candidates (reuses the standard tier)
file_manager file-manager manage_file rename a library file / relink a song’s album-art cover (free, no provider)
wait_for wait-for wait_for_task poll a long sub-generation to completion (free control-flow utility)

lyric_writer and song_namer are not agents — one LLM call, no turn-loop — but they reuse the agent’s standard tier through the shared provider boundary, so they stay in sync with the agent’s model config for free.

The turn-loop

Section titled “The turn-loop”

AgentRunner.run() in app/agent/loop.py drives a provider-neutral ChatModel in a tool-calling loop over core-mcp. process() is synchronous (the fleet contract), so it runs the async loop via asyncio.run.

sequenceDiagram
    autonumber
    participant MCP as core-mcp
    participant W as AgentRunner (this worker)
    participant LLM as LLM tier (Claude / Kimi)
    participant Sub as sub-tool (via core-mcp)
    MCP->>W: QueueTask → fleet claims it
    W->>MCP: list_tools (exclude self + internal)
    W->>W: select advertised (skip/conditional, name-sorted)
    W->>W: load history + resolve @-mentions
    loop each step (until no tool calls, or max_steps)
        W->>LLM: stream_turn(messages, tools)
        LLM-->>W: reasoning deltas + tool calls
        W->>Sub: call_tool(name, args, _message_id) [concurrent]
        Sub-->>W: result (or a still-running handle)
        W->>W: feed result back into messages
    end
    W->>MCP: report_usage(total tokens); final answer + step trace

On every sub-tool call three things happen by design:

  • live — a typed tool_call / tool_progress / tool_result envelope is emitted on the agent’s own task event stream (via TASK_EVENT_TYPE_PARTIAL_OUTPUT), and the model’s reasoning streams as a separate thinking envelope; the frontend switches on kind. No new event enum or consumer-mapper change was needed.
  • durable — a role=TOOL chat message is created and its id passed as the sub-call’s _message_id, so core-mcp projects that generation’s result onto it. The step trace persisted to the assistant message is deliberately URL-free (blob hashes, never signed SAS links) so a reload can re-mint stream URLs client-side without leaking a private URL.
  • traced — OTel spans agent.run → agent.turn → agent.tool_call wrap the run (see Observability).

The default persona (build_default_system_prompt) directs song creation: call lyric_writer first, then in a single turn issue song_namer + song_gen_v5 + album_art together so they run concurrently. A turn’s tool calls are dispatched with asyncio.gather over one MCP session (replies correlate by JSON-RPC id), then folded back in original order so the assistant→tool pairing and step trace stay deterministic. max_steps defaults to 12 (AGENT_MAX_STEPS).

Notable subsystems

Section titled “Notable subsystems”

The agent fetches every tool core-mcp advertises and converts them into the LLM request’s tools block on every reasoning step — a tool’s full description + JSON Schema rides there uncached. ToolPolicy in app/agent/tools.py is the seam that decides what to advertise, applied before to_openai() so a dropped tool’s (possibly huge) description never enters the request:

  • skip — a hard kill-switch (AGENT_SKIP_TOOLS, comma-separated MCP names) that drops a tool without unregistering it in core-mcp. The canonical candidate is stitch, whose description ships a ~55K-char reference catalog + schema that would bloat every turn.
  • conditional — name → predicate(prompt): advertise a tool only when its predicate matches (wired via keyword_predicate, currently unused). This is the groundwork for lazy advertising — move a heavy tool from skip to conditional so it costs nothing on unrelated runs but appears when the prompt needs it.

select_advertised name-sorts the surviving tools so the request’s tools prefix is deterministic run-to-run, which is what lets automatic prefix-based prompt caching hit. Prompt caching (AGENT_PROMPT_CACHE, default on) is regime-aware: the Anthropic route gets explicit cache_control breakpoints on the system persona + tools head; the OpenAI-compatible route caches automatically. Cache telemetry lands on span attributes (llm.cache_read_tokens / llm.cache_hit_ratio) — watch it in SigNoz; 0.0 means a cold miss.

Field guide

Section titled “Field guide”
local
cp .env.example .env        # set ANTHROPIC_API_KEY / OPENAI_API_KEY (or LITELLM_API_KEY)
make sync                    # uv sync
make run                     # python -m app.main

Needs Redis, core-database, core-storage, and core-mcp reachable (REDIS_ADDR, CORE_DATABASE_GRPC, CORE_STORAGE_GRPC, CORE_MCP_URL). Set CORE_DATABASE_USE_TLS=false / CORE_STORAGE_USE_TLS=false for plaintext gRPC locally (both default to TLS, since staging/prod front those services with TLS-only ACA ingress). With OTEL_EXPORTER_OTLP_ENDPOINT unset it logs JSON to stdout and ships no telemetry.

Gotchas

Section titled “Gotchas”

Deploy

Section titled “Deploy”

Both workflows are workers (enable_ingress: false) that call the shared service-template. Push to the branch to deploy: staging → deploy-aca-staging.yml, prod → deploy-aca-prod.yml. Runtime env comes from STAGING_* / PROD_* org secrets/variables (the deploy engine strips the prefix): the provider keys are set as STAGING_ANTHROPIC_API_KEY / STAGING_OPENAI_API_KEY (and the PROD_ equivalents). See Deploy a service.

Section titled “Related”