Networking & TLS

The router has two network sides, and they fail differently. Inbound is the listen socket your agents hit — the side that needs a bind decision and TLS. Outbound is the HTTP client that calls providers — the side where a badly chosen timeout kills a working agent mid-run.

Bind deliberately

server:
  listen: "127.0.0.1:4356"

Binding 0.0.0.0 while skip_auth: true is the one configuration that turns a local convenience into an open relay for your provider keys: any host that can reach the port can spend your credits, with no credential and no attribution. The starter config says so in a comment for a reason.

Only two combinations are safe:

A third option is often the best one: keep listen on loopback and let a reverse proxy on the same host own the public interface. The router is then unreachable except through something that terminates TLS and authenticates.

What the daemon serves

Both MCP routes are mounted only when mcp_servers is non-empty — on a config that declares none, they are absent rather than empty.

Request bodies are capped at 16 MiB. That is generous for text, and reachable if you push large base64 image or document parts through the proxy — a 413 on a multimodal request is this limit, not the provider's.

Reverse proxy with TLS

The router speaks plain HTTP. Terminate TLS in front of it.

The thing that breaks here is always the same: LLM responses are long-lived streams, and a proxy tuned for ordinary web requests will buffer them into uselessness or cut them off mid-generation. Two settings matter more than the rest — disable response buffering, and raise the read timeout well past your slowest expected completion.

server {
    listen 443 ssl http2;
    server_name router.internal.example.com;

    ssl_certificate     /etc/ssl/certs/router.pem;
    ssl_certificate_key /etc/ssl/private/router.key;

    location / {
        proxy_pass http://127.0.0.1:4356;
        proxy_http_version 1.1;

        # Streaming: without these, SSE arrives in one lump at the end
        # — or not at all.
        proxy_buffering off;
        proxy_cache off;
        proxy_set_header Connection "";

        # A reasoning model can think for minutes before the first token.
        proxy_read_timeout 600s;
        proxy_send_timeout 600s;

        client_max_body_size 16m;

        proxy_set_header Host              $host;
        proxy_set_header X-Real-IP         $remote_addr;
        proxy_set_header X-Forwarded-For   $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
    }
}

router.internal.example.com {
    reverse_proxy 127.0.0.1:4356 {
        flush_interval -1          # stream immediately, never buffer
        transport http {
            read_timeout 600s
            write_timeout 600s
        }
    }
    request_body {
        max_size 16MB
    }
}

client_max_body_size / max_size should match the router's own 16 MiB limit — set it lower and multimodal requests fail at the proxy with a different error than they would at the router.

Outbound timeouts

The other side of the connection is the client BitRouter uses to call providers. It's configured globally under upstream.timeouts, in seconds:

upstream:
  timeouts:
    connect_secs: 10        # TCP connect
    read_secs: 120          # idle guard — no bytes for this long, including mid-stream
    pool_idle_secs: 90      # evict idle pooled connections
    tcp_keepalive_secs: 60  # keepalive probe interval
    # total_secs: unset     # no overall wall-clock cap. See below.

Every field is optional and inherits from the level above; an unset global falls back to the built-in default.

read_secs is the one that matters for streaming. It's a per-read idle timeout, not a total — it fires when the upstream has sent nothing for that long, including mid-stream. It's the guard that kills a genuinely hung provider without killing a slow one, and it's what you tune when a reasoning model that pauses to think keeps getting disconnected.

Per-provider overrides live under the provider entry, which is how you give one slow upstream a longer leash without loosening everything:

providers:
  slow-local-vllm:
    timeouts:
      read_secs: 600

Timeouts are not inherited through a provider's derives: chain. A provider that derives from another does not pick up its timeout block — resolution goes to the global upstream.timeouts instead. If a derived provider needs a longer read timeout, state it on the provider itself.

Health checks

GET /health returns 200 with {"status":"ok"} — the right probe for a load balancer or a systemd watchdog. It reports that the HTTP server is up; it does not check upstream provider reachability, so a 200 here does not promise that a given model is routable.

For a check that reflects routing state, use the control socket instead:

bitrouter status -c /etc/bitrouter/bitrouter.yaml   # pid, listen address, routable model count

A model count of zero is a healthy process with no usable providers — an HTTP health check will never catch that.