Migrating from TensorZero to BitRouter

TensorZero is an open-source LLMOps platform — a Rust gateway bundled with observability, evaluations, optimization (fine-tuning, automated prompt engineering), and experimentation, all backed by a ClickHouse data warehouse and driven by a GitOps tensorzero.toml. BitRouter solves a narrower problem on purpose: an agent-native proxy with a single OpenAI-compatible surface, runnable as a local binary (BYOK, no infra) or as a hosted endpoint.

This guide is for teams who adopted TensorZero mainly as a gateway — unified provider access, routing, retries, fallback — and want a leaner, agent-first surface without operating a database. If you lean on TensorZero's optimization and experimentation loop, read What BitRouter intentionally doesn't ship before you commit.

Why migrate?

Two things worth highlighting

1. A proxy, not a platform

TensorZero is built around a feedback loop: route inference, store every call and its metrics in ClickHouse, then evaluate, fine-tune, and A/B test against that history. That's powerful if you run that loop — and it's why TensorZero needs a database and a typed config of functions and variants.

BitRouter deliberately stops at the gateway. There's no ClickHouse to operate — OpenTelemetry gives you spans and metrics for every request (the same OTLP export TensorZero offers), but they go to a backend you already run, not a warehouse the proxy manages.

Configuration is the other difference, and it's one of degree rather than kind. BitRouter is also configured as code — bitrouter.yaml holds providers, routing policy, and gateways, with a published JSON Schema and a validate command for CI. What it doesn't ask you to declare is the shape of your application: there are no functions to define and no variants to enumerate before you can route a request. You can run on provider keys alone and add config only where you want policy under review.

2. Agent-native, and cloud/local share one surface

TensorZero's gateway is provider-facing — it unifies the upstreams. BitRouter adds the agent-facing half: an MCP gateway for tools, an ACP gateway for agent identity and dispatch, and a server-tool loop that runs those tools inside the request.

And the hosted cloud and local binary expose the same OpenAI-compatible endpoint — start local during development, point at api.bitrouter.ai for production (or vice versa) without changing client code. See the Quick Start for both flows.

Migration paths

From an OpenAI-compatible client

If your app already talks to TensorZero's OpenAI-compatible endpoint, migration is a base-URL swap plus dropping the tensorzero::… model prefix for a plain provider/model id:

import openai

client = openai.OpenAI(
    base_url="http://localhost:3000/openai/v1",
    api_key="not-used",
)

response = client.chat.completions.create(
    # TensorZero model / function reference
    model="tensorzero::model_name::openai::gpt-4o-mini",
    messages=[{"role": "user", "content": "Hello"}],
)

import openai

# Local: `bitrouter` (BYOK via env vars) — see /docs/overview/quickstart
# Cloud: base_url="https://api.bitrouter.ai/v1", api_key=$BITROUTER_API_KEY
client = openai.OpenAI(
    base_url="http://127.0.0.1:4356/v1",
    api_key="not-used-in-local-byok",
)

response = client.chat.completions.create(
    model="openai/gpt-4o-mini",
    messages=[{"role": "user", "content": "Hello"}],
)

A TensorZero function (a named prompt template + schema with one or more variants) has no single equivalent — the routing half maps to BitRouter, the templating/variant half stays in your app or moves to a preset. See the feature mapping below.

From the gateway + ClickHouse stack

The infra migration replaces the Compose stack — gateway container plus ClickHouse — with one binary and no database:

# docker-compose.yml runs the gateway + a ClickHouse warehouse
export OPENAI_API_KEY=sk-...
export TENSORZERO_CLICKHOUSE_URL=http://chuser:chpass@clickhouse:8123/tensorzero
docker compose up   # gateway on :3000, ClickHouse on :8123
# config mounted from ./config/tensorzero.toml

# Install (curl, npm, brew, or cargo — see Quick Start)
curl -fsSL https://bitrouter.ai/install.sh | sh

# Set provider keys; BitRouter auto-detects on start
export OPENAI_API_KEY=sk-...
export ANTHROPIC_API_KEY=sk-ant-...

# Interactive wizard (Cloud or local); default local serves on :4356
bitrouter

To skip the local proxy entirely, point clients at https://api.bitrouter.ai/v1 with a BitRouter API key — no binary, no ClickHouse, no infra.

Feature mapping

What BitRouter intentionally doesn't ship

To set expectations honestly: BitRouter is a gateway, not an LLMOps platform. It does not ship TensorZero's optimization and experimentation loop — no built-in supervised fine-tuning, RLHF, automated prompt engineering, dynamic in-context learning, inference-level or workflow evaluations, or adaptive A/B testing, and no ClickHouse-backed UI for replaying historical inferences against alternative prompts.

If your workflow depends on that closed feedback loop — collect inferences and feedback, evaluate, optimize, experiment — TensorZero is the better fit and you should keep it for those workloads. BitRouter is the right move when the value you need from TensorZero is the gateway and the missing half is agent infrastructure: tool/agent gateways, an agent firewall, and pay-per-request autonomy.

Migration checklist

Next steps

Get help

• Discord: Join the community for migration support
• GitHub: Open an issue
• Email: contact@bitrouter.ai for enterprise migration assistance