Every developer on your team pasted a different OpenAI key into a different `.env`. One app hard-coded GPT-4o, another still points at a deprecated endpoint, and finance just asked which project burned $800 last Tuesday. You do not need twelve vendor dashboards — you need one front door that speaks OpenAI-shaped HTTP, routes to the models you actually pay for, and logs who spent what.
New API (QuantumNous/new-api) is a self-hosted AI gateway for exactly that. It sits between your apps, agents, and IDE plugins and upstream providers (OpenAI, Anthropic, Google Gemini, Azure OpenAI, Bedrock, Vertex, DeepSeek, Qwen, and other compatible APIs). Clients keep using familiar paths like /v1/chat/completions and /v1/messages; New API handles channels, retries, quotas, and a web console for keys and costs. With ~51k GitHub stars, it is one of the most forked-on projects in the self-hosted AI space — evolved from the earlier One API lineage, now maintained under QuantumNous with RelayKit protocol conversion and a modern React admin UI.
What it actually does
Think of New API as infrastructure, not a chat product. You are not replacing Open WebUI for humans typing prompts; you are giving Cursor, n8n, internal microservices, and custom agents a single base URL and API keys you control.
- Multi-protocol relay — OpenAI Chat/Responses, Anthropic Messages, Gemini generate/stream endpoints, embeddings, images, audio, rerank, plus WebSocket paths where upstreams support them.
- Channel routing — Map model names to upstream keys, set priorities and weights, fail over when a provider rate-limits you, pin affinity when you care about consistency.
- Org-style access — Users, groups, fine-grained permissions, OAuth/OIDC, passkeys, 2FA, session management — so “everyone uses my personal key” stops being policy.
- Usage and money — Quotas, subscriptions, per-request logs, cache accounting, expression-based pricing tiers — the boring ledger work SaaS gateways charge for.
- Extensions — JavaScript task plugins for async image/video-style jobs; plugin marketplace in the console for optional capabilities.
If you already run Langfuse for traces and evals, New API complements it: Langfuse watches behavior; New API is often the ingress that actually calls the models. Many teams run both on the same VPS or split gateway and observability across containers.
Why self-host New API?
One integration point. Switch from OpenAI to Azure or add a cheaper fallback model without redeploying every service — change channel config in the console, not twelve repos.
Keys stay off laptops. Issue per-team or per-app API keys with quotas instead of sharing a corporate master key in Slack.
Logs and residency. Request metadata and billing logs land in your Postgres (or SQLite for trials), optional separate log DB or ClickHouse for heavy volume. For Canadian companies, that is a defensible place to say where prompt metadata and usage records live — upstream providers still process inference per their terms, but your gateway layer is yours.
Cost visibility before the invoice surprise. Built-in dashboards and usage logs beat guessing from three vendor portals.
License reality check. New API is AGPLv3 (with extra Section 7 terms and UI attribution requirements). Internal gateway on your own infra is the common case. Reselling a public multi-tenant API without complying with AGPL — or operating a regulated public generative AI service without local filing rules — is where their README’s legal warnings matter. Read upstream ToS for every channel you enable.
What running it takes
Fastest poke: Docker with SQLite on localhost port 3000 — image calciumion/new-api:latest, mount a data volume, run the setup wizard for the admin account.
Production-shaped deploy: clone the repo’s docker-compose.yml for New API + PostgreSQL + Redis (MySQL and optional ClickHouse for logs are documented). Before docker compose up, rotate example passwords, set a persistent SESSION_SECRET (openssl rand -hex 32), and for HTTPS consoles set SESSION_COOKIE_SECURE and trusted origin variables as their auth docs describe.
Put nginx or Caddy in front for TLS and WebSocket upgrades — streaming chat breaks on misconfigured proxies. Back up the main database; pin release tags instead of blindly tracking latest on upgrades. Multi-node setups need shared DB, shared session/crypto secrets, and shared Redis if you want global rate limits.
Stack: Go/Gin backend, React 19 console. Resource use is modest compared to running models locally; the gateway is I/O-bound to upstream APIs, not GPU-heavy.
Who it's for (and who should skip it)
Good fit: dev shops standardizing LLM access, agencies routing client projects through keyed quotas, homelabbers juggling several cheap API providers, teams building agents that need OpenAI-compatible clients without binding to one vendor.
Maybe skip it: solo hobbyist with one OpenAI key and one script — overhead may not pay off. If you only need a browser chat UI, start with Open WebUI. If you need deep trace/eval workflows only, Langfuse alone does not replace a gateway — but you might not need New API until multiple apps hit the same models.
Hosting it in Canada
We deploy gateway stacks on Canadian Docker hosting when teams want TLS, backups, and firewall rules around the console while upstream calls still go to US or EU model regions — the split is normal: control plane in Canada, inference where the model lives.
Describe your apps and expected token volume if you want help sizing Postgres, Redis, and log retention without opening the admin UI to the whole internet.
