Route every agentthrough one gateway. 让所有 Agent，走同一个网关。
LLMIngress is an open-source, self-hosted AI gateway. Connect provider API keys, subscription accounts and local model servers, serve them to your agents under stable virtual model names, and manage routing, fallback, limits and usage from one Console. LLMIngress 是一个开源、自托管的 AI 网关。把供应商 API 密钥、订阅账号和本地模型服务接进来，以稳定的虚拟模型名提供给你的各个 Agent；路由、回退、限额和用量，都在一个 Console 里管理。
See the gateway at work.看看网关如何工作。
Scroll sideways to see the other Console pages.横向滚动可查看其他 Console 页面。
Why one
gateway.为什么需要
一个网关。
Every new agent adds another copy; every key rotation or model swap means hunting all of them down.每加一个 Agent 就多一份拷贝；每次换密钥、换模型，都得挨个文件找一遍。
Every agent points at the same address and the same name. Rotate keys or swap providers by editing once, in the Console.所有 Agent 指向同一个地址、同一个名字。换密钥、换供应商，只在 Console 里改一次。
How it solves this.它怎么解决
这些问题。
One endpoint一个入口
No more per-client wiring: four OpenAI- and Anthropic-compatible APIs on one port, and each key sees only the models it was granted.不再挨个客户端配置：四个 OpenAI / Anthropic 兼容端点在同一端口，每把密钥只见被授权的模型。
Virtual models虚拟模型
Model names stop being hard-coded: one stable name maps to an ordered list of real models — swap providers without touching any agent.模型名不再写死：一个稳定名字映射一组真实模型，换供应商不用动任何 Agent。
Fallbacks回退
A provider going down no longer stops the session: on 429, 5xx or timeout the gateway silently falls through to the next candidate.供应商挂了不再中断会话：遇 429 / 5xx / 超时静默切到下一个候选，首字节后绝不重放。
Usage and cost用量与成本
Spend stops being a blind spot: every request is logged with model, tokens, latency and cost, viewable per agent key.花费不再分不清：每个请求按密钥记账——模型、token、延迟、成本，一目了然。
Limits限额
A runaway agent hits a ceiling, not your bill: budget, RPM, TPM and concurrency caps enforced per key at the gateway.失控的 Agent 撞到的是上限而不是账单：每把密钥的预算、RPM、TPM、并发由网关强制执行。
Self-hosted自托管
Keys stop leaking into dotfiles: credentials live encrypted in one place, on your machine, and prompts never touch the logs.密钥不再散落在各个配置文件里：凭据集中加密、只存在你机器上，提示词不落日志。
Supported clients
and providers.支持的客户端
与供应商。
Deploy it,
then connect.先部署，
再接入。
One script brings up Console, Gateway, Worker and database with Docker Compose. Everything stays on your machine.一条脚本用 Docker Compose 起好 Console、Gateway、Worker 和数据库。所有东西都留在你机器上。
THEN, IN THE CONSOLE然后，在 CONSOLE 里
-
01
Add a provider: an API key, a subscription, or local Ollama.加供应商：API 密钥、订阅，或本机 Ollama。
-
02
Create a virtual model and order its candidates.建虚拟模型，排好候选顺序。
-
03
Issue one key per agent and point it at the gateway.给每个 Agent 发一把密钥，指向网关。
git clone https://github.com/IamNotShady/LLMIngress.git cd LLMIngress ./scripts/deploy.sh
CONSOLE :3000 · GATEWAY :4000 · LOCAL-ONLY BY DEFAULT默认仅本机可访问
export ANTHROPIC_BASE_URL="http://localhost:4000"
export ANTHROPIC_AUTH_TOKEN="llmi_cc_4f8a…"
claude # model: my-coding-model (virtual)
# ~/.codex/config.toml
model = "my-coding-model"
model_provider = "llmingress"
[model_providers.llmingress]
base_url = "http://localhost:4000/v1"
env_key = "LLMINGRESS_API_KEY"
curl http://localhost:4000/v1/chat/completions \
-H "Authorization: Bearer llmi_dev_a1c3…" \
-H "Content-Type: application/json" \
-d '{ "model": "my-coding-model",
"messages": [{ "role": "user", "content": "Hello" }] }'
ONE KEY WORKS ON ALL FOUR ENDPOINTS一把密钥在四个端点上都能用
Local Node本地 Node
Run directly with Node and your own PostgreSQL.用 Node 和自己的 PostgreSQL 直接跑，适合改代码。
Server · VPS服务器 · VPS
The same compose file runs on a server; keep gateway and database private.同一份 compose 上服务器；Gateway 与数据库保持内网。
Upgrades升级
git pull and run the script again. Data migrations happen automatically.git pull 后重跑一次脚本，数据迁移自动完成。
How a request
is routed.请求是怎么
被路由的。
A virtual model is a name you define, mapped to an ordered list of real models. Agents only know the name.虚拟模型：你定义的一个名字，映射到一组按序的真实模型。Agent 只认识这个名字。
my-coding-model ├─ 1 deepseek-v3 @ deepseek ├─ 2 qwen3-coder @ openrouter └─ 3 qwen3-coder @ ollama (local)
Below, one request with one fallback.下面是一次带回退的请求过程。
Cursor asks for my-coding-modelCursor 请求 my-coding-model
The agent’s config holds only this name and the gateway address.Agent 的配置里只有这个名字和网关地址。
DeepSeek answers 429DeepSeek 返回 429
Nothing has been sent to the client yet, so the gateway tries the next candidate.还没有数据发回客户端，网关按序尝试下一个候选。
OpenRouter serves qwen3-coderOpenRouter 用 qwen3-coder 完成
The agent sees one successful response; both attempts are recorded.Agent 只看到一次成功响应；两次尝试都被记录。
ILLUSTRATIVE TRACE · FIELDS MATCH THE CONSOLE’S ACTIVITY RECORD示意追踪 · 字段与 Console 活动记录一致
Questions.常见问题。
Is it open source?它是开源的吗？
Yes. Apache-2.0, self-hosted, built for one operator. There is no hosted plan, no account and no telemetry.是。Apache-2.0，自托管，按单人使用来设计。没有托管版本，不需要账号，不做遥测。
Which agents can connect?哪些 Agent 能接？
Anything that lets you set an OpenAI-compatible base URL: Codex, Cursor, OpenCode, Copilot. Claude Code connects through the Anthropic messages endpoint instead.任何能填 OpenAI 兼容地址的客户端：Codex、Cursor、OpenCode、Copilot。Claude Code 则走 Anthropic 的 messages 接口。
Are my prompts stored?提示词会被存下来吗？
No. Logs hold metadata only: model, tokens, cost, latency, status and retries. Prompts, responses, tool arguments and credentials are left out.不会。日志里只有元数据：模型、token、成本、延迟、状态和重试。提示词、响应、工具参数和凭据都不写。
Do my provider keys leave the machine?供应商密钥会离开我的机器吗？
No. Agents hold keys your gateway issued. Provider credentials stay in your database, encrypted with a key only you hold.不会。Agent 拿的是你的网关签发的密钥。供应商凭据留在你的数据库里，用只有你持有的密钥加密。
What does it cost to run?运行它要花多少钱？
The gateway is free. You keep paying the providers their own prices, and the usage page shows what each agent and model spent.网关本身免费。你照旧按供应商的价格付钱，用量页面会显示每个 Agent、每个模型花了多少。
Get started.开始使用。
Clone the repository and run the deploy script. The Console is at localhost:3000, and nothing is exposed beyond your machine by default.克隆仓库，运行部署脚本，Console 就在 localhost:3000。默认不对外暴露任何服务。