OpenAI API Proxy in Python: Build One, Then Point It at Every Model

TL;DR

• An OpenAI API proxy is a small server between your app and the model API. It holds the real key and adds what the API doesn't give you: per-customer keys, rate limits, logging and key rotation. Below is a working FastAPI version in about 50 lines.
• HiAPI's text API is OpenAI-compatible: POST /v1/chat/completions and POST /v1/responses at https://api.hiapi.ai/v1. Point the official openai SDK, or your proxy, at that base URL and you can call DeepSeek, Claude, GPT, Kimi, GLM and Grok text models with one key.
• Images and video are different: they take seconds to minutes, so they run on HiAPI's async POST /v1/tasks. Don't push those through a synchronous proxy.

Step 1: a minimal pass-through proxy

The proxy checks an internal key from your client, swaps in the real upstream key, and streams the response back.

pip install fastapi uvicorn httpx

# proxy.py
import os
import httpx
from fastapi import FastAPI, Request, HTTPException
from fastapi.responses import StreamingResponse
from starlette.background import BackgroundTask

UPSTREAM = os.environ.get("UPSTREAM_BASE", "https://api.openai.com/v1")
UPSTREAM_KEY = os.environ["UPSTREAM_KEY"]
INTERNAL_KEYS = set(os.environ["INTERNAL_KEYS"].split(","))  # comma-separated

app = FastAPI()
client = httpx.AsyncClient(timeout=120.0)

def auth(request: Request) -> str:
    header = request.headers.get("authorization", "")
    key = header.removeprefix("Bearer ").strip()
    if key not in INTERNAL_KEYS:
        raise HTTPException(401, "invalid internal key")
    return key

@app.post("/v1/{path:path}")
async def forward(path: str, request: Request):
    tenant = auth(request)
    body = await request.body()
    upstream_req = client.build_request(
        "POST", f"{UPSTREAM}/{path}", content=body,
        headers={"Authorization": f"Bearer {UPSTREAM_KEY}",
                 "Content-Type": request.headers.get("content-type", "application/json")},
    )
    upstream = await client.send(upstream_req, stream=True)
    # Stream back so long answers and SSE don't time out; close the upstream stream when done
    return StreamingResponse(upstream.aiter_raw(), status_code=upstream.status_code,
                             media_type=upstream.headers.get("content-type"),
                             background=BackgroundTask(upstream.aclose))

Run it:

UPSTREAM_KEY=your-upstream-key INTERNAL_KEYS=tenant-a,tenant-b \
uvicorn proxy:app --host 0.0.0.0 --port 8000

Clients use the normal OpenAI SDK with your proxy as the base URL:

from openai import OpenAI

client = OpenAI(api_key="tenant-a", base_url="http://localhost:8000/v1")
resp = client.chat.completions.create(
    model="gpt-4o-mini",
    messages=[{"role": "user", "content": "Hello"}],
)
print(resp.choices[0].message.content)

Chat, streaming and embeddings all go through the same handler, because the proxy never looks inside the body.

Step 2: add what the upstream API doesn't give you

Per-tenant rate limits. Without them, one busy customer uses up your whole quota.

import time
from collections import defaultdict

_window: dict[str, list[float]] = defaultdict(list)
LIMIT = 60  # requests per 60 s

def check_rate_limit(tenant: str):
    now = time.time()
    _window[tenant] = [t for t in _window[tenant] if now - t < 60]
    if len(_window[tenant]) >= LIMIT:
        raise HTTPException(429, "rate limit exceeded")
    _window[tenant].append(now)

Call it right after auth(). With more than one server process, move the counter to Redis (INCR plus EXPIRE).

Logging. Log tenant, path, status, latency and token counts. Never log the Authorization header.

Key rotation. Load UPSTREAM_KEY from a secret manager and refresh it on a timer. Clients keep their tenant keys while the upstream key changes underneath.

Cost rules. Parse the body and reject requests outside policy, for example an expensive model on a free plan or a max_tokens above your limit.

Step 3: put several model vendors behind the same proxy

A proxy in front of one vendor is easy. Adding Anthropic, DeepSeek or Moonshot means different auth headers, request shapes and error formats, which turns the thin proxy into a translation layer.

HiAPI's text API takes the OpenAI request shape for all of its text models, so the proxy above works unchanged. Set two environment variables:

UPSTREAM_BASE=https://api.hiapi.ai/v1 UPSTREAM_KEY=your-hiapi-key INTERNAL_KEYS=tenant-a,tenant-b \
uvicorn proxy:app --host 0.0.0.0 --port 8000

Or skip the proxy and use the SDK directly:

import os
from openai import OpenAI

client = OpenAI(api_key=os.environ["HIAPI_KEY"], base_url="https://api.hiapi.ai/v1")
resp = client.chat.completions.create(
    model="deepseek-v4-flash",
    messages=[{"role": "user", "content": "Explain sparse attention in two sentences."}],
    max_tokens=400,
)
print(resp.choices[0].message.content)

We ran this with the official openai Python package (3.35.1), once normally and once with stream=True; both calls completed without errors and the SDK parsed the responses as usual. One catch from that test: with max_tokens=20, the reply was empty, because deepseek-v4-flash spent all 20 tokens on reasoning (usage.completion_tokens_details.reasoning_tokens was 22). Give reasoning models a few hundred tokens of room.

Some of the text models on HiAPI:

Model id	Route	Input / output, $ per 1M tokens
deepseek-v4-flash	Chat Completions	$0.14 / $0.28
deepseek-v4.1-flash	Chat Completions and Responses	$0.44 / $1.32
gpt-5.6-luna	Responses only	$0.05 / $0.30
grok-4.6	Chat Completions	$0.84 / $2.52
deepseek-v4-pro	Chat Completions and Responses	$1.50 / $4.50
claude-opus-5-5	Chat Completions	$2.02 / $10.08
kimi-k3	Chat Completions	$2.32 / $11.60
claude-sonnet-4-6	Chat Completions	$3.00 / $15.00

Live HiAPI prices, checked October 20, 2026. GET https://api.hiapi.ai/v1/models lists every model your key can use.

GPT models such as gpt-5.6-luna are on the Responses API only (client.responses.create), so a proxy that forwards /v1/{path} covers both.

An unknown model id returns HTTP 400 in OpenAI's error style, with a request_id to give support:

{"error": {"code": "model_not_found", "type": "hiapi_error",
           "message": "The requested model does not exist or is not available. Check the exact model ID with GET /v1/models and try again. ...",
           "request_id": "..."}}

Step 4: images and video don't belong in a synchronous proxy

A synchronous proxy holds the connection open until the answer arrives. That's fine for text. An image takes tens of seconds and a video takes minutes, which is longer than most load balancers and browsers will wait.

HiAPI runs image and video models on an async endpoint: submit, get a task id, then poll or receive a callback.

curl -s -X POST https://api.hiapi.ai/v1/tasks \
  -H "Authorization: Bearer $HIAPI_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "z-image",
    "input": {"prompt": "a glass data center at golden hour, isometric", "aspect_ratio": "16:9"},
    "callback": {"url": "https://your-app.example.com/hiapi/callback", "when": "final"}
  }'
# => {"code":200,"message":"success","data":{"taskId":"tk-hiapi-..."}}

Your proxy can still forward the submit call, which returns at once. Results then come back through your callback URL, or through GET /v1/tasks/{taskId}. Output links expire after about 7 days, so save the files.

Production checklist

• Strip secrets from logs. Never log Authorization, in either direction.
• Use callbacks for media. Polling thousands of tasks wastes requests. Treat the callback handler as idempotent and key it on taskId, since deliveries can be retried.
• Store task ids before polling. If the process dies, resume polling instead of submitting again and paying twice.
• Leave room for reasoning tokens. Set max_tokens well above the length of the answer you expect.
• Check what the key can use. GET /v1/models returns the list for that key.

FAQ

Can I use the OpenAI Python SDK with HiAPI?

Yes, for text models. Set base_url="https://api.hiapi.ai/v1" and use your HiAPI key. Images and video use /v1/tasks, which the OpenAI SDK doesn't cover; call it with requests or httpx.

Do I still need my own proxy?

Only for the same reasons as with any vendor: per-tenant keys, rate limits and audit logs. You no longer need per-vendor translation inside it.

Does streaming work?

Yes. stream=True returns server-sent events ending in data: [DONE], and the proxy above streams them through without buffering.

How do I rotate HiAPI keys without downtime?

Create a new key in the dashboard, deploy it to your proxy or secret store, then revoke the old one. Both keys work during the switch.

More detail: Chat Completions docs, Responses docs and HiAPI as an OpenAI API proxy alternative.