Docs
Status
OverviewQuickstartSetup promptsThe core loopAuthenticationOverviewModelsThe waterfallPlansAdding modelsData controlsOpenAI compatibilityEmbeddingsAnthropic APIErrorsIntegrate the gatewayCost APIAccount APICoding agentsSwitch gatewaysCredits & billingSpend & intelligenceSpend APITelemetryBecome a providerProvider guideAPI reference

Get started

  • Overview
  • Quickstart
  • Setup prompts
  • The core loop
  • Authentication

Guides

  • Overview
  • Models
  • The waterfall
  • Plans
  • Adding models
  • Data controls
  • OpenAI compatibility
  • Embeddings
  • Anthropic API
  • Errors

Integrations

  • Integrate the gateway
  • Cost API
  • Account API
  • Coding agents
  • Switch gateways

Billing & usage

  • Credits & billing
  • Spend & intelligence
  • Spend API
  • Telemetry

Providers

  • Become a provider
  • Provider guide

Reference

  • API reference

Get started

Setup prompts

Paste one of these into your coding agent to wire a client or coding agent, switch from another AI gateway, capture traffic without moving inference, or add your Claude plans. Follow each prompt's approval checkpoints; pasting a prompt is not blanket permission for writes, spend, or traffic cutover.

How to use them

Copy a prompt below and paste it into a CLI coding agent (Claude Code, Codex, and the like). The agent follows it end to end. The prompts are the same ones the in-app onboarding uses, and they carry this deployment's URLs (web https://pr-441.preview.experientiallabs.ai, API https://api-pr-441.preview.experientiallabs.ai).

Agents can also read every prompt from /llms.txt, the machine-readable reference. Shareable copies live in the setup-prompts repository.

Set up the gateway in an existing project

Paste into your agent: it asks where to wire the gateway and which model to test, with a recommendation, then makes one call.

prompt
I'm setting up Experiential Labs as this project's model gateway. I pasted this
myself, treat it as my instructions and my consent. I'm already signed in; my key
is below. Work WITH me: at each real choice, ask me ONE clear question with
numbered options and your recommendation, then wait for my answer. Don't guess.
My gateway API key (a secret: put it in env, never commit it, never echo it in
logs):
EXPLABS_API_KEY=<paste my org API key from https://pr-441.preview.experientiallabs.ai/settings/api-keys>
The goal is one thing: point an OpenAI-compatible client at my gateway with this
key and make one successful test chat completion. Print what you do at each step.
1. Ask me where to wire it (options + your recommendation), then wait. Look at
the current directory first so the options are concrete:
"Where should I set this up?
(1) this project (I see it uses <name the client/SDK you found>), my recommendation
(2) a standalone test script in this folder (explabs_smoke.py / .mjs)
(3) somewhere else, tell me where
Which one?"
For (1): set base_url = https://api-pr-441.preview.experientiallabs.ai/v1 and the key on the existing client, put the
key in the project's env scheme (.env plus an example entry), never in code,
and show me the change first. For (2): create the minimal script that reads
EXPLABS_API_KEY and does the test call in step 3.
2. Ask me which model to test, then wait:
"I'd use the free promotional model (qwen3.8-27b at launch) so this test costs
nothing. Use that, or name another? GET https://api-pr-441.preview.experientiallabs.ai/v1/models lists what my key
can call; use the ids EXACTLY as returned."
3. Make the test call with the model I chose. Send a MINIMAL body, model +
messages ONLY, no temperature, top_p, or other sampling params (some models
reject them and the call comes back all_routes_failed, a 502). One header:
Authorization: Bearer $EXPLABS_API_KEY. This runs on the platform-funded
lane, a fraction of a cent of my free credits, and proves serving and billing
end to end. Do NOT use any of my own provider keys for this test.
4. Report back what you changed and the test call (the model, that it succeeded,
and its cost). Then ask me which optional follow-ups I want, with your
recommendation, and do only the ones I pick:
(a) Connect my own provider keys (BYOK, Pro) so those calls bill my own accounts
(b) Import my Claude Code / Codex usage history (metadata only)
(c) Review my credits and spend
BYOK and import are quick for me to do at https://pr-441.preview.experientiallabs.ai/settings; balances live at
https://pr-441.preview.experientiallabs.ai/credits.
Read the contract if you build further: https://pr-441.preview.experientiallabs.ai/docs (human docs) and
https://pr-441.preview.experientiallabs.ai/llms.txt (machine-readable: honored and refused parameters, error codes,
streaming caveats). Follow it literally.

Wire your coding agent to the gateway

Paste into Claude Code, Codex, OpenCode, Hermes Agent, Pi, Cline, VS Code Copilot, Conductor, Cursor, or any OpenAI-SDK tool: the agent identifies itself and applies its own verified integration.

prompt
I pasted this into you myself, wire THIS coding agent up to my Experiential
Labs gateway, so my model calls route through https://api-pr-441.preview.experientiallabs.ai and show up in my usage.
Ground rules: never print my full key (first 8 characters at most), ask me
before you edit any config file or shell profile, and if you cannot do a step
(no file access, settings live in a GUI), print the exact manual steps for me
instead. Print what you're doing at each step.
1. Get my key. Use EXPLABS_API_KEY from my environment if it is set; otherwise
ask me to paste one (I can mint it at https://pr-441.preview.experientiallabs.ai/settings/api-keys). It looks
like xpl_ followed by 40 hex characters.
2. Prove the key works before touching any config:
GET https://api-pr-441.preview.experientiallabs.ai/v1/models with header "Authorization: Bearer <key>" -> 200 and
the model slugs I can call. Remember the list; I'll pick models from it.
3. Identify which agent you are and apply YOUR integration:
- Claude Code:
You cannot repoint yourself mid-session, and switching auth under a running
session causes mixed-state errors. First prove the lane without touching any
config, via one isolated subprocess run from this project (so my MCP servers
and plugins load and their tool schemas ride the request. Tool schemas are
where provider strictness bites, so a bare hello proves too little):
ANTHROPIC_BASE_URL="https://api-pr-441.preview.experientiallabs.ai" ANTHROPIC_API_KEY="<my key>" \
ANTHROPIC_MODEL="<slug I pick>" \
claude -p "list your available tools, then read ./README.md and reply
with its first heading"
Pass only if that run completes with tools listed and a real tool use.
Then offer me three placements and apply the one I pick:
(a) append to my shell profile, or (b) print for one-off use:
export ANTHROPIC_BASE_URL="https://api-pr-441.preview.experientiallabs.ai" # no /v1 suffix
export ANTHROPIC_API_KEY="<my key>"
export ANTHROPIC_MODEL="<slug I pick>"
or (c) a named profile beside my normal Claude Code (ask me for the name):
its own config dir ~/.claude-<name> with these three in settings.json "env"
plus "model": "<slug I pick>", and a shell alias
alias claude-<name>='CLAUDE_CONFIG_DIR=~/.claude-<name> claude'
A fresh config dir starts with the first-run onboarding, so skip it: copy
.claude.json from an existing profile dir that uses a gateway key (theme,
trusted projects), else write {"hasCompletedOnboarding": true}; then set
customApiKeyResponses to {"approved": ["<last 20 chars of my key>"],
"rejected": []} so the custom-API-key prompt never appears. Never copy
.credentials.json or a claude.ai sign-in into it.
Use ANTHROPIC_API_KEY, never ANTHROPIC_AUTH_TOKEN: an existing claude.ai
sign-in can outrank AUTH_TOKEN and send the gateway an OAuth token instead.
Then tell me to start a NEW session from a shell with those set (for (c),
a new terminal or after re-sourcing my shell profile). Name the
model by its gateway slug exactly as GET https://api-pr-441.preview.experientiallabs.ai/v1/models lists it (the DOT
form, e.g. claude-fable-5.1; Anthropic's dashed ids like claude-fable-5-1
answer 403 model_not_granted). If the slug is not an Anthropic model, also
export CLAUDE_CODE_MAX_CONTEXT_TOKENS=<max_input_tokens from
GET https://api-pr-441.preview.experientiallabs.ai/api/models/<slug>, rounded down to the nearest 50k>. Claude Code
assumes 200k for names it does not know and compacts early. NEVER use
context_window for that pin: on most models (every maker but Anthropic and
Gemini) it is a total window the provider counts the model's maximum output
inside, so a session pinned to it never compacts before the provider refuses
it and /compact then fails the same way. max_input_tokens already accounts for this per maker, so it is
always the right pin. Warn me plainly: if a session ever shows
"Not logged in · Please run /login", do NOT run /login (it starts Anthropic
OAuth that overrides the gateway). Restart with --resume, and if it
persists, my <config dir>/.claude.json remembers a "No" to the custom-API-key
prompt under customApiKeyResponses.rejected: answer Yes or move the entry to
approved. Images work on image-capable routes and are rejected by name on
text-only ones; extended thinking passes through on all-Anthropic routes
and is translated to reasoning effort elsewhere.
- Conductor:
Conductor launches Claude Code with the environment it captured from my
login shell (interactive login shell, cached per workspace) merged with its
own settings, and passes the per-chat picker choice as --model, so the
model is routed by remapping picker aliases with ANTHROPIC_DEFAULT_*_MODEL
variables (ANTHROPIC_MODEL does not apply to picker selections). Offer me
two placements and apply the one I pick:
(a) per-repo, recommended: merge into this repo's .claude/settings.json
"env" block (committed; Claude Code applies it in every workspace, and it
overrides anything Conductor captured from my shell), one
ANTHROPIC_DEFAULT_<ALIAS>_MODEL entry per picker alias I use:
{ "env": { "ANTHROPIC_BASE_URL": "https://api-pr-441.preview.experientiallabs.ai",
"ANTHROPIC_DEFAULT_OPUS_MODEL": "<slug I pick>" } }
then put ONLY the key machine-local in .conductor/settings.local.toml
under [environment_variables] (make sure that file is
git-ignored before writing my key into it):
ANTHROPIC_API_KEY = "<my key>"
(b) app-wide: print the same variables for me to add by hand in
Conductor -> Settings -> Environment, and remind me to check that
Settings -> Harnesses -> Claude Code uses my API key, not a claude.ai
CLI login.
Warn me either way: settings reach newly created workspaces, not running
ones, and if my shell profile exports a direct Anthropic
ANTHROPIC_API_KEY, Conductor's capture hands it to every agent;
placement (a) overrides it; otherwise I should remove the export.
The key rides ANTHROPIC_API_KEY, never ANTHROPIC_AUTH_TOKEN: an existing
claude.ai sign-in can outrank AUTH_TOKEN, and a set API key also stops
Claude Code from trying to authenticate with Anthropic directly.
- OpenAI Codex CLI:
First prove the lane in a throwaway home so my real ~/.codex (config,
sessions, auth) stays untouched, and test with my REAL tool surface: you
send every mcp_servers/plugin tool schema on each request, and tool schemas
are where provider strictness bites, so a bare hello proves too little.
Copy ~/.codex into a fresh "$(mktemp -d)", append ONLY the
[model_providers.explabs] table below to that copy's config.toml, then run
CODEX_HOME=<that dir> codex exec -m "<slug I pick>" \
-c model_provider=explabs -c model_reasoning_effort=max \
"list your available tools, then read ./README.md and reply with its
first heading"
Pass only if it completes with my MCP/plugin tools listed and a real tool
use. Then add the whole block to ~/.codex/config.toml (create it if
missing, show me the diff before writing) and tell me to start a new Codex
session:
model = "<slug I pick>"
model_provider = "explabs"
model_reasoning_effort = "max" # reasoning models; "max" is the top tier
[model_providers.explabs]
name = "Experiential Labs"
base_url = "https://api-pr-441.preview.experientiallabs.ai/v1"
env_key = "EXPLABS_API_KEY"
wire_api = "responses"
Leave requires_openai_auth unset, and make sure EXPLABS_API_KEY is exported
where I launch you (export KEY=..., a plain KEY=... line is invisible to
child processes like me).
- OpenCode:
Write the provider block into this project's opencode.json (or
~/.config/opencode/opencode.json if I prefer global, ask):
{"provider": {"explabs": {"npm": "@ai-sdk/openai-compatible",
"name": "Experiential Labs",
"options": {"baseURL": "https://api-pr-441.preview.experientiallabs.ai/v1", "apiKey": "{env:EXPLABS_API_KEY}"},
"models": {"<slug>": {"name": "<slug>"}}}}
Set each model's "name" to its slug exactly. OpenCode shows that name in its
status bar, so never put prices, discounts or promotion notes in it: they go
stale, and promotions are applied at billing, not by this config.
Fill limit.context/limit.output and cost ({"input": $/1M, "output": $/1M},
i.e. *_nano_usd_per_million / 1000000000 of the providers entry whose id is
the position-0 default_waterfall model_provider_id) for each slug from
GET https://api-pr-441.preview.experientiallabs.ai/api/models/<slug> so my context window and spend display are right.
- Hermes Agent (Nous Research):
First prove the lane in a throwaway data dir so my real ~/.hermes (config,
sessions, memories) stays untouched: create "$(mktemp -d)", write ONLY the
config below plus a .env line EXPLABS_API_KEY=<my key> (chmod 600) into it,
then run
HERMES_HOME=<that dir> hermes -z "list your available tools, then read
./README.md and reply with its first heading"
Pass only if that run completes with tools listed and a real tool
round-trip. Then merge the same block into ~/.hermes/config.yaml (show me
the diff before writing) and put the key line in ~/.hermes/.env:
providers:
explabs:
base_url: "https://api-pr-441.preview.experientiallabs.ai/v1"
key_env: "EXPLABS_API_KEY"
api_mode: chat_completions
model:
provider: "custom:explabs"
default: "<slug I pick>"
Any catalog slug works as the default, and /model custom:explabs:<slug>
switches mid-session. Warn me plainly: Hermes' session-title side call
sends a temperature value, so on models that pin their sampling the title
falls back to a default; the session itself is unaffected.
- Pi:
First prove the lane in a throwaway home so my real ~/.pi (auth, sessions,
model store) stays untouched: pi resolves its agent dir from HOME, so create
"$(mktemp -d)" and copy my real ~/.pi into it (extensions, skills and
settings ride the test; tool schemas are where provider strictness bites,
so a defaults-only run proves too little; if ~/.pi does not exist yet, just
create <that dir>/.pi/agent). Then merge the block below into that copy's
.pi/agent/models.json and run
HOME=<that dir> EXPLABS_API_KEY=<my key> pi -p --provider explabs \
--model "<slug I pick>" "list your available tools, then read
./README.md and reply with its first heading"
Pass only if that run completes with tools listed and a real tool
round-trip, then DELETE the throwaway home ("rm -rf <that dir>"). The copy
carries my auth and session state and must not outlive the proof. Then
merge the same provider into ~/.pi/agent/models.json
(create it as {"providers": {}} if missing; show me the diff before
writing):
{ "providers": { "explabs": {
"baseUrl": "https://api-pr-441.preview.experientiallabs.ai/v1",
"api": "openai-completions",
"apiKey": "$EXPLABS_API_KEY",
"models": [ { "id": "<slug I pick>", "name": "<slug I pick>",
"reasoning": true, "input": ["text", "image"],
"contextWindow": <from GET https://api-pr-441.preview.experientiallabs.ai/api/models/<slug>>,
"maxTokens": <from the same> } ] } } }
The apiKey stays the literal string "$EXPLABS_API_KEY"; pi interpolates it
from my environment, so make sure the variable is exported where I launch
pi and never write the raw key into the file. Set reasoning/input to the
slug's actual catalog capabilities. Any catalog slug works; I pick models
with --model at launch or /model in-session (models.json reloads each time
/model opens, no restart).
- Cline:
Your settings live in the VS Code UI, so print these for me to set by hand:
API Provider "OpenAI Compatible"; Base URL https://api-pr-441.preview.experientiallabs.ai/v1; API Key = my key (no
Bearer prefix); Model ID = a slug from step 2; and per-model context window /
max output tokens from https://api-pr-441.preview.experientiallabs.ai/api/models/<slug>.
- VS Code (Copilot Chat):
Copilot's bring-your-own-model support is VS Code core (1.122+); chat and
agent mode need no Copilot plan. Find my VS Code user dir (macOS
~/Library/Application Support/Code/User, Linux ~/.config/Code/User, Windows
%APPDATA%\Code\User) and merge this entry into chatLanguageModels.json
there (create the file as a JSON array if missing), one models[] item per
slug I pick:
{ "name": "Experiential Labs", "vendor": "customendpoint",
"apiType": "chat-completions",
"models": [ { "id": "<slug I pick>", "name": "<slug I pick>",
"url": "https://api-pr-441.preview.experientiallabs.ai/v1/chat/completions",
"toolCalling": true, "vision": true,
"modelOptions": { "temperature": null, "top_p": null } } ] }
Set toolCalling and vision to each slug's ACTUAL capabilities from
GET https://api-pr-441.preview.experientiallabs.ai/api/models/<slug> (tool support and image input). Declaring a
capability the model lacks makes Copilot send tool calls or images the
route rejects.
The null modelOptions are required, not decoration: null deletes the field,
and Copilot otherwise sends temperature 0.1 and top_p 1 on every request,
which models that pin their sampling reject with a 400 naming the
parameter. You cannot enter my key (it lives in VS Code's encrypted secret
storage), so print the manual step: in the Chat model picker open Manage
Models, right-click the Experiential Labs model, choose to enter an API
key, and paste my key (<my key>). VS Code stores it encrypted and writes
the ${input:chat.lm.secret.*} references into the entry itself; never
hand-write a raw key into the file.
- Cursor:
Your gateway settings live in the Cursor Settings UI, so print these for me
to set by hand. First tell me the prerequisite: a paid Cursor plan (the Free
plan serves only Auto and refuses named models, which custom models are).
Then: in Cursor Settings -> Models -> API Keys, put my key in
"OpenAI API Key" (no Bearer prefix), enable "Override OpenAI Base URL" and
set it to https://api-pr-441.preview.experientiallabs.ai/v1. Add each slug I pick as a custom model
name in the models list; custom models ride the override in OpenAI format,
so any catalog slug works. Warn me plainly: Cursor relays requests through
its own servers, Tab autocomplete stays on Cursor's models, and if a chat
answers 400 invalid_parameter naming temperature or top_p, that model pins
its sampling (the Claude 5 family pins temperature to 1). Pick a
different slug.
- Any other OpenAI-compatible tool:
Export OPENAI_BASE_URL="https://api-pr-441.preview.experientiallabs.ai/v1" and OPENAI_API_KEY="<my key>" wherever I
launch you, and name models by slug. If your own config wants the values
instead, it needs the same three: base URL https://api-pr-441.preview.experientiallabs.ai/v1, my key, and a slug.
4. Verify end to end. Make one tiny completion with the surface you configured
("reply with the single word: ok", small max output), via your own next
model call if you now route through the gateway, otherwise via curl with my
key. Then tell me it landed and that I can watch every call at
https://pr-441.preview.experientiallabs.ai/telemetry.
5. Recap exactly what you changed (files and values, key shown as xpl_ prefix
only) so I can undo it later.

Set up identities with a provisioning key

Org admins: paste into your agent with a provisioning key and it creates your identities (coding, product, ...) with one inference key each, so spend buckets per identity.

prompt
Set up per-identity API keys for my Experiential Labs organization so
spend is bucketed per identity (for example "coding" vs "product") in usage.
Provisioning key: xpl_<paste-your-provisioning-key-here>
(I replace the placeholder above before sending. It is a PROVISIONING key:
it can manage this org's keys, identities, and budgets. Never print it back
to me, never write it into any file, and do not use it for inference. The
API does not stop you from doing those; these are my rules.)
Base URLs: API https://api-pr-441.preview.experientiallabs.ai (management under /api, inference under /v1),
dashboard https://pr-441.preview.experientiallabs.ai. Every call below sends
Authorization: Bearer <the provisioning key>
plus Content-Type: application/json on writes. On a non-2xx read the JSON
"error" field; a body with "detail" instead is a request-shape failure and
detail[0].loc[-1] names the offending field. Then apply the Fallbacks.
What this key can call: GET /api/whoami; GET, POST /api/orgs/{org_id}/identities; PATCH /api/orgs/{org_id}/identities/{identity_id} (display_name and description only); GET, POST /api/v1/keys; GET, PATCH, DELETE /api/v1/keys/{hash}; GET, PUT /api/orgs/{org_id}/budgets; DELETE /api/orgs/{org_id}/budgets/{budget_id} (identity- and key-scope budgets this key created); GET /api/orgs/{org_id}/usage/by-identity.
Disabling an identity, model grants, org-wide budgets, the default identity's
budget, and any budget an admin set are the admin's, on https://pr-441.preview.experientiallabs.ai/api-keys; if a
step needs one of those, ask me instead.
1. Discover the organization (reads only).
GET https://api-pr-441.preview.experientiallabs.ai/api/whoami -> {org_id, org_slug, org_name}. Keep org_id; every
/api/orgs/{org_id}/... call below uses it. Then
GET https://api-pr-441.preview.experientiallabs.ai/api/orgs/{org_id}/identities -> {identities: [{identity_id,
display_name, active, is_default, ...}]}. The is_default row is the org's
own identity: leave it alone, mint no keys under it, cap nothing on it.
2. Intake, BEFORE any change. Do not send a single POST, PUT, PATCH, or
DELETE until I have answered these and said yes to the plan. Ask me, in
my words, skipping any question my message already answers:
- What identities do I want? Identities are spend buckets, usually one per
team, product, or agent. Ask: "Do you have more than one product?
Which teams or agents will call the API?" Offer these as a starting
point, and take my renames, drops, and additions:
coding - IDE and terminal coding agents
product - the product's own inference traffic
research - experiments, evals, notebooks
- Does each identity need its own inference key, and which tool or
service will hold it (Claude Code, Codex, a server, ...)? One key per
identity is the default; say so and let me change it.
- Do I want a spend budget per identity? If yes: how much (USD) and for
what period, monthly recurring or one calendar month, on the identity
or on its key. If I have no number, propose none: no budget means no cap.
Then show me the plan as ONE table: identity_id, display name, purpose,
key name and the tool it goes to (or "no key"), budget (amount + period,
or "none"). Wait for my explicit "yes". Anything else is an edit: apply it
and show the table again. The first write happens only after that yes.
3. Create each identity (reuse one that exists).
POST https://api-pr-441.preview.experientiallabs.ai/api/orgs/{org_id}/identities
{"display_name": "Coding", "identity_id": "coding", "description": "<one line>"}
201 -> created. 409 (code identity_id_unavailable) -> the id already
exists: if it is in MY list from step 1, reuse that identity; otherwise
(ids are unique platform-wide, and only need to be unique to my tooling)
retry ONCE with details.suggested_id from that response. identity_id
starts with a letter, continues with lowercase letters or digits, groups
joined by ".", "-" or "_", and never starts with "org-". If step 1 listed
the identity with active=false, do not use it: an admin disabled it on
purpose and only the dashboard re-enables it. Tell me and pick another.
4. Mint the inference keys the plan names (one per identity by default).
POST https://api-pr-441.preview.experientiallabs.ai/api/v1/keys
{"name": "<identity>-<tool>", "identity_id": "<identity_id>"}
200 -> {data: {hash, name, identity_id, ...}, key: "xpl_..."}. The plaintext
appears exactly once; hold it only until step 6. Never set "provisioning":
true on these keys (the API would allow it; I do not). Check
data.identity_id matches. An optional DAILY USD cap: "limit": 5 in the body.
5. Budgets, only where the plan has one, in nano-USD ($1 = 1000000000).
"period": "*" recurs every month; "period": "YYYY-MM" caps that one
calendar month only. 0 is a HARD $0 cap, never "no cap"; remove one with
DELETE https://api-pr-441.preview.experientiallabs.ai/api/orgs/{org_id}/budgets/{budget_id}. Replace or delete only
budgets this key created (an admin's row is 403). Never the default identity's.
PUT https://api-pr-441.preview.experientiallabs.ai/api/orgs/{org_id}/budgets
{"period": "*", "scope_kind": "identity", "identity_id": "coding",
"limit_nano_usd": 50000000000} # $50 every month for the identity
{"period": "*", "scope_kind": "key", "api_key_id": "<data.hash>",
"limit_nano_usd": 20000000000} # $20 every month for one key
{"period": "2026-10", "scope_kind": "identity", "identity_id": "coding",
"limit_nano_usd": 5000000000} # $5 for October 2026 only
Only those two scope kinds are this key's to set. Read back with
the calendar month, never "*": GET https://api-pr-441.preview.experientiallabs.ai/api/orgs/{org_id}/budgets?period=YYYY-MM
6. Hand me one table: identity -> key name -> where the key goes. Print each
new key exactly once, in that table, and nowhere else. Never commit a key;
prefer the OS keychain or a .env file that is gitignored. Per tool:
Claude Code: export ANTHROPIC_BASE_URL="https://api-pr-441.preview.experientiallabs.ai" (no /v1 suffix)
export ANTHROPIC_API_KEY="<key>"
export ANTHROPIC_MODEL="<slug from GET https://api-pr-441.preview.experientiallabs.ai/v1/models>"
Codex, Cline, Cursor, the OpenAI SDK, any OpenAI-compatible tool:
export OPENAI_BASE_URL="https://api-pr-441.preview.experientiallabs.ai/v1"
export OPENAI_API_KEY="<key>"
OpenCode: a provider in opencode.json with
"npm": "@ai-sdk/openai-compatible" and
"options": {"baseURL": "https://api-pr-441.preview.experientiallabs.ai/v1", "apiKey": "{env:OPENAI_API_KEY}"}
A server: the same OPENAI_* pair in its secret store. Per-tool
steps: https://pr-441.preview.experientiallabs.ai/docs/coding-agents.
7. Verify every key with one request and show me where the spend lands.
GET https://api-pr-441.preview.experientiallabs.ai/v1/models (bearer required) for a slug, then
POST https://api-pr-441.preview.experientiallabs.ai/v1/chat/completions with Authorization: Bearer <that key>
{"model": "<slug>", "max_tokens": 5,
"messages": [{"role": "user", "content": "ping"}]}
Then, with the provisioning key,
GET https://api-pr-441.preview.experientiallabs.ai/api/orgs/{org_id}/usage/by-identity?window=30d and confirm each
identity row lists its key with a request_count (only keys that carried
traffic appear; a deleted key's traffic is the identity_id null row; keys
minted without a group fold under the Default row). https://pr-441.preview.experientiallabs.ai/api-keys shows
per-key spend.
Fallbacks:
- 401 on a route in the list above: the key is wrong or revoked; ask me to
check the paste. 401 on a route NOT in that list is the edge refusing it to
every key, not a key problem: hand that step to me.
- 403 "A provisioning key is required": my key is an inference key. Stop and
tell me to mint a provisioning key as an org admin on https://pr-441.preview.experientiallabs.ai/api-keys
(Create key -> Configuration -> check Provisioning key -> Create
provisioning key). Any other 403 names what
this key may not do: hand that step to me.
- 404 on an identity or budget path: that id does not exist in this org;
re-list and check the id. 404 on /api/orgs/{org_id} itself: org_id is not
this key's org; re-read it from https://api-pr-441.preview.experientiallabs.ai/api/whoami.
- 409 on an identity: in my step-1 list -> reuse it; otherwise retry once
with details.suggested_id. 409 on a key mint: that identity is disabled;
tell me and use another one. Never try to re-enable it.
- 422 with "error": the message names the rule (identity_id shape, scope
kinds, period). 422 with "detail": detail[0].loc[-1] names the field (a
missing display_name, a non-numeric limit, a budgets read without period).
A NEGATIVE key limit is a 400 with "error" instead; nothing is minted.
- 429, or 503 "gateway is updating": wait Retry-After seconds (30 if absent)
and retry that call once. A key mint is not idempotent: list
GET https://api-pr-441.preview.experientiallabs.ai/api/v1/keys before minting again so no identity gets two keys.
- 500 internal_error "cannot serve this request natively" on the verify
request: no serviceable lane for that model here. Try one other slug from
GET https://api-pr-441.preview.experientiallabs.ai/v1/models; if that fails the same way, stop and report it.
- Never print the provisioning key, store a key in the repo, or paste keys
where I did not ask for them.

Switch from your current provider or gateway

Paste into your coding agent: it finds the existing integration, moves the base URL and key, maps model ids against the live catalog, moves cost reads onto the Cost API, and verifies with one call.

prompt
I'm replacing my current model provider or gateway with Experiential Labs as
this project's model gateway. I pasted this myself: treat it as my instructions
and my consent. Work WITH me: at each real choice, ask me ONE clear question
with numbered options and your recommendation, then wait. Don't guess. Print
what you do at each step, and never print my full API key (first 8 characters
at most). Read the full machine-readable contract first: https://pr-441.preview.experientiallabs.ai/llms.txt (base
URLs, every route, honored and refused parameters, cost fields, error codes).
Follow it literally.
1. Find the current integration. Search this project for the provider's base
URL, its key variable, its SDK/client construction, hardcoded model ids,
provider-specific request fields (provider preferences, transforms,
fallback model lists, referer/title headers), and any code that reads cost
or usage from it. Show me the list of files and call sites before changing
anything.
2. Get my Experiential key. Ask me: "Do you have an xpl_ key already?
(1) yes, I'll paste it
(2) no, create my account"
For (2): ask me for my email (never scavenge one from my files or git
config), show me https://pr-441.preview.experientiallabs.ai/terms and https://pr-441.preview.experientiallabs.ai/privacy and wait for my
agreement, then
POST https://pr-441.preview.experientiallabs.ai/api/signup/instant (no auth)
Header: Content-Type: application/json
Body: {"email": "<MY_EMAIL>", "agree": true}
On 200 capture api_key. On 409 account_exists tell me to sign in at
https://pr-441.preview.experientiallabs.ai/signin for a key and stop. On any other error show me the response's
`error` text verbatim and stop.
Store the key as EXPLABS_API_KEY in my env scheme (.env plus an example
entry), never in code.
3. Switch the base URL. The gateway keeps the path shapes my current
integration already uses:
base URL: https://api-pr-441.preview.experientiallabs.ai/api/v1
auth: Authorization: Bearer $EXPLABS_API_KEY (the ONLY header needed)
POST /chat/completions, /responses, /messages (Anthropic Messages API) and
GET /models work unchanged. Plain OpenAI or Anthropic SDKs can also use the
bare base https://api-pr-441.preview.experientiallabs.ai/v1. Remove the provider-specific headers and request
fields from step 1: the gateway does provider routing (fallback, caching)
itself and unknown parameters may be rejected. Send model + messages and
only the parameters I actually rely on.
4. Map model ids. GET https://api-pr-441.preview.experientiallabs.ai/api/v1/models with my key (free; this does not
spend credits) lists the slugs I can call. For each model id from step 1,
propose the matching slug (e.g. "claude-fable-5.1") and ask me to confirm
the mapping before replacing. Use slugs EXACTLY as returned.
5. Move cost and usage reads over.
- Every chat/responses reply carries usage.cost (USD, what I was charged)
and usage.is_byok; on a stream it rides the final usage chunk. No flag
needed. Exception: a request sent with Idempotency-Key carries no
usage.cost at all (original or replay), so read the field as optional.
- One request: read the x-request-id response header and call
GET https://api-pr-441.preview.experientiallabs.ai/api/v1/generation?id=<that id>.
- Balance: GET https://api-pr-441.preview.experientiallabs.ai/api/v1/credits. Recent activity:
GET https://api-pr-441.preview.experientiallabs.ai/api/v1/activity.
- Billing feed: GET https://api-pr-441.preview.experientiallabs.ai/api/v1/usage is the settled per-request export
(cost_usd, estimated_cost_usd, real_cost_usd, tokens, attribution_label,
api_key_id), newest first inside a lookback window (24h|7d|30d); its
next_cursor pages OLDER within one call and is not a resume checkpoint.
Each run scans from the top and dedupes on the row id.
- If I rebill my own customers, pass their stable id as safety_identifier
(alias: user) on every request; it comes back as attribution_label.
Replace my old provider's cost/usage code with these; show me the diff
first.
6. Key management (only if my code creates or rotates keys programmatically).
https://api-pr-441.preview.experientiallabs.ai/api/v1/keys (GET/POST/PATCH/DELETE) requires a PROVISIONING key: an
xpl_ key minted with "provisioning": true. The first one is created from
the dashboard at https://pr-441.preview.experientiallabs.ai/api-keys (org admin, "Provisioning key" option).
Tell me to mint it and paste it as a separate server-side secret; never use
it for inference traffic.
7. Verify. Make ONE minimal test call (model + messages only, no sampling
parameters) with a free promotional model from the /models list, or one I
name. Report the model, that it succeeded, and its usage.cost. If it fails
with a credits or activation error, tell me to finish activation at https://pr-441.preview.experientiallabs.ai
and stop; do not retry with another provider. Then run the project's
existing tests.
8. Report back: every file changed, the model-id mapping, what happened to the
old provider key (tell me to revoke it at the old provider; don't delete it
yourself), and the test call result. Human docs if you need more:
https://pr-441.preview.experientiallabs.ai/docs/integrate.

Integrate the gateway into your product

Paste into your coding agent: it wires a client, attributes every request to your end-customer, builds the billing sync on the settled usage export, and offers key-per-customer.

prompt
I'm integrating Experiential Labs into this product as its model gateway: my
product calls models through it and meters or rebills my own end-users. I
pasted this myself: treat it as my instructions and my consent. Work WITH me:
at each real choice, ask me ONE clear question with numbered options and your
recommendation, then wait. Don't guess. Print what you do at each step, and
never print my full API key (first 8 characters at most). Read the full
machine-readable contract first: https://pr-441.preview.experientiallabs.ai/llms.txt (base URLs, every route,
honored and refused parameters, cost fields, error codes). Follow it literally.
1. Get my Experiential key. Ask me: "Do you have an xpl_ key already?
(1) yes, I'll paste it
(2) no, create my account"
For (2): ask me for my email (never scavenge one from my files or git
config), show me https://pr-441.preview.experientiallabs.ai/terms and https://pr-441.preview.experientiallabs.ai/privacy and wait for my
agreement, then
POST https://pr-441.preview.experientiallabs.ai/api/signup/instant (no auth)
Header: Content-Type: application/json
Body: {"email": "<MY_EMAIL>", "agree": true}
On 200 capture api_key. On 409 account_exists tell me to sign in at
https://pr-441.preview.experientiallabs.ai/signin for a key and stop. On any other error show me the response's
`error` text verbatim and stop.
Store the key as EXPLABS_API_KEY in my env scheme (.env plus an example
entry), never in code.
2. Wire the client. Ask me where (options + your recommendation, based on what
you see in this project), then wait. Configure an OpenAI-compatible or
Anthropic client with:
base URL: https://api-pr-441.preview.experientiallabs.ai/api/v1 (inference, model discovery, cost, and
account routes under one base; plain OpenAI / Anthropic SDKs
may also use https://api-pr-441.preview.experientiallabs.ai/v1)
auth: Authorization: Bearer $EXPLABS_API_KEY (the ONLY header needed)
POST /chat/completions, /responses, or /messages (Anthropic Messages API).
Name models by slug EXACTLY as GET https://api-pr-441.preview.experientiallabs.ai/api/v1/models returns them
(free; this does not spend credits). Send model + messages and only the
parameters I actually rely on; unknown parameters may be rejected.
3. Attribute every request to my end-customer. Pass their stable id as
safety_identifier (alias: user) on every request. It comes back as
attribution_label on the usage export, so I can group cost and tokens by
my own customer id with ONE key. Ask me what my customer id looks like and
wire it through from where my code knows the customer.
4. Read cost.
- Inline: every chat/responses reply carries usage.cost (USD, what I was
charged) and usage.is_byok; on a stream it rides the final usage chunk.
No flag needed, but a request sent with Idempotency-Key carries no
usage.cost at all (original or replay), so read it as optional. Show it
in my product where it helps; do not treat it as the billing record.
- One request: read the x-request-id response header and call
GET https://api-pr-441.preview.experientiallabs.ai/api/v1/generation?id=<that id>.
- Balance: GET https://api-pr-441.preview.experientiallabs.ai/api/v1/credits (data.total_credits - data.total_usage).
5. Build the billing sync on the settled export. Ask me which billing system I
use (Metronome, Orb, Stripe metered billing, my own tables, or none yet),
then write a scheduled job that:
- reads GET https://api-pr-441.preview.experientiallabs.ai/api/v1/usage?limit=1000 (settled rows, newest first
inside a lookback window: window=24h|7d|30d, default 7d; filterable by
attribution_label, model, api_key_id, status). next_cursor
({cursor_ts, cursor_id, cursor_after}, passed back as query params)
pages OLDER within this one call and is NOT a checkpoint to resume from
next run: each run starts at the top, follows next_cursor until a whole
page is older than the last successful run's start time minus a grace
period covering my longest request, and dedupes on the row id. Schedule
it at least daily; rows older than 30 days are no longer exportable.
- lands each row in a local outbox table keyed on the row id (insert, on
conflict do nothing) in one local transaction, then in a separate
delivery step pushes unsent rows to the billing system and marks them
sent (an external HTTP write cannot share my database transaction; the
outbox makes a crash between the two safe to replay),
- sends one usage event per row: customer from attribution_label,
transaction_id / idempotency key = the row id, timestamp = created_at,
quantity = the cost column I choose (below) plus token counts if I meter
on tokens; the margin is a rate in the billing system, not in this job.
Cost columns: cost_usd is the platform credits charged; estimated_cost_usd
is the attributed value of a BYOK row (served on my own provider key) at
catalog list rates, an estimate and never the provider's invoice;
real_cost_usd is the two added together. Ask me whether to bill
real_cost_usd (accepting the BYOK estimate) or cost_usd plus tokens on
BYOK rows. pricing_known false means unpriced (cost 0), not free.
6. Key-per-customer (optional; ask me). If I want each customer isolated to
their own key with independent revocation and a daily USD cap, that needs
a PROVISIONING key: an xpl_ key minted with "provisioning": true. The
first one comes from the dashboard at https://pr-441.preview.experientiallabs.ai/api-keys (org admin,
"Provisioning key" option); keep it server-side and never use it for
inference. With it: POST https://api-pr-441.preview.experientiallabs.ai/api/v1/keys {name, limit, provisioning}
returns the plaintext key exactly once; GET/PATCH/DELETE
https://api-pr-441.preview.experientiallabs.ai/api/v1/keys/{hash} list, update (disabled:true revokes), and revoke.
A normal inference key gets 403 on every /keys route.
7. Verify. Make ONE minimal test call (model + messages only) with a free
promotional model from the /models list, or one I name, carrying a test
safety_identifier. Report the model, that it succeeded, and its usage.cost;
then confirm the row shows up on GET https://api-pr-441.preview.experientiallabs.ai/api/v1/usage with that
attribution_label. If the call fails with a credits or activation error,
tell me to finish activation at https://pr-441.preview.experientiallabs.ai and stop. Then run the project's
existing tests.
8. Report back: every file changed, where the key lives, how customer
attribution flows, and the billing job's schedule and cursor storage.
Human docs if you need more: https://pr-441.preview.experientiallabs.ai/docs/integrate, https://pr-441.preview.experientiallabs.ai/docs/cost-api,
and https://pr-441.preview.experientiallabs.ai/docs/account-api.

Switch from OpenRouter

Using OpenRouter today? Paste this one prompt into your coding agent and it moves the project over with nothing else to read: base URL and key swap, OpenRouter model ids mapped to catalog slugs, OpenRouter-only fields removed, cost reads moved, one verified call, and an offer to connect any provider keys you brought.

prompt
I'm moving this project off OpenRouter and onto the Experiential Labs gateway.
I pasted this myself: treat it as my instructions and my consent. Work through
it end to end; stop and ask me ONE numbered question only where this prompt
says to. Print what you change at each step. Never print my full API key
(first 8 characters at most). My Experiential key is: xpl_...
(if that still reads "xpl_..." I have not filled it in: ask me for it, or tell
me to mint one at https://pr-441.preview.experientiallabs.ai/api-keys, then continue).
1. Find every place this project talks to OpenRouter. Search for:
- the base URL https://openrouter.ai/api/v1 (openrouter.ai anywhere), env
vars OPENROUTER_API_KEY / OPENROUTER_BASE_URL, and sk-or-v1- keys in
.env*, docker-compose, Makefiles, CI config, and secret stores,
- clients pointed at it: openai or Anthropic SDKs (TS / Python / Go),
ruby-openai (uri_base:), Go net/http, fetch / curl / any HTTP client,
Vercel AI SDK @openrouter/ai-sdk-provider, LangChain, LiteLLM openrouter/ ids,
- coding-agent configs: opencode.json provider blocks, Claude Code
ANTHROPIC_BASE_URL in .claude/settings*.json or shell exports, Cline /
Continue / Cursor "OpenAI Compatible" settings, Codex config.toml,
- hardcoded vendor/model ids (openai/gpt-6-astra, openrouter/auto, ...),
OpenRouter-only fields (provider, transforms, route, models, plugins,
usage), HTTP-Referer / X-Title, previous_response_id, and reads of
/generation, /credits, /auth/key, /models/user, /activity, usage.cost.
List the files and call sites before changing anything.
2. Substitute the base URL and the key. The gateway keeps OpenRouter's paths:
https://openrouter.ai/api/v1 -> https://api-pr-441.preview.experientiallabs.ai/api/v1
Authorization: Bearer $EXPLABS_API_KEY (the ONLY header needed)
Store my key as EXPLABS_API_KEY in this project's env scheme (.env plus the
example file entry), never in code; point every OpenAI-compatible client at
the new base (a plain OpenAI SDK may also use https://api-pr-441.preview.experientiallabs.ai/v1) and
REPLACE URL variables such as OPENROUTER_BASE_URL with it. An Anthropic SDK
appends /v1/messages itself: its base is the bare origin https://api-pr-441.preview.experientiallabs.ai
(never .../v1, which doubles to /v1/v1/messages). Do NOT
delete OPENROUTER_API_KEY or the OpenRouter key from .env, secret stores or
CI: stop reading it, leave it in place; I revoke it at openrouter.ai/keys.
A key COMMITTED in a config file (ANTHROPIC_AUTH_TOKEN in a checked-in
.claude/settings.json) is already leaked: remove it there, never send it on.
Coding agents: Claude Code uses ANTHROPIC_BASE_URL="https://api-pr-441.preview.experientiallabs.ai"
(no /v1); the key rides ANTHROPIC_API_KEY (never ANTHROPIC_AUTH_TOKEN) via
a shell export, the gitignored .claude/settings.local.json, or an
apiKeyHelper reading .env, never the committed .claude/settings.json; remap
EVERY model variable (ANTHROPIC_MODEL, ANTHROPIC_SMALL_FAST_MODEL,
ANTHROPIC_DEFAULT_*_MODEL) to slugs or a background role keeps calling an
expensive model. OpenCode, Cline, Continue, Cursor and Codex take the
OpenAI base https://api-pr-441.preview.experientiallabs.ai/v1.
3. Map model ids. Call GET https://api-pr-441.preview.experientiallabs.ai/api/v1/models with my
key (free, spends no credits). EVERY entry carries canonical_slug, the
plain slug the gateway serves; ids are usually vendor/slug (vendor/slug:free
for a free lane), sometimes a bare slug (gpt-6-astra, grok-4); ~vendor/x-latest
ids are rolling aliases, for projects that used OpenRouter's :latest ids.
For each OpenRouter id from step 1: strip any :nitro, :floor, :online,
:thinking, :extended, :beta or :exacto suffix (those variants do not exist
here; only :free does), then match canonical_slug == the part after the
vendor's slash (openai/gpt-6-astra -> gpt-6-astra, anthropic/claude-sonnet-5
-> claude-sonnet-5, deepseek/deepseek-v4-flash -> deepseek-v4-flash).
No match: look for the same model under a slightly different spelling and
use ONE obvious candidate, noted in the report. openrouter/auto and every
id still unmapped: ask me one question listing each with your best
candidates from the list, then wait. Never guess a model I did not pick.
Use canonical_slug EXACTLY as returned; a vendor/slug id works verbatim
ONLY when it is in the list (x-ai/grok-4 for the bare grok-4 answers 403
model_not_granted: that means "use the canonical_slug", not an entitlement
problem). LiteLLM: model "openai/<slug>" (replace the "openrouter/" prefix)
with api_base https://api-pr-441.preview.experientiallabs.ai/api/v1 and api_key from the new var.
4. Keep the request shapes (OpenAI Chat Completions and Responses bodies, SSE
streaming, tool calling, json_schema, response shapes). Strip EVERY
OpenRouter-only field BEFORE the first call: each one left in is either
silently ignored or its own 400 unsupported_parameter naming it in "param".
- usage: {"include": true} (a guaranteed 400 today); for a stream send
stream_options: {"include_usage": true} instead (non-streams carry usage).
- provider (order, allow_fallbacks, ignore, quantizations, sort, ...):
accepted and silently ignored (not disclosed); the gateway routes and
fails over itself. The one honored key is "provider": {"zdr": true}, a
per-request zero-data-retention demand; keep exactly that if I used it.
- transforms, route, models (fallback list: use its first model), plugins.
- HTTP-Referer / X-Title headers: harmless, not shown; attribute via safety_identifier.
Vercel AI SDK: replace @openrouter/ai-sdk-provider with @ai-sdk/openai
(createOpenAI({ baseURL, apiKey })) or @ai-sdk/openai-compatible.
"reasoning": {"effort"} is translated to reasoning_effort (rewriting it is optional).
Unsupported efforts may only move down if policy permits and the request is admissible; else 400.
"none" means off and can be refused; "minimal" still enables reasoning.
On non-reasoning routes, effort may drop only when policy permits and the request is admissible.
Translated, dropped or adjusted fields are usually listed in the reply's
x-experiential-ignored-parameters JSON body field (not a header): informational.
Anthropic-shaped traffic through OpenRouter (an Anthropic SDK, Claude Code)
moves to the native Messages API, POST https://api-pr-441.preview.experientiallabs.ai/api/v1/messages
(SDK base: the bare origin, step 2), same key; no Idempotency-Key there.
Responses API previous_response_id: continuation resolves ONLY while my
org's capture_prompt_content is on and within the retention window; with
capture off the second turn 400s previous_response_not_found. If the
project chains turns that way, read
GET https://api-pr-441.preview.experientiallabs.ai/api/orgs/<org_id>/telemetry-settings (org_id: /api/whoami);
if off, ask me ONE question: turn it on (PUT the same route,
{"capture_prompt_content": true}; it stores my prompts with Experiential)
or rewrite the chain to resend the full conversation each turn.
5. Move cost reads. Every chat / responses reply carries usage.cost (USD, what
I was charged) and usage.is_byok; on a stream it rides the final usage
chunk (step 4's stream_options). A request sent with Idempotency-Key
carries no usage.cost at all, so read the field as optional. Same key on
all of these, under https://api-pr-441.preview.experientiallabs.ai/api/v1:
GET /generation?id=<the x-request-id response header> one request
GET /usage settled per-request rows (api_key_id, cost) for billing
GET /credits balance GET /key key info (OpenRouter's /auth/key)
GET /activity per-day rollup (not per request)
OpenRouter's /models/user has no equivalent: use /models. /usage is paginated
(50 rows + next_cursor) and, like /activity, ORG-wide: filter ?api_key_id=<my
key's id, on every row> and match my x-request-id, or use /generation?id=.
6. My own provider keys (BYOK). If step 1 found provider keys I routed through
OpenRouter (OPENAI_API_KEY, ANTHROPIC_API_KEY, GEMINI, XAI, DEEPSEEK, AZURE,
AWS/Bedrock, Fireworks, ...), ask whether to connect them with no markup:
GET https://api-pr-441.preview.experientiallabs.ai/api/whoami -> org_id
GET https://api-pr-441.preview.experientiallabs.ai/api/orgs/<org_id>/provider-connections
List existing accounts FIRST. For EACH new account, ask me for a unique
setup_alias (1-63 lowercase letters, digits, hyphens or underscores), unused
across all providers in my org; multiple keys need distinct account names.
POST https://api-pr-441.preview.experientiallabs.ai/api/orgs/<org_id>/provider-connections/<provider>
Content-Type: application/json
{"setup_alias": "<unique-account-name>", "secret": "<that provider key>"}
Never overwrite an existing account. On 409 account_exists choose another
name and retry POST, never bypass it with PUT. PUT is only for a deliberate
rotation I request, with that saved account's exact setup_alias and config.
Provider: openai, anthropic, gemini, xai, deepseek, or any other whose
credential is ONE string; never my OpenRouter key: I am leaving OpenRouter.
azure_openai, bedrock, fireworks, vertex and modal need structured config
or a token pair: connect those at
https://pr-441.preview.experientiallabs.ai/settings/connections instead.
Use INFERENCE keys, never admin keys, as secret; OpenAI/Anthropic admin keys
belong only in optional spend_secret. If spend_error is present,
the serving key WAS saved: report partial success, never repeat POST; retry
only the SAME returned connection.setup_alias with PUT, retaining its serving
secret and config. Read check.status; if not "valid", tell me
check.status_detail.remediation and stop. A 402 pro_required means a FIRST connection needs Pro
(https://pr-441.preview.experientiallabs.ai/credits); an org already holding one keeps connecting.
Saving keys adds no arbitrary models: let me select each account on the
chosen catalog model's waterfall with Use this account. Never echo a key.
7. Verify. Make ONE minimal call (model + messages, a small max_tokens) with
the CHEAPEST mapped slug: a reasoning model can cost cents even for a
smoke, and an empty pricing object in the listing means unknown: read
GET https://api-pr-441.preview.experientiallabs.ai/api/models/<slug>, and if that 404s (an alias)
price the model it resolves to (its canonical_slug) or skip it and smoke a
priced slug. Stream if the project streams (add stream_options:
{"include_usage": true} so the final chunk carries usage.cost); if the
project chains previous_response_id, verify a TWO-turn continuation, not a
single call. Report the model, that it succeeded, and its usage.cost, or
read the cost from GET /api/v1/generation?id=<x-request-id>, which works
either way; then confirm the request's row on GET /api/v1/usage by that
x-request-id (I can also see it at https://pr-441.preview.experientiallabs.ai/logs). On a credits
or activation error, tell me to finish activation at https://pr-441.preview.experientiallabs.ai
and stop; do not retry against OpenRouter. Then run the project's tests.
8. Report back: every file changed; the model-id mapping and the ids I still
need to decide; the OpenRouter-only fields removed; where EXPLABS_API_KEY
lives; that the OpenRouter key stays in place for me to revoke; the
verification result. Full contract: https://pr-441.preview.experientiallabs.ai/llms.txt; human
docs: https://pr-441.preview.experientiallabs.ai/docs/integrate.

Switch from Cloudflare AI Gateway

Assess rules, customer plans, model-group allowances, BYOK and custom providers before moving traffic. The agent asks before writes, spend, privacy changes or cutover, and stops on unsupported requirements or missing live proof.

prompt
Help me assess and then migrate this project's Cloudflare AI Gateway traffic
onto the Experiential Labs gateway. Start with a read-only inventory and a
proposed mapping. This prompt is NOT blanket consent: ask for my explicit
approval before any file or policy writes, credential storage or rotation,
paid probes or other spend, traffic cutover, privacy/retention changes, or
secret revocation. Explain each proposed change and its rollback first.
My Experiential key is: xpl_...
If it is still "xpl_...", ask me for a key or direct me to
https://pr-441.preview.experientiallabs.ai/api-keys. Do not assume an inference key has admin authority.
Never print secrets, put them in committed files, or send an Experiential key
to Cloudflare or an upstream provider. Redact secret values in every report.
1. Inventory before editing. Find Cloudflare AI Gateway URLs (including
gateway.ai.cloudflare.com, provider-specific paths, /compat and custom-*
providers), SDKs, fetch/curl clients, Workers bindings, env variables, CI
and secret-store references. List call sites, endpoints and model ids.
Identify Chat Completions, Responses, Messages, streaming, tool calls,
structured output, continuation, batches and any other used API surface.
Record Cloudflare headers such as cf-aig-authorization, cf-aig-metadata,
cf-aig-cache-ttl, cf-aig-skip-cache, cf-aig-custom-cost and cf-aig-log-level,
plus all other cf-aig-* headers, retry/fallback settings and timeouts.
Separate Cloudflare gateway/admin tokens from direct OpenAI/Anthropic or
other provider inference credentials. A Cloudflare token is not BYOK for
those providers. Inspect references without exposing secret values.
2. Ask for the actual Cloudflare rule export and customer subscription rules.
Record dynamic-routing conditions, rule order, rate/spend limits, metadata
filters/split dimensions and provider/model overrides. Identify the trusted
customer id and the server-side subscription source. Cloudflare custom
metadata labels are NOT authentication or authorization. Neither request
metadata.plan, user nor safety_identifier may select a customer's plan.
The application server must map its authenticated customer to an Experiential
identity; all that identity's keys, including rotated keys, share policy.
Inventory each tier's allowed models, Basic/Frontier (or other) model groups,
exact memberships, separate allowance amounts, currency and cost basis.
These are my authored groups, not universal model classes or platform
Free/Pro subscriptions. Never turn illustrative amounts into defaults.
Ask about reset timezone and window (fixed, rolling, billing-anniversary or
UTC calendar-month), carryover, current-period usage and pending requests,
carry-in at cutover, upgrades/downgrades and over-limit behavior. A zero
allowance means hard zero, not unlimited. Also inventory the shared total
across all groups and the exact action on group exhaustion. An absent/null
aggregate_limit_nano_usd means no shared cap; zero denies. on_exhausted
defaults to deny; next_model requires explicit policy and a certified
server-authored model chain, never an arbitrary client retry. Aggregate
exhaustion and forbidden models still stop. Unknown plans must block mapping,
never become unassigned identities. Do not silently reset usage.
3. Read THIS deployment's live contract and catalog before proposing APIs:
https://pr-441.preview.experientiallabs.ai/llms.txt
https://pr-441.preview.experientiallabs.ai/docs/integrate
https://pr-441.preview.experientiallabs.ai/docs/account-api
https://pr-441.preview.experientiallabs.ai/docs/cost-api
GET https://api-pr-441.preview.experientiallabs.ai/api/whoami
GET https://api-pr-441.preview.experientiallabs.ai/v1/models
GET https://api-pr-441.preview.experientiallabs.ai/api/v1/models
Use the catalog's canonical_slug and verified alias resolution to map every
model, including provider-prefixed ids and custom models. Do not simply strip
a prefix, assume a model exists, or replace one without asking me. Catalog
discovery is not proof of this customer's access or successful serving.
Build a requirement table: existing behavior, proposed mapping, live-doc
evidence, test needed and blocker. Discover exact native model-group,
customer-plan, assignment and allowance API routes from live documentation;
do not invent routes, fields, or claim these controls are already deployed.
Require native identity-bound access and separate per-group allowances,
not application-side checks dressed up as equivalent gateway enforcement.
Confirm admin/delegated assignment authority separately from provisioning
keys and inference keys. Customers must not change plans, mint escape keys,
or read another identity's usage; caller-supplied filters are not isolation.
Verify allowed-model access across public models, aliases, retries/fallback
and every used serving surface; unsupported lanes must refuse plan keys.
Verify shared counters across keys, settled usage plus pending reservations,
pricing revisions, plan-change effective times and preserved spent amounts.
Read whoami's identity_id, customer_plan_identity_id and is_provisioning;
restricted keys may self-read it, but that does not grant management rights.
The implemented period contract supports UTC calendar-month or an explicit
authoritative source interval with source_period_id and timezone-aware start/end.
Explicit periods expire without automatic renewal; omitted budget_period on
a plan change preserves it. Rolling windows are unsupported. Never rename,
overlap or backdate a used source period to refill it. Treat unsupported
reset/carryover rules as blockers, not approximations. Discover the documented
allowance-cutover contract too, without assuming THIS deployment has it.
If documented carry-in is unavailable, ask about a deliberate next-period cutover;
never silently grant a fresh full allowance.
4. Map provider and cost behavior, not just a base URL. Existing Account API
provisioning and Cost API settled exports are useful but do not by themselves
prove subscription-policy parity. Keep wallet charges separate from customer
allowance attribution. BYOK must not debit platform credits; attributed BYOK
inference value uses a frozen rate schedule and is an estimate, NOT a provider
invoice. Inventory input/output, cache-read/write, reasoning, long-context
and custom rates, billable failures, missing usage and pending exposure.
Unknown prices or missing usage must not turn a monetary allowance into free
requests. Verify the allowance API reads the counters enforcement uses.
Subscription events and invoicing stay in my application. For external
billing, poll the documented settled Cost API export with pagination,
durable checkpoints and deduplication; reconcile request ids, late settlement
and retries. Do not invent webhooks or bill from a daily rollup or wallet
balance. Treat inline usage.cost as optional; do not confuse it with a BYOK
allowance estimate. Verify reporting scopes with two customer identities.
List existing provider accounts before proposing BYOK. Ask before saving
anything; use a distinct setup_alias for each new account and create-only
POST from the live contract, never PUT over an existing account. Rotation
needs separate approval. Respect the live plan entitlement for new BYOK
connections; saving a provider key does not add arbitrary model support.
5. Custom providers need their own proof. Cloudflare custom providers can use
arbitrary HTTP paths and native bodies; Experiential's supported adapters
and model-bound custom endpoints are NOT an arbitrary HTTP proxy. Inventory
the exact URL, wire protocol, auth headers, model ids, usage format and rate
units. Use the documented Vault-backed connection/custom-endpoint flow and
its validation, not a new proxy or a made-up Cloudflare provider connection.
Custom endpoint rates are not inferred from public provider prices. Confirm
the live rate_card contract, full schedule and frozen revision. Without an
authored card, custom pricing is unknown, not free. New custom accounts need
an explicit unique setup_alias and create-only POST. For key or rate changes,
create a new custom account and deliberately select Use on the model; do not
rotate it through the generic provider PUT or probe at the official origin.
For Tinfoil (TinFoil), verify the endpoint and SDK's enclave-attestation and
privacy requirements. OpenAI compatibility does NOT prove attestation survives
an intermediary. If the required guarantee cannot be verified, stop that move.
Keep Modal separate from the Cloudflare transfer. Determine whether it uses
Modal proxy authentication, an application bearer, or another supported scheme,
and whether it bills tokens or time/GPU usage. Do not claim token-cost parity
for unsupported billing units. Name any adapter, auth, usage or pricing gap
as a blocker and scope it separately; never silently reroute these services.
6. Preserve privacy and runtime semantics. Inventory prompt/response logging,
retention, zero-data-retention/no-training needs, cache keys/TTL/bypass,
cache hits and their accounting, retries, fallback order and timeout budgets.
Cloudflare response caching is not the same as provider prompt caching.
Map each cf-aig-* header to a documented control or report it unsupported;
never forward Cloudflare auth to the new host or silently drop a required rule.
Check continuation requirements such as previous_response_id against the live
capture_prompt_content contract. Ask before enabling storage; offer resending
the full conversation where supported instead. Never weaken privacy to make
a smoke pass. Check retry/idempotency behavior per API surface, including
ambiguous disconnects; do not assume every retry is free or safe to replay.
7. Present the mapping and blockers, then wait for approval of specific edits.
For supported OpenAI clients the base is https://api-pr-441.preview.experientiallabs.ai/v1 with
Authorization: Bearer $EXPLABS_API_KEY. An Anthropic SDK appends /v1/messages
itself: use the bare origin https://api-pr-441.preview.experientiallabs.ai, not a doubled /v1 base.
Store keys only in the approved local env/secret scheme, never source control.
Preserve unrelated Cloudflare DNS, Workers, Turnstile and other services.
Keep old untracked .env secrets, CI secrets and Cloudflare/provider tokens
in place for rollback until deliberate revocation is approved. A committed
secret is exposed: flag it and ask for removal/rotation, never echo it.
Propose a reversible configuration diff and keep a redacted baseline of
endpoint, model and policy settings. Do not switch production traffic yet.
8. Ask for an isolated test environment, bounded credentials and a request/spend
budget before probes; connection checks may themselves make paid calls.
Pin platform revision, deployed engine versions, schema, configuration and
catalog/pricing revisions. Run local tests first; label mocks as mocks.
With approval, test actual providers and the project's real request shapes,
streams/tools/continuations, two customers, multiple models per group,
independent allowance depletion, forbidden-model refusal before dispatch,
key rotation, concurrent reservations, retries/fallback, interrupted streams,
reset/plan changes, shared aggregate depletion, deny versus next_model,
identity-scoped reports and BYOK without wallet debits. Reconcile request ids,
settled costs and native plus imported pending allowance exposure. A single
successful completion does not establish policy, billing or privacy parity.
Stop before cutover on unsupported requirements or absent live verification.
Do not autoactivate production policy or move traffic. Only after evidence and
a separate cutover approval, use the supported administrator handoff:
- Prepare at most 500 immutable, disjoint source-owned per-request receipts,
each with external_ref, frozen group reference, settled and pending amounts.
Never import cumulative snapshots or requests already counted by the destination.
Pin exact period, source watermark, coverage and evidence digest. Identical
prepare retries are idempotent; changed replays/overlapping exports refuse.
More than 500 receipts is a blocker for this bounded path, not permission
to truncate, aggregate them or invent pagination.
- Prepare closes that identity's destination admission even while global
enforcement is inactive. It does not stop Cloudflare. Explicitly stop source
dispatch and reconcile in-flight ownership before activating with source_frozen
and independent source-stop evidence. This is NOT a transaction across gateways.
Require the documented trusted administrator transport; an ordinary or delegated
provisioning key cannot perform cutover. Do not invent a browser proxy or
supply actor headers if the live contract does not provide an admin entrypoint.
- Rollback must account for usage on BOTH gateways. Freeze destination admission,
drain native reservations and resolve native pending plus all imported pending
while active or frozen, then obtain a complete fresh export before acknowledgment.
Any unresolved native or imported exposure blocks acknowledgment.
Include settled_periods across all historical periods, not just the current one.
Preserve the original JSON and parse/serialize money losslessly with Python
integers/exact decimals or a JavaScript lossless BigInt JSON library. Totals
can exceed 2^53 - 1. Never use ordinary JSON.parse, response.json() or Number
for rollback amounts; rounding cannot be repaired afterward. Send exact JSON
integers, not numeric strings, and return the server-issued digest unchanged.
The export bounds are 500 native outstanding rows, 500 external receipts and
1,000 settled-period rows; overflow is never proof of a complete export.
A changed export_sha256 or export_overflow blocks acknowledgment. Rolled-back
destination admission stays closed; no step resets history or releases unknown
usage. Unresolved prior exposure or destination use in the same period blocks
a new prepare. Never grant extra allowance or replay uncertain paid requests.
Keep old secrets until I deliberately approve revocation after verification.
9. Report pass/fail/blocked/untested for every requirement, changed files, model
mappings, policy/reset/carry-in decisions, credential locations (no values),
tests and their pinned cutoff, billing reconciliation and rollback steps.
Distinguish existing Account/Cost APIs, new native controls and their actual
deployment status, application billing work and unverified provider behavior.
Never claim full Cloudflare parity before this workflow is tested.

Switch from Vercel AI Gateway

Moves AI SDK gateway model strings and OpenAI clients pointed at ai-gateway.vercel.sh onto the gateway, maps creator/model ids to catalog slugs, and replaces OIDC auth with a key in the project's env.

prompt
I'm moving this project's model traffic from Vercel AI Gateway onto the Experiential
Labs gateway. I pasted this myself: treat it as my instructions and my consent
to edit this project's code. Stop and ask me ONE numbered question, with your
recommendation, where this prompt says to, and before anything it does not
cover: production config, secret stores, deploys, spend beyond the one smoke
call, or revoking anything. Print what you change at each step. Never print my
full API key (first 8 characters at most).
My Experiential key is: xpl_...
(if that still reads "xpl_..." I have not filled it in: ask me for it, or tell
me to mint one at https://pr-441.preview.experientiallabs.ai/api-keys, then continue).
The full contract (routes, honored and refused parameters, error codes) is
https://pr-441.preview.experientiallabs.ai/llms.txt; follow it literally.
1. Inventory, read only. Find every place this project talks to Vercel AI Gateway:
- the base URL https://ai-gateway.vercel.sh (/v1 for OpenAI clients, the
bare host for Anthropic clients and ANTHROPIC_BASE_URL), AI_GATEWAY_API_KEY,
VERCEL_OIDC_TOKEN and .env.local files written by vercel env pull,
- Vercel AI SDK calls ("ai" package) whose model is a PLAIN STRING such as
generateText({ model: "anthropic/claude-sonnet-5" }): the AI SDK sends
bare strings through the Vercel gateway by default, so these call Vercel
even though no URL appears in the code; also gateway("...") from "ai" or
"@ai-sdk/gateway", and createOpenAI / createOpenAICompatible clients
pointed at ai-gateway.vercel.sh,
- gateway extensions in request bodies: a top-level "models" fallback list,
a "provider" object (sort and friends), and providerOptions.gateway
(order, only, sort, models, byok, providerTimeouts, caching: "auto"),
- provider keys configured as Vercel BYOK (in the Vercel dashboard or
request-scoped under providerOptions.gateway.byok).
Also list hardcoded model ids, every Vercel AI Gateway-specific header and body field,
retry / fallback / cache settings, coding-agent configs pointed at it, and
code that reads cost or usage from it. Show me the files and call sites
before changing anything. Inspect secret references, never their values.
2. Map Vercel AI Gateway's features before moving anything.
- order / only / sort / providerTimeouts: the gateway picks and fails over
providers itself; drop them. "models" (fallback models): keep the primary
model and tell me which fallbacks are lost.
- caching: "auto" and cache_control markers: see prompt caching below.
- BYOK (dashboard or per request): offered as saved accounts in step 6;
request-scoped credentials have no equivalent and leave the body.
- OIDC: there is no keyless equivalent. On Vercel the key becomes a project
environment variable: ask me to add EXPLABS_API_KEY to the Vercel project
(Production, Preview, Development) myself, or run vercel env add only
after I say so.
The gateway's own equivalents, for the table:
- Fallback, retries and load balancing across providers of one model: the
gateway routes and fails over itself (GET https://api-pr-441.preview.experientiallabs.ai/api/models/<slug>/waterfall
shows the order). A fallback list ACROSS models has no per-request
equivalent.
- Keys per team or customer, spend caps, budgets: a PROVISIONING key (an org
admin mints the first at https://pr-441.preview.experientiallabs.ai/api-keys) mints inference keys with a
daily USD cap (POST https://api-pr-441.preview.experientiallabs.ai/api/v1/keys {"name", "limit"}) and sets monthly
caps per key or identity (PUT https://api-pr-441.preview.experientiallabs.ai/api/orgs/<org_id>/budgets).
- Metadata: the X-Explabs-Tags header (a JSON object, at most 16 labels, on
chat, Responses and Messages calls; never secrets or personal data). End
user: safety_identifier (alias user) in the body. Session or
conversation: the X-Explabs-Session-Id header.
- Logs and dashboards: https://pr-441.preview.experientiallabs.ai/logs and https://pr-441.preview.experientiallabs.ai/spend, the Cost API (step 5),
and live export of every call to my Langfuse project
(PUT https://api-pr-441.preview.experientiallabs.ai/api/orgs/<org_id>/observation-export).
- Privacy: "provider": {"zdr": true} per request, or account-wide
PUT https://api-pr-441.preview.experientiallabs.ai/api/orgs/<org_id>/zero-data-retention (Pro); processing
regions are in Settings, Data controls (https://pr-441.preview.experientiallabs.ai/settings/data-controls).
- Content guardrails: per identity, on the Guardrails tab at https://pr-441.preview.experientiallabs.ai/api-keys.
- Prompt caching: provider prompt caching applies where the route supports
it; Anthropic cache_control markers are kept on supported routes and
disclosed as not forwarded elsewhere (default five-minute markers; a
one-hour marker is refused on platform credits).
- No equivalent: gateway-side response or semantic caching, hosted prompt
templates, metadata-based routing between models, and arbitrary
request/response transforms.
Show me a table: feature in use -> equivalent (or "none") -> what changes.
Then ask me ONE question: proceed with this plan, adjust it, or stop. Wait.
A feature I depend on with no equivalent is a blocker: say so, never drop it
silently.
3. Point the clients at the gateway.
AI SDK, two ways (ask me which; recommend (a) when many call sites use
plain strings):
(a) one provider for every plain string, set once at startup:
import { createOpenAICompatible } from "@ai-sdk/openai-compatible";
globalThis.AI_SDK_DEFAULT_PROVIDER = createOpenAICompatible({
name: "experiential", baseURL: "https://api-pr-441.preview.experientiallabs.ai/v1",
apiKey: process.env.EXPLABS_API_KEY });
then every model string must be an Experiential slug (step 4);
(b) per call site: model: experiential("<slug>") with the same provider
instance, or createAnthropic from "@ai-sdk/anthropic" with baseURL
https://api-pr-441.preview.experientiallabs.ai/v1 for Claude call sites that need Anthropic-only options
(that provider's default base already ends in /v1).
Remove the gateway import once nothing uses it, and delete the
AI_GATEWAY_API_KEY read (keep the variable itself for me to revoke).
OpenAI-compatible clients: base URL https://api-pr-441.preview.experientiallabs.ai/v1 and
Authorization: Bearer $EXPLABS_API_KEY (the ONLY header needed). An
Anthropic SDK appends /v1/messages itself: its base is the bare origin
https://api-pr-441.preview.experientiallabs.ai (never .../v1). A coding agent pointed at Vercel AI Gateway: follow that
agent's section at https://pr-441.preview.experientiallabs.ai/docs/coding-agents instead of guessing.
Store my key as EXPLABS_API_KEY in this project's env scheme (.env plus the
example file entry), never in code. Strip every Vercel AI Gateway-only field before
the first call: left in, each is ignored or refused with 400
unsupported_parameter naming it. Do NOT delete the old Vercel AI Gateway keys from
.env, secret stores or CI: stop reading them; I revoke them myself. A key
committed in a tracked file is already leaked: flag it and remove it there.
4. Map model ids. GET https://api-pr-441.preview.experientiallabs.ai/api/v1/models with my key (free, spends no
credits): every entry carries canonical_slug, the plain slug the gateway
serves. Vercel ids are creator/model (anthropic/claude-sonnet-5,
openai/gpt-6-astra): match canonical_slug against the part after the slash.
One obvious candidate: use it and note it in the report. Anything else:
ask me ONE question listing each id with your best candidates, then wait.
Never guess a model I did not pick. Use canonical_slug EXACTLY as returned.
5. Move cost and usage reads. Every chat / Responses reply carries usage.cost
(USD charged) and usage.is_byok; on a stream it rides the final usage chunk
when the request sets stream_options: {"include_usage": true}. A request
sent with Idempotency-Key carries no usage.cost, so read it as optional.
Same key, under https://api-pr-441.preview.experientiallabs.ai/api/v1:
GET /generation?id=<the x-request-id response header> one request
GET /usage settled per-request rows for billing (org-wide; filter
?api_key_id= or attribution_label)
GET /credits balance GET /activity per-day rollup
Replace the old cost and usage code with these; show me the diff first.
AI SDK code that read providerMetadata.gateway (cost, routing) reads
usage.cost or the Cost API instead.
6. Provider keys (bring your own key). Vercel may hold my provider keys (dashboard BYOK, or providerOptions.gateway.byok in requests). Ask whether to connect
them so those calls bill my own provider accounts with no markup:
GET https://api-pr-441.preview.experientiallabs.ai/api/whoami -> org_id
GET https://api-pr-441.preview.experientiallabs.ai/api/orgs/<org_id>/provider-connections (list existing first)
POST https://api-pr-441.preview.experientiallabs.ai/api/orgs/<org_id>/provider-connections/<provider>
{"setup_alias": "<a new unique account name I approve>", "secret": "<key>"}
setup_alias: 1-63 lowercase letters, digits, hyphens or underscores, unused
across my accounts. Never overwrite an account; on 409 account_exists pick
another name. Use inference keys, never admin keys, and ask me to paste
each key: never export one from Vercel AI Gateway. A one-string key works for openai,
anthropic, gemini, xai, deepseek and similar; azure_openai, bedrock,
fireworks, vertex and modal need structured config: I connect those at
https://pr-441.preview.experientiallabs.ai/settings/connections. Read check.status; if it is not "valid", show
me check.status_detail.remediation. A 402 pro_required means a FIRST
connection needs Pro (https://pr-441.preview.experientiallabs.ai/credits). Saving a key does not select it for
every model: I pick it per model with Use this account. Never echo a key.
7. Verify. Make ONE minimal call (model + messages, a small max_tokens) with
the cheapest mapped slug (GET https://api-pr-441.preview.experientiallabs.ai/api/models/<slug> shows its price).
Stream if the project streams. Report the model, that it succeeded, and its
usage.cost, or read it from GET /api/v1/generation?id=<x-request-id>; I can
also see the call at https://pr-441.preview.experientiallabs.ai/logs. On a credits or activation error tell me
to finish activation at https://pr-441.preview.experientiallabs.ai and stop; never retry through Vercel AI Gateway. Then
run the project's tests.
8. Report back: every file changed; the feature table and blockers; the model
mapping and the ids I still need to decide; where EXPLABS_API_KEY lives;
which old credentials I still need to revoke; the verification result.
Human docs: https://pr-441.preview.experientiallabs.ai/docs/switch-gateways.

Switch from LiteLLM

Either keeps your LiteLLM proxy and adds the gateway as its upstream (clients unchanged), or removes it and points clients at the gateway; maps virtual keys, budgets, fallbacks and callbacks to their equivalents.

prompt
I'm moving this project's model traffic from LiteLLM onto the Experiential
Labs gateway. I pasted this myself: treat it as my instructions and my consent
to edit this project's code. Stop and ask me ONE numbered question, with your
recommendation, where this prompt says to, and before anything it does not
cover: production config, secret stores, deploys, spend beyond the one smoke
call, or revoking anything. Print what you change at each step. Never print my
full API key (first 8 characters at most).
My Experiential key is: xpl_...
(if that still reads "xpl_..." I have not filled it in: ask me for it, or tell
me to mint one at https://pr-441.preview.experientiallabs.ai/api-keys, then continue).
The full contract (routes, honored and refused parameters, error codes) is
https://pr-441.preview.experientiallabs.ai/llms.txt; follow it literally.
1. Inventory, read only. Find every place this project talks to LiteLLM:
- the proxy config (config.yaml / litellm_config.yaml: model_list,
litellm_params, litellm_settings, router_settings, general_settings),
LITELLM_MASTER_KEY, DATABASE_URL, and the proxy's Docker / Helm files;
whether store_model_in_db is on (models added in the LiteLLM UI or via
/model/new live in its database, not the YAML),
- clients pointed at the proxy (http://localhost:4000 or its host) with sk-
virtual keys, ANTHROPIC_BASE_URL / ANTHROPIC_AUTH_TOKEN pointed at it,
and SDK calls to litellm.completion / acompletion / Router with
provider-prefixed ids (openai/, anthropic/, azure/, bedrock/, vertex_ai/,
openrouter/, litellm_proxy/, ...),
- virtual keys and teams (models, max_budget, budget_duration, tpm_limit,
rpm_limit), fallbacks and context_window_fallbacks, caching, the request
"metadata" field, and success_callback / failure_callback integrations.
Also list hardcoded model ids, every LiteLLM-specific header and body field,
retry / fallback / cache settings, coding-agent configs pointed at it, and
code that reads cost or usage from it. Show me the files and call sites
before changing anything. Inspect secret references, never their values.
2. Map LiteLLM's features before moving anything.
- Virtual keys, teams and budgets: under (a) below they stay in LiteLLM;
under (b) they become provisioning-key minted keys and budgets.
- fallbacks across deployments of one model, retries, routing_strategy: the
gateway does this itself; fallbacks across models and
context_window_fallbacks: keep the primary and tell me.
- drop_params: the gateway refuses or ignores unknown fields itself; the
request fields the project sends still need stripping (step 3).
- Request "metadata" (tags, trace ids): X-Explabs-Tags /
X-Explabs-Session-Id. With overwrite_user_with_key_hash on, the "user" the
provider saw was the key hash, not my end user.
- Callbacks: Langfuse becomes the gateway's live export; list the others
for me to decide.
The gateway's own equivalents, for the table:
- Fallback, retries and load balancing across providers of one model: the
gateway routes and fails over itself (GET https://api-pr-441.preview.experientiallabs.ai/api/models/<slug>/waterfall
shows the order). A fallback list ACROSS models has no per-request
equivalent.
- Keys per team or customer, spend caps, budgets: a PROVISIONING key (an org
admin mints the first at https://pr-441.preview.experientiallabs.ai/api-keys) mints inference keys with a
daily USD cap (POST https://api-pr-441.preview.experientiallabs.ai/api/v1/keys {"name", "limit"}) and sets monthly
caps per key or identity (PUT https://api-pr-441.preview.experientiallabs.ai/api/orgs/<org_id>/budgets).
- Metadata: the X-Explabs-Tags header (a JSON object, at most 16 labels, on
chat, Responses and Messages calls; never secrets or personal data). End
user: safety_identifier (alias user) in the body. Session or
conversation: the X-Explabs-Session-Id header.
- Logs and dashboards: https://pr-441.preview.experientiallabs.ai/logs and https://pr-441.preview.experientiallabs.ai/spend, the Cost API (step 5),
and live export of every call to my Langfuse project
(PUT https://api-pr-441.preview.experientiallabs.ai/api/orgs/<org_id>/observation-export).
- Privacy: "provider": {"zdr": true} per request, or account-wide
PUT https://api-pr-441.preview.experientiallabs.ai/api/orgs/<org_id>/zero-data-retention (Pro); processing
regions are in Settings, Data controls (https://pr-441.preview.experientiallabs.ai/settings/data-controls).
- Content guardrails: per identity, on the Guardrails tab at https://pr-441.preview.experientiallabs.ai/api-keys.
- Prompt caching: provider prompt caching applies where the route supports
it; Anthropic cache_control markers are kept on supported routes and
disclosed as not forwarded elsewhere (default five-minute markers; a
one-hour marker is refused on platform credits).
- No equivalent: gateway-side response or semantic caching, hosted prompt
templates, metadata-based routing between models, and arbitrary
request/response transforms.
Show me a table: feature in use -> equivalent (or "none") -> what changes.
Then ask me ONE question: proceed with this plan, adjust it, or stop. Wait.
A feature I depend on with no equivalent is a blocker: say so, never drop it
silently.
3. Point the clients at the gateway.
First ask me ONE question, with your recommendation, then wait:
(a) keep LiteLLM in front and make the gateway its upstream: clients and
virtual keys stay unchanged; each model_list entry becomes
- model_name: <the name my clients already send>
litellm_params:
model: openai/<slug>
api_base: https://api-pr-441.preview.experientiallabs.ai/v1
api_key: os.environ/EXPLABS_API_KEY
(recommended when several teams hold LiteLLM virtual keys). With
store_model_in_db on, change the models where they live (the LiteLLM
UI or its model API), not only the YAML. LiteLLM does not know
catalog prices, so its spend tracking reads 0 for these models
unless I add custom pricing: read cost from the gateway (step 5);
(b) remove LiteLLM from the request path and point the clients at the
gateway directly (recommended for a single app or script).
LiteLLM SDK without the proxy: litellm.completion(model="openai/<slug>",
api_base="https://api-pr-441.preview.experientiallabs.ai/v1", api_key=os.environ["EXPLABS_API_KEY"]). A bare slug
cannot route in LiteLLM; the openai/ prefix selects its OpenAI-compatible
adapter.
OpenAI-compatible clients: base URL https://api-pr-441.preview.experientiallabs.ai/v1 and
Authorization: Bearer $EXPLABS_API_KEY (the ONLY header needed). An
Anthropic SDK appends /v1/messages itself: its base is the bare origin
https://api-pr-441.preview.experientiallabs.ai (never .../v1). A coding agent pointed at LiteLLM: follow that
agent's section at https://pr-441.preview.experientiallabs.ai/docs/coding-agents instead of guessing.
Store my key as EXPLABS_API_KEY in this project's env scheme (.env plus the
example file entry), never in code. Strip every LiteLLM-only field before
the first call: left in, each is ignored or refused with 400
unsupported_parameter naming it. Do NOT delete the old LiteLLM keys from
.env, secret stores or CI: stop reading them; I revoke them myself. A key
committed in a tracked file is already leaked: flag it and remove it there.
4. Map model ids. GET https://api-pr-441.preview.experientiallabs.ai/api/v1/models with my key (free, spends no
credits): every entry carries canonical_slug, the plain slug the gateway
serves. LiteLLM ids carry a provider prefix (openai/gpt-6-astra,
anthropic/claude-..., openrouter/<vendor>/<model>): strip the prefix (and an
OpenRouter vendor) and match canonical_slug. Under (a) keep each model_name
the clients send and change only litellm_params.model to openai/<slug>.
One obvious candidate: use it and note it in the report. Anything else:
ask me ONE question listing each id with your best candidates, then wait.
Never guess a model I did not pick. Use canonical_slug EXACTLY as returned.
5. Move cost and usage reads. Every chat / Responses reply carries usage.cost
(USD charged) and usage.is_byok; on a stream it rides the final usage chunk
when the request sets stream_options: {"include_usage": true}. A request
sent with Idempotency-Key carries no usage.cost, so read it as optional.
Same key, under https://api-pr-441.preview.experientiallabs.ai/api/v1:
GET /generation?id=<the x-request-id response header> one request
GET /usage settled per-request rows for billing (org-wide; filter
?api_key_id= or attribution_label)
GET /credits balance GET /activity per-day rollup
Replace the old cost and usage code with these; show me the diff first.
Code that read LiteLLM spend (/key/info, /team/info, /user/info, the spend
logs) reads the Cost API instead; under (a) both exist, and the gateway's
is the one that matches the bill.
6. Provider keys (bring your own key). litellm_params may reference my provider keys (OPENAI_API_KEY, ANTHROPIC_API_KEY, AZURE_API_KEY, AWS credentials, ...). Ask whether to connect
them so those calls bill my own provider accounts with no markup:
GET https://api-pr-441.preview.experientiallabs.ai/api/whoami -> org_id
GET https://api-pr-441.preview.experientiallabs.ai/api/orgs/<org_id>/provider-connections (list existing first)
POST https://api-pr-441.preview.experientiallabs.ai/api/orgs/<org_id>/provider-connections/<provider>
{"setup_alias": "<a new unique account name I approve>", "secret": "<key>"}
setup_alias: 1-63 lowercase letters, digits, hyphens or underscores, unused
across my accounts. Never overwrite an account; on 409 account_exists pick
another name. Use inference keys, never admin keys, and ask me to paste
each key: never export one from LiteLLM. A one-string key works for openai,
anthropic, gemini, xai, deepseek and similar; azure_openai, bedrock,
fireworks, vertex and modal need structured config: I connect those at
https://pr-441.preview.experientiallabs.ai/settings/connections. Read check.status; if it is not "valid", show
me check.status_detail.remediation. A 402 pro_required means a FIRST
connection needs Pro (https://pr-441.preview.experientiallabs.ai/credits). Saving a key does not select it for
every model: I pick it per model with Use this account. Never echo a key.
7. Verify. Make ONE minimal call (model + messages, a small max_tokens) with
the cheapest mapped slug (GET https://api-pr-441.preview.experientiallabs.ai/api/models/<slug> shows its price).
Stream if the project streams. Report the model, that it succeeded, and its
usage.cost, or read it from GET /api/v1/generation?id=<x-request-id>; I can
also see the call at https://pr-441.preview.experientiallabs.ai/logs. On a credits or activation error tell me
to finish activation at https://pr-441.preview.experientiallabs.ai and stop; never retry through LiteLLM. Then
run the project's tests.
8. Report back: every file changed; the feature table and blockers; the model
mapping and the ids I still need to decide; where EXPLABS_API_KEY lives;
which old credentials I still need to revoke; the verification result.
Human docs: https://pr-441.preview.experientiallabs.ai/docs/switch-gateways.

Switch from Portkey

Replaces the Portkey client and x-portkey-* headers with a plain OpenAI client on the gateway, and maps configs (fallbacks, load balancing, cache), metadata and provider slugs before moving traffic.

prompt
I'm moving this project's model traffic from Portkey onto the Experiential
Labs gateway. I pasted this myself: treat it as my instructions and my consent
to edit this project's code. Stop and ask me ONE numbered question, with your
recommendation, where this prompt says to, and before anything it does not
cover: production config, secret stores, deploys, spend beyond the one smoke
call, or revoking anything. Print what you change at each step. Never print my
full API key (first 8 characters at most).
My Experiential key is: xpl_...
(if that still reads "xpl_..." I have not filled it in: ask me for it, or tell
me to mint one at https://pr-441.preview.experientiallabs.ai/api-keys, then continue).
The full contract (routes, honored and refused parameters, error codes) is
https://pr-441.preview.experientiallabs.ai/llms.txt; follow it literally.
1. Inventory, read only. Find every place this project talks to Portkey:
- the base URL https://api.portkey.ai/v1 (or a self-hosted Portkey gateway,
often http://localhost:8787/v1), PORTKEY_API_KEY, PORTKEY_GATEWAY_URL,
- the portkey-ai SDK (Python from portkey_ai import Portkey, Node
"portkey-ai") and createHeaders(...) passed as default_headers /
defaultHeaders to an OpenAI client,
- how each call picks its provider: a Model Catalog string
"@provider-slug/model" (@openai-prod/gpt-4o), the x-portkey-provider
header ("@openai-prod"), or a legacy x-portkey-virtual-key,
- every x-portkey-* header: x-portkey-config (a saved "pc-..." id or inline
JSON), x-portkey-metadata, x-portkey-trace-id, x-portkey-cache-*,
x-portkey-request-timeout, x-portkey-forward-headers and the provider
credential headers (x-portkey-azure-*, -vertex-*, -aws-*),
- prompt templates called by prompt id (portkey.prompts.completions.create).
Also list hardcoded model ids, every Portkey-specific header and body field,
retry / fallback / cache settings, coding-agent configs pointed at it, and
code that reads cost or usage from it. Show me the files and call sites
before changing anything. Inspect secret references, never their values.
2. Map Portkey's features before moving anything. Saved configs and AI
Providers live in Portkey, not in this repo: ask me to export each config
JSON the project uses, and read inline ones from the code.
- strategy.mode "fallback" / "loadbalance" across providers of one model,
and retry: the gateway does this itself. Targets that name DIFFERENT
models, and mode "conditional" (routing on metadata or params): no
equivalent; keep one model and raise it.
- override_params / default_params: move the values into the request
itself (show me each one); drop_params: strip those fields in code.
- cache (simple or semantic): no equivalent; raise it if hit rates matter.
- input_guardrails / output_guardrails: Experiential guardrails (below);
list each check so I can recreate it.
- x-portkey-metadata: X-Explabs-Tags, with _user as safety_identifier.
x-portkey-trace-id: X-Explabs-Session-Id, or a W3C traceparent for the
Langfuse export.
- Prompt templates: no equivalent; ask me before inlining a template into
code.
The gateway's own equivalents, for the table:
- Fallback, retries and load balancing across providers of one model: the
gateway routes and fails over itself (GET https://api-pr-441.preview.experientiallabs.ai/api/models/<slug>/waterfall
shows the order). A fallback list ACROSS models has no per-request
equivalent.
- Keys per team or customer, spend caps, budgets: a PROVISIONING key (an org
admin mints the first at https://pr-441.preview.experientiallabs.ai/api-keys) mints inference keys with a
daily USD cap (POST https://api-pr-441.preview.experientiallabs.ai/api/v1/keys {"name", "limit"}) and sets monthly
caps per key or identity (PUT https://api-pr-441.preview.experientiallabs.ai/api/orgs/<org_id>/budgets).
- Metadata: the X-Explabs-Tags header (a JSON object, at most 16 labels, on
chat, Responses and Messages calls; never secrets or personal data). End
user: safety_identifier (alias user) in the body. Session or
conversation: the X-Explabs-Session-Id header.
- Logs and dashboards: https://pr-441.preview.experientiallabs.ai/logs and https://pr-441.preview.experientiallabs.ai/spend, the Cost API (step 5),
and live export of every call to my Langfuse project
(PUT https://api-pr-441.preview.experientiallabs.ai/api/orgs/<org_id>/observation-export).
- Privacy: "provider": {"zdr": true} per request, or account-wide
PUT https://api-pr-441.preview.experientiallabs.ai/api/orgs/<org_id>/zero-data-retention (Pro); processing
regions are in Settings, Data controls (https://pr-441.preview.experientiallabs.ai/settings/data-controls).
- Content guardrails: per identity, on the Guardrails tab at https://pr-441.preview.experientiallabs.ai/api-keys.
- Prompt caching: provider prompt caching applies where the route supports
it; Anthropic cache_control markers are kept on supported routes and
disclosed as not forwarded elsewhere (default five-minute markers; a
one-hour marker is refused on platform credits).
- No equivalent: gateway-side response or semantic caching, hosted prompt
templates, metadata-based routing between models, and arbitrary
request/response transforms.
Show me a table: feature in use -> equivalent (or "none") -> what changes.
Then ask me ONE question: proceed with this plan, adjust it, or stop. Wait.
A feature I depend on with no equivalent is a blocker: say so, never drop it
silently.
3. Point the clients at the gateway.
The Portkey client is OpenAI-shaped: replace Portkey(...), or an OpenAI
client built with createHeaders / PORTKEY_GATEWAY_URL, by a plain OpenAI
client (base URL https://api-pr-441.preview.experientiallabs.ai/v1, apiKey from EXPLABS_API_KEY) and keep the call
sites. Remove every x-portkey-* header and the config attachment.
OpenAI-compatible clients: base URL https://api-pr-441.preview.experientiallabs.ai/v1 and
Authorization: Bearer $EXPLABS_API_KEY (the ONLY header needed). An
Anthropic SDK appends /v1/messages itself: its base is the bare origin
https://api-pr-441.preview.experientiallabs.ai (never .../v1). A coding agent pointed at Portkey: follow that
agent's section at https://pr-441.preview.experientiallabs.ai/docs/coding-agents instead of guessing.
Store my key as EXPLABS_API_KEY in this project's env scheme (.env plus the
example file entry), never in code. Strip every Portkey-only field before
the first call: left in, each is ignored or refused with 400
unsupported_parameter naming it. Do NOT delete the old Portkey keys from
.env, secret stores or CI: stop reading them; I revoke them myself. A key
committed in a tracked file is already leaked: flag it and remove it there.
4. Map model ids. GET https://api-pr-441.preview.experientiallabs.ai/api/v1/models with my key (free, spends no
credits): every entry carries canonical_slug, the plain slug the gateway
serves. Portkey sends the provider's own model id (gpt-4o,
claude-sonnet-4-...) and picks the provider from the slug; a model may be
written "@provider-slug/model". Take the model part, read the model a
config's override_params sets when there is one, and match canonical_slug.
One obvious candidate: use it and note it in the report. Anything else:
ask me ONE question listing each id with your best candidates, then wait.
Never guess a model I did not pick. Use canonical_slug EXACTLY as returned.
5. Move cost and usage reads. Every chat / Responses reply carries usage.cost
(USD charged) and usage.is_byok; on a stream it rides the final usage chunk
when the request sets stream_options: {"include_usage": true}. A request
sent with Idempotency-Key carries no usage.cost, so read it as optional.
Same key, under https://api-pr-441.preview.experientiallabs.ai/api/v1:
GET /generation?id=<the x-request-id response header> one request
GET /usage settled per-request rows for billing (org-wide; filter
?api_key_id= or attribution_label)
GET /credits balance GET /activity per-day rollup
Replace the old cost and usage code with these; show me the diff first.
Portkey analytics and log exports have no counterpart in code: the Cost API
and https://pr-441.preview.experientiallabs.ai/spend replace them.
6. Provider keys (bring your own key). Portkey AI Providers (and legacy virtual keys) hold my provider credentials. Ask whether to connect
them so those calls bill my own provider accounts with no markup:
GET https://api-pr-441.preview.experientiallabs.ai/api/whoami -> org_id
GET https://api-pr-441.preview.experientiallabs.ai/api/orgs/<org_id>/provider-connections (list existing first)
POST https://api-pr-441.preview.experientiallabs.ai/api/orgs/<org_id>/provider-connections/<provider>
{"setup_alias": "<a new unique account name I approve>", "secret": "<key>"}
setup_alias: 1-63 lowercase letters, digits, hyphens or underscores, unused
across my accounts. Never overwrite an account; on 409 account_exists pick
another name. Use inference keys, never admin keys, and ask me to paste
each key: never export one from Portkey. A one-string key works for openai,
anthropic, gemini, xai, deepseek and similar; azure_openai, bedrock,
fireworks, vertex and modal need structured config: I connect those at
https://pr-441.preview.experientiallabs.ai/settings/connections. Read check.status; if it is not "valid", show
me check.status_detail.remediation. A 402 pro_required means a FIRST
connection needs Pro (https://pr-441.preview.experientiallabs.ai/credits). Saving a key does not select it for
every model: I pick it per model with Use this account. Never echo a key.
7. Verify. Make ONE minimal call (model + messages, a small max_tokens) with
the cheapest mapped slug (GET https://api-pr-441.preview.experientiallabs.ai/api/models/<slug> shows its price).
Stream if the project streams. Report the model, that it succeeded, and its
usage.cost, or read it from GET /api/v1/generation?id=<x-request-id>; I can
also see the call at https://pr-441.preview.experientiallabs.ai/logs. On a credits or activation error tell me
to finish activation at https://pr-441.preview.experientiallabs.ai and stop; never retry through Portkey. Then
run the project's tests.
8. Report back: every file changed; the feature table and blockers; the model
mapping and the ids I still need to decide; where EXPLABS_API_KEY lives;
which old credentials I still need to revoke; the verification result.
Human docs: https://pr-441.preview.experientiallabs.ai/docs/switch-gateways.

Switch from Helicone

Moves traffic off Helicone's AI Gateway or provider proxies, maps Helicone-* properties, user and session headers to request tags, and offers the provider keys Helicone forwarded as bring-your-own-key accounts.

prompt
I'm moving this project's model traffic from Helicone onto the Experiential
Labs gateway. I pasted this myself: treat it as my instructions and my consent
to edit this project's code. Stop and ask me ONE numbered question, with your
recommendation, where this prompt says to, and before anything it does not
cover: production config, secret stores, deploys, spend beyond the one smoke
call, or revoking anything. Print what you change at each step. Never print my
full API key (first 8 characters at most).
My Experiential key is: xpl_...
(if that still reads "xpl_..." I have not filled it in: ask me for it, or tell
me to mint one at https://pr-441.preview.experientiallabs.ai/api-keys, then continue).
The full contract (routes, honored and refused parameters, error codes) is
https://pr-441.preview.experientiallabs.ai/llms.txt; follow it literally.
1. Inventory, read only. Find every place this project talks to Helicone, and
which of its two shapes each call uses:
- the AI Gateway: base URL https://ai-gateway.helicone.ai with
HELICONE_API_KEY as the bearer, and model strings that pin providers
("gpt-4o-mini/openai"), name a deployment ("gpt-4o-mini/azure/<id>"),
chain fallbacks with commas, or exclude providers with "!",
- the provider proxies: oai.helicone.ai, anthropic.helicone.ai,
gateway.helicone.ai or any other *.helicone.ai base, where MY provider
key (OPENAI_API_KEY, ANTHROPIC_API_KEY) is the bearer and a Helicone-Auth
header carries the Helicone key,
- every Helicone-* header: Helicone-Property-*, Helicone-User-Id,
Helicone-Session-Id / -Path / -Name, Helicone-Cache-Enabled,
Helicone-Retry-Enabled, Helicone-Fallbacks, Helicone-Prompt-Id,
Helicone-RateLimit-Policy, Helicone-Target-Url,
- async logging that records calls made elsewhere (@helicone/helpers,
HeliconeManualLogger, OpenLLMetry exporters).
Also list hardcoded model ids, every Helicone-specific header and body field,
retry / fallback / cache settings, coding-agent configs pointed at it, and
code that reads cost or usage from it. Show me the files and call sites
before changing anything. Inspect secret references, never their values.
2. Map Helicone's features before moving anything.
- Proxy calls ran on MY provider key and Helicone only observed: after the
move they run on Experiential credits unless I connect that key in step 6.
Say this plainly before I approve.
- Provider pins, fallback chains and exclusions within one model: the
gateway routes and fails over itself; a chain across DIFFERENT models:
keep the first model and tell me. An exclusion I rely on for policy
reasons is a data-control question: ask me.
- Helicone-Property-*: X-Explabs-Tags. Helicone-User-Id: safety_identifier.
Helicone-Session-Id: X-Explabs-Session-Id.
- Helicone-Cache-Enabled: no equivalent; raise it if hit rates matter.
Helicone-Retry-Enabled / Helicone-Fallbacks: the gateway does this
itself. Helicone-RateLimit-Policy: keys and budgets (below).
- Helicone-Prompt-Id and Helicone prompt management: no equivalent; raise it.
- Async logging of calls that should stay where they are: do not move
those; seeing them without moving inference is the "Capture traffic"
prompt (https://pr-441.preview.experientiallabs.ai/docs/setup-prompts#capture-traffic).
The gateway's own equivalents, for the table:
- Fallback, retries and load balancing across providers of one model: the
gateway routes and fails over itself (GET https://api-pr-441.preview.experientiallabs.ai/api/models/<slug>/waterfall
shows the order). A fallback list ACROSS models has no per-request
equivalent.
- Keys per team or customer, spend caps, budgets: a PROVISIONING key (an org
admin mints the first at https://pr-441.preview.experientiallabs.ai/api-keys) mints inference keys with a
daily USD cap (POST https://api-pr-441.preview.experientiallabs.ai/api/v1/keys {"name", "limit"}) and sets monthly
caps per key or identity (PUT https://api-pr-441.preview.experientiallabs.ai/api/orgs/<org_id>/budgets).
- Metadata: the X-Explabs-Tags header (a JSON object, at most 16 labels, on
chat, Responses and Messages calls; never secrets or personal data). End
user: safety_identifier (alias user) in the body. Session or
conversation: the X-Explabs-Session-Id header.
- Logs and dashboards: https://pr-441.preview.experientiallabs.ai/logs and https://pr-441.preview.experientiallabs.ai/spend, the Cost API (step 5),
and live export of every call to my Langfuse project
(PUT https://api-pr-441.preview.experientiallabs.ai/api/orgs/<org_id>/observation-export).
- Privacy: "provider": {"zdr": true} per request, or account-wide
PUT https://api-pr-441.preview.experientiallabs.ai/api/orgs/<org_id>/zero-data-retention (Pro); processing
regions are in Settings, Data controls (https://pr-441.preview.experientiallabs.ai/settings/data-controls).
- Content guardrails: per identity, on the Guardrails tab at https://pr-441.preview.experientiallabs.ai/api-keys.
- Prompt caching: provider prompt caching applies where the route supports
it; Anthropic cache_control markers are kept on supported routes and
disclosed as not forwarded elsewhere (default five-minute markers; a
one-hour marker is refused on platform credits).
- No equivalent: gateway-side response or semantic caching, hosted prompt
templates, metadata-based routing between models, and arbitrary
request/response transforms.
Show me a table: feature in use -> equivalent (or "none") -> what changes.
Then ask me ONE question: proceed with this plan, adjust it, or stop. Wait.
A feature I depend on with no equivalent is a blocker: say so, never drop it
silently.
3. Point the clients at the gateway.
Replace the Helicone base URL with https://api-pr-441.preview.experientiallabs.ai/v1 (Anthropic SDKs: https://api-pr-441.preview.experientiallabs.ai) and
the bearer with EXPLABS_API_KEY; remove Helicone-Auth and every Helicone-*
header. Do not chain the gateway behind a Helicone proxy.
OpenAI-compatible clients: base URL https://api-pr-441.preview.experientiallabs.ai/v1 and
Authorization: Bearer $EXPLABS_API_KEY (the ONLY header needed). An
Anthropic SDK appends /v1/messages itself: its base is the bare origin
https://api-pr-441.preview.experientiallabs.ai (never .../v1). A coding agent pointed at Helicone: follow that
agent's section at https://pr-441.preview.experientiallabs.ai/docs/coding-agents instead of guessing.
Store my key as EXPLABS_API_KEY in this project's env scheme (.env plus the
example file entry), never in code. Strip every Helicone-only field before
the first call: left in, each is ignored or refused with 400
unsupported_parameter naming it. Do NOT delete the old Helicone keys from
.env, secret stores or CI: stop reading them; I revoke them myself. A key
committed in a tracked file is already leaked: flag it and remove it there.
4. Map model ids. GET https://api-pr-441.preview.experientiallabs.ai/api/v1/models with my key (free, spends no
credits): every entry carries canonical_slug, the plain slug the gateway
serves. On the proxies the model is the provider's own id; on the AI
Gateway take the part before the first "/" of the first entry in a chain
(gpt-4o-mini/openai -> gpt-4o-mini) and match canonical_slug.
One obvious candidate: use it and note it in the report. Anything else:
ask me ONE question listing each id with your best candidates, then wait.
Never guess a model I did not pick. Use canonical_slug EXACTLY as returned.
5. Move cost and usage reads. Every chat / Responses reply carries usage.cost
(USD charged) and usage.is_byok; on a stream it rides the final usage chunk
when the request sets stream_options: {"include_usage": true}. A request
sent with Idempotency-Key carries no usage.cost, so read it as optional.
Same key, under https://api-pr-441.preview.experientiallabs.ai/api/v1:
GET /generation?id=<the x-request-id response header> one request
GET /usage settled per-request rows for billing (org-wide; filter
?api_key_id= or attribution_label)
GET /credits balance GET /activity per-day rollup
Replace the old cost and usage code with these; show me the diff first.
6. Provider keys (bring your own key). The proxies used my own provider keys, and the AI Gateway may hold BYOK keys in Helicone's Provider Settings. Ask whether to connect
them so those calls bill my own provider accounts with no markup:
GET https://api-pr-441.preview.experientiallabs.ai/api/whoami -> org_id
GET https://api-pr-441.preview.experientiallabs.ai/api/orgs/<org_id>/provider-connections (list existing first)
POST https://api-pr-441.preview.experientiallabs.ai/api/orgs/<org_id>/provider-connections/<provider>
{"setup_alias": "<a new unique account name I approve>", "secret": "<key>"}
setup_alias: 1-63 lowercase letters, digits, hyphens or underscores, unused
across my accounts. Never overwrite an account; on 409 account_exists pick
another name. Use inference keys, never admin keys, and ask me to paste
each key: never export one from Helicone. A one-string key works for openai,
anthropic, gemini, xai, deepseek and similar; azure_openai, bedrock,
fireworks, vertex and modal need structured config: I connect those at
https://pr-441.preview.experientiallabs.ai/settings/connections. Read check.status; if it is not "valid", show
me check.status_detail.remediation. A 402 pro_required means a FIRST
connection needs Pro (https://pr-441.preview.experientiallabs.ai/credits). Saving a key does not select it for
every model: I pick it per model with Use this account. Never echo a key.
7. Verify. Make ONE minimal call (model + messages, a small max_tokens) with
the cheapest mapped slug (GET https://api-pr-441.preview.experientiallabs.ai/api/models/<slug> shows its price).
Stream if the project streams. Report the model, that it succeeded, and its
usage.cost, or read it from GET /api/v1/generation?id=<x-request-id>; I can
also see the call at https://pr-441.preview.experientiallabs.ai/logs. On a credits or activation error tell me
to finish activation at https://pr-441.preview.experientiallabs.ai and stop; never retry through Helicone. Then
run the project's tests.
8. Report back: every file changed; the feature table and blockers; the model
mapping and the ids I still need to decide; where EXPLABS_API_KEY lives;
which old credentials I still need to revoke; the verification result.
Human docs: https://pr-441.preview.experientiallabs.ai/docs/switch-gateways.

Switch from Databricks AI Gateway

Inventories serving endpoints and AI Gateway governance (rate limits, usage and inference tables, guardrails) and moves only the chat traffic you approve; custom model endpoints and data you must keep in the workspace stay put.

prompt
I'm moving this project's model traffic from Databricks AI Gateway onto the Experiential
Labs gateway. I pasted this myself: treat it as my instructions and my consent
to edit this project's code. Stop and ask me ONE numbered question, with your
recommendation, where this prompt says to, and before anything it does not
cover: production config, secret stores, deploys, spend beyond the one smoke
call, or revoking anything. Print what you change at each step. Never print my
full API key (first 8 characters at most).
My Experiential key is: xpl_...
(if that still reads "xpl_..." I have not filled it in: ask me for it, or tell
me to mint one at https://pr-441.preview.experientiallabs.ai/api-keys, then continue).
The full contract (routes, honored and refused parameters, error codes) is
https://pr-441.preview.experientiallabs.ai/llms.txt; follow it literally.
1. Inventory, read only. Find every place this project talks to Databricks
model serving, and which gateway generation each call uses:
- workspace hosts (*.cloud.databricks.com, *.azuredatabricks.net,
*.gcp.databricks.com), DATABRICKS_HOST, DATABRICKS_TOKEN, OAuth
service-principal settings and ~/.databrickscfg profiles (never print
their values),
- Unity AI Gateway calls: base URL <workspace>/ai-gateway/mlflow/v1 (model
= a Unity Catalog name such as system.ai.<model>), and its native OpenAI,
Anthropic and Gemini API paths,
- legacy serving-endpoint calls: base URL <workspace>/serving-endpoints
(model = the endpoint name), /serving-endpoints/<name>/invocations,
WorkspaceClient().serving_endpoints.get_open_ai_client(),
mlflow.deployments.get_deploy_client("databricks"), databricks-langchain
ChatDatabricks,
- ai_query(...) in SQL, notebooks and jobs, and the governance in use:
budgets, rate limits, usage tracking, inference tables, guardrails,
fallbacks, traffic splits, request tags (the
Databricks-Ai-Gateway-Request-Tags header) and extra_body usage_context.
Also list hardcoded model ids, every Databricks AI Gateway-specific header and body field,
retry / fallback / cache settings, coding-agent configs pointed at it, and
code that reads cost or usage from it. Show me the files and call sites
before changing anything. Inspect secret references, never their values.
2. Map Databricks AI Gateway's features before moving anything.
- Scope first: ask me which call sites move. Pay-per-token system.ai models
and external-model endpoints can move; custom model serving endpoints
(my own registered models and agents) and ai_query batch pipelines stay
on Databricks unless I say otherwise. A custom endpoint I want served here
is a custom model (https://pr-441.preview.experientiallabs.ai/docs/integrate#custom-endpoints).
- Governance: if prompts and responses must stay inside the workspace
(inference tables, Unity Catalog lineage, private networking or a
compliance boundary), moving that traffic out is a blocker; ask before
proposing it.
- Budgets, rate limits and endpoint permissions: keys and budgets (below).
- Usage tables (system.ai_gateway.usage, system.serving.endpoint_usage):
the Cost API export (step 5); offer a scheduled job that lands
/api/v1/usage rows in a Delta table if I want them in Databricks.
- Inference tables (payload logging): the prompt and response storage
switch at https://pr-441.preview.experientiallabs.ai/settings/data-controls plus the Langfuse export.
- Request tags / usage_context: X-Explabs-Tags. Fallbacks and traffic
splits across providers of one model: the gateway does this itself.
- Guardrails: Experiential guardrails (below); list each one.
The gateway's own equivalents, for the table:
- Fallback, retries and load balancing across providers of one model: the
gateway routes and fails over itself (GET https://api-pr-441.preview.experientiallabs.ai/api/models/<slug>/waterfall
shows the order). A fallback list ACROSS models has no per-request
equivalent.
- Keys per team or customer, spend caps, budgets: a PROVISIONING key (an org
admin mints the first at https://pr-441.preview.experientiallabs.ai/api-keys) mints inference keys with a
daily USD cap (POST https://api-pr-441.preview.experientiallabs.ai/api/v1/keys {"name", "limit"}) and sets monthly
caps per key or identity (PUT https://api-pr-441.preview.experientiallabs.ai/api/orgs/<org_id>/budgets).
- Metadata: the X-Explabs-Tags header (a JSON object, at most 16 labels, on
chat, Responses and Messages calls; never secrets or personal data). End
user: safety_identifier (alias user) in the body. Session or
conversation: the X-Explabs-Session-Id header.
- Logs and dashboards: https://pr-441.preview.experientiallabs.ai/logs and https://pr-441.preview.experientiallabs.ai/spend, the Cost API (step 5),
and live export of every call to my Langfuse project
(PUT https://api-pr-441.preview.experientiallabs.ai/api/orgs/<org_id>/observation-export).
- Privacy: "provider": {"zdr": true} per request, or account-wide
PUT https://api-pr-441.preview.experientiallabs.ai/api/orgs/<org_id>/zero-data-retention (Pro); processing
regions are in Settings, Data controls (https://pr-441.preview.experientiallabs.ai/settings/data-controls).
- Content guardrails: per identity, on the Guardrails tab at https://pr-441.preview.experientiallabs.ai/api-keys.
- Prompt caching: provider prompt caching applies where the route supports
it; Anthropic cache_control markers are kept on supported routes and
disclosed as not forwarded elsewhere (default five-minute markers; a
one-hour marker is refused on platform credits).
- No equivalent: gateway-side response or semantic caching, hosted prompt
templates, metadata-based routing between models, and arbitrary
request/response transforms.
Show me a table: feature in use -> equivalent (or "none") -> what changes.
Then ask me ONE question: proceed with this plan, adjust it, or stop. Wait.
A feature I depend on with no equivalent is a blocker: say so, never drop it
silently.
3. Point the clients at the gateway, only at the call sites I approved.
An OpenAI client gets base URL https://api-pr-441.preview.experientiallabs.ai/v1 and EXPLABS_API_KEY instead of the
workspace URL and DATABRICKS_TOKEN; ChatDatabricks becomes ChatOpenAI
(langchain-openai) with the same base URL and key; mlflow deployments
clients become a plain OpenAI client. A Unity AI Gateway Responses call
stays a Responses call (the gateway serves /v1/responses). Leave every
DATABRICKS_* setting in place for what stays.
OpenAI-compatible clients: base URL https://api-pr-441.preview.experientiallabs.ai/v1 and
Authorization: Bearer $EXPLABS_API_KEY (the ONLY header needed). An
Anthropic SDK appends /v1/messages itself: its base is the bare origin
https://api-pr-441.preview.experientiallabs.ai (never .../v1). A coding agent pointed at Databricks AI Gateway: follow that
agent's section at https://pr-441.preview.experientiallabs.ai/docs/coding-agents instead of guessing.
Store my key as EXPLABS_API_KEY in this project's env scheme (.env plus the
example file entry), never in code. Strip every Databricks AI Gateway-only field before
the first call: left in, each is ignored or refused with 400
unsupported_parameter naming it. Do NOT delete the old Databricks AI Gateway keys from
.env, secret stores or CI: stop reading them; I revoke them myself. A key
committed in a tracked file is already leaked: flag it and remove it there.
4. Map model ids. GET https://api-pr-441.preview.experientiallabs.ai/api/v1/models with my key (free, spends no
credits): every entry carries canonical_slug, the plain slug the gateway
serves. A Databricks model is a Unity Catalog name or an endpoint name,
not a model id: for each one, find what it serves (system.ai.<model> names
the model; for an endpoint read its served entities or external_model.name
in the workspace UI or GET /api/2.0/serving-endpoints/<name>, with my
approval and my own credentials), then match canonical_slug.
One obvious candidate: use it and note it in the report. Anything else:
ask me ONE question listing each id with your best candidates, then wait.
Never guess a model I did not pick. Use canonical_slug EXACTLY as returned.
5. Move cost and usage reads. Every chat / Responses reply carries usage.cost
(USD charged) and usage.is_byok; on a stream it rides the final usage chunk
when the request sets stream_options: {"include_usage": true}. A request
sent with Idempotency-Key carries no usage.cost, so read it as optional.
Same key, under https://api-pr-441.preview.experientiallabs.ai/api/v1:
GET /generation?id=<the x-request-id response header> one request
GET /usage settled per-request rows for billing (org-wide; filter
?api_key_id= or attribution_label)
GET /credits balance GET /activity per-day rollup
Replace the old cost and usage code with these; show me the diff first.
6. Provider keys (bring your own key). External-model endpoints hold my provider keys in Databricks secrets; never read a Databricks secret yourself. Ask whether to connect
them so those calls bill my own provider accounts with no markup:
GET https://api-pr-441.preview.experientiallabs.ai/api/whoami -> org_id
GET https://api-pr-441.preview.experientiallabs.ai/api/orgs/<org_id>/provider-connections (list existing first)
POST https://api-pr-441.preview.experientiallabs.ai/api/orgs/<org_id>/provider-connections/<provider>
{"setup_alias": "<a new unique account name I approve>", "secret": "<key>"}
setup_alias: 1-63 lowercase letters, digits, hyphens or underscores, unused
across my accounts. Never overwrite an account; on 409 account_exists pick
another name. Use inference keys, never admin keys, and ask me to paste
each key: never export one from Databricks AI Gateway. A one-string key works for openai,
anthropic, gemini, xai, deepseek and similar; azure_openai, bedrock,
fireworks, vertex and modal need structured config: I connect those at
https://pr-441.preview.experientiallabs.ai/settings/connections. Read check.status; if it is not "valid", show
me check.status_detail.remediation. A 402 pro_required means a FIRST
connection needs Pro (https://pr-441.preview.experientiallabs.ai/credits). Saving a key does not select it for
every model: I pick it per model with Use this account. Never echo a key.
7. Verify. Make ONE minimal call (model + messages, a small max_tokens) with
the cheapest mapped slug (GET https://api-pr-441.preview.experientiallabs.ai/api/models/<slug> shows its price).
Stream if the project streams. Report the model, that it succeeded, and its
usage.cost, or read it from GET /api/v1/generation?id=<x-request-id>; I can
also see the call at https://pr-441.preview.experientiallabs.ai/logs. On a credits or activation error tell me
to finish activation at https://pr-441.preview.experientiallabs.ai and stop; never retry through Databricks AI Gateway. Then
run the project's tests.
8. Report back: every file changed; the feature table and blockers; the model
mapping and the ids I still need to decide; where EXPLABS_API_KEY lives;
which old credentials I still need to revoke; the verification result.
Human docs: https://pr-441.preview.experientiallabs.ai/docs/switch-gateways.

Switch from Kong AI Gateway

Either repoints Kong's ai-proxy upstream at the gateway (Kong keeps its auth and policies, clients unchanged) or moves clients off Kong, with the key held in a Kong vault reference, never in committed config.

prompt
I'm moving this project's model traffic from Kong AI Gateway onto the Experiential
Labs gateway. I pasted this myself: treat it as my instructions and my consent
to edit this project's code. Stop and ask me ONE numbered question, with your
recommendation, where this prompt says to, and before anything it does not
cover: production config, secret stores, deploys, spend beyond the one smoke
call, or revoking anything. Print what you change at each step. Never print my
full API key (first 8 characters at most).
My Experiential key is: xpl_...
(if that still reads "xpl_..." I have not filled it in: ask me for it, or tell
me to mint one at https://pr-441.preview.experientiallabs.ai/api-keys, then continue).
The full contract (routes, honored and refused parameters, error codes) is
https://pr-441.preview.experientiallabs.ai/llms.txt; follow it literally.
1. Inventory, read only. Find every place this project talks to Kong AI Gateway:
- Kong configuration: declarative kong.yaml / decK files, Helm values,
KongPlugin / KongClusterPlugin resources, or Konnect, with the ai-proxy or
ai-proxy-advanced plugin and its companions (ai-rate-limiting-advanced,
ai-semantic-cache, ai-prompt-guard, ai-prompt-decorator,
ai-prompt-template, ai-request-transformer, ai-response-transformer),
- per plugin: config.route_type, config.model.provider and name,
config.model.options (upstream_url and provider options),
config.auth (header_name, header_value, allow_override), config.llm_format
and config.logging,
- the routes clients call (the Kong proxy host and route paths) and the
consumer credentials they send (key-auth apikey header, JWT, OIDC),
- provider keys referenced by the plugins and the vault references holding
them.
Also list hardcoded model ids, every Kong AI Gateway-specific header and body field,
retry / fallback / cache settings, coding-agent configs pointed at it, and
code that reads cost or usage from it. Show me the files and call sites
before changing anything. Inspect secret references, never their values.
2. Map Kong AI Gateway's features before moving anything.
- ai-rate-limiting-advanced, ai-prompt-guard, ai-semantic-cache, the
decorators, templates and transformers: under (a) below they keep running
in Kong; under (b) rate limits become keys and budgets, prompt guard
becomes guardrails, and the cache, templates and transformers have no
equivalent.
- ai-proxy-advanced balancing and fallbacks across providers of one model:
the gateway does this itself; across different models: raise it.
- Kong consumers and their credentials: under (a) they stay; under (b)
each consumer that needs its own spend becomes its own Experiential key.
- logging.log_statistics / log_payloads: the Cost API and the Langfuse
export (below).
The gateway's own equivalents, for the table:
- Fallback, retries and load balancing across providers of one model: the
gateway routes and fails over itself (GET https://api-pr-441.preview.experientiallabs.ai/api/models/<slug>/waterfall
shows the order). A fallback list ACROSS models has no per-request
equivalent.
- Keys per team or customer, spend caps, budgets: a PROVISIONING key (an org
admin mints the first at https://pr-441.preview.experientiallabs.ai/api-keys) mints inference keys with a
daily USD cap (POST https://api-pr-441.preview.experientiallabs.ai/api/v1/keys {"name", "limit"}) and sets monthly
caps per key or identity (PUT https://api-pr-441.preview.experientiallabs.ai/api/orgs/<org_id>/budgets).
- Metadata: the X-Explabs-Tags header (a JSON object, at most 16 labels, on
chat, Responses and Messages calls; never secrets or personal data). End
user: safety_identifier (alias user) in the body. Session or
conversation: the X-Explabs-Session-Id header.
- Logs and dashboards: https://pr-441.preview.experientiallabs.ai/logs and https://pr-441.preview.experientiallabs.ai/spend, the Cost API (step 5),
and live export of every call to my Langfuse project
(PUT https://api-pr-441.preview.experientiallabs.ai/api/orgs/<org_id>/observation-export).
- Privacy: "provider": {"zdr": true} per request, or account-wide
PUT https://api-pr-441.preview.experientiallabs.ai/api/orgs/<org_id>/zero-data-retention (Pro); processing
regions are in Settings, Data controls (https://pr-441.preview.experientiallabs.ai/settings/data-controls).
- Content guardrails: per identity, on the Guardrails tab at https://pr-441.preview.experientiallabs.ai/api-keys.
- Prompt caching: provider prompt caching applies where the route supports
it; Anthropic cache_control markers are kept on supported routes and
disclosed as not forwarded elsewhere (default five-minute markers; a
one-hour marker is refused on platform credits).
- No equivalent: gateway-side response or semantic caching, hosted prompt
templates, metadata-based routing between models, and arbitrary
request/response transforms.
Show me a table: feature in use -> equivalent (or "none") -> what changes.
Then ask me ONE question: proceed with this plan, adjust it, or stop. Wait.
A feature I depend on with no equivalent is a blocker: say so, never drop it
silently.
3. Point the clients at the gateway.
First ask me ONE question, with your recommendation, then wait:
(a) keep Kong as the edge and repoint each ai-proxy plugin (or each
ai-proxy-advanced target) at the gateway: clients, consumers and Kong
policies stay unchanged. Per plugin, after checking these names
against the installed Kong version's ai-proxy schema:
config.route_type: llm/v1/chat (llm/v1/responses for Responses)
config.model.provider: openai
config.model.name: <slug>
config.model.options.upstream_url: https://api-pr-441.preview.experientiallabs.ai/v1/chat/completions
(https://api-pr-441.preview.experientiallabs.ai/v1/responses for a responses route)
config.auth.header_name: Authorization
config.auth.header_value: a Kong vault reference whose secret is the
whole value "Bearer <my key>" (a reference replaces the field's
entire value), never the key written into committed config
config.auth.allow_override: false
Ask me which vault (env, AWS / GCP / Azure secret manager, HashiCorp)
holds it. Recommended when Kong enforces auth or policies for other
teams;
(b) move clients off Kong for model traffic and point them at the gateway
directly; other Kong routes stay as they are.
OpenAI-compatible clients: base URL https://api-pr-441.preview.experientiallabs.ai/v1 and
Authorization: Bearer $EXPLABS_API_KEY (the ONLY header needed). An
Anthropic SDK appends /v1/messages itself: its base is the bare origin
https://api-pr-441.preview.experientiallabs.ai (never .../v1). A coding agent pointed at Kong AI Gateway: follow that
agent's section at https://pr-441.preview.experientiallabs.ai/docs/coding-agents instead of guessing.
Store my key as EXPLABS_API_KEY in this project's env scheme (.env plus the
example file entry), never in code. Strip every Kong AI Gateway-only field before
the first call: left in, each is ignored or refused with 400
unsupported_parameter naming it. Do NOT delete the old Kong AI Gateway keys from
.env, secret stores or CI: stop reading them; I revoke them myself. A key
committed in a tracked file is already leaked: flag it and remove it there.
4. Map model ids. GET https://api-pr-441.preview.experientiallabs.ai/api/v1/models with my key (free, spends no
credits): every entry carries canonical_slug, the plain slug the gateway
serves. Kong pins one model per plugin (config.model.name) or lets the
client send it: map each to canonical_slug. Under (a) the route paths the
clients call stay the same.
One obvious candidate: use it and note it in the report. Anything else:
ask me ONE question listing each id with your best candidates, then wait.
Never guess a model I did not pick. Use canonical_slug EXACTLY as returned.
5. Move cost and usage reads. Every chat / Responses reply carries usage.cost
(USD charged) and usage.is_byok; on a stream it rides the final usage chunk
when the request sets stream_options: {"include_usage": true}. A request
sent with Idempotency-Key carries no usage.cost, so read it as optional.
Same key, under https://api-pr-441.preview.experientiallabs.ai/api/v1:
GET /generation?id=<the x-request-id response header> one request
GET /usage settled per-request rows for billing (org-wide; filter
?api_key_id= or attribution_label)
GET /credits balance GET /activity per-day rollup
Replace the old cost and usage code with these; show me the diff first.
6. Provider keys (bring your own key). The plugins authenticate to my providers with my own keys (config.auth, often through a vault). Ask whether to connect
them so those calls bill my own provider accounts with no markup:
GET https://api-pr-441.preview.experientiallabs.ai/api/whoami -> org_id
GET https://api-pr-441.preview.experientiallabs.ai/api/orgs/<org_id>/provider-connections (list existing first)
POST https://api-pr-441.preview.experientiallabs.ai/api/orgs/<org_id>/provider-connections/<provider>
{"setup_alias": "<a new unique account name I approve>", "secret": "<key>"}
setup_alias: 1-63 lowercase letters, digits, hyphens or underscores, unused
across my accounts. Never overwrite an account; on 409 account_exists pick
another name. Use inference keys, never admin keys, and ask me to paste
each key: never export one from Kong AI Gateway. A one-string key works for openai,
anthropic, gemini, xai, deepseek and similar; azure_openai, bedrock,
fireworks, vertex and modal need structured config: I connect those at
https://pr-441.preview.experientiallabs.ai/settings/connections. Read check.status; if it is not "valid", show
me check.status_detail.remediation. A 402 pro_required means a FIRST
connection needs Pro (https://pr-441.preview.experientiallabs.ai/credits). Saving a key does not select it for
every model: I pick it per model with Use this account. Never echo a key.
7. Verify. Make ONE minimal call (model + messages, a small max_tokens) with
the cheapest mapped slug (GET https://api-pr-441.preview.experientiallabs.ai/api/models/<slug> shows its price).
Stream if the project streams. Report the model, that it succeeded, and its
usage.cost, or read it from GET /api/v1/generation?id=<x-request-id>; I can
also see the call at https://pr-441.preview.experientiallabs.ai/logs. On a credits or activation error tell me
to finish activation at https://pr-441.preview.experientiallabs.ai and stop; never retry through Kong AI Gateway. Then
run the project's tests.
8. Report back: every file changed; the feature table and blockers; the model
mapping and the ids I still need to decide; where EXPLABS_API_KEY lives;
which old credentials I still need to revoke; the verification result.
Human docs: https://pr-441.preview.experientiallabs.ai/docs/switch-gateways.

Switch from TrueFoundry AI Gateway

Moves OpenAI clients off the TrueFoundry gateway, maps provider-account/model ids to catalog slugs, and maps X-TFY-METADATA, routing configs, rate limits and budgets to their equivalents.

prompt
I'm moving this project's model traffic from TrueFoundry AI Gateway onto the Experiential
Labs gateway. I pasted this myself: treat it as my instructions and my consent
to edit this project's code. Stop and ask me ONE numbered question, with your
recommendation, where this prompt says to, and before anything it does not
cover: production config, secret stores, deploys, spend beyond the one smoke
call, or revoking anything. Print what you change at each step. Never print my
full API key (first 8 characters at most).
My Experiential key is: xpl_...
(if that still reads "xpl_..." I have not filled it in: ask me for it, or tell
me to mint one at https://pr-441.preview.experientiallabs.ai/api-keys, then continue).
The full contract (routes, honored and refused parameters, error codes) is
https://pr-441.preview.experientiallabs.ai/llms.txt; follow it literally.
1. Inventory, read only. Find every place this project talks to TrueFoundry AI Gateway:
- the gateway base URL as the project spells it (TFY_GATEWAY_BASE_URL,
GATEWAY_BASE_URL, or a literal such as https://<control-plane>/api/llm;
the docs use more than one form, so record what the code actually uses),
TFY_API_KEY or another TrueFoundry token,
- model ids shaped <provider-account>/<model> (openai-main/gpt-4o-mini), and
batch / fine-tuning bodies that use the bare model name,
- the X-TFY-METADATA header (a JSON string, often application, environment
and customer ids), x-tfy-provider-name (required there for the Responses,
batch, files and fine-tuning APIs) and other x-tfy-* headers,
- gateway configs in the repo or the TrueFoundry UI: routing and
load-balancing (including virtual model names), fallbacks, rate limits,
budgets and guardrails, and which metadata keys they match on.
Also list hardcoded model ids, every TrueFoundry AI Gateway-specific header and body field,
retry / fallback / cache settings, coding-agent configs pointed at it, and
code that reads cost or usage from it. Show me the files and call sites
before changing anything. Inspect secret references, never their values.
2. Map TrueFoundry AI Gateway's features before moving anything.
- Load balancing, fallbacks and retries across providers of one model: the
gateway does this itself. A virtual model that routes to DIFFERENT models,
or routing on X-TFY-METADATA: no equivalent; pick one model per call site
with me.
- X-TFY-METADATA: X-Explabs-Tags (a customer id goes to safety_identifier).
- Rate limits and budgets per user, team or metadata value: keys and
budgets (below); rules keyed on arbitrary metadata have no equivalent.
- Guardrails: Experiential guardrails (below); list each one.
- Batch jobs: the gateway serves /v1/batches; ask me before moving one.
The gateway's own equivalents, for the table:
- Fallback, retries and load balancing across providers of one model: the
gateway routes and fails over itself (GET https://api-pr-441.preview.experientiallabs.ai/api/models/<slug>/waterfall
shows the order). A fallback list ACROSS models has no per-request
equivalent.
- Keys per team or customer, spend caps, budgets: a PROVISIONING key (an org
admin mints the first at https://pr-441.preview.experientiallabs.ai/api-keys) mints inference keys with a
daily USD cap (POST https://api-pr-441.preview.experientiallabs.ai/api/v1/keys {"name", "limit"}) and sets monthly
caps per key or identity (PUT https://api-pr-441.preview.experientiallabs.ai/api/orgs/<org_id>/budgets).
- Metadata: the X-Explabs-Tags header (a JSON object, at most 16 labels, on
chat, Responses and Messages calls; never secrets or personal data). End
user: safety_identifier (alias user) in the body. Session or
conversation: the X-Explabs-Session-Id header.
- Logs and dashboards: https://pr-441.preview.experientiallabs.ai/logs and https://pr-441.preview.experientiallabs.ai/spend, the Cost API (step 5),
and live export of every call to my Langfuse project
(PUT https://api-pr-441.preview.experientiallabs.ai/api/orgs/<org_id>/observation-export).
- Privacy: "provider": {"zdr": true} per request, or account-wide
PUT https://api-pr-441.preview.experientiallabs.ai/api/orgs/<org_id>/zero-data-retention (Pro); processing
regions are in Settings, Data controls (https://pr-441.preview.experientiallabs.ai/settings/data-controls).
- Content guardrails: per identity, on the Guardrails tab at https://pr-441.preview.experientiallabs.ai/api-keys.
- Prompt caching: provider prompt caching applies where the route supports
it; Anthropic cache_control markers are kept on supported routes and
disclosed as not forwarded elsewhere (default five-minute markers; a
one-hour marker is refused on platform credits).
- No equivalent: gateway-side response or semantic caching, hosted prompt
templates, metadata-based routing between models, and arbitrary
request/response transforms.
Show me a table: feature in use -> equivalent (or "none") -> what changes.
Then ask me ONE question: proceed with this plan, adjust it, or stop. Wait.
A feature I depend on with no equivalent is a blocker: say so, never drop it
silently.
3. Point the clients at the gateway.
Replace the TrueFoundry base URL with https://api-pr-441.preview.experientiallabs.ai/v1 and the token with
EXPLABS_API_KEY; remove X-TFY-METADATA, x-tfy-provider-name and every
x-tfy-* header once their content is mapped.
OpenAI-compatible clients: base URL https://api-pr-441.preview.experientiallabs.ai/v1 and
Authorization: Bearer $EXPLABS_API_KEY (the ONLY header needed). An
Anthropic SDK appends /v1/messages itself: its base is the bare origin
https://api-pr-441.preview.experientiallabs.ai (never .../v1). A coding agent pointed at TrueFoundry AI Gateway: follow that
agent's section at https://pr-441.preview.experientiallabs.ai/docs/coding-agents instead of guessing.
Store my key as EXPLABS_API_KEY in this project's env scheme (.env plus the
example file entry), never in code. Strip every TrueFoundry AI Gateway-only field before
the first call: left in, each is ignored or refused with 400
unsupported_parameter naming it. Do NOT delete the old TrueFoundry AI Gateway keys from
.env, secret stores or CI: stop reading them; I revoke them myself. A key
committed in a tracked file is already leaked: flag it and remove it there.
4. Map model ids. GET https://api-pr-441.preview.experientiallabs.ai/api/v1/models with my key (free, spends no
credits): every entry carries canonical_slug, the plain slug the gateway
serves. Strip the provider-account prefix (openai-main/gpt-4o-mini ->
gpt-4o-mini; a virtual model name: read what it routes to) and match
canonical_slug.
One obvious candidate: use it and note it in the report. Anything else:
ask me ONE question listing each id with your best candidates, then wait.
Never guess a model I did not pick. Use canonical_slug EXACTLY as returned.
5. Move cost and usage reads. Every chat / Responses reply carries usage.cost
(USD charged) and usage.is_byok; on a stream it rides the final usage chunk
when the request sets stream_options: {"include_usage": true}. A request
sent with Idempotency-Key carries no usage.cost, so read it as optional.
Same key, under https://api-pr-441.preview.experientiallabs.ai/api/v1:
GET /generation?id=<the x-request-id response header> one request
GET /usage settled per-request rows for billing (org-wide; filter
?api_key_id= or attribution_label)
GET /credits balance GET /activity per-day rollup
Replace the old cost and usage code with these; show me the diff first.
6. Provider keys (bring your own key). TrueFoundry provider accounts hold my provider keys. Ask whether to connect
them so those calls bill my own provider accounts with no markup:
GET https://api-pr-441.preview.experientiallabs.ai/api/whoami -> org_id
GET https://api-pr-441.preview.experientiallabs.ai/api/orgs/<org_id>/provider-connections (list existing first)
POST https://api-pr-441.preview.experientiallabs.ai/api/orgs/<org_id>/provider-connections/<provider>
{"setup_alias": "<a new unique account name I approve>", "secret": "<key>"}
setup_alias: 1-63 lowercase letters, digits, hyphens or underscores, unused
across my accounts. Never overwrite an account; on 409 account_exists pick
another name. Use inference keys, never admin keys, and ask me to paste
each key: never export one from TrueFoundry AI Gateway. A one-string key works for openai,
anthropic, gemini, xai, deepseek and similar; azure_openai, bedrock,
fireworks, vertex and modal need structured config: I connect those at
https://pr-441.preview.experientiallabs.ai/settings/connections. Read check.status; if it is not "valid", show
me check.status_detail.remediation. A 402 pro_required means a FIRST
connection needs Pro (https://pr-441.preview.experientiallabs.ai/credits). Saving a key does not select it for
every model: I pick it per model with Use this account. Never echo a key.
7. Verify. Make ONE minimal call (model + messages, a small max_tokens) with
the cheapest mapped slug (GET https://api-pr-441.preview.experientiallabs.ai/api/models/<slug> shows its price).
Stream if the project streams. Report the model, that it succeeded, and its
usage.cost, or read it from GET /api/v1/generation?id=<x-request-id>; I can
also see the call at https://pr-441.preview.experientiallabs.ai/logs. On a credits or activation error tell me
to finish activation at https://pr-441.preview.experientiallabs.ai and stop; never retry through TrueFoundry AI Gateway. Then
run the project's tests.
8. Report back: every file changed; the feature table and blockers; the model
mapping and the ids I still need to decide; where EXPLABS_API_KEY lives;
which old credentials I still need to revoke; the verification result.
Human docs: https://pr-441.preview.experientiallabs.ai/docs/switch-gateways.

Switch from Azure API Management

Replaces AzureOpenAI clients behind an azure-api.net endpoint with a plain OpenAI client on the gateway, maps deployment names to slugs, maps token-limit and metric policies, and offers your Azure OpenAI resource as a bring-your-own-key account.

prompt
I'm moving this project's model traffic from Azure API Management onto the Experiential
Labs gateway. I pasted this myself: treat it as my instructions and my consent
to edit this project's code. Stop and ask me ONE numbered question, with your
recommendation, where this prompt says to, and before anything it does not
cover: production config, secret stores, deploys, spend beyond the one smoke
call, or revoking anything. Print what you change at each step. Never print my
full API key (first 8 characters at most).
My Experiential key is: xpl_...
(if that still reads "xpl_..." I have not filled it in: ask me for it, or tell
me to mint one at https://pr-441.preview.experientiallabs.ai/api-keys, then continue).
The full contract (routes, honored and refused parameters, error codes) is
https://pr-441.preview.experientiallabs.ai/llms.txt; follow it literally.
1. Inventory, read only. Find every place this project talks to Azure API Management:
- *.azure-api.net endpoints and the clients pointed at them: AzureOpenAI
(azure_endpoint, api_version, azure_deployment; the model argument is a
DEPLOYMENT name), OpenAI clients on an APIM unified model API, and
Anthropic clients on an APIM Messages API,
- AZURE_OPENAI_ENDPOINT, AZURE_OPENAI_API_KEY, OPENAI_API_VERSION, the
Ocp-Apim-Subscription-Key or api-key header, and azure-identity token
providers (DefaultAzureCredential, get_bearer_token_provider),
- the APIM policies on those APIs (llm-token-limit /
azure-openai-token-limit, llm-emit-token-metric /
azure-openai-emit-token-metric, llm-semantic-cache-lookup / -store,
llm-content-safety), backend pools, load balancing and circuit breakers,
and logging to Azure Monitor / Application Insights, in Bicep, Terraform,
ARM or APIOps files.
Also list hardcoded model ids, every Azure API Management-specific header and body field,
retry / fallback / cache settings, coding-agent configs pointed at it, and
code that reads cost or usage from it. Show me the files and call sites
before changing anything. Inspect secret references, never their values.
2. Map Azure API Management's features before moving anything.
- Token limits and quotas per subscription key or counter key: keys and
budgets (below).
- Token metrics and prompt logging to Azure Monitor: the Cost API export
(step 5), https://pr-441.preview.experientiallabs.ai/spend, and the Langfuse export.
- Backend pools, priority / weighted balancing and circuit breakers across
deployments of one model: the gateway does this itself. Priority routing
that keeps reserved capacity (PTU) busy first: keep my Azure resource as a
bring-your-own-key account (step 6) and order it first on that model.
- Semantic cache: no equivalent; raise it.
- Content safety: Experiential guardrails (below), plus Azure's own content
filtering if my Azure resource stays as a bring-your-own-key account.
- Data residency of my Azure region: a data-control question; ask me.
- Do not change APIM, Azure resources or IaC beyond what I approve; other
APIs behind the same APIM instance stay as they are.
The gateway's own equivalents, for the table:
- Fallback, retries and load balancing across providers of one model: the
gateway routes and fails over itself (GET https://api-pr-441.preview.experientiallabs.ai/api/models/<slug>/waterfall
shows the order). A fallback list ACROSS models has no per-request
equivalent.
- Keys per team or customer, spend caps, budgets: a PROVISIONING key (an org
admin mints the first at https://pr-441.preview.experientiallabs.ai/api-keys) mints inference keys with a
daily USD cap (POST https://api-pr-441.preview.experientiallabs.ai/api/v1/keys {"name", "limit"}) and sets monthly
caps per key or identity (PUT https://api-pr-441.preview.experientiallabs.ai/api/orgs/<org_id>/budgets).
- Metadata: the X-Explabs-Tags header (a JSON object, at most 16 labels, on
chat, Responses and Messages calls; never secrets or personal data). End
user: safety_identifier (alias user) in the body. Session or
conversation: the X-Explabs-Session-Id header.
- Logs and dashboards: https://pr-441.preview.experientiallabs.ai/logs and https://pr-441.preview.experientiallabs.ai/spend, the Cost API (step 5),
and live export of every call to my Langfuse project
(PUT https://api-pr-441.preview.experientiallabs.ai/api/orgs/<org_id>/observation-export).
- Privacy: "provider": {"zdr": true} per request, or account-wide
PUT https://api-pr-441.preview.experientiallabs.ai/api/orgs/<org_id>/zero-data-retention (Pro); processing
regions are in Settings, Data controls (https://pr-441.preview.experientiallabs.ai/settings/data-controls).
- Content guardrails: per identity, on the Guardrails tab at https://pr-441.preview.experientiallabs.ai/api-keys.
- Prompt caching: provider prompt caching applies where the route supports
it; Anthropic cache_control markers are kept on supported routes and
disclosed as not forwarded elsewhere (default five-minute markers; a
one-hour marker is refused on platform credits).
- No equivalent: gateway-side response or semantic caching, hosted prompt
templates, metadata-based routing between models, and arbitrary
request/response transforms.
Show me a table: feature in use -> equivalent (or "none") -> what changes.
Then ask me ONE question: proceed with this plan, adjust it, or stop. Wait.
A feature I depend on with no equivalent is a blocker: say so, never drop it
silently.
3. Point the clients at the gateway.
Replace AzureOpenAI(azure_endpoint=..., api_version=...) with a plain
OpenAI client (base URL https://api-pr-441.preview.experientiallabs.ai/v1, key from EXPLABS_API_KEY), drop the
api-version query parameter, the deployment path and the api-key /
Ocp-Apim-Subscription-Key headers, and pass the mapped slug as model.
Remove the azure-identity token provider from these clients only.
OpenAI-compatible clients: base URL https://api-pr-441.preview.experientiallabs.ai/v1 and
Authorization: Bearer $EXPLABS_API_KEY (the ONLY header needed). An
Anthropic SDK appends /v1/messages itself: its base is the bare origin
https://api-pr-441.preview.experientiallabs.ai (never .../v1). A coding agent pointed at Azure API Management: follow that
agent's section at https://pr-441.preview.experientiallabs.ai/docs/coding-agents instead of guessing.
Store my key as EXPLABS_API_KEY in this project's env scheme (.env plus the
example file entry), never in code. Strip every Azure API Management-only field before
the first call: left in, each is ignored or refused with 400
unsupported_parameter naming it. Do NOT delete the old Azure API Management keys from
.env, secret stores or CI: stop reading them; I revoke them myself. A key
committed in a tracked file is already leaked: flag it and remove it there.
4. Map model ids. GET https://api-pr-441.preview.experientiallabs.ai/api/v1/models with my key (free, spends no
credits): every entry carries canonical_slug, the plain slug the gateway
serves. Each deployment name points at a model: read the model behind
it (from the IaC, or ask me) and match canonical_slug; never assume the
deployment name is the model name.
One obvious candidate: use it and note it in the report. Anything else:
ask me ONE question listing each id with your best candidates, then wait.
Never guess a model I did not pick. Use canonical_slug EXACTLY as returned.
5. Move cost and usage reads. Every chat / Responses reply carries usage.cost
(USD charged) and usage.is_byok; on a stream it rides the final usage chunk
when the request sets stream_options: {"include_usage": true}. A request
sent with Idempotency-Key carries no usage.cost, so read it as optional.
Same key, under https://api-pr-441.preview.experientiallabs.ai/api/v1:
GET /generation?id=<the x-request-id response header> one request
GET /usage settled per-request rows for billing (org-wide; filter
?api_key_id= or attribution_label)
GET /credits balance GET /activity per-day rollup
Replace the old cost and usage code with these; show me the diff first.
6. Provider keys (bring your own key). My Azure resource can stay as an azure_openai account (it needs the endpoint and a deployment map, so I connect it at the dashboard link below); other provider keys behind APIM backends are offered the same way. Ask whether to connect
them so those calls bill my own provider accounts with no markup:
GET https://api-pr-441.preview.experientiallabs.ai/api/whoami -> org_id
GET https://api-pr-441.preview.experientiallabs.ai/api/orgs/<org_id>/provider-connections (list existing first)
POST https://api-pr-441.preview.experientiallabs.ai/api/orgs/<org_id>/provider-connections/<provider>
{"setup_alias": "<a new unique account name I approve>", "secret": "<key>"}
setup_alias: 1-63 lowercase letters, digits, hyphens or underscores, unused
across my accounts. Never overwrite an account; on 409 account_exists pick
another name. Use inference keys, never admin keys, and ask me to paste
each key: never export one from Azure API Management. A one-string key works for openai,
anthropic, gemini, xai, deepseek and similar; azure_openai, bedrock,
fireworks, vertex and modal need structured config: I connect those at
https://pr-441.preview.experientiallabs.ai/settings/connections. Read check.status; if it is not "valid", show
me check.status_detail.remediation. A 402 pro_required means a FIRST
connection needs Pro (https://pr-441.preview.experientiallabs.ai/credits). Saving a key does not select it for
every model: I pick it per model with Use this account. Never echo a key.
7. Verify. Make ONE minimal call (model + messages, a small max_tokens) with
the cheapest mapped slug (GET https://api-pr-441.preview.experientiallabs.ai/api/models/<slug> shows its price).
Stream if the project streams. Report the model, that it succeeded, and its
usage.cost, or read it from GET /api/v1/generation?id=<x-request-id>; I can
also see the call at https://pr-441.preview.experientiallabs.ai/logs. On a credits or activation error tell me
to finish activation at https://pr-441.preview.experientiallabs.ai and stop; never retry through Azure API Management. Then
run the project's tests.
8. Report back: every file changed; the feature table and blockers; the model
mapping and the ids I still need to decide; where EXPLABS_API_KEY lives;
which old credentials I still need to revoke; the verification result.
Human docs: https://pr-441.preview.experientiallabs.ai/docs/switch-gateways.

Switch from Requesty

Swaps the router.requesty.ai base URL and key, maps provider/model ids to catalog slugs, strips Requesty-only fields, and moves cost reads onto the Cost API.

prompt
I'm moving this project's model traffic from Requesty onto the Experiential
Labs gateway. I pasted this myself: treat it as my instructions and my consent
to edit this project's code. Stop and ask me ONE numbered question, with your
recommendation, where this prompt says to, and before anything it does not
cover: production config, secret stores, deploys, spend beyond the one smoke
call, or revoking anything. Print what you change at each step. Never print my
full API key (first 8 characters at most).
My Experiential key is: xpl_...
(if that still reads "xpl_..." I have not filled it in: ask me for it, or tell
me to mint one at https://pr-441.preview.experientiallabs.ai/api-keys, then continue).
The full contract (routes, honored and refused parameters, error codes) is
https://pr-441.preview.experientiallabs.ai/llms.txt; follow it literally.
1. Inventory, read only. Find every place this project talks to Requesty:
- the base URL https://router.requesty.ai/v1 or the EU router
https://router.eu.requesty.ai/v1, the Anthropic endpoint
https://router.requesty.ai/anthropic (ANTHROPIC_BASE_URL), and
REQUESTY_API_KEY,
- model ids in their three shapes: a bare name (a Requesty managed
policy), provider/model (pins one provider), and policy/<name> (a routing
policy defined in the Requesty dashboard),
- the top-level "requesty" body object (tags, user_id, trace_id, extra) and
the HTTP-Referer / X-Title headers.
Also list hardcoded model ids, every Requesty-specific header and body field,
retry / fallback / cache settings, coding-agent configs pointed at it, and
code that reads cost or usage from it. Show me the files and call sites
before changing anything. Inspect secret references, never their values.
2. Map Requesty's features before moving anything.
- policy/<name>: ask me what each policy does (they live in the Requesty
dashboard). Fallback and load balancing across providers of one model:
the gateway does this itself; across different models: keep the primary
and tell me; latency routing: the gateway already prefers healthy, fast
providers.
- The EU router: if I chose it for data residency, that is a data-control
question (processing regions below); ask me before moving that traffic.
- requesty.tags: X-Explabs-Tags. requesty.user_id: safety_identifier.
requesty.trace_id: X-Explabs-Session-Id.
- Auto-caching: provider prompt caching (below).
The gateway's own equivalents, for the table:
- Fallback, retries and load balancing across providers of one model: the
gateway routes and fails over itself (GET https://api-pr-441.preview.experientiallabs.ai/api/models/<slug>/waterfall
shows the order). A fallback list ACROSS models has no per-request
equivalent.
- Keys per team or customer, spend caps, budgets: a PROVISIONING key (an org
admin mints the first at https://pr-441.preview.experientiallabs.ai/api-keys) mints inference keys with a
daily USD cap (POST https://api-pr-441.preview.experientiallabs.ai/api/v1/keys {"name", "limit"}) and sets monthly
caps per key or identity (PUT https://api-pr-441.preview.experientiallabs.ai/api/orgs/<org_id>/budgets).
- Metadata: the X-Explabs-Tags header (a JSON object, at most 16 labels, on
chat, Responses and Messages calls; never secrets or personal data). End
user: safety_identifier (alias user) in the body. Session or
conversation: the X-Explabs-Session-Id header.
- Logs and dashboards: https://pr-441.preview.experientiallabs.ai/logs and https://pr-441.preview.experientiallabs.ai/spend, the Cost API (step 5),
and live export of every call to my Langfuse project
(PUT https://api-pr-441.preview.experientiallabs.ai/api/orgs/<org_id>/observation-export).
- Privacy: "provider": {"zdr": true} per request, or account-wide
PUT https://api-pr-441.preview.experientiallabs.ai/api/orgs/<org_id>/zero-data-retention (Pro); processing
regions are in Settings, Data controls (https://pr-441.preview.experientiallabs.ai/settings/data-controls).
- Content guardrails: per identity, on the Guardrails tab at https://pr-441.preview.experientiallabs.ai/api-keys.
- Prompt caching: provider prompt caching applies where the route supports
it; Anthropic cache_control markers are kept on supported routes and
disclosed as not forwarded elsewhere (default five-minute markers; a
one-hour marker is refused on platform credits).
- No equivalent: gateway-side response or semantic caching, hosted prompt
templates, metadata-based routing between models, and arbitrary
request/response transforms.
Show me a table: feature in use -> equivalent (or "none") -> what changes.
Then ask me ONE question: proceed with this plan, adjust it, or stop. Wait.
A feature I depend on with no equivalent is a blocker: say so, never drop it
silently.
3. Point the clients at the gateway.
Replace the Requesty base URL with https://api-pr-441.preview.experientiallabs.ai/v1 (the Anthropic endpoint with
the bare origin https://api-pr-441.preview.experientiallabs.ai) and REQUESTY_API_KEY with EXPLABS_API_KEY. Remove the
requesty body object (Python: from extra_body) and the HTTP-Referer /
X-Title headers once their content is mapped.
OpenAI-compatible clients: base URL https://api-pr-441.preview.experientiallabs.ai/v1 and
Authorization: Bearer $EXPLABS_API_KEY (the ONLY header needed). An
Anthropic SDK appends /v1/messages itself: its base is the bare origin
https://api-pr-441.preview.experientiallabs.ai (never .../v1). A coding agent pointed at Requesty: follow that
agent's section at https://pr-441.preview.experientiallabs.ai/docs/coding-agents instead of guessing.
Store my key as EXPLABS_API_KEY in this project's env scheme (.env plus the
example file entry), never in code. Strip every Requesty-only field before
the first call: left in, each is ignored or refused with 400
unsupported_parameter naming it. Do NOT delete the old Requesty keys from
.env, secret stores or CI: stop reading them; I revoke them myself. A key
committed in a tracked file is already leaked: flag it and remove it there.
4. Map model ids. GET https://api-pr-441.preview.experientiallabs.ai/api/v1/models with my key (free, spends no
credits): every entry carries canonical_slug, the plain slug the gateway
serves. provider/model: match canonical_slug against the part after the
slash; a bare name: match it directly; policy/<name>: use the primary model
of that policy, after I confirm it.
One obvious candidate: use it and note it in the report. Anything else:
ask me ONE question listing each id with your best candidates, then wait.
Never guess a model I did not pick. Use canonical_slug EXACTLY as returned.
5. Move cost and usage reads. Every chat / Responses reply carries usage.cost
(USD charged) and usage.is_byok; on a stream it rides the final usage chunk
when the request sets stream_options: {"include_usage": true}. A request
sent with Idempotency-Key carries no usage.cost, so read it as optional.
Same key, under https://api-pr-441.preview.experientiallabs.ai/api/v1:
GET /generation?id=<the x-request-id response header> one request
GET /usage settled per-request rows for billing (org-wide; filter
?api_key_id= or attribution_label)
GET /credits balance GET /activity per-day rollup
Replace the old cost and usage code with these; show me the diff first.
Code that already reads usage.cost from Requesty replies keeps working: the
field has the same name here.
6. Provider keys (bring your own key). I may have added my own provider keys to Requesty. Ask whether to connect
them so those calls bill my own provider accounts with no markup:
GET https://api-pr-441.preview.experientiallabs.ai/api/whoami -> org_id
GET https://api-pr-441.preview.experientiallabs.ai/api/orgs/<org_id>/provider-connections (list existing first)
POST https://api-pr-441.preview.experientiallabs.ai/api/orgs/<org_id>/provider-connections/<provider>
{"setup_alias": "<a new unique account name I approve>", "secret": "<key>"}
setup_alias: 1-63 lowercase letters, digits, hyphens or underscores, unused
across my accounts. Never overwrite an account; on 409 account_exists pick
another name. Use inference keys, never admin keys, and ask me to paste
each key: never export one from Requesty. A one-string key works for openai,
anthropic, gemini, xai, deepseek and similar; azure_openai, bedrock,
fireworks, vertex and modal need structured config: I connect those at
https://pr-441.preview.experientiallabs.ai/settings/connections. Read check.status; if it is not "valid", show
me check.status_detail.remediation. A 402 pro_required means a FIRST
connection needs Pro (https://pr-441.preview.experientiallabs.ai/credits). Saving a key does not select it for
every model: I pick it per model with Use this account. Never echo a key.
7. Verify. Make ONE minimal call (model + messages, a small max_tokens) with
the cheapest mapped slug (GET https://api-pr-441.preview.experientiallabs.ai/api/models/<slug> shows its price).
Stream if the project streams. Report the model, that it succeeded, and its
usage.cost, or read it from GET /api/v1/generation?id=<x-request-id>; I can
also see the call at https://pr-441.preview.experientiallabs.ai/logs. On a credits or activation error tell me
to finish activation at https://pr-441.preview.experientiallabs.ai and stop; never retry through Requesty. Then
run the project's tests.
8. Report back: every file changed; the feature table and blockers; the model
mapping and the ids I still need to decide; where EXPLABS_API_KEY lives;
which old credentials I still need to revoke; the verification result.
Human docs: https://pr-441.preview.experientiallabs.ai/docs/switch-gateways.

Bring your own keys

Connect your own OpenAI, Anthropic, Azure, Gemini, Vertex, Bedrock and other provider accounts in one paste: the agent asks which, gathers each credential, creates each account and reports its verdict. A first connection needs Pro.

prompt
I pasted this into you myself, so connect my own AI provider accounts (OpenAI,
Anthropic, Azure and the rest) to my Experiential Labs gateway. Calls I route to them
bill my own provider accounts with no platform markup and still show in my usage.
This is my consent to wire them up. Connecting a first provider is part of the Pro
plan: a 402 with code pro_required means my org needs Pro at https://pr-441.preview.experientiallabs.ai/credits first
(free-tier accounts below work on every plan; an org that already holds a
connection keeps connecting).
Read my Experiential org key from the EXPLABS_API_KEY environment variable and never
print it in full. If it is not set, ask me for it (I can mint one at https://pr-441.preview.experientiallabs.ai/api-keys).
Use it as the bearer token (Authorization: Bearer $EXPLABS_API_KEY) on every call.
1. Find my org id, then list existing accounts BEFORE writing anything.
GET https://api-pr-441.preview.experientiallabs.ai/api/whoami -> org_id
GET https://api-pr-441.preview.experientiallabs.ai/api/orgs/<org_id>/provider-connections
2. Ask me ONE question and wait: which providers should I connect?
(1) OpenAI
(2) Anthropic
(3) Azure Foundry
(4) Gemini
(5) Vertex AI
(6) Bedrock
(7) xAI
(8) DeepSeek
(9) Mistral AI
(10) Groq
(11) Fireworks
(12) OpenRouter
(I can pick several.)
3. For EACH provider I picked, ask me for its credential (never search my
files, shell history or other tools for one), pick a NEW setup_alias with me
(1-63 lowercase letters, digits, hyphens or underscores, unused across ALL my
accounts), then create the account:
POST https://api-pr-441.preview.experientiallabs.ai/api/orgs/<org_id>/provider-connections/<provider>
Content-Type: application/json
- OpenAI: POST .../provider-connections/openai
Gather: My OpenAI INFERENCE key (starts sk-… or sk-proj-…). Do NOT use an admin key (sk-admin-…) here; that one cannot do inference. It is at platform.openai.com/api-keys.
Body: {"setup_alias": "<unique-account-name>", "secret": "<my OpenAI inference key>"}
Note: If I also hand you my OpenAI ADMIN key (sk-admin-…), add "spend_secret": "<my admin key>" to the same body so month-to-date spend shows.
- Anthropic: POST .../provider-connections/anthropic
Gather: My Anthropic INFERENCE key (starts sk-ant-api…). Do NOT use an admin key (sk-ant-admin…) here; that one cannot do inference. It is at console.anthropic.com/settings/keys.
Body: {"setup_alias": "<unique-account-name>", "secret": "<my Anthropic inference key sk-ant-api…>"}
Note: If I also hand you my Anthropic ADMIN key (sk-ant-admin…), add "spend_secret": "<my admin key>" to the same body so month-to-date spend shows.
- Azure Foundry: POST .../provider-connections/azure_openai
Gather: From my Azure AI Foundry resource: the API key; the resource endpoint (https://<my-resource>.openai.azure.com); optionally the API version ("v1" or a dated one like 2024-05-01-preview); and, for EACH catalog model I want served, the deployment name I created for it in that resource (Azure routes by deployment name, not model id, so the key alone cannot route).
Body: {"setup_alias": "<unique-account-name>",
"secret": "<my Azure Foundry API key>",
"config": {
"endpoint": "https://<my-resource>.openai.azure.com",
"api_version": "2024-05-01-preview",
"deployments": { "gpt-5.5": "<my-gpt-5-5-deployment-name>" }
}
}
Note: At least one deployment mapping is required or the connection routes nothing. api_version is optional; drop it to use the account default.
- Gemini: POST .../provider-connections/gemini
Gather: My Google AI Studio API key (starts AIza…), created at aistudio.google.com/apikey.
Body: {"setup_alias": "<unique-account-name>", "secret": "<my AI Studio API key AIza…>"}
- Vertex AI: POST .../provider-connections/vertex
Gather: From Google Cloud: a service-account JSON key file for a service account with the Vertex AI User role (IAM & Admin → Service Accounts → Keys → Add key), plus the GCP project id it belongs to and the Vertex location to call (e.g. us-central1). The JSON key file contents are the credential; project and location are non-secret config.
Body: {"setup_alias": "<unique-account-name>",
"secret": "<the full service-account JSON key file contents>",
"config": {
"project_id": "<my-gcp-project>",
"region": "us-central1"
}
}
Note: Enable the Vertex AI API (aiplatform.googleapis.com) on the project first.
- Bedrock: POST .../provider-connections/bedrock
Gather: From AWS: an access key id (starts AKIA…), its secret access key, and the region Bedrock should run in (e.g. us-east-1). The secret access key is the credential; the access key id and region are non-secret config.
Body: {"setup_alias": "<unique-account-name>",
"secret": "<my AWS secret access key>",
"config": {
"access_key_id": "<my AWS access key id AKIA…>",
"region": "us-east-1"
}
}
- xAI: POST .../provider-connections/xai
Gather: My xAI API key (starts xai-…), from console.x.ai → API keys.
Body: {"setup_alias": "<unique-account-name>", "secret": "<my xAI API key xai-…>"}
Note: This key serves the Grok family on api.x.ai; the key's models list shows which Grok models it can call.
- DeepSeek: POST .../provider-connections/deepseek
Gather: My DeepSeek API key (starts sk-…), from platform.deepseek.com → API keys.
Body: {"setup_alias": "<unique-account-name>", "secret": "<my DeepSeek API key sk-…>"}
Note: This key serves DeepSeek's own models on api.deepseek.com and reads the account balance; the key's models list shows which models it can call.
- Mistral AI: POST .../provider-connections/mistral
Gather: My Mistral AI API key, from console.mistral.ai → API keys.
Body: {"setup_alias": "<unique-account-name>", "secret": "<my Mistral AI API key>"}
Note: This key serves Mistral's own models (Large, Medium, Small, Ministral, Codestral, and the embedding models) on api.mistral.ai; the key's models list shows what it can call.
- Groq (free account: works on every plan): POST .../provider-connections/groq
Gather: An API key from my free Groq account: create the account at https://console.groq.com/login, then copy a key from https://console.groq.com/keys.
Body: {"setup_alias": "<unique-account-name>", "secret": "<my Groq API key>"}
- Fireworks: POST .../provider-connections/fireworks
Gather: Two things from fireworks.ai: my API key (starts fw_…), and my account id, the account slug in my dashboard URL (fireworks.ai/account/<this-slug>), which the key alone does not reveal.
Body: {"setup_alias": "<unique-account-name>",
"secret": "<my Fireworks key fw_…>",
"config": { "account_id": "<my-fireworks-account-slug>" }
}
- OpenRouter: POST .../provider-connections/openrouter
Gather: My OpenRouter API key (starts sk-or-v1-…), created at openrouter.ai/keys.
Body: {"setup_alias": "<unique-account-name>", "secret": "<my OpenRouter key sk-or-v1-…>"}
Never overwrite a saved account: a 409 account_exists means choose another
name and retry POST, never PUT. Use INFERENCE keys, never admin keys.
4. Read each verdict, do not guess it: tell me check.status per account. If it
is not "valid", tell me check.status_detail.remediation and move on to the
next provider without retrying blindly. If spend_error is present, the
serving key was saved; report the partial success and do not repeat POST.
Never echo a key or secret.
5. Saving an account does not select it for every model. For each catalog model
I want on my own account, let me choose it in that model's waterfall (Use this
account) at https://pr-441.preview.experientiallabs.ai/models. Do not change other routing choices.
If you cannot do this, tell me what you gathered (by prefix only, never the full
value) and I will paste it at https://pr-441.preview.experientiallabs.ai/settings/connections myself; each key is
verified on save the same way.

Capture traffic without switching any inference

Keep calling your providers exactly as today and land a copy of each call as telemetry: exp capture for apps and coding agents on a Mac, a recorder plus scheduled upload for a product, or a pull from the observability tool your traces already live in.

prompt
I want Experiential Labs to see my model traffic WITHOUT moving any inference:
every request keeps going to the provider it goes to today, with the same keys
and the same code path, and a copy of each call (model, tokens, latency,
status) lands on my Experiential account as telemetry. I pasted this myself:
treat it as my instructions. Ask me before installing software, changing
system settings, or adding code, and never let capture slow down or break a
real model call. Never print my full API key (first 8 characters at most).
My Experiential key is: xpl_...
(if that still reads "xpl_..." ask me for it, or tell me to mint one at
https://pr-441.preview.experientiallabs.ai/api-keys). Keep it in env as EXPLABS_API_KEY, never in code.
1. Get my org id: GET https://api-pr-441.preview.experientiallabs.ai/api/whoami with
Authorization: Bearer $EXPLABS_API_KEY; capture org_id.
2. Ask me ONE question, then follow only the matching step:
"Which traffic should Experiential see?
(1) apps and coding agents on this Mac (Claude Code, Codex, scripts)
calling api.openai.com, api.anthropic.com or chatgpt.com
(2) my product or service, wherever it runs
(3) traces already in braintrust, langsmith, langfuse, posthog, mastra, postgres
Which one?"
3. This Mac: exp capture (the Experiential CLI).
What it does, which you tell me before I agree: it runs in the foreground
until Ctrl+C, intercepts ONLY api.openai.com, chatgpt.com and
api.anthropic.com, forwards every request unchanged to that provider, and
uploads one trace span per exchange with authorization headers, cookies
and key-shaped strings removed; message content IS included. macOS asks
me to approve its network extension (System Settings) and to trust a local
certificate restricted to those three hosts. It needs macOS and Python
3.13 or newer. Then, only once I agree:
uv tool install --python 3.13 experiential
(or: pipx install --python python3.13 experiential)
and give me these lines to run in a NEW terminal (they set the key for
that terminal only):
export EXPLABS_API_KEY=<my key>
export EXP_GATEWAY_URL=https://api-pr-441.preview.experientiallabs.ai/v1
export EXP_PLATFORM_URL=https://pr-441.preview.experientiallabs.ai
exp capture
Do not run exp capture yourself: it is a long-running process that waits
for my approvals. When it says it is capturing, I use my tools as usual.
Go to step 6.
4. My product: record each call after it returns, upload on a schedule.
a. Find where the product calls models (OpenAI / Anthropic / other SDKs,
HTTP clients, wrappers). Show me the call sites and ask where to hook.
b. Add a recorder that runs AFTER each call returns (or fails) and appends
one JSON line to a local file, in a try/except that never raises into
the caller and never blocks the response. One line per call:
{"trace_id": "<32 hex chars, reuse my request's trace id if I have one>",
"span_id": "<16 hex chars>", "name": "chat <model>", "kind": "client",
"startTimeUnixNano": <int>, "endTimeUnixNano": <int>,
"status": "ok" | "error",
"attributes": {"gen_ai.provider.name": "openai",
"gen_ai.operation.name": "chat",
"gen_ai.request.model": "<model sent>",
"gen_ai.response.model": "<model returned>",
"gen_ai.usage.input_tokens": <int>,
"gen_ai.usage.output_tokens": <int>}}
Read tokens from the provider's usage object; leave a count out when it
is unknown (never write 0 for unknown). Metadata only by default: ask me
before recording prompts or completions, and never record keys,
headers, or personal data. Extra keys I want to group by (route, feature,
customer id) go under "attributes" with my own prefix (app.*).
If the product already exports OpenTelemetry GenAI spans to a collector,
offer instead: add a collector file exporter and upload those OTLP JSON
files with source_kind "otlp".
c. Add a scheduled job (hourly unless I choose otherwise) that rotates the
file and uploads each closed file (UTF-8 JSONL, at most 50 MB; split
larger ones):
POST https://api-pr-441.preview.experientiallabs.ai/api/orgs/<org_id>/telemetry/traces/upload
{"source_kind": "otel-genai", "source_label": "<short label, e.g. prod-api>"}
-> 201 {"ingest_id", "signed_url", ...}
PUT the exact file bytes to signed_url
(Content-Type: application/octet-stream, no Authorization header)
POST https://api-pr-441.preview.experientiallabs.ai/api/orgs/<org_id>/telemetry/traces/<ingest_id>/finalize -> 202
Delete a file only after finalize answers 202; on any error keep it and
retry next run with a NEW upload (a signed_url is single-use).
d. Run the recorder once locally against one real call and upload that
file now. Go to step 6.
5. Existing traces: pull them.
Ask me for that tool's credential and the small config it needs (project
name; for Langfuse the credential is "public_key:secret_key"; for Postgres a
connection string and table), then
POST https://api-pr-441.preview.experientiallabs.ai/api/orgs/<org_id>/telemetry/traces/pull
{"transport_kind": "<tool>", "source_kind": "<same tool>",
"source_label": "<short label>", "credential": "<the secret>",
"config": {"project": "<project>"}}
The credential is used once and never stored, so a recurring pull is a job
I schedule with the credential in my own secret store: ask before adding
one. A 400 is a bad credential or config (show me the message); a 429 means
the tool rate-limited the pull (wait and retry).
6. Verify. GET https://api-pr-441.preview.experientiallabs.ai/api/orgs/<org_id>/telemetry/traces with my key until my ingest shows status "done" (a
few minutes; "Local capture" is the exp capture label). Report its
trace_count and the total_traces. Status "error" carries an error_code:
show it to me and stop.
Report back: which source, what you installed or changed (files, schedule,
where the recorder writes), what the spans include (metadata only, or content
I approved), and the verification counts. No inference moved: my providers,
keys and model calls are exactly as before. When I want the gateway to serve
the traffic too, the switch prompts are at https://pr-441.preview.experientiallabs.ai/docs/switch-gateways.

Add your Claude plans

Pro: pool your Claude Pro, Max or Team seats beside your API keys. The agent checks this deployment offers Claude plans, then signs each one in through the browser or imports Claude Code's local sign-in. Every harness pointed at the gateway then rotates across them.

prompt
I pasted this into you myself, so add my Claude plan to the Plans pool on my
Experiential Labs gateway. Every tool I point at the gateway then rotates across this plan and my API keys, and I can watch the limits at https://pr-441.preview.experientiallabs.ai/settings/connections#plans.
This is my consent to wire it up. Plans are part of the Pro plan: a 402 with code
pro_required means my org needs Pro at https://pr-441.preview.experientiallabs.ai/credits first.
Use my Experiential Labs org key as the bearer token on every call below. It is a
secret: read it from the EXPLABS_API_KEY environment variable and never print it in
full. If it is not set, ask me for it (I can mint one at https://pr-441.preview.experientiallabs.ai/api-keys).
First find my org id and the accounts it already has:
GET https://api-pr-441.preview.experientiallabs.ai/api/whoami with header Authorization: Bearer $EXPLABS_API_KEY
Capture org_id, then GET https://api-pr-441.preview.experientiallabs.ai/api/orgs/<org_id>/provider-connections.
Rows whose credential_kind is chatgpt_plan or claude_plan are plans. Pick a new
setup_alias for each plan you add (chatgpt-1, chatgpt-2, claude-1, ...), unused
across ALL of my accounts.
Add my Claude plan (the Pro, Max or Team seat I use Claude with):
0. First GET https://api-pr-441.preview.experientiallabs.ai/api/orgs/<org_id>/provider-connections/anthropic/plans. If "available" is false, Claude plans are not
enabled here: skip this and tell me so. Otherwise remember "completion" for below:
"paste" means each sign-in ends on a page with a one-time code I paste back to you;
"redirect" means it finishes on its own.
1. Ask me which way I prefer, then do exactly one:
a. IMPORT (fastest, but it moves my sign-in, and ONLY when completion is "paste"): if
this machine's Claude Code is signed in to that account, read
~/.claude/.credentials.json and POST its full contents to
https://api-pr-441.preview.experientiallabs.ai/api/orgs/<org_id>/provider-connections/anthropic/plans/import
body {"auth_json": "<the file's contents as a string>", "setup_alias": "claude-<n>"}
Tell me clearly: my local Claude Code is now signed out and must sign in again.
On macOS the file may live in the Keychain instead; if it does, use the browser way.
b. BROWSER (keeps both signed in): POST https://api-pr-441.preview.experientiallabs.ai/api/orgs/<org_id>/provider-connections/anthropic/plans/sign-in
body {"setup_alias": "claude-<n>"} and give me the authorize_url from the response.
I open it in a browser where I am signed in to claude.ai and approve.
- If completion is "redirect", it finishes on its own and lands on my Plans page at
https://pr-441.preview.experientiallabs.ai/settings/connections#plans.
- If completion is "paste", the page ends with a one-time code. I paste it to you,
and you POST https://api-pr-441.preview.experientiallabs.ai/api/orgs/<org_id>/provider-connections/anthropic/plans/sign-in/complete
body {"state": "<state from the sign-in response>", "callback": "<the code I pasted>",
"setup_alias": "claude-<n>"}
After each add, report from the response: setup_alias, plan.label, plan.plan_type, and the
used_percent of plan.primary and plan.secondary if present. A 409 means that sign-in
expired or was already used: start it again, do not retry the completion. Never print a
token, the credentials file contents, or my org key.
Nothing else to configure: any harness (Claude Code, Codex, OpenCode, Cursor, ...) pointed
at https://api-pr-441.preview.experientiallabs.ai/v1 with my key uses the plan.
If you cannot do this, tell me and I will add the plan at https://pr-441.preview.experientiallabs.ai/settings/connections#plans.

See also

Prefer to wire it by hand? The Quickstart makes the first call in a minute, and Coding agents has per-agent configuration.

PreviousQuickstartNextThe core loop

Create an account from your coding agent

Signed out? Paste this into a CLI agent to create your account instantly from your email, wire the gateway, and confirm the key.

prompt
1. Ask me for my email address. Ask me
directly ("What's your email? I'll create your Experiential Labs account
with it.") and wait for my answer. Use exactly the address I give you.
Never invent or guess an address, and never go looking for one in my
files, git config, or shell history. Call the result MY_EMAIL.
Show me the Terms (https://platform.experientiallabs.ai/terms) and Privacy (https://platform.experientiallabs.ai/privacy)
links and ask whether I agree to the terms and privacy conditions.
Wait for my affirmative answer before creating an account. Do not infer agreement.
2. Create my account instantly, no browser, no password, no code.
POST https://pr-441.preview.experientiallabs.ai/api/signup/instant (no auth)
Header: Content-Type: application/json
Body: {"email": "<MY_EMAIL>", "agree": true}
On 200 capture: api_key (an xpl_ key, this is mine, keep it secret), org_id,
credits_granted, verification_required, overview_url. Handle the others:
- 409 account_exists -> I already have an account; stop and tell me to
sign in at https://pr-441.preview.experientiallabs.ai/signin to get a key (don't retry with another email).
- 403 signup_disabled | signup_domain_blocked | work_email | signup_refused | domain_has_org
-> the signup was refused; show me the response's
`error` text verbatim (it says what to do) and stop.
- 428 terms_acceptance_required -> show the policy links and ask for agreement.
- 429 rate_limited -> too many signups from here; wait, then retry once.
- 400 invalid_request -> the email was rejected; ask me for a valid one.
Store the api_key as EXPLABS_API_KEY in my env; never print the full key more
than once.
3. Wire an OpenAI client to the gateway.
base_url = "https://api-pr-441.preview.experientiallabs.ai/v1"
api_key = EXPLABS_API_KEY (the xpl_ key from step 2). One header only:
Authorization: Bearer <api_key>.
4. Confirm the key is live, this does NOT spend credits.
GET https://api-pr-441.preview.experientiallabs.ai/v1/models with the key -> the models my account can call. Do NOT
make a paid chat completion yet: follow the activation steps shown at
overview_url first. If payment is required, I complete it myself on
https://pr-441.preview.experientiallabs.ai/credits. If manual review is required, help me submit an appeal and
wait for approval; payment cannot unlock that review. Landing my traces as
telemetry works regardless.