Connect the proxy

CacheGap is strictly BYOK: you bring your own provider key and keep your direct billing relationship with the provider. We charge for the optimization software, never for token access. Change one base URL and every request is restructured to maximize the provider's native prompt cache, losslessly.

STEP 01 · Get your keys

Create an account at /signup, then generate your CacheGap API key (sk_cg_…) and add your encrypted provider key in the dashboard. The key is shown once at generation: save it then.

STEP 02 · Change one base URL

Point your existing SDK at CacheGap and authenticate with your CacheGap key:

# OpenAI SDK
client = OpenAI(base_url="$CACHEGAP/v1", api_key="sk_cg_…")

# Anthropic SDK
client = Anthropic(base_url="$CACHEGAP", api_key="sk_cg_…")

That's it. POST /v1/chat/completions and POST /v1/messages behave exactly like the provider's (same request shape, same response shape, streaming preserved), only optimized.

Pick your stack below. Each block runs as-is after inserting your CacheGap key ($CACHEGAP is your proxy origin, e.g. https://proxy.yourdomain.com); your provider key is already stored, encrypted, from Step 01.

OpenAI SDK · Python

# .env
OPENAI_API_KEY=sk_cg_...
OPENAI_BASE_URL=$CACHEGAP/v1

from openai import OpenAI

client = OpenAI()  # picks up OPENAI_API_KEY and OPENAI_BASE_URL from the env

stream = client.chat.completions.create(
    model="gpt-5-mini",
    messages=[{"role": "user", "content": "hello"}],
    stream=True,  # streaming passes through untouched
)
for chunk in stream:
    print(chunk.choices[0].delta.content or "", end="")

OpenAI SDK · Node

# .env
OPENAI_API_KEY=sk_cg_...
OPENAI_BASE_URL=$CACHEGAP/v1

import OpenAI from 'openai';

const client = new OpenAI(); // picks up OPENAI_API_KEY and OPENAI_BASE_URL

const res = await client.chat.completions.create({
  model: 'gpt-5-mini',
  messages: [{ role: 'user', content: 'hello' }],
});
console.log(res.choices[0].message.content);

Anthropic SDK · Python

# .env
ANTHROPIC_API_KEY=sk_cg_...
ANTHROPIC_BASE_URL=$CACHEGAP

import anthropic

client = anthropic.Anthropic()  # picks up ANTHROPIC_API_KEY and ANTHROPIC_BASE_URL

msg = client.messages.create(
    model="claude-sonnet-4-6",
    max_tokens=256,
    messages=[{"role": "user", "content": "hello"}],
)
print(msg.content[0].text)

Anthropic SDK · Node

# .env
ANTHROPIC_API_KEY=sk_cg_...
ANTHROPIC_BASE_URL=$CACHEGAP

import Anthropic from '@anthropic-ai/sdk';

const client = new Anthropic(); // picks up ANTHROPIC_API_KEY and ANTHROPIC_BASE_URL

const msg = await client.messages.create({
  model: 'claude-sonnet-4-6',
  max_tokens: 256,
  messages: [{ role: 'user', content: 'hello' }],
});
console.log(msg.content);

LangChain

Pass the base URL explicitly (env pickup varies across LangChain versions):

from langchain_openai import ChatOpenAI
from langchain_anthropic import ChatAnthropic

llm = ChatOpenAI(model="gpt-5-mini",
                 base_url="$CACHEGAP/v1", api_key="sk_cg_...")
# or
llm = ChatAnthropic(model="claude-sonnet-4-6",
                    base_url="$CACHEGAP", api_key="sk_cg_...")

print(llm.invoke("hello").content)

Vercel AI SDK

The AI SDK appends the endpoint path itself, so both providers take the versioned base here ($CACHEGAP/v1), unlike the native Anthropic SDK:

import { createOpenAI } from '@ai-sdk/openai';
import { createAnthropic } from '@ai-sdk/anthropic';
import { generateText } from 'ai';

const openai = createOpenAI({ baseURL: '$CACHEGAP/v1', apiKey: 'sk_cg_...' });
const anthropic = createAnthropic({ baseURL: '$CACHEGAP/v1', apiKey: 'sk_cg_...' });

const { text } = await generateText({ model: openai('gpt-5-mini'), prompt: 'hello' });
console.log(text);

STEP 03 · Read your savings in your terminal

Every request produces proof you can read without opening the dashboard. Send anything with curl -i and the response headers tell you what happened:

$ curl -i $CACHEGAP/v1/messages \
    -H "content-type: application/json" \
    -H "x-api-key: sk_cg_..." \
    -d '{"model":"claude-sonnet-4-6","max_tokens":128,
         "messages":[{"role":"user","content":"hello"}]}'

HTTP/1.1 200 OK
x-cachegap-cache: kv_read            <- the provider served the prefix from its prompt cache
x-cachegap-baseline-usd: 0.00826800  <- same request at list input rates, no caching
x-cachegap-actual-usd: 0.00150450    <- what it actually cost, write premiums included
x-cachegap-savings-usd: 0.00676350   <- baseline minus actual
x-cachegap-optimized: 1              <- the optimizer changed this request

Every x-cachegap header

At most one of the four mode flags appears per response, with precedence fail-open, then shadow, then passthrough, then optimized. An exact-match cache hit carries no mode flag at all; the x-cachegap-cache value tells that story.

The _cachegap body field

Non-streaming JSON responses also carry a non-breaking _cachegap field with the same numbers plus token counts:

"_cachegap": {
  "cache": "kv_read",
  "baseline_usd": 0.008268,
  "actual_usd": 0.0015045,
  "savings_usd": 0.0067635,
  "input_tokens": 2756,
  "cached_tokens": 2688,
  "output_tokens": 40
}

Set STRICT_BODY_PARITY=true on the proxy to omit it and keep response bodies byte-identical to the provider's; the headers still carry the proof. The field is also absent on provider errors (those pass through untouched) and whenever measurement fails.

Streaming

Streams get the mode flag up front, but the dollar figures only exist once the stream ends, so they arrive as a trailing SSE comment after the final event:

data: {"type":"message_stop"}

: cachegap {"cache":"kv_read","baseline_usd":0.008268,"actual_usd":0.0015045,...}

SSE comments (lines starting with :) are ignored by spec-compliant clients, so nothing breaks. If measurement fails the comment is simply absent; the receipts endpoint below is the authoritative record either way.

Receipts: the authoritative record

curl "$CACHEGAP/v1/usage/receipts?limit=5" -H "x-api-key: sk_cg_..."
curl "$CACHEGAP/v1/usage/summary" -H "x-api-key: sk_cg_..."

Every request persists a receipt: model, tokens (input, cached-read, cache-write, output), baseline, actual, savings, and the cache hit type. limit takes 1 to 200 (default 50). Spot-check any line against the usage block on your own provider invoice; the numbers reconcile per request, which is the whole point.

Schema isolation guarantee

The exact-match cache key is a hash of the entire request body with keys canonicalized, scoped by customer, provider, and user. Anything that changes the output contract (response_format, tools, tool_choice) changes the key, so a cached response can never violate the schema you asked for. Structured-output requests only ever match exactly; when the opt-in semantic cache ships later, they stay excluded from semantic matching entirely.

← run the free audit on a real request first