Prompt compression API
Half the context.
The same bytes.
Your model does not need all two thousand reviews to find the one reporting a fire hazard. tokz returns byte offsets into the text you already have — not a summary, not a paraphrase — and your SDK reassembles locally, so every byte the model reads is a byte you sent.
Every account starts with 1 GB of payload free, no card. Two lines to integrate, and no key set means calls run uncompressed rather than failing.
Asked: What should we do when we hit the rate limit?
Welcome to the API reference, regenerated from the OpenAPI schema on every release. SDKs for TypeScript, Python and Go are listed on the downloads page. Every endpoint is rate limited per project key, not per signed-in user. Requests over the limit return 429 with a Retry-After header in seconds. Timestamps are RFC 3339 in UTC, and we never return a local offset. Webhook signatures are HMAC-SHA256 over the raw body. When you hit the limit, wait the full Retry-After interval before sending the request again. If anything here is unclear, the developer forum is the fastest way to reach us.
5 sentences dropped
3 kept · 5 dropped
On structured payloads, we send back offsets — not text
On structured payloads — tool output, JSON, logs, diffs — tokz returns byte ranges, not a shortened copy: nothing to drift, nothing to store. Prose is the exception: extractive chunks with a provenance map, still checkable, but the response does contain text.
- 01
You POST the payload
Tool output, retrieved documents, a transcript. It is parsed, scored and ranked in process. No model is called — there is no LLM anywhere in the request path.
- 02
You get offsets back
An ordered list of byte ranges to keep, the runs that were dropped, and a SHA-256 of the source so a map can never be applied to text it was not built from.
- 03
The SDK reassembles locally
Bytes are sliced from the copy already in your process. On the structural path verbatim is not promised, it is structural: the server emits no text, so it has no way to alter one.
- Byte-exact
- every fragment is a literal substring of what you sent
- Deterministic
- same input, byte-identical output; cache-aware policies protect changing prefixes
- Stateless
- nothing retained — not in a table, a log, or an error message
What survives, on real payloads
Everyday exports, cut to roughly half, then asked the question the data existed to answer. The one line that matters is the one that lives.
| Tool output | Sent | Question it still answers |
|---|---|---|
| Product reviews0.9 KB | 45% | Is anyone reporting a defect? battery swelled and cracked the casing |
| Online orders1.2 KB | 41% | Which order went wrong? lost in transit, refund pending |
| Website uptime checks1.2 KB | 42% | When did the site go down, and why? 12:00 — database connection refused |
| Expense claims1.3 KB | 41% | Which claim breaks policy? £4,800 taxi — exceeds policy limit |
| Freezer temperatures0.8 KB | 42% | Which freezer is failing? freezer-12 — door left open |
| Hotel bookings1.3 KB | 44% | Which booking was a no-show? room 112 — held, unpaid |
No LLM in the request path
Compression is a parse and a sort, not a generation. A small classifier scores prose in-process — it decides what to keep and cannot write anything — so no request ever waits on a language model.
0.13 µs
per byte, structural, warm
Under 10 ms to ~64 KB, about 45 ms at 350 KB. Linear in input size, so a payload twice as big costs twice as much and never more.
100%
of structured payloads still parse
After compression, the JSON is still JSON. Truncation baselines at the same ratio: 0–19%.
Dropping bytes is not a one-way door
Every elision carries its position and its length, so the run that was cut is still addressable. expand() returns it from the copy you already hold — no second API call, no key, no network. Compression you cannot undo is a decision you make once and live with; this one you can walk back per request.
expand() is pure and local. It is exported from the SDK and does not talk to us at all.
import { expand } from "@tokz/sdk";
// `source` is your original payload — the one the span map was built from.
// Elision 0 is the run of Running pods the compressor dropped.
const dropped = expand(source, result.spanMap, { elision: 0 });
// Byte-for-byte what was there. The hash on the span map is checked first, so a
// map can never be applied to text it was not built from.Two lines
Wrap the client you already have. Every call site stays the same.
$ npm i @tokz/openai
import OpenAI from "openai";
import { withTokz } from "@tokz/openai";
const openai = withTokz(new OpenAI(), {
apiKey: process.env.TOKZ_API_KEY!,
});
// Everything below is unchanged. Bulky tool results get
// compressed on the way out; responses come back untouched.Pricing
What it costs, and when it pays
- $0.10per 1M tokens
- Metered exactly in bytes at $0.025 per million. Tokens are the quote, bytes are the ledger.
- 1 GBfree at signup
- $25 of credit, no card. It does not expire.
- $0everything else
- No seat licence, no monthly floor, no charge for expand().
Questions worth asking
How is this different from the compaction the model providers ship?
Provider compaction runs an extra sampling pass to summarise your context, bills you for it, and then drops the original — so you cannot reconstruct what the model actually saw. tokz calls no LLM at all, returns offsets into text you still hold, and is reversible.
Does compression break my prompt cache?
No, and that is the point of determinism. The same input produces byte-identical output, so a compressed prefix stays stable across calls and keeps hitting. A non-deterministic compressor invalidates the cache on every request.
What happens if tokz is down or my key is missing?
Your call runs uncompressed. The wrappers fall back to the original text rather than failing the request, and a circuit breaker bounds a dead API to one trial per reset window instead of a timeout on every payload. On the raw SDK the fallback is opt-in — compress() throws, and resilientCompress() is the helper that degrades — because a client that silently returns your input when it failed is hard to debug.
When is it not worth using?
On cheap models, and on context your prompt cache already holds. Compression pays above roughly $0.18 per million input tokens on uncached payload; below that the fee exceeds the saving. The estimator says so plainly.
Find out if it is worth it
The estimator uses your bill, your model and your cache rate — and it will tell you when compression does not pay. Below roughly $0.18 per million input tokens, it does not.
Open the estimator