GPT-6 Astra Prompt Caching Guide: How to Reduce Repeated Context Costs

Large agent prompts often repeat the same system policy, tool definitions, product documentation, examples, and conversation history. Re-sending that material is sometimes unavoidable; paying full input cost and latency for an identical prefix is not. GPT-6 Astra supports prompt caching in the Responses API so repeated prefixes can be reused.

Caching is an optimization, not memory. It does not make a model remember a customer between requests, and it does not change what the model sees. The request still needs the relevant input. The difference is that an eligible, identical prefix can be served from cache at a lower cached-input rate.

What GPT-6 Astra caches

The cache is prefix-based. OpenAI can reuse tokens from the beginning of a prompt when the next request starts with matching content. A useful mental model is a document whose stable chapters come first and whose request-specific appendix comes last.

Put these near the front:

• stable developer instructions;
• tool schemas in a stable order;
• long reference documents used across requests;
• canonical examples and output rules.

Put these near the end:

• the current user message;
• timestamps, request IDs, and temporary state;
• retrieved passages that change on every call;
• per-user preferences that are not shared.

One timestamp inserted near the top can invalidate everything after it. Likewise, generating tool arrays from an unordered map can produce semantically identical but byte-different prefixes. Build prompts deterministically.

Implicit and explicit caching

GPT-5.6 and later models expose prompt_cache_options. In implicit mode, the service identifies a reusable breakpoint automatically. This is the easiest starting point and is suitable when your prompt has one large, stable prefix.

import OpenAI from "openai";

const client = new OpenAI();

const response = await client.responses.create({
  model: "gpt-6-astra",
  prompt_cache_key: "support-agent:v4",
  prompt_cache_options: { mode: "implicit", ttl: "30m" },
  input: [
    { role: "developer", content: "Stable policy and operating instructions..." },
    { role: "user", content: "Why was my invoice duplicated?" }
  ]
});

The currently documented TTL value is 30m, and 30 minutes is the default. Do not design around undocumented durations. OpenAI also marks the older prompt_cache_retention field as deprecated.

Explicit mode gives the application more control. You add prompt_cache_breakpoint content items at boundaries worth preserving. This is useful when a prompt is assembled from several stable blocks followed by volatile material. A request can write at most four breakpoints, and the service considers up to the latest 80 breakpoints. More breakpoints are not automatically better: each write has a cost, and fragmented prefixes can be harder to reason about.

A practical prefix architecture

For a production agent, use four layers:

1. Identity and safety

Place the durable role, safety constraints, and response contract first. Version this block deliberately. A policy edit should create a new cache key instead of silently mixing measurements from two versions.

2. Tool definitions

Tool schemas are often large and repeated. Keep names, descriptions, properties, and ordering stable. Remove unused tools where possible; this lowers both uncached and cached context and reduces tool-selection ambiguity.

3. Shared knowledge

Add durable manuals, taxonomies, style guides, or product documentation. If the knowledge changes frequently, file search may be better than embedding it all in every prompt. Cache stable operating knowledge; retrieve changing facts.

4. Dynamic request state

Append user input, current records, live search results, and temporary state. This placement protects the reusable prefix from routine changes.

For a creative workflow, the stable layer might contain animation production rules and a house style, while the dynamic tail contains the current scene. A platform such as Elser AI could apply the same principle to repeated story-bible or character-consistency context without implying that caching itself creates consistency.

Measure savings instead of assuming them

Inspect usage.input_tokens_details.cached_tokens and cache_write_tokens. A high cached-token count shows that part of the prefix was reused. Cache-write tokens reveal the cost of creating or refreshing entries.

For GPT-6 Astra, the published model pricing at the verification date lists cached input below normal input and cache writes above normal input. That creates a break-even question: a prefix reused repeatedly can save money; a prefix written once and never reused can cost more. Pricing changes, so calculate with the current model page rather than hard-coding numbers into planning spreadsheets.

Track at least:

• cache hit rate by prompt version;
• cached, written, and total input tokens;
• p50 and p95 time to first token;
• cost per completed task, not merely per request;
• cache misses caused by releases.

A dashboard grouped only by model hides the cause of misses. Include a cache key or prompt-version dimension in your own telemetry, but never put personal data or secrets in cache keys.

Seven common cache-miss causes

Dynamic content appears too early

Move dates, user IDs, and retrieved material after the reusable content.

Tool schemas change order

Sort tools and schema properties deterministically in the application build step.

Prompts are “equivalent” but not identical

Whitespace, examples, or serialization can differ. Generate shared blocks from versioned artifacts rather than ad hoc strings.

The prefix is too short

Minimum cacheable length varies by model. Very short prompts may not benefit. Confirm through usage data.

Compaction changed the prefix

Compaction helps fit long conversations but produces a different context representation. Expect reuse patterns to change after compaction and measure around milestone boundaries.

Too many low-value breakpoints

Breakpoints should correspond to meaningful reusable layers. Four allowed writes are a ceiling, not a target.

A cache key is too broad or too narrow

One key for every unrelated workflow produces weak grouping; a unique key per request prevents reuse. Prefer a semantic key such as legal-review:v3:us.

A safe rollout plan

Start with implicit mode on one high-volume workflow. Stabilize prompt construction, record token details, and compare two weeks of cost and latency. Then consider explicit breakpoints if the prompt has multiple reusable layers or frequent dynamic tails. Evaluate quality alongside savings: aggressive removal of context is not caching, and it may reduce answer quality.

Cache only content you are already permitted to send to the API. Caching does not replace data classification, tenant isolation, access controls, or retention decisions. Keep secrets out of prompts whenever a tool can fetch them just in time.

FAQ

Does prompt caching reduce output-token cost?

No. It applies to eligible repeated input. Output is generated normally and billed at the output rate.

Does previous_response_id make prior turns free?

No. OpenAI states that earlier input tokens in a response chain are still billed as input. Prompt caching may reduce eligible repeated-prefix cost, but conversation chaining and caching are separate mechanisms.

Should I cache retrieved search results?

Only when they are stable and genuinely reused. Live results normally belong in the dynamic tail. For controlled corpora, file search can avoid embedding an entire collection in every prompt.

Can I rely on a cache hit?

Treat caching as an opportunistic optimization. Your application must remain correct on a miss.

Conclusion

The highest-value GPT-6 Astra caching strategy is architectural: stable instructions and tools first, volatile state last, deterministic serialization, deliberate versioning, and measurement through token details. Begin with implicit caching, add explicit breakpoints only when data supports them, and optimize cost per successful task rather than chasing a he