We keep blowing past our OpenAI API token budget every Thursday afternoon, what are the best practices for caching and semantic routing to cut costs?
Download run
Fan-out diagram
The searches the AI actually ran for this prompt.
SEED QUERY
We keep blowing past our OpenAI API token budget every Thursday afternoon, what are the best practices for caching and semantic routing to cut costs?
2 real searches · 1 real-search pass
Badge and edge weight = share of passes that produced the query. Hover a query for its intent and routing format.
Query classes
Google's semantic query class for every sub-query: what kind of answer each search is after, simulated vs real.
Classifying 2 queries…
Searches and sources
ChatGPT2 real searches · gpt-6
OpenAI API prompt caching documentation cached input tokens automatic caching cache retention 2026 Google ↗
OpenAI API embeddings documentation semantic search vector stores batch API pricing model routing official docs Google ↗
- Vector embeddings · developers.openai.com
- Retrieval · developers.openai.com
- Prompt caching · developers.openai.com
- Prompt caching · developers.openai.com
- Batch API · developers.openai.com
- Batch API · developers.openai.com
- Flex processing · developers.openai.com
Queries
2 queries · click a column to sort
| Query / type | Search | Intent | Query class | Routing format | Cluster | Freq ↓ | Weight |
|---|---|---|---|---|---|---|---|
OpenAI API prompt caching documentation cached input tokens automatic caching cache retention 2026 OBSERVED | Google ↗ AI Mode ↗ | — | — | — | — | 1x of 1 | |
OpenAI API embeddings documentation semantic search vector stores batch API pricing model routing official docs OBSERVED | Google ↗ AI Mode ↗ | — | — | — | — | 1x of 1 |