Can someone explain how managing LLM token budgets works because our monthly OpenAI bill is way too high right now.
Download run
Fan-out diagram
The searches the AI actually ran for this prompt.
SEED QUERY
Can someone explain how managing LLM token budgets works because our monthly OpenAI bill is way too high right now.
1 real search · 1 real-search pass
OpenAI API documentation prompt caching input output tokens usage pricing token optimization officialGoogle ↗
OBSERVED
Badge and edge weight = share of passes that produced the query. Hover a query for its intent and routing format.
Query classes
Google's semantic query class for every sub-query: what kind of answer each search is after, simulated vs real.
Classifying 1 queries…
Searches and sources
ChatGPT1 real searches · gpt-6
OpenAI API documentation prompt caching input output tokens usage pricing token optimization official Google ↗
- Prompt caching · developers.openai.com
- Prompt caching · developers.openai.com
- Reviewing API usage and costs · help.openai.com