We keep hitting memory limit errors on our ClickHouse cluster during nightly batch loads and I need to know how to fix it without scaling up.
Download run
Fan-out diagram
The searches the AI actually ran for this prompt.
SEED QUERY
We keep hitting memory limit errors on our ClickHouse cluster during nightly batch loads and I need to know how to fix it without scaling up.
1 real search · 1 real-search pass
ClickHouse official documentation memory overcommit settings max_memory_usage max_bytes_before_external_group_by external sort insert block size background mergesGoogle ↗
OBSERVEDOther / navigational
Badge and edge weight = share of passes that produced the query. Hover a query for its intent and routing format.
Query classes
Google's semantic query class for every sub-query: what kind of answer each search is after, simulated vs real.
ChatGPT real searches (1)
Classes come from the query-classifier (a local Laya model). 1 of 1 calls are low-confidence (under 50%) — hover a query in the table to see its confidence.
Searches and sources
ChatGPT1 real searches · gpt-6
ClickHouse official documentation memory overcommit settings max_memory_usage max_bytes_before_external_group_by external sort insert block size background merges Google ↗
- Supercharging your large ClickHouse data loads - Part 2: Tuning a large data load for speed · clickhouse.com
- Getting started with ClickHouse? 13 mistakes and how to avoid them · clickhouse.com
- clickhouse-docs/knowledgebase/memory-limit-exceeded-for-query.mdx at main · ClickHouse/clickhouse-docs · github.com
- ClickHouse/src/Core/ServerSettings.cpp at master · ClickHouse/ClickHouse · github.com
- ClickHouse concurrency: how to size for user-facing analytics | Engineering | ClickHouse Resource Hub · clickhouse.com
Queries
1 queries · click a column to sort