100 GB, Two Ways: Measuring Interactive Questions over Records and Documents
An engineering evaluation on one workstation: a TPC-H-derived workload of 866 million rows and a synthetic corpus of 11 million passages, what a small memory budget does to them, and why loading costs more than querying.
Roozbeh Bandpey
Gründer & CEOA quality lead asks how many units of one part number failed final inspection last quarter. The answer is in an operational database with a few hundred million rows, built to record transactions one at a time and slow at any question that has to read all of them. A maintenance engineer asks where the torque procedure for the same part was last changed. That answer is in a folder tree of PDFs, spreadsheets and saved mail. Both would say they have "about 100 GB of data", and both expect an answer while the conversation is still going.
We wanted to know, with numbers, what it takes to answer both kinds of question interactively at that size. This post reports an engineering evaluation: a workload derived from TPC-H at scale factor 100 for records, and a synthetic corpus of 11 million passages for documents, both on one workstation.
Two things to know before the numbers. First, the record results with larger resource budgets come from an evaluation of the next version of our analytics layer. They are not the behaviour of the release customers run today, which is sized for files attached to a page; we show that starting point as well. Second, everything ran on one machine that was doing other work at the same time. There are no cloud numbers in this post and no claims about cloud hardware.
"100 GB" means two different things
Records are rows and columns: orders, inspections, sensor readings, an export of an operational database. The questions asked of them are how many, how much and which rows. Content is what people write and read: procedures, reports, contracts, mail. The questions asked of it are what it says and where that is written.
The two need different machinery. A count over 600 million rows wants a columnar format and an engine that scans it with every core. A question about what a procedure says wants an index over passages of text, by words and by meaning. Embedding 600 million rows as vectors would answer no question that a WHERE clause does not answer better, and loading PDFs into tables answers nothing at all.
So we keep them apart, under one roof. Records go to a lake: Apache Parquet files read by an embedded columnar engine through SQL. Content goes to a document index: passages with a full-text index and a vector index in a relational database. What makes it one system is what sits above both: one catalog that names every table and every folder of files, and one permission model that decides every read. A table's description (its name, columns and comments) is itself indexed as content, so an assistant finds the right table by meaning and then queries its rows in the lake.
Diagram key: solid arrows are data or control; the dotted arrow is the one bridge between the two stores. A table's description is searchable as content. Its rows never are.
Three things make this hard at 100 GB.
- An operational database is slow at analytical questions by design. Copying its rows into a columnar format fixes that only if the engine reading them has room to work: threads, memory, and somewhere to put intermediate results that do not fit in memory.
- Search over documents has to apply permissions inside the ranking, not after it. Otherwise the best ten results can all be passages the reader may not see.
- Getting the data in is a different problem from querying it, with its own bill.
The path of a question
Both kinds of question take the same first steps, which is why one catalog matters more than which engine sits behind it.
Diagram key: the assistant first asks the catalog what exists and is told only about what this person may read. Every query that follows runs with the same person's rights.
How we measured
Hardware. One 16-core Apple M4 Max workstation (12 performance cores, 4 efficiency cores) with 128 GB of memory and an internal SSD. Other engineering work shared the machine throughout; its load average was between 5 and 30. We report single runs unless we say otherwise.
Records. We generated the TPC-H dataset at scale factor 100 with the generator that ships with the engine: eight tables, 866 million rows, 600 million of them in the largest. We wrote each table as 16 Parquet files with Zstandard compression, in the order the generator emits rows, which is by key. Then we ran TPC-H's 22 queries and 16 further questions of our own that look like what people ask an assistant: one record by key, rows by a column the files are not sorted by, a month's total, a top 20, a substring anywhere in a text column.
Each run is one engine instance with a fixed budget of threads and memory, reading the Parquet files directly. The files were in the operating system's cache. We ran every statement once per budget; a repeat of one budget gave the same total within 1 % and single statements within 9 %.
This workload is derived from the TPC-H Benchmark and as such is not comparable to published TPC-H Benchmark results, as these results do not comply with the TPC-H Benchmark specification. Nothing here is audited. We ran the queries one at a time, with no throughput test and no refresh functions.
Where 866 million rows sit as Parquet
Gigabytes per table, Zstandard-compressed, 16 files a table.
**Documents.** We used our own benchmark for the document index: one tenant with 1,000,000 synthetic documents of ten passages each, about 250 words a passage, drawn from a 5,000-word vocabulary with skewed frequencies so that some words are in almost every passage and some are rare. A second tenant's million passages share the same indexes. The reader in the test can read one workspace in full and five projects of another, and 25,000 documents carry a classification above the reader's clearance. We ran 200 queries of one or two words, each with a query vector, and report percentiles. The vectors in this corpus are random. That is the hardest case for an approximate index's speed, and it means this test measures latency and size only. **It says nothing about recall**, which needs real embeddings and judged results.
Diagram key: the upper row is the record workload, the lower row the document workload. Both end in a wall-clock time per question, measured from outside the engine.
Starting point: a budget sized for a file on a page
The analytics layer in our current release was built for a narrower job: someone attaches a spreadsheet or a CSV to a page and asks a question about it. Each question gets a small, fixed budget and must stay in memory. That is the right shape for files of megabytes, and it keeps one person's question from taking a machine away from everyone else.
We ran the 100 GB workload under that budget, unchanged, to see what happens.
The starting budget on the 22 analytical queries
A small in-memory budget, sized for a file attached to a page.
The result is clean. Nothing was slow. The queries that fit ran in seconds, and the ones that did not fit stopped almost at once. The same held for our own 16 questions: 14 completed and 2 ran out of memory. The limit was memory, never time.
Spilling to disk is necessary, and not enough
A columnar engine that runs out of memory during a join or a large aggregation has a standard answer: write part of the intermediate state to disk and continue. We allowed that, changing nothing else, and re-ran the eight failures. For contrast, the chart also shows the same eight under a larger budget.
The eight queries that failed, once they may spill to disk
Seconds per query. Same small budget with spilling allowed, against 8 threads and 16 GB.
All eight now complete. Two of them are still far outside anything a person would call interactive. Allowing spill turns "cannot answer" into "answers eventually", which matters for a batch job and does not help a conversation.
Memory per thread decides the heavy joins
The next question was whether those two slow queries wanted more threads or more memory. We ran four budgets that separate the two: 8 threads on 4 GB, 4 threads on 8 GB, 8 threads on 16 GB and 16 threads on 32 GB.
The heaviest joins as the memory budget grows
Seconds per query under four budgets, spilling allowed in all four.
For the heaviest join, memory per thread mattered more than the thread count. Eight threads that share 4 GB spend their time writing to and reading from disk; total CPU time for the whole workload rose by about 30 % in that run. This agrees with the usual guidance for columnar engines: budget memory per thread, more for joins than for aggregations, and give the engine a fast local disk for whatever is left.
All 22 queries under three budgets
With 2 GB per thread and spilling allowed, we ran the full set at 4, 8 and 16 threads.
All 22 queries under three budgets
Seconds per query; within a row, darker is slower. One run each, file cache warm.
This is the main record result. On one workstation, every one of the 22 queries over 866 million rows completes in 17.2 s or less with four threads, and in 7.5 s or less with eight. Most are far below that: with eight threads, 19 of the 22 finish in under three seconds.
Lookups, scans, joins and text behave differently
The 22 queries are analytical by construction. People also ask for one order, or for every row that mentions a word. We grouped eight kinds of question to see which ones a budget changes.
Eight kinds of question over 600 million rows
Seconds per question under two budgets: a record by its key, rows by a column the files are not sorted by, one month's total, a group over the whole table, ten million groups, an exact distinct count, a multi-table join, and a substring in 600 million text fields.
Two of these deserve a comment.
A lookup by key does not need an index. The files are written in key order, and Parquet stores the lowest and highest value of each column for every block of rows. The engine reads those and skips everything else, so one order among 600 million rows comes back in 0.03 s under any budget. A lookup by a column the files are not sorted by has to read that whole column: 1.3 s on four threads.
Brute force over text is viable inside the lake. Matching a substring against all 600 million comment fields, 15.9 GB of text, takes 4.2 s on eight threads, decompression included. For comparison, a widely used command-line search tool reports about 1.9 GB/s on one core for a common word in a 13 GB file held in memory (ripgrep README, read on 18 November 2026). Our run is about 0.5 GB/s per thread from compressed Parquet. So a free-text column of tens of gigabytes can simply be scanned. That does not carry over to 100 GB of documents, which are not plain text, do not fit in memory, and cannot be scanned across permissions.
Scaling with threads
Seconds per question at 4, 8 and 16 threads, 2 GB of memory a thread.
Documents: search stays fast at 11 million passages
How many passages 100 GB of documents contain depends on how much of the 100 GB is text. By our estimate, an office mix, where a megabyte of file holds a few tens of kilobytes of text, gives about one million, and text-heavy files such as mail and text PDFs give about ten million. We tested eleven. The 2 million figures below are from an earlier run of the same benchmark.
The search is one statement. It takes the best full-text matches and the nearest vectors, both filtered by what the reader may see, and fuses the two rankings.
Document search latency at 2 million and 11 million passages
Milliseconds, 200 queries, words and meaning fused, permissions applied in the ranking.
At 11 million passages the vector index was 8.7 GB and the full-text index 3.2 GB, both in proportion to what we had measured at 2 million. Building the two after a bulk load took 32 minutes. The whole database, with passages, their full-text entries and every index, was 78 GB. The vectors are compact, a few hundred dimensions at half precision; with full-size vectors the index would be several times larger.
What surprised us: the word index bent before the vector index
We expected the vector index to be the first thing to struggle. It was not. Its share of a search took 30 to 35 ms at 11 million passages, even for a reader who may see only a small slice of a workspace.
What bent was full-text search, in one specific case that our own benchmark is built to catch: a search inside a single workspace, which is how an assistant searches, by a reader who sees only a few projects of it, for a word of middling frequency. Such a word matched about 380,000 passages of the tenant. The database keeps a map of matching rows in working memory; at that many matches the map no longer fit and fell back to tracking whole pages instead of rows. Every row on those pages then had to be checked against the query again, about two million of them. That one search took 3.7 s.
At 2 million passages the same test passes with room to spare. At 11 million it fails, in every one of eight passes, so it is not a cold cache. We traced it, tried two changes on the same data, and timed the search statement alone over 200 queries.
Search inside one workspace: as built, and with two changes
Milliseconds for the search statement alone, 200 queries at 11 million passages.
The first change is a guard: give the statement enough working memory that the map stays exact. The second is the real fix: let the full-text index know which workspace a passage belongs to, so a search inside one workspace collects that workspace's matches only, a quarter of them in this corpus, and then discards 2,500 rows the reader may not see instead of 108,000.
Two lessons. Filtered vector search gets the attention, and at this size it was the well-behaved half. And the failure only appears with a reader who has partial rights and a tenant large enough for common words to match hundreds of thousands of passages. A benchmark with one all-seeing user would have passed.
Loading is where the time and the money go
Query time turned out to be the smaller problem in both halves. Loading is the larger one, and it differs between them.
Records cost time, not money. Converting CSV to Parquet ran at 1.4 GB/s on eight threads, 1.1 GB/s when also sorting: 5 GB of CSV, 37.5 million rows, in under five seconds. At that rate the conversion of 100 GB is a matter of minutes, and the real bound is how fast the source database gives up its rows, which we did not measure. No model is involved, apart from embedding a short description of each table.
Documents cost model calls. We derived the figures below from token counts and public list prices; we called no provider. A passage is about 500 tokens to embed. Extracting entities and facts from it with a language model is about 640 tokens in and 120 out. Prices are list prices per million tokens on OpenAI's and Google's pricing pages, read on 18 November 2026: USD 0.02 to 0.13 for embeddings, USD 0.10 to 0.30 in and 0.50 to 2.50 out for small text models, and USD 1.00 to 1.50 in and 5.00 to 7.50 out for mid-tier ones. The embedding prices and the lower text-model prices are from the first page; the higher text-model prices are from the second.
Model cost of loading 10 million passages
US dollars at the lowest and highest list price in each class, read on 18 November 2026. No provider was called.
Embedding everything is cheap. Having a model read everything is not, and at ten million passages it is also 7.6 billion tokens to push through a provider's rate limits. So it makes sense to treat documents in tiers, and to let use decide how far each one goes.
Diagram key: Listed means only the file's name, path and dates are known, which costs almost nothing. Searchable means its text is indexed by words and meaning. Understood means a model has read it and its facts are linked to the passages that state them.
A file that nobody asks about never needs more than a listing. The first question about it pays for a fetch, a conversion and one embedding call, which we estimate at seconds to a few tens of seconds and have not measured; every later question is an ordinary search. This is a design we are evaluating, not a description of the current release.
Limits of this evaluation
- One machine, shared. A workstation core and a cloud virtual CPU are not the same unit. These numbers do not transfer to a cloud instance by counting threads, and we make no cloud claim.
- Warm cache. The Parquet files were in memory. The first question after a restart reads from disk and is slower; we did not measure by how much.
- One question at a time. No concurrency on the record side. A budget is only half of a serving design; the other half is how many budgets a machine runs at once.
- A derived workload. TPC-H data is uniform and synthetic. Real tables have skew, wide text and messy types, and compress differently.
- Latency without recall. Random vectors measure the speed and size of the document index, not whether it finds the right passage.
- A synthetic corpus. Real word frequencies differ from our 5,000-word vocabulary, and the full-text result depends on them.
- Derived costs. Token counts are our estimates and prices are list prices on one day.
- Not the current release. The larger record budgets and the two full-text changes come from this evaluation.
What we take from it
Read in order, the figures make one argument.
- 100 GB of records is 23 GB of Parquet, and one machine holds it comfortably.
- A budget sized for attached files fails honestly at this size: 14 of 22 queries complete and 8 stop for memory within two seconds.
- Spilling to disk makes everything complete and leaves two queries at 57 and 83 seconds. It is necessary and not sufficient.
- Memory per thread closes the gap: the heaviest join drops from 33.6 s to 5.5 s on the same eight threads with four times the memory.
- With 2 GB per thread, all 22 queries finish in at most 17.2 s on four threads and 7.5 s on eight. A lookup by key takes 0.03 s on any budget, and a substring over 15.9 GB of text takes 4.2 s.
- Document search holds at 11 million passages: 101 ms at the 95th percentile, latency only.
- The full-text half needed attention before the vector half. Scoped to one workspace, it went from 3.4 s to 72 ms at the 95th percentile once the index knew the scope.
- Loading dominates. Records cost minutes of conversion plus whatever the source allows. Documents cost USD 100 to 650 to embed ten million passages and thousands more to have a model read them, which is why reading should be earned by use.
The measurements this run could not give are the ones that matter next: the same workload on cloud instances, cold starts, several questions at once, and recall on real embeddings with judged results.
TPC-H is a trademark of the Transaction Processing Performance Council.
