A request log grouped by exact query string is useless. Grouping by shape is what makes the single most actionable number in search visible for the first time.
A search box produces a different string for every visitor, so “top queries” is a list of one-hit wonders and the expensive pattern underneath never surfaces. A shape is the query with its literals removed and its structure kept — so two searches for different words in the same field are one row, and the same word searched in two different fields are two rows.
Figure 1 — why grouping matters. On an ungrouped list a shape that always returns nothing is invisible, because each of those ten thousand requests looks unique.
- Slowest requests — a single call, the highest query times in the range, sorted by the platform, exact and not a sample. Each row carries its shape, so a one-off outlier can be told apart from the worst instance of a pattern that runs constantly.
- Shapes by volume — what is actually being asked of this index.
- Shapes that return nothing — the one to act on.
A shape cannot be faceted: it has to be computed from the full request, which means reading documents, which means the work is unbounded and belongs in something that can be stopped and resumed rather than in one request. So each table is a stepped job of twenty steps reading four hundred requests each.
That is the honest cost of the feature: twenty bounded reads of somebody else’s platform, about fifteen seconds of polling. It is a cap, not a target — a step that reaches the end of the log finishes the job early, so a quiet index completes in one step and only a busy one pays the full twenty. On an index busier than that the tables describe a sample of the most recent traffic in the range.
Distinct shapes retained are capped at 250; past that an unseen shape increments an overflow counter instead of being added, so the numbers on screen never go backwards and the page can say how many were left out. The aggregates are chosen to be mergeable — counts and sums by addition, the maximum by comparison, the latency histogram bucket for bucket — because a percentile computed per page cannot be combined with another page’s at all.
Because it is a job, it survives a page refresh: a reloaded page re-attaches to the newest scan and the table picks up where it was, rather than restarting eight thousand reads. Nothing scanned is stored beyond the job’s own working state, which is swept with the job. Demo mode refuses the start outright, because there is no fabricated search data to scan.
- A renamed or missing field. Something in your application still asks for it; the index no longer has it.
- A filter that can never match. Two clauses that exclude each other, usually after a schema change.
- An interface asking a question the schema cannot answer. A facet or a sort on a field that was never set up for it.
- Genuinely absent content, which is a product finding rather than a bug — and the one case where the right answer is to add the content rather than fix the query.
The truncated request text shown in a shape row was chosen byte for byte by whoever queried the index, so it is truncated server-side and escaped at every point it is rendered.