Two Solr 9 cores, provisioned and managed by Loghound. Field names are a contract: the parser writes them, the scorer reads them and the panel queries them, so renaming one is a breaking change to three components at once.
| Core | One document per | Queried by |
|---|---|---|
| hits | log line | The session drill-down, the latency percentiles, and the fingerprint facet the scorer runs |
| sessions | session, plus one per day for the rollup | Everything else in the dashboard |
The names are generated per installation rather than fixed — see provisioning. The identifiers stored in these fields are on identifiers and labels.
These configsets are pushed when the indexes are provisioned, and upgrading the code does not re-push them. A release that adds a field therefore writes a field the live schema does not declare — and the catch-all in section 04 silently discards it. bin/loghound-schema is what puts that right, and upgrading explains why running it is not optional.
- Not stored by default; retrieve from docValues. The dashboard facets and sorts on nearly every field, so docValues is being paid for regardless. Storing as well writes the value a second time. Only fields a human reads in the drill-down are stored.
- No norms on any string field. Norms exist for relevance scoring by length, and nothing here is ranked on a string field — they are filtered and faceted. At forty-six string fields and ten million documents that would be 460 MB of nothing.
- No term frequencies or positions on string fields. They answer “how often” and “how near”, and an exact match on an address needs neither. Roughly halves the postings size for those fields.
- No stock sprawl. Solr’s default configset ships a catchall plus about thirty dynamic fields and a rule copying every value into it. On a schema with seventy named fields that makes the catchall the largest single component of the index — for a field nothing in Loghound queries. Both are deleted.
Figure 1 — two cores with mirror-image workloads, which is why they are configured differently. Ten to twenty times fewer documents on the right is what makes “bots by network” one cheap query rather than a scan of the hit stream.
| Field | Why it is shaped that way |
|---|---|
| the hit id | A hash of the installation id, the source file and the byte offset. Deterministic, so replaying a rotated file overwrites instead of duplicating. The installation id is in it for a reason: two machines tailing identically named files would otherwise produce identical ids, and a duplicate key in Solr is a delete-and-add — each would silently have overwritten the other’s traffic, one request at a time, with no error anywhere. |
| the address | The most expensive docValues field in the schema, and non-negotiable: the distinct count per fingerprint is the product’s headline metric. |
| the query string | Stored only — not indexed, no docValues. Query strings are unbounded attacker-controlled text with a near-unique value per request; indexing them would build a term dictionary the size of the corpus and faceting on it would be meaningless. Search terms are its deliberate opposite. |
| the response time | Absent when the log format does not carry it, never zero — a zero would drag every latency percentile toward the floor. |
| request headers | Present only when the operator logs them, and absent rather than empty string when they are not. An empty string is a real value that facets, sorts and matches, and it would make “no header” indistinguishable from “an empty header” — two rules worth 70 points or more turn on exactly that distinction, and both go silent rather than guessing. |
| the coordinate | Indexed only, so it answers bounding-box queries but cannot be retrieved or aggregated — which is why the map is country resolution. It is also absent rather than wrong when the service returned only a country-level centroid. |
| the timezone | From the geolocation service when it names one, otherwise derived locally from the country for the 216 territories that have exactly one zone. Absent for a multi-zone country, because guessing one would make the mismatch rule fire on people who did nothing wrong. |
| the JA4 fingerprint | Defined and nothing writes it. It needs a TLS-terminating proxy in front exporting it into a header. Defined now so enabling it later is a configuration change rather than a reindex, and an always-absent field costs zero bytes. |
| the path list on a session | Capped at fifty. A crawler walking two hundred thousand URLs would otherwise put two hundred thousand terms into one document, which is where a document goes from large to breaking the merge. The true distinct count is carried separately, so nothing is fabricated — but the interface must label the list a sample. |
| the fingerprint count | Absent when it could not be computed, and the rule that reads it must not treat absent as one. |
| the reasons list | Never empty. A clean session carries a code saying nothing fired. |
Discriminated by a document-type field rather than being a third core, which would triple the provisioning for a few hundred documents a year. It is recomputed from the sessions on every scorer run, never incremented: a client retries a request whose response was lost, and a retried increment double-counts, whereas recomputing with one facet query is idempotent, exact and self-healing — and it is what makes a shared pair converge rather than fork.
These are estimates from field-count arithmetic rather than measurements of a live index, shown as a breakdown so you can check the reasoning and adjust for your own traffic. Real numbers depend heavily on how many distinct paths and addresses you see.
| Per million hit documents | plain combined | recommended format |
|---|---|---|
| docValues | ~105 MB | ~140 MB |
| Inverted index | ~60 MB | ~85 MB |
| Stored fields, excluding the raw line | ~105 MB | ~110 MB |
| The raw line copy | ~65 MB | ~165 MB |
| The catchall | ~275 MB | ~285 MB |
| Everything on | ~0.6 GB | ~0.79 GB |
| Raw copy and catchall off | ~0.27 GB | ~0.34 GB |
The sessions core is much smaller: at eight to fifteen hits per session, roughly 50–90 MB per million hits. Daily rollups are about 100 KB a year, which is why keeping them forever is the default.
What to turn off, in order
- The catchall. The biggest single win, roughly 45% of the hits core. You lose the free-text search box in the session explorer; every facet, filter and chart keeps working.
- The raw line copy. About 11% on plain combined, 21% on the recommended format. You lose the ability to re-derive documents under a future ruleset.
- The retention window. The only lever that bounds growth rather than reducing a constant.
- Drop the stored query string if you never look at query strings.
Deleted documents do not return disk immediately — the index reclaims on merge. Both cores are configured so that happens steadily rather than in one stall, and nothing forces a full merge, which would rewrite the whole index and need double the disk while it ran.
The stock configset loads extraction, clustering, language detection, templating and data-import libraries. Every one is an unauthenticated code path behind a request handler, and the extraction one in particular has a long history of remote code execution triggered by a hostile document. Loghound needs none of them, and not loading a library is a stronger control than not defining its handler.
No streaming, SQL, export, replication, extraction, debug-dump, terms, term-vector, analysis, spelling, suggest, browse or elevation handlers. Each removal is annotated in the configset with what it would have handed an attacker. The short version: the streaming and SQL handlers execute a server-side language with network and database sources; the export handler dumps the entire core in one response; the replication handler serves the raw index files; the terms handler enumerates the term dictionary, which on this schema means every address and every URL in the index without a query.
Remote streaming and stream bodies are written out as disabled explicitly even though they are already the defaults, so that copying a stock configuration over this one is a visible change rather than a silent regression — and the Solr client refuses to send any streaming parameter independently. Two locks on the same door.
The one catch-all that remains
A dynamic field that swallows anything not named in the schema and indexes none of it. The trade-off, stated plainly: a schema and code that drift apart degrade quietly — a value from a future version vanishes instead of failing the request, which is right for a running ingest daemon, because the alternative is one bad field killing an entire batch and stalling the tailer. The cost is that a typo in a field name is silent, and so is a live index left behind by an upgrade. If you are developing new fields, comment that line out and let Solr reject them loudly until you are done — and in production run bin/loghound-schema after every upgrade, which is the supported way to catch the same problem.
Both cores use a five-second soft commit for visibility and a sixty-second hard commit that flushes segments to disk and truncates the transaction log without opening a new searcher. That second half is the important one: it gives durability with no cache flush and no warming stall, and visibility stays entirely the soft commit’s job. Getting it backwards is the classic way to make a Solr node spend all its time warming.
The two cores differ on caches because their workloads are mirror images: hits is write-heavy, read-rare and append-only, so nothing is autowarmed there — with a five-second soft commit a searcher lives five seconds, and warming it costs more than the queries it would serve. Sessions is read-heavy, write-light and facet-dominated, with writes arriving once a minute and the dashboard reusing the same handful of filters, so warming pays for itself.
Both use the higher-compression codec rather than the faster default, because log lines are extremely repetitive: it typically reaches four to six times on the raw line where the default reaches two to three. The cost is processor time on stored-field retrieval, and stored fields are retrieved only in the drill-down, one screen at a time.