Bot forensics

The reason codes as a facet you can click, and declared crawlers kept apart from evasive ones.

A facet on the reasons themselves. This is where you find out why each verdict was reached, ranked by how often each signal fires on your own traffic.

01 · Six cards, six independent requests
CardWhat it shows
Declared vs evasiveThe split that matters most, first, because the two are opposite facts
Signals firedThe signal codes as bars, ranked by frequency. This is the card to tune from.
Verdict distributionHow your traffic falls across the five verdicts
Score histogramWhere sessions sit on the nought-to-hundred scale, so you can see how close the mass is to a threshold
Bot classesHeadless, scripted, proxy fleet, spoofed User-Agent, monitor, declared, AI
Crawler roll-callWhich named crawlers visited, and how much

The signal facet is the expensive one, and separating it is why the split, the verdict distribution and the crawler roll-call are on screen long before it lands.

02 · Declared crawlers are kept separate, always
They are a service you want, not a cost you are paying

Search crawlers and AI crawlers identify themselves and get the verdict bot — they are bots. But conflating “a search engine indexed four hundred pages” with “someone is scraping you from two hundred residential addresses” is precisely what makes other tools useless for this. The separation is enforced in the ruleset rather than by a checkbox here, because every consumer of a session document must see the same split.

An AI-crawler flag is set for around thirty named agents, so “how much of my content is being taken for model training” is one facet click away. Note that several of those are user-triggered fetchers rather than training crawlers — they fetch a page because a person asked an assistant about it — and “an AI crawler visited” and “somebody asked an AI about your page” are not the same event.

A declared crawler that fails forward-confirmed reverse DNS is an entirely different matter: that is a 95-point signal and the class becomes spoofed User-Agent. Claiming to be a major crawler when you are not is the single most common form of crawler impersonation.

03 · The cross-tabulation

This view carries one pivot, folded into a request it was already making so it costs no extra round trip: bot class against network type. A declared crawler on a hosting network is ordinary; a headless browser on consumer broadband is not.

Every cell links through to the session explorer filtered by both dimensions at once, each keeping the operator it already had. The inner facet is limited, so the cells of a row do not add up to the row total — and the shortfall is printed as its own muted cell rather than left to be inferred.

04 · Reading it honestly

The signal facet is the starting point for tuning, and the procedure is deliberately slow: take the most frequent signal, pull twenty sessions that fired it, and read them. If they are humans, lower that weight. If they are bots, leave it alone.

Never act on a score alone

A session called a bot on an automation marker is not arguable. A session called likely bot on no-JS plus a timezone mismatch is a privacy-conscious VPN user until proven otherwise. False positives is the catalogue of who the defaults get wrong, and identifiers and labels says what each signal actually tests.

Two signal codes on this facet are worth no points at all and are not accusations: one says the session had not ended when it was scored, and one says it was measured on one plane only. Both are present so the verdict explains itself rather than appearing without a reason.

Loghound is open source and MIT licensed. Questions about the Opensolr half — the account, the indexes, the plan — go to opensolr.com/contact; questions about the software itself belong on GitHub.

Loghound Documentation