Bot forensics

The reason codes as a facet you can click, and declared crawlers kept apart from evasive ones.

A facet on the reasons themselves. This is where you find out why each verdict was reached, ranked by how often each signal fires on your own traffic.

01 · Six cards and a cross-table
01 Declared versus evasive

The split that matters most, first, because the two are opposite facts

02 Why each session was scored

The reason codes as bars, ranked by how often they fire on visits judged bot or likely bot. A visit fires several rules, so the bars add up to more than the visits. This is the card to tune from.

03 Verdict distribution

How your traffic falls across the five verdicts

04 Score distribution

Where visits sit on the 0 to 100 scale. A healthy ruleset has two humps; a pile-up in the middle means the weights need tuning

05 Bot classes

Visits judged bot or likely bot by kind: headless, scripted, proxy fleet, spoofed User-Agent, undeclared client, monitor, declared crawler, AI crawler

06 Declared crawlers, by name

Which named crawlers visited, how much, and whether reverse DNS verified the name

The reason facet is the expensive one, and loading it separately is why the other cards are on screen long before it lands.

02 · Declared crawlers are kept separate, always
They are a service you want, not a cost you are paying

Search crawlers and AI crawlers identify themselves and get the verdict bot — they are bots. But conflating “a search engine indexed four hundred pages” with “someone is scraping you from two hundred residential addresses” is precisely what makes other tools useless for this. The separation is enforced in the ruleset rather than by a checkbox here, because every consumer of a session document must see the same split.

An AI-crawler flag is set for around thirty named agents, so “how much of my content is being taken for model training” is one facet click away. Note that several of those are user-triggered fetchers rather than training crawlers — they fetch a page because a person asked an assistant about it — and “an AI crawler visited” and “somebody asked an AI about your page” are not the same event.

A declared crawler that fails forward-confirmed reverse DNS is an entirely different matter: that is a 95-point signal and the class becomes spoofed User-Agent. Claiming to be a major crawler when you are not is the single most common form of crawler impersonation.

03 · The cross-tabulation

This view carries one pivot, folded into a request it was already making so it costs no extra round trip: bot class against network type. A declared crawler on a hosting network is ordinary; a headless browser on consumer broadband is not.

Every cell links through to the Sessions view filtered by both dimensions at once, each keeping the operator it already had. The inner facet is limited, so the cells of a row do not add up to the row total — and the shortfall is printed as its own muted cell rather than left to be inferred.

04 · Reading it honestly

The reason facet is the starting point for tuning, and the procedure is deliberately slow: take the most frequent signal, pull twenty sessions that fired it, and read them. If they are humans, lower that weight. If they are bots, leave it alone.

Never act on a score alone

A session called a bot on an automation marker is not arguable. A session called likely bot on no-JS plus a timezone mismatch is a privacy-conscious VPN user until proven otherwise. False positives is the catalogue of who the defaults get wrong, and Signal and reason codes maps every code to its label.

Some codes on this facet are worth no points and are not accusations: for example one says the visit had not ended when it was scored, and one says it was measured on one plane only. They are there so every verdict explains itself.

Analysis views

Loghound is open source and MIT licensed. Questions about the Opensolr half — the account, the indexes, the plan — go to opensolr.com/contact; questions about the software itself belong on GitHub.

Loghound Documentation