Loghound — traffic analytics and bot forensics
Loghound is an open-source, MIT-licensed traffic analyser and bot detector for Apache, nginx and Caddy access logs. It reads the logs your web server already writes, correlates them with an optional JavaScript beacon, stores everything in Solr, and answers two questions that conventional analytics answers badly or not at all.
It is a separate product from the Opensolr control panel. You run it on your own server; Opensolr is simply one of the two places its index can live.
1. It detects bots that every other analytics tool counts as human. Not a User-Agent blocklist — headless Chrome on rotating residential proxies, which is what modern scraping actually looks like.
2. It measures real time on site. Conventional analytics reports how long a tab existed. Loghound reports how long the page was visible, and how long somebody was actually interacting with it.
Source, issues and releases: github.com/phpcip/loghound. Product overview: opensolr.com/loghound.
The three-plane model
Detection happens on three independent planes. Each one alone is defeatable, and essentially every existing tool uses exactly one of them. The power is in correlating them: a scraper has to defeat all three simultaneously, and the things that defeat plane 3 make plane 2 harder, not easier.
| Plane | Source | What it sees |
|---|---|---|
| 1. Transport | the access log line | Headers, ASN, reverse DNS, protocol, status, timing |
| 2. Behaviour | correlation across lines | Asset ratio, cache behaviour, session shape, inter-request timing, and cross-IP fingerprint clusters |
| 3. Execution | the JavaScript beacon | Whether JavaScript runs at all, whether the engine matches what the User-Agent claims, whether a human is driving |
Log-only analysers see plane 1. JavaScript-only analytics sees plane 3 — and trusts it: a client that runs the script is a human, as far as it is concerned. That is precisely why headless Chrome on residential proxies is counted as human traffic almost everywhere.
The fingerprint cluster — the strongest single signal
fp_hash_s is a SHA-1 of a normalised header tuple: User-Agent, Accept, Accept-Language, Accept-Encoding, Sec-CH-UA, Sec-CH-UA-Platform, the four Sec-Fetch-* headers, and the protocol version.
It deliberately excludes the IP address, and that is the entire point. A rotating-proxy fleet changes its exit address on every request and changes nothing else, because the whole value of the fleet is that it is one automation stack wearing many addresses. Hashing everything except the address collapses the fleet back into a single row.
fp_ips_24h_i is the number of distinct addresses that shared one fingerprint within a twelve-hour window either side, computed by the scorer with a single Solr JSON Facet query per distinct fingerprint in the batch. Five or more, on networks that are not mobile, fires fp_cluster_proxy_fleet at 80 points — a bot verdict on that signal alone.
Carrier-grade NAT genuinely puts thousands of real people behind a handful of addresses, so fp_cluster_proxy_fleet skips networks classified as mobile. The consequence worth knowing: an ASN misclassification turns into a batch of false positives. If a whole organisation shows up as a fleet, check the network type for their ASN before you believe it.
What you get with and without the beacon
The beacon is one line of JavaScript and it is optional. Loghound works without it — but both of the things it exists to do need it.
| Logs only | Logs + beacon | |
|---|---|---|
| Transport plane | Full | Full |
| Behaviour plane | Full | Full |
| Execution plane | Blind | Full |
| Headless automation | Inferred from behaviour | Proven — driver properties, WebGL renderer, engine-versus-User-Agent contradiction |
| Time on site | log_span_ms only — the weak number every log analyser reports | Wall, visible and engaged time, measured separately |
| The last page of a session | Invisible — there is no later request to measure against | Measured, because the beacon flushes on pagehide |
Add it before </body> on the site you are measuring:
No dependencies, no build step, no cookies by default, passive and throttled event listeners. The ?v= is the cache buster: b.js is served with a long cache lifetime, so a new version has to be a new URL.
The four timing numbers
Conventional analytics pings while the page is merely loaded. A tab left open in a background window therefore reports as engagement, which is how "average time on page: 4m 12s" gets into a report about a page nobody was looking at. Loghound keeps four numbers and never conflates them.
| Field | What it actually means |
|---|---|
log_span_ms_l | Last request minus first request. This is what log-only tools call "time on site". It is a lower bound, and it is structurally blind to the last page of every session. Present for every session. |
wall_ms_l | The page was open this long. This is the number conventional analytics reports. It includes every minute the tab spent hidden. |
visible_ms_l | Accumulates only while document.visibilityState is visible and the window is focused. |
engaged_ms_l | Accumulates only while visible and within 30 seconds of a real interaction — scroll, click, keypress, pointer movement. The honest number. |
When no beacon arrived, the three beacon fields are absent from the session document, never zero. A zero is a measurement: it would enter every average, median and percentile as a genuine "0 seconds engaged" observation, and a few thousand beacon-less bot sessions would drag a site's reported engagement to nearly nothing. Every chart in the panel states the population it covers, for the same reason.
All three clocks are accumulated from performance.now() deltas, never Date.now(), so an NTP step or a daylight-saving change cannot be added to somebody's reading time. The final flush uses navigator.sendBeacon() on pagehide, never unload — unload disables the browser's back/forward cache and does not fire reliably on mobile.
Installing it
PHP 8.1 or newer with curl, json, pcre, sqlite3 and mbstring; Solr 9; systemd; and read access to your access logs. No Composer, no npm, no build step.
- Clone it and look before you leap.
--dry-runprints every command it would run and every file it would write, and changes nothing. - Run the installer. It creates the service user, lays out the tree with the right ownership and modes, installs the hardened systemd units, writes a dedicated PHP-FPM pool and a vhost for whichever web server you already run, and validates both before enabling them.
- Confirm the detected log format. The wizard reads your Apache or nginx configuration for
LogFormat/CustomLogpairs, then shows you the mapping with five of your own log lines rendered as parsed records and a confidence percentage. Nothing is ever ingested with a silently guessed format. - Choose where the index lives — managed Opensolr, or a Solr you run yourself.
- Add the beacon to the site you are measuring, if you want the execution plane.
The install location is configurable with --prefix=DIR (or the LOGHOUND_PREFIX environment variable), and every generated artefact follows it — the vhost document root, the FPM pool paths, the systemd unit paths, the deny rules. The default is /opt/loghound; the reference install runs from /var/www/workspace/loghound, because a server-management platform that discovers applications by scanning a base directory then gives it deploy, backups and log viewing for free.
Ingest runs as a systemd service, so a request appears in the panel about five seconds after it was served. Cron's floor is sixty seconds, which is exactly what this is built to beat. Scoring and retention are systemd timers. On a box with no systemd the installer falls back to cron for those two and says plainly that the ingest daemon then needs a supervisor of your own.
Your log files are opened read-only
Every source file is opened 'rb' and never written to, truncated, rotated, renamed or deleted. No other mode string appears anywhere near a source path in the codebase, and the test suite asserts that a source file's size is unchanged after a full read. Whatever else is already reading those files — fail2ban, a log shipper, your own scripts — is unaffected, and so is logrotate.
The tailer survives every rotation shape that occurs in the wild: rename-and-create (the old inode is drained to end-of-file before the switch), copytruncate, rotation that happened while the daemon was stopped, and outright deletion while the file is held open — which is the subtle one, because on Linux an unlinked file a process still holds keeps reading forever with no error, so a naive tailer looks perfectly healthy while ingesting nothing.
On ext4, a rotation that renames the log, compresses it, unlinks the original and creates a fresh file routinely hands the new file the inode number the old one just freed. The stored (device, inode, offset) cursor then matches a file that no longer exists, and resuming at that offset silently skips the beginning of the new file. A valid cursor always sits immediately after a newline, because only complete lines are ever committed — so reading the single byte before the cursor is enough to tell a genuine resume from a recycled inode. When it fails, the source restarts at zero and the event is counted as a missed rotation rather than hidden.
The recommended LogFormat
Loghound works out of the box on plain combined, with zero configuration. Detection is materially better if you log a few more headers, because the fingerprint is only as discriminating as the tuple it is built from.
Then apache2ctl configtest && systemctl reload apache2, and re-run loghound-setup so it picks up the new format. There is an equivalent nginx log_format in the repository, and JSON log formats are first-class rather than an afterthought.
Be honest with yourself about the trade. These lines are roughly two to three times longer than combined. On a site doing ten million requests a day that is real disk; on most sites it is not. Decide with du, not with a feeling. You do not have to take all of it — every header is independent.
What each extra header buys you
| Log this | Field | What becomes possible |
|---|---|---|
%v:%p | host_s | Per-vhost separation. Without it, every site sharing a log file is one dataset. |
%D | dur_us_l | The whole Performance view — p50/p95/p99 by path. Also shows you a scraper hammering your most expensive endpoint. |
%O | bytes_l | Bytes actually sent rather than response-body size, so bandwidth attribution per bot is accurate. |
Accept | accept_s | Sharpens the fingerprint considerably. Accept strings vary a lot between real browser versions and are almost always wrong or absent in scripted clients. |
Accept-Language | accept_lang_s | Fingerprint and visitor hash. A whole fleet sharing one Accept-Language while claiming to be spread across a dozen countries is a strong tell. |
Accept-Encoding | accept_enc_s | Fingerprint. Scripted clients routinely send a short or unusual list. |
Sec-CH-UA | sec_ch_ua_s | The single most valuable addition. A Chrome User-Agent claim can be verified against the client hint Chrome itself sends. Spoofers change the User-Agent and forget this one constantly. Fires ua_secch_mismatch, worth 75. |
Sec-CH-UA-Platform | sec_ch_platform_s | Cross-checks the claimed operating system. "Windows" in the User-Agent and "Linux" in the hint is a headless container in a Windows costume. Fires platform_mismatch, worth 70. |
Sec-Fetch-Site | sec_fetch_site_s | Distinguishes a real navigation from a sub-resource fetch and from a cross-site request. Catches direct asset fetching that never loaded the page. |
Sec-Fetch-Mode | sec_fetch_mode_s | navigate versus cors versus no-cors. A "browser" whose HTML fetch is not a navigation is not browsing. |
Sec-Fetch-Dest | sec_fetch_dest_s | Lets asset classification come from the browser rather than from guessing at file extensions. |
Sec-Fetch-User | sec_fetch_user_s | Present only when a navigation was triggered by real user activation. Its absence on a top-level HTML request is meaningful. |
X-Forwarded-For | xff_s | Essential behind a proxy or load balancer. Without it every visitor appears to come from the proxy, and IP-based clustering and geolocation become meaningless. Configure trusted_proxies as well. |
%H | proto_s | HTTP/1.1 versus HTTP/2 versus HTTP/3. Part of the fingerprint. |
SSL_PROTOCOL, SSL_CIPHER | tls_proto_s, tls_cipher_s | A coarse TLS fingerprint. Not JA4, but the cipher a client offers still separates browser families from Go, curl and Python clients. |
If you only add two things
Add Sec-CH-UA and Sec-CH-UA-Platform. They cost about forty bytes a line and they are worth 145 points of the ruleset. With plain combined those two rules cannot fire at all, and the fingerprint collapses to User-Agent plus protocol — still useful, but far coarser in both directions.
Where the index lives
Setup asks one question: managed Opensolr, or a Solr you run yourself. Two cores are used: one document per log line, and one document per session — which is what the dashboard actually queries.
On the managed path you give it the email address and API key from your Opensolr control panel and pick a region from the list the platform returns. It creates both indexes, pushes the configset to each (schema first, then solrconfig.xml — reversed, the core reloads against a config referencing field types the old schema does not define and the reload fails), reads the connection details back, and verifies. You never have to learn what a configset is.
Opensolr index names are unique across the entire platform, not per account, so a hardcoded name would collide for the second person who ever installed this. Both cores share one 8-hex install id — loghound_9f3c17ab_hits and loghound_9f3c17ab_sessions. If either name is taken, the whole pair is retried with a fresh id, and any index the failed attempt already created is deleted first, so a collision never leaves orphaned indexes on an account you are billed for. On your own Solr the names are yours to choose.
The Opensolr API key is an account credential — it can create, reconfigure and delete every index on the account. Loghound reads it with terminal echo off, never prints it back (not even masked: a masked key in a scrollback is still a key in a scrollback), never writes it to the install log, and strips it out of any error message the platform echoes back. It lives in config/loghound.php, mode 0640, in a 0700 directory the web server user cannot reach.
The shipped configsets deliberately load no contrib jars and define no /stream, /sql, /export, /replication, /update/extract, /terms or /browse handlers, each removal annotated in the file with what it would hand an attacker.
How the scoring works
Weighted, additive, 0 to 100, capped. Every point added carries a reason code, and a verdict with no reasons is a bug — bot_score_f is always accompanied by bot_reasons_ss, and a clean session carries no_bot_signals rather than an empty list. Weights and thresholds are overridable in the configuration, and rule_version_i is written onto every session so a historical verdict stays traceable to the ruleset that produced it.
| Code | Weight | Fires when |
|---|---|---|
automation_marker | 100 | A driver property that does not exist in a browser a person is using — navigator.webdriver, a chromedriver cdc_ property, a Playwright, Puppeteer, Selenium, Nightmare, Phantom or DOM-automation binding. |
ua_declared_bot | 100 | The User-Agent honestly identifies itself as a crawler. Verdict bot, class declared_crawler or ai_crawler — not a threat, see below. |
rdns_claim_failed | 95 | The User-Agent declares Googlebot, Bingbot or similar, and forward-confirmed reverse DNS fails. That is impersonation, not automation. |
headless_renderer | 90 | The WebGL renderer is SwiftShader, llvmpipe, Mesa OffScreen or Microsoft Basic Render — software rasterisers. Not 100, because a VM or a machine with broken graphics drivers can land here legitimately. |
beacon_forged | 90 | The claimed timings are impossible against the token's issue time. The payload is recorded rather than discarded, because the lie is the evidence. |
ua_claim_failed | 85 | The engine lacks a feature that shipped in the Chrome version the User-Agent claims, or has one that shipped after it. A spoofed User-Agent cannot retrofit V8. |
fp_cluster_proxy_fleet | 80 | Five or more distinct addresses, on non-mobile networks, shared one header fingerprint within twenty-four hours. The signal this project is built around. |
ua_secch_mismatch | 75 | Sec-CH-UA is absent from, or contradicts, a Chrome User-Agent claim. |
platform_mismatch | 70 | Sec-CH-UA-Platform contradicts the operating system the User-Agent claims. |
no_js_on_html | 70 | An HTML 200 was served, the User-Agent claims a real browser, and no beacon ever arrived. Weighted below the threshold on purpose — ad blockers and script blockers produce exactly this from real people. |
hosting_asn_browser_ua | 45 | A consumer-browser User-Agent arriving from a hosting network. People do browse from servers, so this is a stacking signal. |
periodic_timing | 45 | Inter-request gaps with almost no variance across four or more requests. Humans are irregular; a timer is not. |
no_interaction | 40 | A beacon arrived, the session ended, and there was not one scroll, mousemove, keypress, click or touch. |
tz_mismatch | 35 | The browser timezone disagrees with the one derived from the address. Low, because every VPN user and everybody on a plane trips it. |
no_304_on_repeat | 30 | Assets were re-requested and no conditional request was ever sent. Browsers cache; this client does not. |
no_assets | 25 | HTML was fetched and zero sub-resources followed. The classic scripted-client signature, but a warm cache looks similar. |
single_page_10s | 15 | One page, ten seconds or less. Deliberately weak — plenty of humans bounce. It exists to push a session that already has other signals over a threshold. |
Thresholds: 80 and above is bot, 60–79 likely_bot, 40–59 unknown, 20–39 likely_human, below 20 human. bot_class_s says what kind: headless, scripted, declared_crawler, ai_crawler, monitor, spoofed_ua, proxy_fleet or none.
unknown is a real answer and it is used. A session with 45 points is not "probably a bot"; it is a session Loghound does not have enough evidence about, and saying so is more useful than guessing.
Googlebot, Bingbot, GPTBot, ClaudeBot, PerplexityBot and the rest identify themselves. They are classified as bots — counting them as human traffic is how dashboards end up lying to you — but they are presented separately from evasive traffic, always, and the separation is enforced in the scorer rather than by a checkbox in the interface, so every consumer sees the same split. Conflating "Googlebot indexed 400 pages" with "someone is scraping you from 200 residential addresses" is exactly what makes conventional tools useless here: one is a service you want, the other is a cost you are paying. A self-declared crawler that fails forward-confirmed reverse DNS is a different fact entirely — that is rdns_claim_failed, class spoofed_ua.
Tuning
Run it for a week without changing anything. Then open the bot-forensics view, take the most frequent reason code, pull twenty sessions that fired it and read them. If they are humans, lower that weight; if they are bots, leave it alone. Bump rule_version and write down what you changed and why.
Do not tune towards a bot percentage you expected. There is no correct number. Sites in the same industry and of the same size legitimately differ by an order of magnitude, and tuning until the number looks reasonable is how you end up with a detector that reports whatever you already believed.
Privacy and retention
Loghound is self-hosted software, not a service. There is no vendor with a copy of your traffic, no telemetry and no phone-home. Nothing leaves your machine except the enrichment lookups — geolocation, ASN, reverse DNS, network whois — and every one of those can be switched off individually. With all of them off and your own Solr, nothing leaves the box at all.
The beacon sets no cookies by default and stores nothing on the visitor's device. It records no keystrokes, no form contents, no clipboard, no page text and no mouse coordinates: it counts that interaction happened, how much and what kind. There is no session replay and no plan to add one. The query string is dropped before the payload is built, because real sites routinely carry session tokens and reset codes in query strings.
| IP mode | What is stored | The trade |
|---|---|---|
full (default) | The address as-is | Best detection — the fleet signal counts truly distinct addresses. It is personal data under GDPR, and you need a lawful basis. Default because the raw address is already sitting in your own access logs. |
truncate | IPv4 to /24, IPv6 to /48 | Country and city geolocation, ASN and netname still work. The cluster rule counts distinct addresses, so truncation both under-counts a fleet sharing a /24 and merges unrelated people — expect noisier verdicts. A defensible middle ground. |
hash | Keyed hash with a daily-rotating salt | Cross-day correlation of an individual becomes impossible. Same-day sessions, clustering and forensics still work. You lose geolocation, ASN, netname and reverse DNS entirely — there is no address left to look up. |
Retention is a real deletion job, not a paragraph saying you ought to have one. privacy.retention_days defaults to 90 and a systemd timer issues an actual delete-by-query against both cores every day, along with the expired enrichment caches. privacy.rollup_forever keeps the daily rollup documents, which are aggregate counts with no per-visitor field in them, so long-range trends survive the deletion of the detail underneath them.
It reads the files that exist. Rotation configs commonly keep fourteen or fifty-two files, but a cron entry can also simply delete them, and that is not visible in the logrotate configuration at all. If you care about long-range history, keep it in Loghound rather than in your web server — and run the tailer once with --from-start before your rotation window discards what you still have.
What it does not catch
A bot detector that oversells itself is worse than none, because you stop looking.
A well-funded, careful scraper
Give every session a genuinely unique and self-consistent header fingerprint, run a real non-headless browser, synthesise human-like pointer movement and pacing, never reuse an exit address — and Loghound will call it human. Everything here raises the cost of scraping. None of it makes scraping impossible.
Privacy-conscious humans get flagged
The biggest source of false positives, and it deserves to be first. A visitor running a script blocker or a strict browser mode may never load the beacon, and from the log plane that is indistinguishable from a non-JS client. That is why no_js_on_html is 70 and not 100.
Traffic that never reaches your origin
A request served from a CDN edge or a cache in front of you produces no origin log line and does not exist as far as Loghound is concerned. If you run a CDN, you are analysing your cache misses.
Low-and-slow scrapers
One page, one address, one fingerprint, once a day, from a residential network. It is below every threshold in the ruleset by construction. The fingerprint-cluster signal needs a fleet to detect a fleet.
API and native-app traffic
There is no beacon in a native application, so the execution plane is permanently blind there and every client looks non-JS. Point Loghound at that vhost's log and read the transport plane instead.
It is not a firewall
Loghound observes and reports. It issues no challenges, rate-limits nothing and writes no firewall rules. Deciding what to do about what it finds is your job — which is exactly why every verdict ships with its reasons attached.
Never act on bot_score_f alone. Look at bot_reasons_ss and ask whether that specific combination describes a person you would recognise. A session that is bot on automation_marker is not arguable. A session that is likely_bot on no_js_on_html plus tz_mismatch is a privacy-conscious VPN user until proven otherwise.
Where to go next
github.com/phpcip/loghound — MIT licensed. SPEC.md is the full technical specification, docs/DETECTION.md covers every rule and a worked example on real captured traffic, and docs/BEACON.md documents every field the beacon sends.
opensolr.com/loghound — what it is, the finding that prompted it, and a fair comparison against the analytics tools you may already be running.
Managed Solr hosting if you would rather not run a Solr, or Creating Indexes to see what the control panel gives you around it.
Open an issue on GitHub, or write to support@opensolr.com. Security problems do not go in the public issue tracker — see docs/SECURITY.md.