Loghound Project Structure - The Repository and the Programs

For a developer who has just cloned the repository: the four programs, what every folder holds, and the classes in the library. How data moves and where to change things are on the next two pages. It stays high level on purpose; SPEC.md in the repository goes deeper.

Want to work on it with us?

Loghound is open source and contributions are welcome. To become a contributor to this or any other Opensolr open source project, write to support@opensolr.com and tell us what you would like to work on.

01 · The big picture in one minute

Loghound is plain PHP 8.1+ with no Composer packages, no npm and no build step. It is really four programs sharing one library in src/:

The ingest daemon

bin/loghound-tail runs all the time, reads your access logs as they grow, and writes one document per request into the hits index.

The scorer

bin/loghound-score runs every minute, closes finished visits, merges beacon data, scores each visit and writes it into the sessions index.

The beacon

public/b.js runs in visitors' browsers and posts to public/collect.php, which only stages the data locally.

The panel

public/index.php serves the web interface; every card on a page loads its own data from the two indexes.

Everything Loghound learns lives in two Opensolr indexes it provisions for you; small working state (file positions, open visits, staged beacon rows) lives in a local SQLite file.

02 · The repository, top to bottom

bin/

The command-line programs: the daemon, the scorer, retention, the schema check and the setup wizard.

src/

All PHP classes, namespace Loghound\, loaded by src/autoload.php.

public/

The web root: the panel, the beacon collector, the beacon script, CSS, JavaScript and vendored assets.

solr/

The configuration of the two indexes, hits/conf and sessions/conf.

config/

loghound.example.php, the documented template. The live loghound.php is written by setup and never committed.

install/

install.sh, uninstall.sh, the systemd units and timers, web server and PHP-FPM examples.

tests/

A dependency-free test runner, the tests, anonymised log fixtures and the secret scanner.

tools/

panel-preview.php, for looking at the panel with demo data while developing.

var/

Runtime state, never committed: the SQLite database, status files, setup jobs.

docs/

Operator and developer documentation in Markdown: install, beacon, detection, panel, schema, privacy, security, attacks.

SPEC.md

The binding technical specification. Field names, paths and signatures in it are contracts.

README.md, CONTRIBUTING.md, CHANGELOG.md, LICENSE

The front page, the rules for contributions, the release history (Keep a Changelog, semantic versions), and the MIT licence.

03 · The programs in bin/

loghound-tail

Tail, parse, enrich, group into visits, and write batches into the hits index.

When it runs: Always, as loghound-tail.service

loghound-score

Close idle visits, merge beacon data, apply the scoring rules, write visits into the sessions index, rebuild daily rollups.

When it runs: Every 60 seconds, by a timer

loghound-retention

Delete what is older than the retention period, keep the indexes within the plan, purge old local state.

When it runs: Daily at 03:20, by a timer

loghound-schema

Compare the live schemas with this release; --apply uploads them.

When it runs: By hand, after an upgrade

loghound-setup

The interactive setup wizard; it shares its logic with the browser installer.

When it runs: By hand, once

Every command and option is on the command reference; the services and timers on running it.

04 · The library in src/

Top-level classes

Configuration and state

Load, validate and save the configuration (every default is in Config::defaults()); the local SQLite store for file positions, open visits, caches, staged beacon rows and rate limits.

Classes: Config, State

Reading logs

Follow files across rotation; compile an Apache or nginx log format; find the logs and their format; turn a line into a normalised request.

Classes: Tail, LogFormat, LogDetect, Parser

Visits and rules

Group requests into visits; skip traffic that is never recorded; the operator's own attack patterns; where a visit really came from.

Classes: Sessionizer, Exclusions, AttackPatterns, RefererOrigin

Solr and Opensolr

The only way code talks to Solr, with its query safety checks; the Opensolr account API; the Opensolr request log; query shapes; plan quota and size-based retention.

Classes: Solr, Opensolr, OpensolrLog, OpensolrShape, Quota

Beacon

Validates beacon data and merges it into visits.

Classes: Beacon

Web support

Escaping, safe paths, CSRF and authentication; the panel's read cache; asset versions; safe CSV export; the failure report.

Classes: Security, Cache, Assets, Csv, Diagnostics

Subfolders

src/Enrich/ — adds context to a request: Geo (country and timezone), Asn (network, netname, reverse DNS), Ua (browser and declared bots).
src/Score/ — detection: Signals extracts the evidence, Rules holds the weighted rules and the verdict, Attacks spots hostile requests.
src/Panel/ — the web interface: one controller class per view (Overview, Bots, Fingerprints, Sessions, Seo...), plus Layout (routes, navigation, page chrome), Gateway (the panel's only route to Solr), Query, Facets, Sorting, Paging and the long-running Jobs.
src/Setup/ — installing: the browser Installer, detection and requirement checks, index provisioning and schema sync, resumable jobs, reset and teardown.
src/Auth/ — panel sign-in extras: two-factor codes, the QR code for the authenticator, “stay signed in”.
src/Live/ — the live view, reading logs as they are written without touching ingest.
src/Geo/Countries.php and src/Beacon/Doc.php — the one country table, and the one description of the beacon shown wherever it is documented.

How a log line, a visit, a beacon report and a panel page travel through the code: How data moves. Where to change something, the tests and the contribution rules: Changing the code.

For developers

Project structure
How data moves
Changing the code

Project structure