Loghound Log Sources and Format Detection

Loghound works out of the box on Apache with zero configuration, because it reads your web server’s own configuration to find the logs and their exact format. Nothing is ever ingested with a silently guessed format.

Figure 1 — the detection ladder. Each rung is used only when the one above it cannot answer, and whichever it lands on you are shown the mapping and five real lines before anything is ingested.

01 · The four rungs

Read your web server configuration. Parsing apache2.conf, httpd.conf and the enabled sites for LogFormat definitions and CustomLog pairs, or nginx’s log_format and access_log. This yields the exact parser, the exact file list and the virtual host each file belongs to. Deterministic, with no guessing, and it is the headline setup feature.
Score sample lines against the known-format library when the configuration is unreadable. The last two hundred lines of each candidate file are scored against every known format and the highest full-parse rate wins. The scoring is structural rather than regex luck: the first field has to parse as an address, the bracketed field as a date, the status as three digits.
Generate a candidate from your own sample when nothing matches.
Your own regular expression with named groups as the escape hatch, refused at save time unless it compiles and runs fast.

Formats in the library: Apache combined, vhost_combined and common; nginx combined and nginx with forwarded addresses; Caddy JSON; HAProxy HTTP log; AWS ALB; CloudFront; Traefik JSON; Kubernetes ingress-nginx. JSON log formats are first class — if you already emit JSON, point Loghound at it and the detector recognises it.

02 · Confirming the mapping

Whichever rung it lands on, you are shown the file, where the format came from, a confidence percentage, the token-to-field mapping, and five of your own log lines rendered as parsed records, with one button to accept.

Read the five records before you press it

If a value is in the wrong column the format is wrong, and confirming it would fill your index with nonsense. The screen also lists what your format does not log and what each gap costs — which scoring rules will stay silent. The recommended LogFormat closes all of them.

03 · Naming sources yourself

Sources live in the configuration and can be edited by hand. A path may be a glob, and globs are re-evaluated on every poll, so a new vhost’s log file is picked up without restarting the daemon:

'sources' => [
    [
        'path'   => '/var/log/apache2/example_com_access.log',
        'format' => 'apache_combined',
        'host'   => 'example.com',
    ],
    [
        'path'   => '/var/log/nginx/*access*.log',
        'format' => 'nginx_combined',
    ],
],

format is either a library name or the literal LogFormat or log_format string from your own configuration. host is an optional virtual host override, for a format that does not log the vhost itself.

allowed_log_roots is a security control, not a convenience

A configured path is resolved with realpath() and must sit inside one of these roots or it is refused; /var/log is the default. Without it, the log-path setting would be an arbitrary-file-read primitive. Globs are re-checked at read time, not only when they are saved, and the prefix comparison is against the resolved root with a trailing separator, so /var/log-evil cannot match /var/log and a symlink cannot escape the tree.

04 · Living alongside your other log tooling

You are probably already running something against these files — fail2ban, an abusive-address scanner, a log shipper, your own scripts. So, plainly:

Loghound opens every source file read-only and never writes to, truncates, rotates, renames or deletes one. Source files are opened 'rb' and no other mode string appears anywhere near a source path in the codebase; a test in the suite asserts a source file’s size is unchanged after a full read.
Everything Loghound writes goes under its own var/ — the SQLite state database, the status file, the capped bad-line sample.
Reading the same file from several processes is safe. Loghound holds an open descriptor and reads forward from a recorded offset; it takes no locks and does not care who else is reading.
Never point Loghound at its own vhost’s access log. It would ingest its own beacon traffic and inflate every number it reports.

Rotation, deletion and the genuinely nasty case — inode recycling on ext4 — are covered under how far back it can see.

Log sources & detection