Storage & bandwidth

Two limits that behave nothing alike, and only one of them can actually switch the panel off.

Storage & bandwidth shows how much history your Opensolr plan holds and how much bandwidth is left this month. Its three pages: Bandwidth this month, How far back the data goes and The two retention limits. The two limits behave nothing alike, and only one of them can switch your dashboard off.

01 · The two limits
DISK — MANAGED, NOT A DANGERReclaimable by deleting old data.The rolling trim runs BEFORE a write,so the index never reaches the quota.Reported as: how far back your data goesBANDWIDTH — THE HARD LIMITCannot be reclaimed by deletinganything. Accrues with use andresets on the 1st of the month.Reported as: a live meter with a warning DISK: MANAGED, NOT ADANGERReclaimable by deleting old data.The rolling trim runs BEFORE awrite, so the index neverreaches the quota.Reported as: how far backyour data goesBANDWIDTH: THE HARDLIMITCannot be reclaimed by deletinganything. Accrues with use andresets on the 1st of the month.Reported as: a live meter witha warning

Figure 1 — why the two are presented differently. Averaging a blocked index with an idle one would tell an operator they are fine when they are not, so the worst of the two indexes drives the strip.

Going over either quota is a total blackout, not a throttle

The platform answers every request against that index with a refusal — reads included, not just writes — until usage returns under the limit or the plan is upgraded. It clears by itself roughly seventeen minutes after usage is back under. That is why the bandwidth warning has to arrive early: after the limit, the panel that would have shown it is dark.

02 · Disk is a fact, not an alarm

Because the rolling trim runs before a write, the index never reaches its disk quota and the blackout never triggers on disk. So disk is reported as a fact about how far back the data goes — “roughly eleven days of traffic retained, limited by your plan” — rather than as a risk.

The single exception is a trim that failed while the index is genuinely near the limit. Then the blackout really is coming, and that is loud.

03 · Where the numbers come from, and what “estimated” means

One call to the Opensolr control plane per refresh, which reports the index size and bandwidth against their allowances. That call is expensive — the platform derives the index size from a live status request — and Loghound writes batches every couple of seconds, so calling it per batch would cost far more than the ingestion it is protecting.

So the answer is cached with both a time and a document threshold, and between refreshes the size is projected locally from the bytes of log actually read, using an expansion factor learned from the platform’s own reported growth. Every projected figure is flagged as estimated and says so on screen.

The meter is requested by every view, not just this one

It feeds the always-visible bandwidth strip, and it answers from the cache on all but one call in every refresh window — which matters, because the panel’s own queries consume the bandwidth this meter reports. The two indexes have independent quotas and are reported separately.

04 · What to do when bandwidth is climbing
  • Turn on Cached queries. Bandwidth is metered on what Solr sends back, and on a Loghound installation that is almost entirely the panel’s own reads.
  • Look at what else is reading it. Anything else you have pointed at those indexes counts too.
  • Or move to a plan with more. See the plans; the meter links there too.

When the history held is shorter than you want

  • Turn off the full-text catchall if you do not use free-text search in Sessions. It is the biggest single part of the hits index; Index size gives the order to turn things off in.
  • Turn off the raw line copy if you will not need to re-score history under a future ruleset.
  • Or move to a plan with more disk. The rolling trim keeps more history as soon as the quota grows.

Analysis views

Loghound is open source and MIT licensed. Questions about the Opensolr half — the account, the indexes, the plan — go to opensolr.com/contact; questions about the software itself belong on GitHub.

Loghound Documentation