Account Resources

🚦 Opensolr Traffic Bandwidth Limit: Explained

What’s the Deal with the Traffic Bandwidth Limit?

At Opensolr, we don’t count the number of requests you make—because, let’s face it, not all requests are created equal.
Instead, we use a Traffic Bandwidth Limit to keep things fair. You’re only billed (on a pre-paid plan) for the outgoing bytes sent from your Opensolr index to your site or app.

Translation:

• 1 GB of traffic could be a million ultra-efficient requests (if you optimize your queries)
• ...or it could be just one monster request (if you don’t).
  Yes, size matters!

Why Am I Seeing High Search Traffic Bandwidth?

• Bots and web crawlers love to visit your site’s search pages—sometimes a little too much.
• That traffic is then passed on to our servers, and can quickly add up.
• If your bandwidth seems sky-high overnight, odds are you’re the (un)lucky recipient of a bot party... or maybe even an attack.

🛠️ Solution: Outsmart the Bytes!

• First, read these tips to save bandwidth.
• Next, check our Best Practices Guide for power-user advice.

Bonus: Opensolr transparently logs every single request. You get full access to see all the action, via:

• Our Automation API
• The Opensolr Index Control Panel

📊 Real-World Examples

1. API - Logs & Analytics

• Get, facet, and analyze your requests by any Solr-supported field.
• Example: facet all results by IP and path—see who’s eating your bandwidth.

Learn more in the API Docs

2. Index Control Panel Analytics

• See metrics on traffic spikes, popular queries, and more.
• Diagnose what’s hot—and what’s not—on your search.

Read the Analytics Blog Post

3. Tail the Logs Like an Old-School Sysadmin

• Use your Index Control Panel to view the last 1,000 lines of the live request log.
• Spot traffic in real time. Block, optimize, or celebrate as needed.
• Great for identifying bottlenecks, surprise traffic, or just showing off.

🥷 Pro Tips (Because You’re Not Just Any Solr User)

• Bots aren’t going away—get friendly with your logs.
• Optimize requests, use filters, and cut down on payloads.
• Share your log horror stories. We’ve all been there.

Want deeper insights or custom advice? Contact our team. We love a good bandwidth optimization challenge!

🧠 Solr RAM & Memory Management: Best Practices (or, “How Not to Blow Up Your Server”)

Solr is a beast—it loves RAM like a dog loves a steak. If your Solr server is gobbling up memory and crashing, don’t panic! Here’s what you need to know, plus battle-tested ways to keep things lean, mean, and not out-of-memory.

Why Does Solr Use So Much RAM?

Solr eats memory to build search results, cache data, and keep things fast.
But:

• Bad configuration or huge, inefficient requests can cause even the biggest server to choke and burn through RAM.
• Sometimes, small indexes on giant machines will still crash if your setup isn’t right.
• Good news: Opensolr has self-healing—if Solr crashes, it’ll be back in under a minute. Still, prevention is better than panic.

Essential Best Practices

1. Save Transfer Bandwidth (and Memory)

Want to save bandwidth and RAM? Read these tips.
Optimizing your queries is a win-win: less data in and out, and less stress on your server.

2. Don’t Ask Solr to Return 10 Million Results

• Requesting thousands of docs in one go?
  That makes Solr allocate all that data, and cache it, too.
• Solution: Keep the rows parameter below 100 for most queries.
  Example:

  &rows=100
  

3. Paginate Responsibly (Or: Don’t Scroll to Infinity)

• If you’re paginating over millions of docs (like &start=500000&rows=100), Solr has to allocate a ton of memory for all those results.
• Solution: Try to keep start under 50,000 if possible.
• The more stored fields you have in your schema, the more RAM will be used for large paginations.

4. Heavy Faceting, Sorting, Highlighting, or Grouping? Use docValues=true

• Operations like faceting, sorting, highlighting, and grouping can be memory hogs.

• Solution: Define your fields with docValues="true" in schema.xml.

• Example:

  <field name="name" docValues="true" type="text_general" indexed="true" stored="true" />
  

• For highlighting, you may want even more settings:

  <field name="description" type="text_general" indexed="true" stored="true" docValues="true" termVectors="true" termPositions="true" termOffsets="true" storeOffsetsWithPositions="true" />
  

5. Don’t Go Cache-Crazy

Solr caches are great... until they eat all your memory and leave nothing for real work.

• The big four:

  • filterCache: stores document ID lists for filter queries (fq)
  • queryResultCache: stores doc IDs for search results
  • documentCache: caches stored field values
  • fieldCache: stores all values for a field in memory (dangerous for big fields!)

• Solution: Tune these in solrconfig.xml and keep sizes low.

• Example:

  <filterCache size="1" initialSize="1" autowarmCount="0"/>
  

6. Using Drupal?

• Always update your Search API Solr module to the latest version!
• There’s a known bug that can cause headaches—stay patched!

Final Wisdom

• RAM is precious. Don’t let Solr treat it like an all-you-can-eat buffet.
• Optimize requests, paginate wisely, and keep configs tight.
• If Solr OOMs (“Out of Memory”)—Opensolr’s got your back, but wouldn’t you rather avoid the drama?

Questions? Want a config review or more tips? Contact the Opensolr team!