If indexing fails at regular intervals, every time your CMS or import job runs, the cause is often a single document, not the server.
Failed batches are sent again
Most CMS integrations send documents in batches and mark the whole batch as not indexed when a request fails, then send it again on the next run. One document that cannot be indexed then fails every run, and takes every other document of its batch down with it.
The signs
Failures that line up with your indexing schedule; update requests that time out or fail while searches keep working; one request in your logs far larger than the others.
Find the document
Run one indexing pass with a batch size of 1 to 10. The document that still fails is the one. Look at the size of each of its fields: a rendered page with embedded pictures, a pasted log, or an attached file are the usual suspects.
Cap field sizes before sending
A search index rarely needs more than the first 50 to 100 KB of a page. Truncate long text in your indexing code, and never send binary data. Solr can enforce a ceiling too, with a standard update processor:
<updateRequestProcessorChain name="limits" default="true"> <processor class="solr.TruncateFieldUpdateProcessorFactory"> <str name="typeClass">solr.TextField</str> <int name="maxLength">100000</int> </processor> <processor class="solr.LogUpdateProcessorFactory"/> <processor class="solr.RunUpdateProcessorFactory"/> </updateRequestProcessorChain>
Then reindex the document
Once the document is fixed or truncated, the next run indexes it normally, and the rest of its batch with it. See also never put binary data or base64 into Solr.
Solr best practices
When one document keeps crashing Solr (this page)