Document Extraction (rtf:true) — Index PDFs, Word, Excel, and Other Files via the Ingestion API

How It Works

The extracted text fills the text field automatically. You provide the title, description, and any other metadata you have. The full enrichment pipeline (embeddings, sentiment, language detection, derived fields) runs on the extracted text just like any other document.

Supported Formats

Example Payload

// Mix regular docs and RTF docs in the same batch:
{
  "email": "you@example.com",
  "api_key": "your_api_key",
  "core_name": "my_index",
  "documents": [
    {
      // Regular document — you provide the text
      "title": "Product Announcement",
      "description": "New features for Q1 2026",
      "text": "We are excited to announce...",
      "uri": "https://example.com/blog/announcement"
    },
    {
      // RTF document — text extracted from the PDF automatically
      "rtf": true,
      "title": "2025 Annual Report",
      "description": "Company financials and key metrics",
      "uri": "https://example.com/docs/annual-report-2025.pdf",
      "timestamp": 1735689600,
      "category": "Reports"
    },
    {
      // RTF document — Word file from an internal server
      "rtf": true,
      "title": "Employee Handbook",
      "description": "Company policies and procedures",
      "uri": "https://intranet.example.com/hr/handbook.docx",
      "og_image": "https://example.com/img/handbook-cover.png"
    }
  ]
}

What Happens

1. The rtf:true flag is detected on the document
2. The file at uri is fetched over HTTP/HTTPS
3. The file type is detected from the actual content, never from the URL or the file name: a Word file served as a generic download, a spreadsheet with no extension, a PDF behind a document ID all work
4. Text is extracted with a reader for that format: every page of a PDF; the paragraphs, tables, text boxes, headers and footers of a Word file; every sheet of a spreadsheet; every slide and its notes
5. Pictures inside the document, scanned pages included, are read with OCR and their words are added to the text
6. The extracted text populates the text field
7. All other fields you provided (title, description, timestamp, etc.) are kept as-is
8. The full enrichment pipeline runs: embeddings, sentiment, language detection, derived fields
9. The document is pushed to your Solr index

Scanned Documents and Images (OCR)

On every plan, the extraction reads more than the text layer of a file:

• A PDF is indexed one document per page: each page document has the PDF's uri, title, description and every field you sent, its own text (the page's own text first, then what was read in its pictures), its own language and its own vector, plus page_i (the page number) and uri_id_s (the id of the PDF, the same on all its pages). A search finds the very page that answers it, and the hosted search page links straight to that page of the PDF. A scanned PDF (pages that are only pictures) gives the reading of every page
• Sending the same PDF again replaces all of its pages: the earlier pages, and an earlier whole-document version, are removed before the new pages are written
• Each page's id is md5("<document id>#page=N"), where the document id is the id you sent (or the MD5 of the normalised uri). The document id itself is no longer a document in the index: its pages are, and uri_id_s holds it on each of them. All pages of one PDF: fq=uri_id_s:"<document id>"; one page: add fq=page_i:N
• On each page, the language is detected on that page's text; when it cannot be, it is the language you sent, else the one detected on the document
• These fields are kept by the *_i, *_s and phonetic_* dynamic fields of the Opensolr schema. A custom schema that sends unknown fields to an ignored catch-all (the Drupal Search API schema does) drops them: the pages are written, but without their page number and without the link to their PDF. Add those dynamic fields to such a schema first
• Each picture of a PDF is read in the language of its page, detected automatically, among more than 100 languages; when no language is certain, English
• A picture of a PDF with no text in it (a photo, a chart without labels) gets a short description of what it shows, in English, so it can be found by its meaning
• The pictures inside a Word, Excel, PowerPoint or OpenDocument file are extracted and read, and their words are added at the end of text under an Images heading
• An image file (PNG, JPEG, GIF, WEBP, TIFF, BMP) sent as the document is read whole
• Pictures smaller than 300 pixels on a side are skipped (icons, bullets, logos), and a picture repeated on every page of a PDF is read once

Requirements for RTF Documents

• rtf must be set to true (boolean, not a string)
• uri must be a valid http:// or https:// URL pointing to the document file
• The file must be publicly accessible (or accessible from the Opensolr server)
• Maximum file size: the document size limit of your plan (the same limit as the web crawler's page size)
• title and description are recommended but the text field will be extracted from the document, so at minimum you need rtf:true and uri

If Extraction Fails

If the file cannot be fetched (bad URL, unreachable server, over the size limit of your plan), that specific document is marked as failed in the job results with a descriptive error message. Other documents in the same batch continue processing normally. A file that arrives but cannot be read (an unsupported format such as a zip archive, an empty or corrupted file) is not a failure: the document is indexed with the text you sent (empty if you sent none), with the title, description and every other field you sent, and the detected type in content_type.

Check the job status via the API or the Ingestion Queue page to see per-document results.

See it live: a live search over real PDFs whose words sit only inside pictures.