Data Ingestion is the second way to fill your Opensolr Index for enterprise site search. Instead of Opensolr reading your website, you send your records yourself: products from a database, articles from a CMS, files from a private folder. Each document gets the same treatment as a page read from your site, so it shows up in your hosted search page, your chat bot and your API answers like any other page.
How it works
- Your code sends up to 50 documents in one request, with your account email and API key.
- Opensolr checks the batch and answers at once with a job id.
- The job waits its turn in your Data Ingestion queue. In the background, Opensolr reads your files, adds meaning vectors, language and sentiment, and writes every document to your index.
- You follow the job under Account > Data Ingestion, or with the API.
Pushing and reading your site can feed the same index: read your public pages with Crawl your site and push what is not on a public page.
What you need
- An Opensolr Index on a site search server. When you create the index, pick a server marked Web Crawler and AI Hybrid Vector Search. Any other index answers
ERROR_, with the regions where you can create one. Details: What your index needs.CORE_ NOT_ WEBCRAWLER_ ENABLED - Your account email and an API key. See API authentication.
- AI on your plan, for meaning vectors. The free plan and every current plan include 500 AI requests a month (plans, API Quota). Past the allowance, documents are still indexed and found by their words, only without vectors.
On Drupal or WordPress you do not write any of this: the Opensolr Search module and the Opensolr Search plugin push your content each time you save it. See Drupal and WordPress. The LangChain, LlamaIndex, Haystack, MCP and Laravel Scout packages push for you too: Developer packages.
The pages of this group
- Your first push: one request, the answer, and how to follow the job.
- Document fields: the four required fields, the optional ones, and your own fields.
- Files and documents: send a PDF, Word, Excel or scanned file by its address, or the file itself.
- PDFs page by page: every page of a PDF becomes a result of its own.
- One result per PDF: show each PDF once, with its best pages under it.
- Update and remove documents: send a document again to replace it, delete it by its id.
- Send a JSON file: upload a .json file instead of writing the request body.
- Limits and refusals: what is checked before a batch is queued, and the answers.
- Where to see the queue: Account > Data Ingestion in your Opensolr account, or the direct address.
- The Data Ingestion queue: follow, pause, stop, fix and retry your jobs.
- Why ingestion failed: where the reason of a failed job is shown, the common errors and the fix.