Every batch Opensolr accepts becomes a job in your Data Ingestion queue. The queue page shows each job of each index, what it did with every document, and lets you pause, stop, fix and retry.
Open it from the Account menu > Data Ingestion. The entry is there once you have an Opensolr Index on a site search server.
The life of a job
| State | What it means | Buttons |
|---|---|---|
| Pending | Waiting its turn. Jobs start by themselves, oldest first. | Pause, Cancel |
| Processing | Its files are read, its documents enriched and written. | Pause, Stop |
| Completed | Finished. If some documents failed, it also counts as partially failed. | Retry, Delete |
| Failed | Finished, and every document failed. | Retry, Delete |
| Paused | Held by you. Resume carries on where it stopped. | Resume, Delete |
| Stopped | Cancelled or stopped by you. | Retry, Delete |
- Pause and Stop let the documents being written at that moment finish. What is written stays in your index.
- Retry runs the whole job again, with the same documents and the same ids, so nothing is duplicated.
- Delete removes the job and its documents from the queue, not from your index.
- If meaning vectors cannot be made for a moment, a job goes back to Pending with a note saying so, and later carries on where it stopped. When your plan includes vectors, no document is written without one.
The page
Each index with jobs has a row with its counters (pending, processing, completed, failed, partially failed, paused, stopped) and these buttons:
- Run, when jobs are pending: starts them now, then opens the progress.
- Retry Failed Jobs (n) and Retry Partially Failed (n): put every such job of the index back in the queue at once.
- Progress: the current run in percent, documents ok, in error and in all, and the active jobs in the order they will run. It refreshes every 2 seconds; closing it stops nothing.
Click the index to open its jobs, ten per page: Job ID, Status, Progress, Docs (ok / err / total), Submitted, Completed, Result / Error and Actions. Sort by Docs, Submitted or Completed. Click the failed or partially failed counter to list only those jobs; click it again to list all.
Auto-refresh reloads the page every 10 seconds while a job is processing (your browser remembers the choice); Refresh reloads it now.
Fix a job and send it again
- Click the Job ID. The details show the counts, the times, the Error, the Result with the reasons of any failed documents (for a failed job, every document marked Indexed or Failed) and a link to Check Solr Error Log.
- Under Payload (Documents Sent), change the documents. The payload can be edited while the job is failed, stopped or paused.
- Click Save Payload. It is checked like a new request (Limits and refusals).
- Click Retry (failed or stopped job) or Resume (paused job).
How long jobs are kept
Jobs are kept 30 days from the moment you sent them. After that, a job is removed from the queue with its documents, whatever its state, unless it is processing at that moment. What it wrote stays in your index.
Everything on this page also works through the API: Push and follow your data. Technical reference: Data Ingestion queue.