API - Re-queue Crawled URLs for Re-crawling

Web Crawler

Puts every URL the crawl has read back in its queue, so the next run reads all pages again: changed pages are updated, and pages that now answer an error leave the index.

Endpointhttps://opensolr.com/solr_manager/api/move_done_to_todo
MethodGET or POST
Authemail and api_key of your account, as GET or POST parameters: Authentication.

01 · Parameters

core_nameRequired

The name of your index.

02 · Example

curl -s "https://opensolr.com/solr_manager/api/move_done_to_todo?email=YOUR_EMAIL&api_key=YOUR_API_KEY&core_name=my_index"

03 · Answer

{
    "status": true,
    "msg": "Re-queued 4180 URLs for re-crawling.",
    "moved": 4180
}

The pages are read by the next crawl run: a crawl schedule picks them up by itself, or start one with start_crawl. Only some document types: start_crawl with requeue=pdf,docx.

04 · Errors

NOT_OWNER_ERRORHTTP 200

The index is not yours.

ERROR_AUTHENTICATION_FAILEDHTTP 403

The email and API key do not match.

ERROR_RATE_LIMIT_PER_MINUTEHTTP 429

Too many calls. Wait for the Retry-After seconds: Rate limits.

05 · Related

One page only: delete_crawler_url.

Every error code and HTTP status of the API: API errors. Calls per minute and per hour: Rate limits.