Control the Crawl of Your Site by API

Start, pause, refresh and watch the crawl of your site

Control the crawl

Run the crawl of your site from your own code or from your CMS: start it, pause it, have a page read again the moment you change it, and watch the numbers live. It is the same crawl you control in the Control Panel of your index.

Every call here goes to https://opensolr.com and works on an index in a region with site search (What your index needs). The three calls marked "signed" also carry a signature made with your API key.

Start, pause, stop

start_crawl

Start reading your site from its start URLs, on a schedule or once; how far and how fast it goes are part of the call.

Reference
pause_crawl

Pause a running crawl; the schedule stays and starts it again on its next run.

Reference
stop_crawl

Stop the crawl and remove its schedule.

Reference
crawler_active

Tell whether the crawl is running right now.

Reference
crawl_schedule_status

Tell whether the index has a crawl schedule.

Reference
flush_crawl_buffer

Send the pages read so far into the index now, instead of waiting for the next batch.

Reference

Keep pages fresh

move_done_to_todo

Queue every page already read to be read again, so texts and prices update and pages that are gone leave the index. Keep Index Fresh does this for you on a schedule.

Reference
add_crawl_url_signed

Signed: add an https start URL that is ready to crawl at once, with no verification file on your site.

Reference
delete_crawler_url

Signed: make the crawl forget one page, so the next run reads it again, for example right after you edit it.

Reference

Watch it

get_crawl_stats

The live numbers of the crawl of your site.

Reference
tail_crawl_log

The last lines of the crawl log.

Reference
get_settings_dashboard

Signed: everything a CMS settings page shows in one call: plan and usage, crawl numbers, schedule, your indexes and regions.

Reference

The same controls in the Control Panel: Start, pause, stop, flush, Keep your index fresh and Recrawl a page the moment it changes.

Site search API pages

Back to Enterprise Site Search