Documentation
Select a category on the left, to get your answers quickly
Get LIVE Web Crawler Stats
- GET https://opensolr.com/solr_manager/api/get_crawl_stats
- Parameters:
- email - your opensolr registration email address
- api_key - your opensolr api_key
- core_name - the name of the index/cluster you wish to get the live web crawler stats for
-
Example: https://opensolr.com/solr_manager/api/get_crawl_stats?email=PLEASE_LOG_IN&
api_key=PLEASE_LOG_IN&core_name=my_solr_core
Trigger Start the Web Crawler
- GET https://opensolr.com/solr_manager/api/start_crawl
- Parameters:
- email - (required) your opensolr registration email address
- api_key - (required) your opensolr api_key
- core_name - (required) the name of the core you wish to start the web crawler process for
- follow_docs - (optional) follow documents and images? (yes/no). Default: no
- clean - (optional) start fresh, or resume from where you left off? (yes/no). Default: no.
- auth_username - (optional) if your starting URLs are using Basic HTTP Auth, you can enter the username here.
- auth_password - (optional) if your starting URLs are using Basic HTTP Auth, you can enter the password here.
- mode - (optional) the mode to follow links. possible values are: 1 or 2 or 3 or 4. Default value is 1.
- Follow all DOMAIN links (full site crawl. follows all subdomains *.site.com, and all paths *.site.com/*)
- Follow all HOST links (limits crawl to HOST only site.com and all it's paths, doesn't follow other subdomains)
- Follow all PATH links (limits crawl to given PATH, site.com/path/ will not go outisde the /path/)
- Follow the initial URL only. not scraping any other links.
-
Example: https://opensolr.com/solr_manager/api/start_crawl?email=PLEASE_LOG_IN&
api_key=PLEASE_LOG_IN&core_name=my_solr_core
Trigger Stop the Web Crawler
-
GET https://opensolr.com/solr_manager/api/stop_crawl
-
Parameters:
-
email - your opensolr registration email address
-
api_key - your opensolr api_key
-
core_name - the name of the core you wish to stop the web crawler process for
-
Example: https://opensolr.com/solr_manager/api/stop_crawl?email=PLEASE_LOG_IN&
api_key=PLEASE_LOG_IN&core_name=my_solr_core