API - Add a Crawl URL (HMAC signed)

Web Crawler

Adds a start URL to the crawl of your Opensolr Index, active and verified at once: the signature proves the URL is yours, so no verification file is needed.

Endpointhttps://opensolr.com/solr_manager/api/add_crawl_url_signed
MethodGET or POST
Authemail and api_key of your account, as GET or POST parameters: Authentication. Plus a signature: see below.

01 · Parameters

core_nameRequired

The name of your index.

urlRequired

The start URL. It must start with https://.

signatureRequired

Hex HMAC-SHA256 of url followed by core_name, keyed with your API key.

02 · Example

URL="https://www.example.com/"
SIG=$(printf '%s' "${URL}my_index" | openssl dgst -sha256 -hmac "YOUR_API_KEY" | sed 's/^.* //')
curl -s "https://opensolr.com/solr_manager/api/add_crawl_url_signed" \
  --data-urlencode "email=YOUR_EMAIL" --data-urlencode "api_key=YOUR_API_KEY" \
  --data-urlencode "core_name=my_index" --data-urlencode "url=$URL" --data-urlencode "signature=$SIG"

03 · Answer

{
    "status": true,
    "msg": "URL_ADDED_AND_VERIFIED"
}

A URL already in the list answers URL_ALREADY_EXISTS_ACTIVATED and is switched on again. The next crawl run reads it.

04 · Errors

ERROR_MISSING_PARAMSHTTP 200

core_name, url or signature is missing.

ERROR_URL_MUST_BE_HTTPSHTTP 200

The URL does not start with https://.

ERROR_INVALID_SIGNATUREHTTP 200

The signature does not match.

ERROR_URL_NOT_ALLOWEDHTTP 200

The URL does not lead to a public address.

ERROR_URL_QUOTA_EXCEEDEDHTTP 200

The index has as many start URLs as your plan allows.

ERROR_NOT_CORE_OWNERHTTP 200

The index is not yours.

ERROR_AUTHENTICATION_FAILEDHTTP 403

The email and API key do not match.

ERROR_RATE_LIMIT_PER_MINUTEHTTP 429

Too many calls. Wait for the Retry-After seconds: Rate limits.

05 · Related

Read a changed page again: delete_crawler_url. How to sign: Authentication.

Every error code and HTTP status of the API: API errors. Calls per minute and per hour: Rate limits.