Documentation > Web Crawler

API - Get LIVE Web Crawler Stats

Web Crawler
Web Crawler API

Get LIVE Web Crawler Stats

Get real-time statistics for your web crawler: pages crawled, pages queued, errors, traffic used, and more.

Endpoint

GET https://opensolr.com/solr_manager/api/get_crawl_stats

Parameters

ParameterStatusDescription
emailRequiredYour Opensolr registration email address
api_keyRequiredYour Opensolr API key
core_nameRequiredThe name of the index you wish to get live crawler stats for

Code Examples

$_ cURL

curl -s "https://opensolr.com/solr_manager/api/get_crawl_stats?email=YOUR_EMAIL&api_key=YOUR_API_KEY&core_name=my_solr_core"

PHP PHP

$params = http_build_query([
    'email'     => 'YOUR_EMAIL',
    'api_key'   => 'YOUR_API_KEY',
    'core_name' => 'my_solr_core',
]);
$response = file_get_contents("https://opensolr.com/solr_manager/api/get_crawl_stats?{$params}");
$stats = json_decode($response, true);
print_r($stats);

Py Python

import requests

response = requests.get("https://opensolr.com/solr_manager/api/get_crawl_stats", params={
    "email": "YOUR_EMAIL",
    "api_key": "YOUR_API_KEY",
    "core_name": "my_solr_core",
})
stats = response.json()
print(stats)
Use this endpoint to monitor crawler progress in real time. It's useful for building dashboards or deciding when to pause/resume the crawler.

Related Documentation

Start CrawlerLaunch the web crawler with full control over mode, threads, and rendering. Pause CrawlerTemporarily halt crawling while keeping the cron schedule. Resume CrawlerContinue crawling immediately from where it left off. Check StatusCheck whether crawler processes are currently active. Get Live StatsReal-time crawler statistics: pages crawled, queued, errors. Flush BufferForce pending documents from the crawler buffer into Solr.

Need help with the Opensolr Web Crawler? We are here to help.

Contact Support
Try it live · no signup, no code
See the Web Crawler and AI Hybrid Search working in 60 seconds

Paste any website URL into the RAG in 60 Seconds sandbox. The Web Crawler fetches your pages, Opensolr generates the vector embeddings, and you get keyword + vector hybrid search plus a RAG answer on your own content — live, on a temporary index, with the keyword-only, vector-only and hybrid results shown side by side. No account, no OpenAI key, nothing to install.

Try AI Hybrid Search in 60 seconds →