Crawl Status: Live Numbers of the Crawl

What the crawl is doing right now, and what failed

Crawl Status

See what the crawl is doing right now: how many pages it found, how many wait in line, what failed and why, without leaving Drupal.

Where: Opensolr Search › Data Crawler tab › Crawl Management › Crawl Status

The numbers

NumberWhat it counts
Discovered PagesPages read successfully.
In QueueAddresses still waiting to be read.
ErrorsAddresses that could not be read.
Downloaded MBHow much the crawl has downloaded from your site.
Solr BufferDocuments read and about to be written to the index.
In Solr IndexDocuments in your Opensolr Index right now.

Below the numbers

  • Pages left out, when there are any: Out of Scope, Redirect Outside (the page sent the crawl to another site), Oversize, Dynamic.
  • Content Types: one bar split by kind of document, web pages, PDFs, Word, spreadsheets, presentations, with each count and share.
  • The failed addresses, in lists that open on a click: 404 Not Found, Errors, Blocked Scope, Redirect Outside, Oversize Pages.

Refresh

The dialog refreshes itself every 5 seconds. Change it with the menu at the top (Off, 5, 10, 15, 30 or 60 seconds); your browser remembers the choice. Refresh updates it at once.

Data Crawler