Prove You Own the Site You Crawl

Show that the site is yours

Opensolr crawls a site only after you show that you control it. You do it once for each start URL, by putting one small file on your site. Opensolr finds it on its own.

Steps

1. In the Crawl URLs panel of the WebCrawler tab, click URL Verification.
2. Click this file to download your verification file.
3. Upload it to your site so that it opens at https://example.com/opensolr-verification/ followed by the file name shown in the panel.
4. Wait. The panel says it: Verification runs every 5 minutes. When your file is found, the Verified column of the row turns to Yes.

For a subdomain, put the file on the main domain: for https://blog.example.com, the file goes to https://example.com/opensolr-verification/.

Good to know

• Upload the file exactly as you downloaded it: Opensolr compares what is inside.
• Each Opensolr Index has its own file. A second index on the same site needs its own file next to the first.
• A verified start URL stays verified.
• Until at least one active start URL is verified, a start answers ERROR_PLEASE_VERIFY_YOUR_URLS.

No file when your CMS adds the site

The Opensolr Search module for Drupal and the Opensolr Search plugin for WordPress add your sitemap as a start URL with a call signed by your API key. The signature proves the request comes from you, so that start URL is verified at once, with no file to upload. Your own code can do the same: Recrawl a page from your CMS.