Add Start URLs: Home Page, Sitemap or Feed

Where the crawl begins

A start URL is the address the crawl of your site begins from. One per site is enough: Opensolr follows the links from there, and every page it reaches within the crawl mode goes into your Opensolr Index.

Add one

1. In the WebCrawler tab of your Opensolr Index, find the Crawl URLs panel.
2. Type the address, like https://example.com or https://example.com/sitemap.xml.
3. Click Add URL.
4. Prove you own the site, once.

Only https:// addresses are accepted, on the public internet. Your plan sets how many start URLs you can have across all your Opensolr Indexes: https://opensolr.com/pricing.

What to add

The list

Each start URL is a row with four columns: URL, State (Active or Inactive), Verified (Yes or No) and Settings, a cog that opens the rules of that site.

• Tick rows, then click Delete, Activate or Deactivate. An inactive start URL stays in the list and is left out of the next crawl.
• URL Verification shows where to put your verification file.

A new start URL is Active and not yet Verified. A crawl reads only the start URLs that are both Active and Verified.