Select a category on the left, to get your answers quickly
The opensolr web crawler now indexes and follows any file type in your web root.
To learn more about what fields are indexed, simply create a new opensolr index, go to Config Files Editor, and select schema.xml.
All the fields inside the schema.xml are indexed.
You would simply have to register for a new account, create a new index, go to the CRAWLER tab, add your starting ROOT URL, and then go to Setup Crawler and start the crawling process by clicking the Start Crawl button.
The crawler will follow ANY document types in your starting ROOT URL, and index them and, also have them ready for search in a nice WEB Search UI interface.
You may choose to crawl the entire domain name, which would also include all subdomains of the url you have selected, or only restrict the crawling to the hostname, or simply to the PATH. More information is given in the Setup Crawler tab about each of these options.
Every time you start the crawler, it will recrawl all urls, and will replace the old content with the new one for each URL.
Therefore, this could be considered as updating your website crawled content.