Sitemap Re-Discovery on Resume: Every Child Sitemap Re-Fetched for New URLs
Sitemap re-discovery on Resume. When you resume a finished crawl, the crawler now re-fetches all XML sitemaps — not just the top-level sitemap index, but every child sitemap too (e.g. sitemap-products1.xml through sitemap-products22.xml). Any new URLs found in those sitemaps get queued and crawled automatically. This means your index stays up to date as your site adds new pages, without needing a full re-crawl.