Managing sitemaps
Once you’ve added a sitemap to your workspace, you can manage it directly in ContentGen.io under 'Settings'. This gives you full control over how your site is crawled and ensures the platform always works with the most up-to-date data.
For each sitemap, you can:
- Add crawl rules – Exclude or include specific URLs or patterns to fine-tune what ContentGen.io scans.
- View status – Check whether the sitemap has been successfully processed.
- See page count – Monitor how many pages have been discovered.
- Adjust crawl frequency – Control how often ContentGen.io should revisit the sitemap.
- View last crawl details – See the date and time of the most recent crawl.
- Access crawl history – Review previous crawls and results.
- Trigger a manual crawl – Kick off an immediate crawl if you’ve made site changes.
- Delete sitemaps – Remove outdated or incorrect sitemaps from your workspace.
Keeping your sitemaps up to date ensures ContentGen.io has the best possible view of your site for content generation and optimisation.
How can I provide more context for ContentGen.io with Segments?