All the free sitemaps generators found were bloated with useless features and not so relilable.
Moreover, using the site crawling system to make a sitemap file can lead to an high load on your server, slowing down the navigation of your visitors.
It's like doing a DoS attack on your own site.
And this is not a good thing.
Working on this software, with this idea in mind: let's crawl a local version of the site using an URL substitution and writing the real URL on the sitemap.
Obviously, you can crawl your site directly (also through a proxy if needed).
Comments