Our website crawler will find all pages within your website as it can handle all kinds of links and redirects. Neither the HTML <base> tag nor any <a> variation of valid link addresses will confuse it. Enable a few options and our sitemap generator can find links in <img>, <frame>, <iframe>, <form>, <select>, <input> and many other HTML tags. You can also have it process all kinds of CSS file references and most Javascript generated links.
When crawling websites, our sitemap generator supports robots.txt file directives disallow and crawl-delay. Likewise, the crawler engine also handles nofollow, noindex and canonical attribute values in <meta> and <a> tags.
Website crawling and sitemap generator speed: Our high performance website crawler is able to sustain a high speed, even when crossing 10.000 and 100.000 pages. Do you have websites larger than that? No problem, you can configure our website crawler and tune its speed and memory usage to handle much larger websites.
Create all kinds of sitemaps including text, HTML, RSS and XML using our sitemapper tool. It is easy to use, yet highly configurable when you need it. You can create HTML sitemaps fully customizable with multiple columns and pages using the HTML sitemap generator. Are you more interested in generating XML sitemaps? The XML sitemap builder can calculate priority values for all pages in your website when creating the XML sitemap.
05-11-2009, 09:40 AM
Micro-Sys A1 Sitemap Generator v2.0.1
Similar Threads
Linear Mode
