The SEO Spider is free SEO website crawler tool for Windows, Mac OS and Linux. Using this software, you can analyse, identify flaws and improve your website onsite SEO to rank high in search engine. Basic version is free and the paid version comes with some advanced features.
It allows you to find broken links in your website and server errors, find temporary and permanent redirects in your website. You can also analyse page titles and meta descriptions for long, short, missing, or duplicated contents. It supports finding the duplicate URLs and partially duplicated elements and low content pages in your website. It collects data using CSS Path, XPath or regex. You can manage your robots.txt, meta robots or X-Robots-Tag directives such as ‘noindex’ or ‘nofollow’, as well as canonicals and rel=“next” and rel=“prev”. The Pro version removes the 500 URI crawl limit and enables configuration options.
Using Screaming Frog SEO Spider tool, you can generate XML Sitemaps and integrate with Google Analytics. Just download from the below link and install it. For Ubuntu it is available as .deb file.
