Site migration is without a doubt a very important stage in the life cycle of a website. You have to pay attention to many details in order to keep your SEO machine running and to avoid damaging anything. Site Audit, SEMrush’s unique multi-tool for technical SEO, now provides you with everything you need to relocate your site safe and sound.
But there’s more. Site move is a very complex process and you’re going to need a good checklist to keep everything under control. Following the recommendations of Google, we have developed a complete site migration checklist for each type of site migration with tips from the top SEO experts. You can use it along with SEMrush tools. Check it out!
Now let’s take a closer look at the new Site Audit features.
Crawled Pages export
We’ve added an option to export the Crawled Pages report data, taking into account filtering and sorting applied. You can also export relative URLs instead of absolute ones.
Crawling by URL list
We’ve added a new option for indicating a crawl source – List of URLs. The tool works with both absolute and relative URLs. You can now upload your list using .txt or .csv formats from Google Analytics, Google Search Console, Backlink Analytics, or Position Tracking. A file, which we’ll crawl all URLs (based on your limits) from, should be less than 100 Mb.
Crawling of a password-protected site
It’s now possible to crawl your site with Site Audit if you have to protect it with a password and login for security reasons.
Crawling of a site closed for robots
Previously, it was impossible to crawl a site, closed to robots.txt file. From now on, you can easily crawl your site after confirming your rights to it (as we’re concerned about the legitimacy of the robots.txt circumvent).
What do you think of this update? Your comments, thoughts and ideas are priceless to us. Please send them to site-audit-feedback@semrush.com.
New analyst report recognizes Semrush’s competitive intelligence, robust analytics, and vision for SEO as the “engine of digital discoverability”
Product News
July 2, 2025
Optimize for AI Search with Site Audit
Semrush has introduced a brand-new AI Search category in the Site Audit tool, helping SEO professionals and web teams prepare their websites to be found and cited by AI search engines.
Product News
July 1, 2025
New AI Traffic Tool: Stay Ahead in the Era of AI-Powered Discovery
The Semrush Traffic & Market Toolkit just added a powerful new tool that gives digital marketers and analysts an unprecedented view into one of the most transformative shifts in online discovery: traffic driven by AI assistants. With the debut of the AI Traffic dashb...