Bulk 404 Check
Enter your domain. We read the pages the Wayback Machine has archived for it and check which of them now return 404.
Why look for lost pages?
A page that once answered and now returns 404 sends everyone who still has its address to an error page: visitors with a bookmark, sites that link to it, and search engines that remember it. Pages go missing in redesigns, migrations and clean-ups, and nobody lists what was lost. The Wayback Machine does keep a list of what your site once had, which is what this tool starts from.
What the tool does
- Reads the pages the Wayback Machine's index holds for your domain: HTML pages that answered with a 200 when they were archived.
- Asks your live site about each one with a HEAD request, and confirms any 404 or 410 with a GET before it counts.
- Lists the pages that now return 404 or 410, each with a link to the last copy the Wayback Machine holds, so you can see what was there. The results download as a CSV.
What it checks and what it leaves out
- Up to 50 pages per run, the ones nearest the top of the site first. When the Wayback Machine holds more, the page says how many it found.
- It leaves out the home page, addresses with a query string, and site machinery such as admin, login, feeds and the REST API.
- It reads only the status your site answers. A page that answers 200 with a “not found” message counts as live, and a page that redirects counts as live and shows where it went.
- The checks come from SEOBandwagonBot, which obeys robots.txt and Crawl-delay (see how the bot behaves). A page your robots.txt closes to bots is listed as not checked, never as missing.
- Each visitor gets 6 scans an hour. The Wayback Machine is sometimes slow or busy; when it does not answer, nothing is checked and you can run it again.
What to do with the result
For each page that returns 404, decide whether it should exist. If a close match is live, redirect the old address to it with a 301. If the page earned links or traffic and has no match, restore it from the archived copy. If it should be gone, a 410 says so more plainly than a 404. For the links on one live page, use the link analyzer.
Related
Lost pages are one problem of many
A free audit shows how the rest of your site stacks up.