A site or section that “disappeared” may be unavailable, removed from the index, consolidated to different canonical URLs, hidden by a Search Console removal, affected by security or manual action, damaged during a migration, or simply ranking below where someone checked.
Treat a broad disappearance as an incident. Confirm its scope with Search Console evidence before changing content or submitting repeated indexing requests.
Define the incident precisely
Record:
- exact Search Console property and protocol;
- first known affected date and last known healthy date;
- whether the scope is one URL, a template, directory, subdomain, or entire domain;
- Web or another Search type;
- representative high-value URLs and known queries;
- launches, migrations, redirects, robots, CDN, security, or CMS changes;
- whether clicks, impressions, and indexed counts changed together.
“Our homepage is not first for its name” is a ranking question. “Impressions became zero and URL Inspection says every tested page is unavailable” is an indexing or serving incident. They require different responses.
Confirm that disappearance is real
Use several signals:
- Check the Performance report for impressions and clicks across a finalized range.
- Compare the affected section with the rest of the property.
- Inspect representative URLs, including the homepage, a strong category or hub, and several detail pages.
- Review the Page Indexing report and submitted sitemap filters.
- Search for a few exact URLs or use a
site:query only as a rough confirmation.
Google notes that the site: operator does not necessarily return every indexed URL. A page can also be indexed but not rank for the query someone tried. URL Inspection and Performance data are stronger property evidence.
If only one manual search fails while impressions continue normally, investigate personalization, location, device, query intent, or ranking changes rather than declaring deindexing.
Check availability before SEO hypotheses
Test affected URLs as an unauthenticated user and from more than one network or region where possible.
Confirm:
- DNS resolves to the intended infrastructure;
- HTTPS certificates and redirects work;
- the final URL returns a stable
200for indexable pages; - the site does not return
401,403,429,5xx, or a soft 404 to Googlebot; - CDN, firewall, bot protection, or rate limiting is not blocking Google;
- the response contains the expected primary content;
- rendering does not depend on a failed API or JavaScript bundle;
- server logs show Googlebot requests and their responses.
A widespread 5xx, access block, empty template, or DNS failure can explain a fast sitewide loss. Restore service first; content edits are irrelevant while Google cannot retrieve usable pages.
Compare indexed and live URL Inspection evidence
For each representative URL, examine the indexed result and then run Test live URL.
Indexed result
This shows the version Google used for indexing, including discovery, last crawl, fetch result, indexability, user-declared canonical, and Google-selected canonical.
Live test
This tests whether the current URL can probably be fetched and indexed. It does not prove that Google will index the URL and does not check every condition, including all quality, security, manual-action, removal, or serving signals.
Interpret the pair:
| Indexed evidence | Live evidence | Likely next action |
|---|---|---|
| Not indexed; live fetch fails | Current technical/access problem | Fix response, DNS, robots, firewall, or rendering |
| Not indexed; live is indexable | Historical issue may be fixed | Verify signals, request indexing for samples, monitor recrawl |
| Indexed as another canonical | Consolidation or duplication | Inspect the selected canonical and align signals |
| Indexed and live healthy | Not a basic indexability failure | Check removals, policy evidence, ranking, intent, and query demand |
| Indexed version healthy; live now blocked/noindex | Recent release regression | Roll back or fix before Google recrawls more URLs |
Sample several URLs. One successful homepage inspection does not prove that a directory template is healthy.
Read the Page Indexing pattern
Look for a change that matches the incident date and scope:
- Server error (5xx): availability or capacity failure.
- Blocked due to 401/403/other 4xx: authentication, firewall, or access policy.
- URL marked
noindex: meta tag or HTTP header deployed intentionally or accidentally. - Blocked by robots.txt: Google cannot crawl content or see page-level directives.
- Page with redirect: confirm the destination is intended and indexable.
- Soft 404: the page looks missing or insufficient despite a success status.
- Duplicate / alternate canonical: Google selected or was told to index another URL.
- Crawled or discovered – currently not indexed: not proof of one technical error; inspect quality, duplication, discovery, crawl, and site patterns.
Filter by a submitted sitemap containing only canonical URLs for the affected area. Remember that report examples are limited; totals and patterns matter more than assuming the listed examples are exhaustive.
Audit index-control signals as a system
For affected canonical pages, align:
200response;- no accidental
noindexmeta tag orX-Robots-Tag; - crawl permission in robots.txt;
- self-referential canonical where appropriate;
- canonical inclusion in XML sitemaps;
- crawlable internal links to the canonical URL;
- no redirect chain, loop, or mass redirect to an irrelevant page;
- correct mobile/desktop and
hreflangrelationships where used.
Robots.txt is not a removal or canonicalization mechanism. A disallowed URL can sometimes remain indexed without content, and Google cannot see a noindex directive on a page it is forbidden to crawl.
Check generated HTML and HTTP headers, not only CMS settings. Template, CDN, and edge rules can differ from what an editor sees.
Check urgent UI-only evidence
Several causes are not available through the Search Analytics API.
Removals
Review the Removals tool for temporary URL or prefix removals submitted by any authorized owner or full user. A removal can hide matching results without changing the live page. Cancel an incorrect request and audit access.
Manual actions
Open Manual Actions and read the exact scope. A manual action can affect a portion of the site or all pages. Fix the underlying spam-policy issue across the affected scope before submitting one documented reconsideration request.
Security issues
Open Security Issues and check messages. Hacked content, malware, phishing, or harmful downloads can create warnings or suppress normal visibility. Secure the site, remove the compromise everywhere, and follow the report’s review process.
Do not assume an algorithmic “penalty” when these reports are clear, and do not assume a clear report proves the site has no ranking or technical problem.
Investigate migrations and ownership changes
If URLs, hostname, protocol, platform, or hosting changed near the loss, treat the disappearance as a migration incident.
Check:
- old-to-new URL mapping;
- direct permanent redirects to equivalent destinations;
- absence of redirect chains, loops, and mass redirects to the homepage;
- new URLs returning
200and being indexable; - canonical, sitemap, internal-link, and
hreflangupdates; - old and new Search Console properties;
- Change of Address use only where applicable;
- server capacity for increased recrawling;
- staging
noindexor robots rules removed at launch.
Google expects temporary fluctuations during URL-changing moves. A sustained loss concentrated in URLs with broken mappings is evidence of implementation failure, not normal migration timing.
For a hosting-only change with unchanged URLs, inspect DNS propagation, firewall behavior, server logs, availability, and forgotten staging blocks. A temporary crawl-rate dip can be normal; persistent fetch failure is not.
Separate index loss from ranking loss
If URL Inspection says important canonicals are indexed and Page Indexing totals remain stable, the pages may not have disappeared from the index.
Then analyze:
- whether impressions fell for the same queries;
- whether demand changed outside the site;
- whether a different site page is now shown;
- country, device, and Search-type concentration;
- content or template changes;
- broad Google Search updates;
- spam-policy compliance and people-first usefulness.
Search Console can show the pattern but usually cannot prove a ranking-system cause. Avoid mass rewrites or URL changes during an incident without evidence.
Prioritize recovery by cause
| Cause | Immediate response | Verification |
|---|---|---|
| Site unavailable or blocked | Restore stable access and capacity | Live inspection plus server logs |
Accidental noindex/robots rule |
Remove the unintended directive safely | Rendered HTML/headers and live test |
| Broken redirects or canonicals | Correct one-to-one mappings and align signals | Crawl mappings and inspect destinations |
| Incorrect removal request | Cancel the request and review access | Removals status and result recovery |
| Manual action | Fix all scoped violations and request review | Manual Actions report messages |
| Security compromise | Contain, clean, harden, and request review | Security report and independent scan |
| New or recently changed page | Improve discovery and wait for processing | Sitemap, internal links, later inspection |
| Indexed but lost visibility | Analyze query/page patterns and demand | Finalized Performance comparisons |
Request indexing only after a representative URL is healthy. Repeated requests do not repair a sitewide cause or guarantee indexing.
Monitor recovery
Track daily:
- Web impressions and clicks for the property and affected section;
- indexed totals and relevant Page Indexing reasons;
- successful fetches and status codes in server logs;
- old/new URL visibility during a move;
- Manual Action, Security Issue, Removal, and message status;
- a fixed sample of critical URLs in URL Inspection.
Define closure before the incident begins to recover—for example, all critical templates fetch and render correctly, unintended directives are gone, no urgent UI issue remains, and finalized impressions have returned to an agreed range for several days.
Common incident mistakes
- Treating one manual search or incomplete
site:output as proof of deindexing. - Editing content before checking whether the server works.
- Testing only the homepage.
- Confusing the indexed URL result with the live test.
- Assuming “URL can be indexed” guarantees it will appear.
- Blocking a page in robots.txt while expecting Google to see
noindex. - Ignoring Removals, Manual Actions, Security Issues, and Search Console messages.
- Redirecting every missing URL to the homepage.
- Repeatedly requesting indexing without fixing the shared cause.
- Declaring normal migration fluctuations acceptable without validating mappings.
Related guides
- Diagnose a sudden traffic drop
- The Page Indexing report: complete guide
- URL Inspection: complete guide
- Diagnose migration traffic loss