A page Google can't reach can't appear in search results. The four most common causes are a robots.txt rule that blocks it, a noindex tag that tells Google to leave it out, internal links that lead to error pages, and a homepage that doesn't load cleanly. Second Opinion checks all four, and three of them carry more weight than any other check in the Website Score, because each one can stop everything else from mattering.
You can have the best page on the internet. If Google can't open it, nobody finds it.
That's what this group of checks is about. They're not exciting, and they're easy to skip.. but they come first, because a page that can't be reached makes every other improvement on it pointless.
The four ways a page goes missing
- The homepage doesn't load cleanly. An error, a broken redirect, or the wrong kind of response. If this one fails, nothing else on the site gets a fair look.
- robots.txt blocks it. A small file at the root of your site tells crawlers where they can go. One wrong line can block a page, or the whole site.
- A noindex tag keeps it out. A line in the page's code that tells Google not to list it. It often gets left on by accident after a site launches.
- Links lead to dead pages. A menu item or button that goes to an error page stops visitors and crawlers in the same spot.
Why they carry so much weight
Three of these four carry more weight than any other check in your Website Score. Google documents each one plainly: an error page gets dropped from its index, a blocked page doesn't get fetched, and a noindex page gets left out of results.
There's no partial version of invisible. That's why they're weighted the way they are.
How this happens to careful people
Almost always by accident. A developer blocks the site while it's being built and forgets to unblock it at launch. A page gets moved and the old link stays in the menu. A plugin adds a noindex tag to a page type you didn't know existed.
Nobody did anything wrong on purpose. It's just invisible from the inside.
Check your own site in twenty minutes
You don't need special tools for any of this. A browser and your own website will do.
-
Open your homepage the way a stranger would
Type your web address into a private browser window, with the www and without it. Then try it on your phone.
You should land on your real homepage every time, without an error, a warning, or a stop at some other page first.
Every version of your address opens the same homepage, with no error or security warning.
-
Read your robots.txt
Add /robots.txt to the end of your web address and open it. It's a short text file. If it's missing, that's fine.
Look for a line that says Disallow: / on its own. That blocks the whole site. A Disallow line followed by the address of a page you care about blocks that page.
If you find either, it's a job for whoever maintains your site, and it's usually a one-line fix.
Your robots.txt either doesn't exist or doesn't block any page you want people to find.
-
Look for noindex on the pages that matter
Open your homepage and your main service pages. On each one, right-click, choose View page source, and search the page (Ctrl+F, or Cmd+F on a Mac) for the word noindex.
If it shows up on a page you want in search results, that page is keeping itself out. Most website builders have a setting called something like "Hide from search engines".. check whether it's switched on.
The word noindex doesn't appear in the source of any page you want people to find.
-
Click every link in your menu and footer
Click through your main menu, your footer, and the buttons on your homepage. Every one should open a real page.
Your Second Opinion report lists any broken internal links it found, with the page they sit on and where they point. Start with those.
Every link in your menu, your footer, and on your homepage opens a real page.
-
Re-run the audit
Once the fixes are live, run a fresh Second Opinion audit. We read your site live, so these checks update right away. Google takes longer to come back and notice.. usually days, sometimes a few weeks.
Your new report shows these four checks passing, and you know Google's own listing may lag behind it.