How To Find Orphaned Files In A Website

Finding webpages that have no links is difficult, only non incommunicable.

If at that place are pages on your website that users and search engines tin't attain, this is a problem y'all need to fix.

Fast.

These types of pages accept a name: orphan pages.

In this post, you lot'll learn what orphan pages are, why fixing them is important for SEO, and how to discover every orphan folio on your site.

What Is an Orphan Page?

A page without any links to it is chosen an orphan page.

Search engines, like Google, commonly find new pages in one of ii ways:

• The crawler follows a link from another page.
• The crawler finds the URL listed in your XML sitemap.

So if you want Google to crawl and alphabetize your page, they demand to be able to find it.

Why Are Orphan Pages an SEO Issue?

Search engines can't find orphan pages through links, so orphan pages oft become unindexed and never prove up in search results.

Even if your orphan pages are listed in your XML sitemap, they are yet a problem for SEO.

Are Orphan Pages Bad?

Orphan pages aren't great for either users or crawlers.

Users can't reach those pages through your site'due south natural structure so if there's important or useful data on those pages, it'southward wasted.

This can create a frustrating user experience.

With no internal links, no authority is passed to the pages, and search engines accept no semantic or structural context in which to evaluate the page.

Without whatever way of knowing where the page fits into your site as a whole, it can be more difficult to decide which queries the page is relevant for.

Orphan vs. Dead Finish Pages

Before nosotros dive into orphan pages, let's take a moment to briefly clarify the difference betwixt 2 SEO terms that tin crusade defoliation.

As nosotros've already established, an orphan page is a webpage that isn't linked to past, or reachable from, whatever other page on the same website.

A dead-finish page, on the other hand, is a webpage that doesn't link to any other internal webpages or any external websites, thus creating a "dead end."

When people land on this page, they can either hit back or just carelessness the site.

When search engine crawlers country on the folio, they have nowhere to become, and no link equity can be passed.

Today, with and so many templates and themes available, it'southward more difficult to create a expressionless finish – but hardly impossible.

A dead end can easily be remedied by adding links to your on-folio content, or making sure that sidebar or footer navigation is populated on every page.

All clear? Good.

At present let's find your orphan pages.

1. Place Your Crawlable Pages

You'll need a listing of all of the URLs that currently can exist reached past crawling your site'southward links.

Yous volition need your own crawler – an SEO spider, to do this. ScreamingFrog is a good option.

Whatsoever crawler you utilise, make sure it is set to crawl only pages that are indexable past search engines.

By that, I mean that information technology should not crawl pages that are:

• Noindexed
• Hidden from search engines by robots.txt.

Start the clamber from the homepage of the site.

Make sure to apply the canonical URL, including proper https or http, and world wide web or not-www.

Once yous accept crawled your site, export the URLs to a spreadsheet similar this:

2. Resolve ii Common Causes of Orphan Pages

There are two common causes of orphan pages that should be immediately addressed and dealt with.

Both these causes are essentially page duplicates that should automatically redirect consistently to simply one URL.

If they don't, information technology's likely that some versions of the folio are not linked to and every bit a effect are orphans.

In this example, the fact that they are orphans isn't the primary issue, the fact that they are duplicates is.

These may come up later while y'all are looking for orphan pages, and need to exist dealt with, then it'due south a good idea to get them out of the way beforehand.

Non-Canonical https/http or www/non-www

Every public page on your site should ideally employ http or https consistently (preferably https), and www or non-www consistently.

To cheque if this is the case, try typing all of these variations of your site's homepage into your browser:

• https://www.case.com
• http://www.instance.com
• https://example.com
• http://instance.com

All 4 variations should redirect automatically to the exact same URL.

For consistency, that folio should be canonical to itself.

If ane of these variations does not redirect properly, it tin exist a sign of like problems on the wider site.

Bank check other URLs, using that variation, to come across if it's a more than widespread outcome.

Yous should test a few other pages of your site, and check your site's .htaccess file to make sure that redirects for these are ready upwards properly.

Here is how to force https in .htaccess. If you lot do this, verify that every folio on your site has SSL capabilities, or your users will go a scary browser warning.

Here is how to force www or non-world wide web. Once again, verify that this won't create any server errors.

Trailing Slashes

Another affair to watch out for is the consistent utilize of trailing slashes.

For example, these 2 URLs may produce the same content, but the URLs are not identical:

• https://example.com/page1/
• https://example.com/page1

Check a few pages on your site both with and without the trailing slash, and make sure that they redirect automatically to the same URL, and that they do and then consistently.

Verify that this is ready up properly in .htaccess.

Hither's how to force a abaft slash in .htaccess.

three. Get a Listing of URLs from Google Analytics

Crawlers, by definition, will have a difficult fourth dimension finding orphan pages.

Then using whatsoever SEO tool to find i is bound to exist problematic.

One of the best places to start looking for orphan pages is your own Google Analytics data (or whatsoever other analytics packages you lot utilize).

As long as the pages in question take Google Analytics installed, if the page has ever been visited, in that location is a record of it somewhere in Google Analytics.

To go a comprehensive list of URLs, from the left sidebar, go to Behavior > Site Content > All Pages.

Considering our orphan pages are difficult to discover, the number of times they accept been visited is likely to exist quite low.

Click "Pageviews" so that the arrow is pointing upwardly, indicating that the list of URIs is sorted in ascending order from least to most pageviews.

This will move the pages most likely to be orphans to the top:

To make sure our list is as comprehensive as possible, go to the date range at the summit right.

Set the starting date dorsum to a time before Google Analytics was in place and click the Apply button:

Now nosotros will need to expand our list of URLs as much equally possible.

In the bottom right, click the Testify rows dropdown bill of fare and select the highest number of rows.

Our biggest obstacle is that Analytics tin can only list upwardly to 5,000 URLs at a time:

If you have more than this, you volition have to export v,000 pages at a time until you have all your Google Analytics visitor data.

All the same, we are sorting pageviews by ascending, so our listing should hopefully include all, and volition most likely include about orphan URLs that take had a visitor.

It will likely take a bit of time for Analytics to fetch all of the data.

Be patient and don't attempt to rush things, or yous will risk crashing your browser.

Once the URLs are loaded, head up to the top right, select export, and export a Google Sheet, Excel file, or CSV spreadsheet to get your URLs.

If you're slightly more technical, you can use the Google Analytics API to speed upwardly this process; try using the pageviews metric against the pagePath dimension.

Now copy the URLs from your exported analytics file into your orphan page spreadsheet, like so:

We volition need to get these into URL format in social club for them to be useful.

To practice this, insert a new column and paste down the homepage URL, like and then:

And employ the concat() formula to combine these together into a URL in the adjacent column over:

Then just drag the formula downward to become the total list of URLs:

4. Identify Your Orphan URLs

To identify our orphan URLs, we volition demand to compare the listing of Crawlable URLs and the listing of found Analytics URLs in our spreadsheet.

In our hypothetical example, it's obvious that https://example.com/11 is an orphan page, but in reality you will most ever take far more URLs to sift through, and nosotros will need to automate the process of identifying our orphan URLs.

To exercise this, nosotros need a formula that checks if each URL in our Analytics list is also found in our listing of Crawlable URLs.

Here is an instance of a formula that will accomplish this:

The "match" formula we have used in cell E2 hither is:

=match(D2,$A$2:$A$11,0)

This formula checks if the URL in cell D2 is in the range $A$2:$A$11.

(If y'all're not too familiar with spreadsheets, the dollar signs are there to make certain that when we drag the formula downward the column, the range won't alter.)

The value "0" tells Google Sheets that the columns aren't necessarily sorted. (Run across the Google Sheets documentation.)

If at that place is a match, the formula returns its position in the range, which in this instance is the offset position in the range.

What nosotros're more interested in, however, is if at that place isn't a lucifer.

As you lot can see, the formula returns the error "#Northward/A" for https://example.com/11, because it is not found in our list of Crawlable URLs. This ways it is an orphan folio.

To become a list of our orphan pages, then, all nosotros need to do is sort our Match column to collect all of the "#N/A" results in one place.

We tin can and so copy our listing of orphan URLs and paste them to a new sheet where we tin can accost how to fix them.

5. Other Places to Await for Orphan URLs

You can repeat this procedure for identifying orphan URLs using information sources other than Google Analytics.

Any of the following tools will have a list of pages crawled from your site:

• SEMrush
• Ahrefs
• Moz Link Explorer
• Raven Tools

I would not recommend signing upwardly for any of them exclusively to wait for orphan pages, because they volition demand to somehow crawl these pages in order to find them.

SEMrush and Ahrefs have specific tools and practices to help you discover orphaned pages.

It is possible that in some cases these tools will find pages that aren't directly crawlable because they were found using other means, usually at some point in history when the folio was crawlable:

Piece of work with your dev team to see if they can get the consummate list of URLs on the site directly from the server, since this should be the most complete list available anywhere.

Y'all can also look through your log files to observe this information.

Log files contain data about:

• Who has visited your website.
• Where they came from.
• What pages they visited.

Yous can perform a second crawl of your site, ignoring directives similar "nofollow" and "noindex", and compare it to your original crawl.

At that place may be pages that are only accessible by crawlers who ignore those directives, and those can be some other source of orphan pages.

Finally, you can get a list of URLs from the Google Search Console's Search Analytics report.

Even though these pages are obviously indexed if they are showing up here, you may still find pages that aren't crawlable from your internal links that will need to be fixed.

Decision: Finding & Fixing Orphan Pages

Orphan pages tin't be indexed past search engines if they don't prove upwardly in your sitemap – and they tin create other SEO issues fifty-fifty if they do.

When you have gone through these steps and found your orphan pages, ask yourself some questions:

• Is this page important? If it is, observe where to integrate information technology. If non, remove it.
• Is this folio ranking for whatever keywords, despite existence an orphan page? If it is, find where to integrate it. If not, remove it.
• Where should the page exist within your website's taxonomy?
• Is this page a duplicate or near duplicate? Consider folding that content into a similar folio that isn't an orphan.
• Is this page optimized? Could it exist optimized and better linked from?
• Has the page been linked to from external sources?

Use the methods outlined in this mail service to find your orphan pages and get this issue resolved.

More Resources:

• Site Structure & Internal Linking in SEO: Why Information technology'due south Important
• SEO UX Play: Information Architecture & Linking Bureaucracy
• 7 Reasons Why an HTML Sitemap Is a Must-Accept

---

Image Credits

Featured Paradigm: E2M Solutions
All screenshots taken by author

Source: https://www.searchenginejournal.com/find-orphan-pages/276207/

Posted by: schuleroulk1944.blogspot.com

Share this post