Why Rand Fishkin’s nofollow post was wrong

The SEO method is pretty clear and simple: Experiment, evaluate, adjust. Unfortunately, some people toss in an unwarranted fourth step: “Reach inappropriate conclusions and proclaim revolutionary breakthroughs in SEO science”.

Let’s look at Rand Fishkin’s recent premature declaration that his implementation of Rel=Nofollow on many internal links magically increased the number of search referrals to SEOmoz. Although Rand inserted a few ifs, ands, buts, and ors into his blog post, he was pretty much convinced that SEOmoz had finally proven that using Rel=Nofollow on your internal links is the latest way to ensure search engine success.

Both Danny Sullivan and I challenged Rand’s assumption, and first Danny then I pointed out that our sites (SearchEngineLand and SEO Theory) have experienced dramatic increases in traffic since the beginning of the year. Neither of us has implemented Rel=Nofollow on internal links.

The whole idea of using Rel=Nofollow to control the flow of PageRank throughout a Web site is very specious. Matt Cutts said, when asked if Rel=Nofollow could be used that way, that it could be used that way. To date I have yet to see him tell people to run out and nofollow their “About Us”, “Contact Us”, and other important pages on their sites (NOTE: See update at bottom).

Last year at SMX Advanced Matt told an audience that “your Web site gets only so much PageRank”, and from there the wrong conclusions began spewing out of the SEO community. How much is “so much”? Matt never bothered to explain that abstract comment, but it does dove-tail nicely with Google’s practice of Web Apartheid, where (by my estimates) as much as 70-80% of all Web pages appear to be trapped in Google’s Supplemental Index.

How Google calculates the “so much” PageRank per Web site doesn’t matter. People have gone off the deep end and concluded that they can massage that “so much” to boost the search visibility for some of their pages at the expense of others. Conceptually, that makes sense only if you can show that of the 20-30% of your pages that probably are in the Main Web Index some of them should not be.

Now, some sites (like SEOmoz) have huge natural backlink profiles. It stands to reason that they have a statistically valid chance of getting more than 20-30% of their pages into the Main Web Index. But what if my estimate is wrong? What if only 50% of the Web is in the Supplemental Index? What if only 25% of the Web is in the Supplemental Index? It still doesn’t matter. SEOmoz should have a higher percentage of Main Web Index pages than the average Web site.

Regardless of where you draw the line, you’re pretty much saying that X-percent of all linking resources on the Web do not pass value. Add to that the many Main Web Index pages that have — apparently — lost their ability to pass value either because people use Javascript, nofollow, or violated some guideline and Google stripped those pages of that ability.

If the dividing line between Main Web and Supplemental Index pages is 50%, you still have fewer than 50% of all Web documents that are capable of passing value. Link rich sites like SEOmoz tend to soak up a disproportionate amount of that value being passed. Hence, the need to “sculpt PageRank” for a site like SEOmoz is minimal at best.

There are some legitimate reasons to use rel=nofollow within internal content in a Web that has been mutilated by link spam. All personal profile pages should probably be nofollowed just to discourage people from signing up for profiles (not that a relatively orphaned profile page on the average social media or forum site will pass value anyway, but the perception of possible value stimulates the bogus link economy).

You might also want to program your search results pages to use nofollow if they link out to other sites. After all, you have no control over the content on those sites. But site search tools should NOT use nofollow. If you nofollow your links to profile pages, how can you use a third-party tool to allow your visitors to find profile pages? You need to maintain an internal search database to ensure that visitors to your site can find what THEY are interested in looking for.

The worst mistake an SEO can make is assuming that they know better than the visitor what the visitor wants to see. When you poke around someone else’s Web site, the fact you click away from their most prominent pages — from their conversion pages — in your quest to find out more shows that you place some value in the rest of the Web site, even if it’s only transient value that won’t persist through to your next visit.

If you want to use Nofollow in your internal navigation, make sure visitors can find the destination pages. Just because you have Nofollowed links doesn’t mean people will see the links. Many people habitually use site search tools to find what they are looking for, so if you shoot yourself in the foot with Nofollow your site search will pretty much suck.

Most Web sites have no legitimate need or reason to attempt to sculpt PageRank. First of all, there are no reliable tests that show you which of your pages are in the Supplemental Index. Secondly, there is no way to know how much PageRank you have (don’t even waste your time by glancing at the Toolbar — it tells you nothing useful). Thirdly, there is no way to know how your PageRank is flowing throughout your site.

People have attempted to draw up “PR Maps” — page listings with Toolbar PR values. Those maps are completely useless because they don’t capture PR values in tandem. Most people never even stop to think about how often each page on their site is crawled, re-indexed, or whatever. A page’s internal PageRank can change quickly but its Toolbar PR value (which Google says is based on more than just linkage) may be relatively stable or artificial.

Since you cannot map the real PageRank assigned to your pages, you cannot track PageRank flow. If you cannot track PageRank, you cannot knowledgably influence it. You can shoot in the dark, make wild guesses, and reach unfounded conclusions. But you cannot prove that you successfully manipulate PageRank.

The only visible effect of having more PageRank than the other guy is that you should have a higher percentage of pages in the Main Web Index. PageRank by itself rarely affects search results directly. Most competitive queries are iinfluenced more by link anchor text than by anything else (although some content-lean pages can shoot to the top of competitive search results quickly).

Still, having more pages in the Main Web Index is the trick. They’ll automatically rank higher than Supplemental Pages regardless of which pages are most relevant to the query. So concentrating your internal PageRank into fewer pages doesn’t make any competitive sense. You’re cutting yourself out of long-tail queries.

Nonetheless, Rand claimed that SEOmoz has seen a dramatic increase in search referrals over the past few weeks. But then, so have SearchEngineLand and SEO Theory. One site implemented Nofollow on internal links and two sites did not. That questions the value of Rand’s conclusion.

In fact, to show cause and effect in a system where you don’t control all conditions, there is only one valid test: repetition. If you think a change you’ve made to your site has caused something to change in search results, in order to find out whether there is a connection between your change and the search results change you have to repeat the process.

So, to test Nofollowed navigation you need to:

1. Insert nofollow on your selected links. Understand that if you allow alternative links to stand unaltered you’re probably NOT going to reshape your PageRank flow.
2. Wait for the Nofollowing pages to be indexed.
3. Determine if the Nofollowed destination pages drop out of the search results (they should if you did this right).
4. Remove the nofollows from your linking pages.
5. Wait for the restored linking pages to be indexed.
6. Determine if the destination pages return to the search results.
7. Insert the nofollows again.
8. Wait for the pages to be indexed again.
9. Determine if the destination pages drop out of the search results.

Two iterations won’t provide you with conclusive results but if the second iteration produces a different apparent result from the first then you can stop right there. Either you did something wrong or your nofollows don’t matter.

If you appear to have replicated the predicted results, go for a third iteration. The third iteration should clinch the test for you.

But even showing that placing Rel=Nofollow on your pages can drop pages from the index doesn’t show that you’ll get more search referrals. Suppose you have X amount of PageRank and you have mystically determined which of your pages are in the Main Web Index.

You could drop all your Supplemental Pages from the index by Nofollowing them (assuming no one else is linking to them). Of course, that doesn’t make your Main Web Index pages any more visible. It just increases the average internal PageRank per page but they are already fully indexed so they don’t become relevant to any more queries.

You could pick a few of your Supplemental Pages and try to pass sufficient PageRank to them that they move into the Main Web Index. Assuming one could do this with complete control, I would move one page at a time. Eventually you would reach a threshold point where at least one page would drop out as you move a new page in.

So you would have to Nofollow all but one of your Supplemental pages and point more links to it. Maybe that would work, maybe not.

In the end, if you can increase the number of pages in the Main Web Index, you’ll do it by pointing more links to them, not by using Rel=Nofollow. Why? Because if you eliminate Rel=Nofollow from the process I just described it still works. Simply pointing more internal links to more pages helps your internal PageRank flow through your site.

You still may not have enough PageRank to get every page into the Main Web Index. Most Web sites don’t seem to accumulate that much PageRank. But most Web sites don’t naturally optimize their internal linking anyway. Internal links have only begun to earn some respect from the SEO community after several years of people depending on external links. Most so-called optimized sites these days are actually very poorly optimized.

So, did Rand directly affect his search referrals by implementing Rel=Nofollow? Probably not, but let’s look at the four reasons your search results change:

There are four reasons for why your search results change:

1. You do something with your Web site
2. Other people do something with their Web sites
3. The search engines do something with their algorithms and/or data
4. People change what they are searching for

Since the beginning of December we know that Google has made algorithmic changes. We also know that people (including SEOmoz) have been changing their Web sites (simply adding new blog posts to your site changes it). And we know that Google claims to see 20-25% new queries every month.

Since the beginning of December, SEOmoz, SearchEngineLand, and SEO Theory have all attracted new natural links. Since the beginning of December SEOmoiz, SearchEngineLand, and SEO Theory have all increased their internal linkage. Since the beginning of December the types of queries that bring people to SEO Theory have gradually shifted to include new expressions I don’t recall seeing before.

Using Rel=Nofollow on internal links is just another unproven, cheesy, how-to-game-Google link trick. Matt Cutts endorsed it a a means of sculpting PageRank but since no one can knowledgably sculpt PageRank anyway it’s not likely to help spammers and therefore won’t hurt Google in Google’s point of view.

Of course, Google’s search results already suck because they refuse to show the most relevant content first. But that’s a point to return to another day.

UPDATE: In a discussion of this post at Sphinn, g1msd and Halfdeck found an interview Eric Enge conducted with Matt Cutts last year. In that interview, Eric said:

“What we’ve been doing is working with clients and telling them to take pages like their about us page, and their contact us page, and link to them from the home page normally, without a NoFollow attribute, and then link to them using NoFollow from every other page. It’s just a way of lowering the amount of link juice they get. These types of pages are usually the highest PageRank pages on the site, and they are not doing anything for you in terms of search traffic.”

To which Matt replied, “Absolutely. So, we really conceive of NoFollow as a pretty general mechanism.”

Eric’s advice is ill-conceived, at least as presented in that article, in that it doesn’t guarantee either A) that PageRank to the “About Us” page will be reduced or B) that the “About Us” page won’t appear in search results for the client’s name.

In fact, a well-optimized site will use internal link anchor text in the form of “About Company X” rather than “About Us”. If PageRank were really making a difference, you’d find your “About Us” page ranking highly for…ABOUT US.

Until your “About Us” page hits the top 1,000 in THAT query, you don’t need to be agonizing over how much PageRank your “About Us” page is accumulating. But if it REALLY bothers you, then just put some links on your “About Us” page to other portions of your site and send that PageRank flowing to where you think it should be.

You’ll accomplish more through positive linking than through negative linking.