How to fix your nofollow screwup

You manage a large Web site with thousands or millions of pages of content and sometime in the last two years you resorted to using rel=’nofollow’ on internal links in the hope of “improving” your search performance. Having drunk the kool-aid you looked at metrics and concluded that maybe it worked. And you felt it continued working even long after Google actually deprived you of all that PageRank you thought you were channeling.

So here you are today, wondering if you should remove the nofollows.

What Happened To The “Evaporated” PageRank?
The internalized nofollow has not been concentrating PageRank on favored pages the way people hoped it would for over a year. That PageRank has, according to Matt Cutts, sort of been “… going into the reset vector ….” He doesn’t explain clearly what happens, but one is left to infer that evaporated PageRank may be “redistributed to the rest of the web” (Note: These quotes are drawn from comments on Matt Cutts’ blog post, which I linked to above).

If the PageRank is indeed being redistributed across the Web (let’s speculate only across the Main Web Index), then in a way you’re indirectly distributing part of your “evaporated” PageRank evenly across all your Google-indexed pages — this might be a more efficient distribution method in terms of allocating PageRank through fewer iterations. It just wouldn’t distribute much PageRank to your own site.

Then again, if you’re not the only person distributing PageRank to the entire Main Web Index, then all your indexed pages might benefit from all the pooled evaporated PageRank (we could also call this Recycled PageRank but I like P.E.P. as an acronym). Large content sites would thus draw down more P.E.P. than small content sites. Hence, your indexed pages would have more PageRank gravity than other people’s indexed pages.

Speculative though it is, that strikes me as a cool concept. Google could have found itself tracking PageRank flow to otherwise link-poor sites that had just barely gotten into the Main Web Index. Think of a spam site that has not actually been penalized by Google but which suddenly gets a lot of PageRank. If it has millions of pages of content, it might draw enough P.E.P. to start climbing into monetized search visibility. Wouldn’t that be some nice payback for the black hat community?

Why Undoing Internal Nofollow Strategies May Fail
This week, Rand Fishkin noted on SEOmoz that he removed internal nofollows on some client sites and saw drops in search referral traffic. He restored the nofollows and the traffic appeared to return.

While that’s not enough work to establish cause-and-effect, I would not want to put a client site at risk by removing the nofollows again just to pass the scientific validity test. There is too much at stake with commercial sites for SEOs to be getting experimental on the clients’ time.

Still, it makes sense that if you remove a LOT of internalized nofollows all at once, you’ll see a sudden shift in the way a search engine values your Web content. The internalized nofollows didn’t actually fix any problems to begin with — they only masked those problems. After all, people were looking at the wrong kind of metrics for PageRank sculpting.

Instead of measuring PageRank (which no one in the SEO industry is able to do), SEOs had to measure search referrals and conversions. They were trying to remove “incidental” pages (About Us, Contact Us, Terms of Service, blog archives, etc.) and other types of content people actually search for but which don’t usually produce desired results (conversions). In many cases on large content sites a lot of algorithmically duplicate content pages might have fallen out of the index and search confusion might have been resolved.

That’s a backwards way of fixing site structure, and clearly you don’t need as much PageRank as you thought you did because handing a huge chunk of your PageRank over to the rest of the Main Web Index didn’t hurt you.

In the final analysis, internalized nofollow only acted like a carpet hiding a dirty floor. Restoring the old internal link structures throws the carpet out and re-exposes the dirty floor.

What SEOs Should Have Done Instead of Using Internalized Nofollow
If the SEO community had not resorted to the bad practice of hiding bad site structure through internalized nofollow, a lot of broken site structures would have been fixed by now. Instead, they remain broken and many sites that turned to internalized nofollow have sacrificed their precious PageRank for nothing.

There are always two good options for any site that has poor architecture:

• Rebuild the site with a good crawlable architecture that emphasizes the truly most important content
• Add a layer of alternative architecture to the site to improve crawl and shift importance to the right pages

Some SEOs complain they cannot get their clients to add more content or rebuild their sites. You know, there are still ways to improve the crawl and importance for a site’s pages: get more links.

Changing internal links to use nofollow was naive, cheap, and in the long run has cost those clients a lot of value. Think of all the link building those clients did in the first place to accumulate PageRank. The return on investment for those links dropped significantly when Google changed the way it calculates PageRank for pages that have nofollowed links.

Alternative architectures are easy to construct. In some industries they are used extensively for providing users with optimized navigation systems to move through large inventories by region, manufacturer, function, etc. It’s not like I’m proposing something radical and new — alternative site architectures have been used and safely indexed by search engines for years.

How To Develop An Alternative Site Architecture
I’ve seen people do this in more wrong ways than I can count. I’ll use a few simple examples to illustrate the technique but understand that each site needs to be carefully evaluated based on its own content.

Building an alternative architecture should not be influenced by existing architecture. The existing architecture is irrelevant to the alternative architecture. If you find yourself “blocked” by existing site architecture, you’re doing it wrong.

1. Create an alternate site one page at a time
2. Create an alternate site map
3. Create an alternate veneer
4. Create new crawler-friendly content

There are other ways but I think four examples is sufficient.

Create an alternate site one page at a time – Start with an alternative home page. You do nothing to the existing home page. You just set up an alternative home page and build some links to it. From this home page, you begin branching out into your site through new content pages. Develop new content one section at a time and switch your alternative home page’s links to point to the new section as it goes online. Each new section can then begin to accrue value and when you’re ready you 301-redirect the old section pages to the new section pages.

You don’t have to give up your old site’s performance to do this. If you’re worried about “duplicate content”, don’t create duplicate content. Use this as an opportunity to implement a new site design, too. If you’re satisfied with where things are now, take your time and do this right. You’ll have to do it eventually anyway and you can at least get some more search visibility by creating new content.

Create an alternate site map – I’m actually talking about HTML sitemaps. If you’re going to update your content and site navigation, then updating your XML sitemap makes sense. If you’re not changing the navigation and content then resetting suggested crawl priorities in the XML sitemap won’t help you (except that you want to give the new HTML sitemaps high priorities).

Don’t take down your old HTML sitemaps until the new HTML sitemaps are indexed. Then use 301-redirects to move the HTML sitemap value to the new pages.

If your pages don’t link to your HTML sitemaps or you don’t have HTML sitemaps, then simply adding a link to the HTML sitemaps from every page should fix a lot of your problem. If you can get a client to selectively put nofollow on their internal links you should be able to get them to add a link to their HTML sitemaps.

The more internal links an HTML sitemap gets, the more important it becomes, the more influential it becomes.

Create an alternate veneer – This is like creating a new face mask for your site. Your deep content pages don’t change. You just build a tree of new section/category pages that are more efficiently structured (and which you index in your site search). You want to give the veneer its own HTML sitemap (and XML sitemap) AND you want to include it in the main site’s HTML and XML sitemaps.

Veneers are often placed on sub-domains. People call them different things: portal pages, alternate UIs, etc. They are often used for different regions, different languages, different browsers, etc. Veneers index the same content people see through the main site, but they have their own unique copy.

Question: Should the indexed content link back to the veneer?

Answer: In an ideal world, yes. If you cannot do that, discuss using frames with the client the way Google Image Search and About.com use frames. Forget all the nonsense you’ve been told about how frames are not search engine friendly, blah blah blah. If any part of the site is indexed in Google Image Search, linked to by About.com, or otherwise referenced by other services, those pages are already being served up by external veneers.

There is nothing sneaky about using a veneer but it needs to be clear to the user that it’s a veneer and it needs to provide value of its own. The navigation system is often the only value that a third-party veneer offers but you can include small footprint widgets. Be creative and relevant.

Creating a veneer based on frames may require you to create an alternate page for every existing content page. Fortunately, you can choose from two types of frames: iframes and traditional frames. The only thing is you cannot do away with the existing site’s navigation structure, so be sure you don’t create user confusion. I would go for the traditional About.Com/Google Image Search style of veneer — most people are used to those structures.

Create New Crawler-friendly Content
You could just add a blog to the site (or another blog if the site already hosts blogs) and use that to link to content pages. Be creative with your blog posts but use them to index content on a daily, weekly, monthly basis. Write blog posts that index your blog posts. Danny Sullivan, Andy Beal, and other people whose names you know and trust have been this with their own sites for years.

It’s not as easy to write content about content as it is to autogenerate a new navigation veneer, but you might find some clients are more comfortable with this approach.

The Final Word: You MUST Fix The Nofollow Problem You Created
I would consider it to be unacceptable for any SEO to simply leave things as they are in the long run on a client site. Yes, Matt Cutts says you can do that if you’re happy with the way things are, but that’s not responsible SEO. If you cannot rebuild the site to fix the original problem (bad architecture, inadequate or wrongly focused navigation), then you need to grow the site out of the mess.

It was a bad idea to begin with — I’ve always said so. I had no way of knowing in advance that Google was going to pull the rug out from under everyone’s feet. What I did know and understand, however, is that people were whipping out the nofollow attribute for the wrong reasons.

I have no doubt that a lot of mediocre and bad SEOs are going to point to Matt Cutts’ comment in the SEOmoz video for years to come to justify their laziness. Matt will probably take a lot of heat for giving the appearance of supporting bad SEO. Nonetheless, there are other voices in the field you can listen to besides mine.

For example, Vanessa Fox criticized the ‘do nothing’ approach and said: “I think the bigger reason to do away with a internal nofollow policy is in terms of future maintenance. Similar to the advice I give people about meta keywords tags. If I see a site with a bunch of keywords in their meta keywords tags, it’s not going to hurt them, but if long term, they spend a lot of time maintaining that tag on every page, it could take time away from doing things that are more valuable.”

(NOTE: If the Bing-Yahoo! deal goes through, only Ask among the major search engines will still honor the keywords meta tag, making it even less important than it has become today.)

When you’re confronted with a site that doesn’t perform in search, limiting its options is NOT the same as optimizing it for search.