Why Brent Payne’s PageRank Sculpting report is bogus

There is no doubt that Brent D. Payne is a very smart guy and an extremely successful search engine optimization strategist. He is the Director of SEO for Tribune Technology, managing the strategies for over 50 news sites. That’s not a sit-on-your-backside-and-watch-the-clicks-come-in kind of job by any means.

Nonetheless, we have entered into a new phase in the PageRank Wars with the latest salvo from Brent at SMX East 2009. Brent seemed to be the heavy hitter in today’s effort to revive PageRank Hoarding — I mean, PageRank Sculpting — I mean, PageRank Consolidation with his Revisiting PageRank & Siloling presentation.

We can boil it all down to two slides Brent showed the audience (which were reproduced in the SE Roundtable live blog report I linked to above). Basically the graphics show peaks in traffic around specific stories in the first slide and peaks in columns on the second slide.

The slides both say, “More testing is needed”. Neither slide makes egregious claims to proof that PageRank Sculpting works.

Bloggers sum up his point as: “At the Tribune Co., they created a tool that allowed producers to assign a value to every page created on the site, level 1-5. The highest level pages would be stripped of follow links. It worked well and they found they could easily rank for breaking news stories.”

According to the blog reports, the Tribune group did not change their methodology until after Matt Cutts dropped his June 15, 2009 bombshell at SMX Advanced 2009 — more than a year after Google started evaporating PageRank from nofollowed links. In the new approach, they simply removed the links rather than nofollowed them.

Technically, this is not PageRank Sculpting. If the links were removed then the front pages were not pointing to more authoritative sources — and that’s an entirely different algorithmic issue.

I am confident that people will point to Bruce Payne’s PageRank Sculpting presentation and say, “See! It works! The Tribune people proved that PageRank Sculpting works!”

In fact, all the Tribune people have proved is that they were as unaware of the ineffectiveness of their attempts to sculpt PageRank as the rest of the SEO community. After all, if they were really doing it right, they would have noticed soon after Google made such an extraordinary change in the way it handles PageRank.

Here is why Brent Payne’s presentation was bogus:

1. You cannot track and measure PageRank.
2. News sites are treated differently from normal Web sites.
3. You cannot distinguish between the Main Web Index and the Supplemental Index.
4. Crawling is not indexing. Indexing is not ranking.
5. When did Brent Payne change his method? Before or after Matt Cutts’ announcement? (AFTER)

It cannot be said often enough: no one outside of Google can measure PageRank. The Toolbar data is only updated infrequently. Google implies (or I and other people infer) that Google recalculates PageRank more often than the Toolbar PR is updated.

But I can tell you, from having worked with a news site for even only a few months, that there are profound differences between what Google does with a news site and what Google does with the average non-news site. With a news site you can just about sneeze your way onto the front page of most search results. You may not stay there, but you can get substantial rankings improvements once you are admitted into Google News.

Part of that comes from Google’s Universal Search Injections, where news stories, videos, and others stuff are seeded between, above, or below organic search listings. But part of the rankings boost comes from the fact that news sites are vetted resources and they seem to transfer into the Main Web Index very quickly and easily.

Despite claims to the contrary by some people in the SEO community, Google’s Supplemental Results Index still seems to play a big role in Google’s search results management. Now, it’s been a long time since I’ve seen a Googler discuss the SRI — maybe it no longer exists any more. I can’t say. But they clearly still distinguish between pages that accrue PageRank (and which rank well for strong keywords) and pages that don’t accrue (or pass) PageRank (and which are outranked by less relevant content).

And presenters at the conference have reiterated that you get “only so much PageRank for your site”, as Matt Cutts once put it. So either the PageRank is spread very thinly and evenly across all pages or it lumps up in some pages. That’s a combination of your internal navigation and backlink profile, so for sites that earn a lot of deep links attempting to sculpt PageRank would be an insane idea anyway.

News sites, as it turns out, tend to earn a LOT of deep links. So do blogs, of course.

And it seems to be a common fallacy in many SEO presentations that people treat crawling and indexing as if they are the same thing. A search engine may indeed fetch your low PageRank page and not index it because it’s going into the supplemental index — the supplemental crawler got it. Google promised last year to increase the frequency with which it updates the Supplemental Index and they seem to have delivered on that promise. Perhaps that is why so many SEO pundits have become sloppy about treating crawling and indexing equally.

But neither is indexing equivalent to ranking. You can be indexed, indexed quickly, and still not rank. Why? All sorts of reasons. PageRank is not the only player on the team and it’s far from being the only player that counts.

So anyone who thinks they are sculpting PageRank, if they cannot even tell when Google introduces a major change to the way PageRank value is flowed, but have to wait until after the announcement to make changes — they’re fooling themselves. That’s a horrible way to manage a process — waiting for a search engine to tell you what you’re doing is a total screwup.

There are many reasons for why SEO PageRank sculpting tests fail to prove anything:

1. You cannot track and measure PageRank. You have no way of doing this.
2. Publishing nothing but a chart proves nothing. Most PageRank tests are reported on blogs or in conferences as pretty charts.
3. Publishing the data behind the chart allows other people to check your analysis. I’ve asked people to publish that data but the data itself could be flawed, contrived, or incomplete. Still, your analysis of your data may be flawed no matter how good the data is. People need to see the data.
4. Publishing the detailed methodology for your test allows people to replicate it. Furthermore, you have to use a methodology that other people generally find to be acceptable. Not 10% of the community or just your best drinking buds, but a solid 70-80% consensus of people who analyze data need to sign off on the methodology.
5. You must use criteria other people can easily match. If you don’t want to publish your data, you have to show people how to obtain data essentially like your own.
6. You must not add or remove any pages from your site. This is one of the biggest sticking points for PageRank Sculptors. They keep trying to prove this idea works on live sites that are constantly changing.
7. You must not attract any new links to your site. Again, trying to prove you can sculpt PageRank on a living site just doesn’t work. You have no way of filtering out influences from a multitude of other factors.
8. You must keep your site in a static condition for at least three Toolbar PR updates.

If you’re going to use Toolbar PR data, you must capture at least three data points because the TBPR is randomly derived from one of an undisclosed number of internal PageRank valuations. Three data points is not much of a trend indicator but that’s the minimum number of points you can use to observe a trend.

Data sculpting is not PageRank sculpting. People make assumptions in their SEO tests. Rand Fishkin’s tests, for example, have been challenged by more than one person as being biased. He assumes links are the most important factor in SEO so he sets out to prove they are. That’s not a very reasonable or scientific approach to making a logical case for anything. You have to use a truly objective approach so that you’re not just presenting data that is favorable to your point of view.

So you want to test PageRank Sculpting for yourself
Yes, Virginia, you can test the concept of PageRank Sculpting. Let us know how you do. In order to be credible, you have to publish the details of your methodology and all criteria used for determining that value is or is not passing between links.

It would be great but not necessary if you actually published your data, too.

Here are a few things to avoid doing: creating sites for nonsense terms no one searches on. Not obtaining any value-passing links to the sites. Using queries that provide fewer than 1,000 search result listings.

Assuming you set up your test correctly, you can look for these signs that PageRank sculpting may be helping. I have indicated approximately how much value each signal should have in your assessment:

1. The number of search referrals increases. (HIGH)
2. The number of Long Tail search referrals increases. (HIGH)
3. The number of pages/visit remains stable. (HIGH)
4. Average time spent on site remains stable. (HIGH)
5. More pages appear for deep-text searches. (HIGH)
6. Your Sitelinks change according to your determination. (LOW)
7. The reported number of indexed pages increases. (LOW)
8. The total deep results for a site-specific query increases. (LOW)

I assigned HIGH value to signals that probably are dependent on PageRank-related activity in the search index OR that are INdependent of PageRank-related activity. I assigned LOW values to signals that are not really trustworthy because they are more dependent upon undisclosed algorithmic factors.

Well, I don’t expect the PageRank Hoarding/Sculpting/Consolidating community to break out into tears over what I have written. Being the dedicated choir that they are I am sure they’ll keep singing long after the church has finished burning down and the ashes are blown away by the wind.

But there may be a little bit of common ground that we can seek with them. You cannot sculpt your PageRank but you might be able to manage your internal navigation better. Remember, all this PageRank sculpting nonsense became popular because incompetent SEOs could not figure out how to keep their (or their clients’) “About Us” pages from ranking first in search results.

The herald of a new era in internal navigation management may in fact be none other than Rand Fishkin.

Yes, I said Rand Fishkin and I am NOT kidding. Rand stands on the brink of providing the SEO community with a truly useful tool. Of course, there are other link counters out there (like Majestic SEO). Ideally, any of these companies that crawl the Web and try to show you backlink profiles can move toward creating a truly useful resource.

First, let’s make it perfectly clear that LinkScape, Majestic SEO, and similar tools cannot:

• Tell you anything about Google’s PageRank … or Yahoo!’s WebRank … or Microsoft’s StaticRank.
• Tell you which pages Google has in its Main Web Index. Same for Yahoo! and Microsoft. Heck, let’s throw Ask in here, too.
• Tell you which pages are in Google’s Supplemental Index. Yadayadayada.

Okay, you may now stop gritting your teeth, you Link Bunnies. Here is what companies like SEOmoz and Majestic SEO CAN do for you (provided they have the wherewithal to make it happen):

• They can build a tool that lets you sculpt MozRank (or MajesticRank or SomeSEOCompany’sRankValueThingee).
• They can evolve their (insert RankValueThing here) into a usable metric for link quality. That is, you can say, “I want to only work with link that achieve this XRank regardless of what the search engines think of them.” Remember, links can create visibility and pass traffic to you.
• They can give you the means to play with link models. This would be equivalent to simulating a black hole on a supercomputer. It won’t be like the real thing, but it can vastly improve your “What If” analysis.

Keep in mind that using models doesn’t prove anything because models are biased to the designer/tester’s assumptions. Too many people mistake SEOmoz’s Ranking Factors Survey for some sort of guide to Google’s ranking algorithm. That Ranking Factors Survey is nothing more than an opinion poll and all it really tells us is that the SEO community cannot agree on much of anything about search engine algorithms.

Still, we’ve seen people pay a little more attention to the fundamental principles of search engine optimization lately. I have pretty much expected to see the PageRank Sculptors come slinking out of their burrows to make a new attempt at selling snake oil but to be honest, I have never denied that in theory it should be possible to do this — I’ve just always maintained that no one outside of Google is in a position to do it.

People will continue to share bogus test results, specious claims, and other useless rhetoric to shore up their faith and belief in the One True PageRank Methodology. That is no substitute for good search engine optimization, however.

And as any good search engine optimization specialist knows, it’s NOT all about links — not even close.