Hypertext Matching Is Alive And Well

Google founders Larry Page and Sergey Brin married Citation Analysis to Textual Analysis in their foundational paper Anatomy Of A Large-Scale Hypertextual Search Engine — the so-called Backrub paper which introduced Google.

While most people through the years have focused on the PageRank algorithm the paper introduced it did talk at considerable length about the IR factors used to determine relevance in documents. In 2007 Danny Sullivan wrote:

PageRank gets its name from Google cofounder Larry Page. You can read the original ranking system to calculate PageRank here, if you want. Check out the original paper about how Google worked here, while you’re at it. But for dissecting how Google works today, these documents from 1998 and 2000 won’t help you much. Still, they’ve been pored over, analyzed and unfortunately sometimes spouted as the gospel of how Google operates now.

At the time Danny rightly pointed out that a lot of things have changed since 1998-2000; however Google still talks like the original Backrub project remains the core of its technology.

I don’t mean just PageRank — I mean the combination of Citation Analysis and Textual Analysis, or what Google (and other IR resources) calls “Hypertext-Matching Analysis”. As in the Backrub paper Google is talking about “also [analyzing] page content. However, instead of simply scanning for page-based text …, [Google's] technology analyzes the full content of a page and factors in fonts, subdivisions and the precise location of each word. We also analyze the content of neighboring web pages to ensure the results returned are the most relevant to a user’s query.”

Notice the last sentence, which discusses the analysis of content on neighbor pages. Of course, we can now point to many Google queries where trust is now a significant factor.

People are sure to guess that there is a connection between Google’s PageRank algorithm and its Trust filters. For example, we know from Google’s admissions that PageRank is the primary factor used to determine whether pages go into the Supplemental Results Index or the Main Web Index. One might reasonably ask how much trust should be conferred upon a page that only receives links from Supplemental Index pages.

Hypertextual Matching Analysis, however, has now evolved into looking not only at how each document uses words but also at how its neighbor pages use words.

There is really nothing new in all of this, except that we can say we’re living in the age of PageRank 2.0 (where not all links are permitted to pass PageRank), Trust 2.0 (where trusted sites are actually favored in at least some search results), and Hypertext Matching 2.0 (where a correlation between content on linking and destination pages is desired).

Let’s assume for the sake of discussion that we can obtain all the PageRank-passing, Trusted links we desire. Is that sufficient to manipulate the rankings in search results? Or do we now need to devote significant attention to the content in which links are embedded?