• Register
• Log in

SEO Theory - SEO Theory and Analysis Blog

Algorithm analysis, Web community relationship analysis, SEO practices and techniques, industry news, etc.

• Home
• About SEO Theory
• SEO Glossary

The Theory of Search Engine Optimization

Posted by Michael Martinez on January 16, 2008 in SEO Theory

Search engines organize the data they collect according to indexing criteria and ranking criteria. Indexing criteria determine what information the search engine will store and how that information will be handled. Examples of indexing criteria include: indexing only pages that pass one or more quality tests, omitting portions of page content, etc. Examples of ranking criteria include: giving extra weight to use of query terms in titles, page URLs, and bolded text; giving credit for use of query terms in link anchor text pointing to the page; etc.

A search engine influences the results it serves to users by adjusting either the indexing criteria or the ranking criteria or both. Other factors can impact a search engine’s results, however. For example, a search engine that distributes its database across multiple data centers may not always have syncronous results across its network. And a search engine’s load balancing algorithms may cause some query resolution tasks to terminate prematurely before evaluating all available relevant results during periods of high traffic.

From the search engine’s point of view, the content of search results are determined by infrastructure, indexing criteria, and ranking criteria. The search engine’s goal (ideally) is to serve the most relevant and satisfying results to its users possible. Results cannot simply be relevant, they must be satisfying so that the users feel confident in using the search engine’s service. Of course, satisfying results need to be relevant, but they need not be the most relevant nor even the most authoritative or exhaustive results available.

A search engine therefore measures the quality of search results according to user satisfaction rather than according to objective criteria. In The Wikipedia Principle - How We Devalue Web Content I showed that “a search engine intentionally promotes low quality content that is minimally acceptable to searchers because it costs less to do that than to promote better content.”

Given a choice between search engine David and search engine Edward, a user will choose one over the other either based on the quality of search results (from the user’s perspective) or on the speed of the delivery of results if the discernible quality of results is approximately equal. That is, the less expert a user is in a topic, the less likely the user is to be discriminating about the quality of the content a search engine provides. Hence, query resolution time will be more important than the details of the content if both search engines provide approximately equivalent value in their results.

If David provides the faster query resolution experience, more users will tend to use David than Edward. However, if Edward significantly improves the discernible quality of its resuls over David’s results, users will tend to use Edward more than David despite the difference in query resolution times. Users want to spend as little time searching as possible, but in practice most people experience search fatigue (or search engine fatigue) at some level or another.

You get tired of looking for information the search engines cannot provide you. Hence, if search engine Edward takes longer to provide you the results you’re looking for but search engine David provides less value then you’ll want to use search engine Edward rather than David.

Which means that the user is empowered to optimize their search experience, either by altering queries or by changing search engines (and users actually do both).

As professional search optimizers we tend to look at search engine optimization in a very insular fashion. SEO Theory’s SEO Glossary, for example, defines search engine optimization as “The art of designing or modifying Web pages to rank well in search engines”. This is the traditional definition but it is not the correct technical definition.

In fact, search engine optimization occurs in three ways: through changes made by the search engine, through changes made by the searcher, and through changes made by the content providers. Therefore, to speak of search engine optimization we have to look at everything that everyone does. And that brings us back to the Theorem of Four SEO Influences. There are four reasons why your search results change:

1. You do something with your site
2. Someone else does something with their site
3. The search engines do something with their data
4. People search for something different

The last reason should be restated as, “People change how they search” — which implies that they may change what they search for or they may change where they search or they may change when they search. The searchscape is not a fixed asset. It evolves on an hourly basis if not more frequently. The search engines you had available to you this morning differ from the search engines you have available to you tonight.

Search engine optimization is thus more aptly described as “the art of utilizing a search engine to produce a desired listing of resources”. The search engines desire to produce the most satisfying listings, the users desire to produce the most relevant and informative listings, and the Web content providers desire to produce the most advantageous listings.

Hence, anyone can optimize for search and everyone does but most people do it subconsciously without awareness or understanding of the principles involved. Furthermore, the optimization strategies of the three estates of search (Engines, Users, and Content Providers) clash, causing conflict and confusion. The most visible conflict arises between the search engines’ strategies and Web content providers’ strategies, but I think any search engineer would be quick to point out that users (searchers) create some pretty surprising queries.

Search engine optimization thus embraces multiple objectives but it nonetheless achieves the same result for all practitioners: search engine optimization influences search results to be more satisfying to the optimizer’s experience.

Furthermore, search engine optimization can only be achieved algorithmically. That is, we set out to solve a problem: how to serve satisfying results, to how to find what we feel are reliably informative results, how to make our content appear in desired search results. We develop methods to solve that problem: for search engnies, there are infrastructure methods, indexing methods, and ranking methods; for searchers there are search tool selection methods and query methods; for Web content providers there are content design methods and link development methods.

Search engine optimization is therefore the use or application of algorithms to influence the predictable content and quality of search engine results according to the chosen criteria of the optimizer. And that is The Theory of Search Engine Optimization. It’s really not rocket science, as some people have said, but it really is more complex than most people tend to believe.