SERPology – The Psychology of Search Optimization

by Michael Martinez on October 16, 2007

A working definition for psychology is “the study of the behavior of animals”. Some people distinguish between animal and human psychology but what I’m really interested in is “the study of the behavior of search results”.

A search result is neither an animal nor a human. It has no mind of its own but does it behave. But let’s back up here a bit. Do you know what I mean by “search result”?

That’s a concept, not a page. A SERP is a search engine results page as you all well know but I’m not talking about the search results pages we monitor. I’m talking about the search results and how they behave.

A search result is an aspect of a query space, sort of like a property of an object. For example, let’s say you have a table. A property of the table would be its color, or its height, or its shape. In objected oriented programming (which, believe it or not, greatly impacts Web page design) every object has attributes, including methods. A method tells an object to peform some action, such as to tell you its name, or to close itself, or to launch a copy of itself, to change its color, etc. Web pages can be treated as objects (hence, we work with the document object model in Javascript) and we can modify the attributes of pages through the DOM command structure.

A search result is very much like an attribute of the query space object. It’s not quite a method, but there are methods associated with the search result portion of the query space. I realize this is abstract but sometimes it helps to think in abstractions before you clear the board and get to work.

The search result is where your optimized site shows up in a query space. You ask a search engine to show you a search result when you’re optimizing. You’re not really looking for information other than what the search engine tells you. The search result has several properties associated with it, such as:

  1. The query you use
  2. The position of your highest ranking page
  3. The URL of your highest ranking page
  4. The relevance of your search listing to your page
  5. The search engine you run the query on

Search listing relevance is not something we normally discuss in SEO. After all, it’s a listing for a Web page, so how can it not be relevant to the page, right? But what you want to look for is how relevant the search listing is to the page’s current state. Is the listing based on an older version of the page? This is not about freshness so much as it’s about lag times.

How long does it take for the search engine to update its listing for a given page? How long does it take for your optimizaton to change the search result? Given two pages on a site, A and B, which page do you want to appear first in the search results? How long does it take for the search engine to move page A above page B or vice versa if you make changes with that goal in mind?

The search result won’t behave predictably if you don’t look at lag times. You have to document them wherever and whenever possible to get a feel for how quickly you can make changes in your search result. The search index is like a fast-moving river with huge torrential rains of data flowing into it every day. The river moves quickly but you can watch your Web site cling to the shoreline and fight the current. Your Web page may get stuck in an eddy or backwater and then you have to give it a kickstart.

There are sites that can bring new content into the index quickly and sites that have to wait months to get the new content to show up. You have to know which kind of site you’re optimizing. You have to clock the lag times in the search result.

The search result may also not respond to your optimization the way you feel it should. You can even point a lot of links to one page and have a second page show up instead. Why? You need to figure that out. Maybe your links are ineffective. Maybe your target page is linking to the other page and making that other page seem more important. Maybe other people are linking to the other page. I’ve seen search engines get very, very confused when two very similar pages both have a lot of inbound links. Shame on the search engines for relying so much on linkage, but shame on me for not taking that into consideration.

The search result never lies. It’s the most honest, least egotistical aspect of the query space. The search result tells you exactly what is there and nothing else. And it has lots of brothers and sisters. You have search results for every query on every search engine. The search result is not your friend or ally. It’s the game you’re hunting. It’s the lion you need to kill. It’s the bear you stalk. It’s the mountain you climb.

It’s you against the search result. There is no one else involved. And it’s always one-on-one but never mano-a-mano. That is, no matter how smart you are, no matter how crafty you become, no matter how great and skilled in the arts of search engine optimization you may be, the search result is always completely and totally stupid, inflexible, and about as energetic as a boulder blocking your path.

You choose the search result that you optimize for. No one else can choose it for you. You pick the query (keywords), you do the work, you pick the search engine you monitor. Most people may be concerned only with Google but if you’re not monitoring your search result on other services you’re making the task harder on yourself.

Comparing search results to each other tells you things about your Web page that you don’t know to tell yourself. If you focus on building link anchor text you may never realize that your page is highly optimized for something you didn’t intend. So you don’t just want to check queries that you’re targeting, you want to check queries that you are NOT targeting. You don’t just want to check Google, you want to check Live, Yahoo!, and Ask.

Because even though it’s just you against your search result, you’re really in a very competitive game. Other people are chasing their own search results. Some people chase a lot of search results. Every query space has a lot of search results.

I’ve pointed out in the past that search results change for four reasons:

  1. You do something
  2. Someone else does something
  3. The search engine does something
  4. People search for something else

Your search result dies when people stop using that query. So your goal is not necessarily to kill the search result, but rather to tame it and make it do what you want. On the other hand, if you cannot compete for a particular keyword, killing a search result may be better. Getting out of a too-competitive query space may free up resources that can provide a better return on your investment elsewhere.

A search result has only a few behaviors:

  1. It changes often, weaving and bobbing
  2. It never changes
  3. It changes only occasionally
  4. It changes on a regular though not frequent basis

If you do nothing at all, you’re not going to change the search result. You have to do something to the search result. It’s an object. You can pick it up, throw it away, or turn it over. You can put it down, walk away from it, or give it to someone else.

The search result is only a thing that can be altered. But each search result is different and you have to learn what it takes to alter the search result. Think of the search result as a ball of energy you hold in your hand. It’s radiating green and you want it to radiate blue. You have to change something to make it radiatie blue.

Neither you nor I know how to change the color of a ball of energy, but if we could hold one in our hands long enough we would eventually figure out at least some of its properties. Do this, it does that. Make an action, it reacts.

You don’t have to understand the principle of gravity to know that a ball rolls down hill. All you have to do is stand at the top of the hill and drop the ball. You’ll see it roll down the hillside. Knowing the ball will roll down hill tells you something about both the ball and gravity, but you don’t need to comprehend gravity if you just want to roll balls down hillsides.

You don’t have to know why a search engine does what it does as long as you can document a cause-and-effect behavior between search engines and Web documents. Looking at the search result without worrying about the query space that the search result is part of helps you understand what your own abilities are. You can learn how to tweak a Web site quickly by playing with a search result.

You can define your own methods, your own search result object model. “If I do this, the search result will do that”. Cataloguing the actions you can take to affect a search result empowers you.

Search optimization is really about documening consistent behaviors and choosing which behaviors you want to see. You document causes and effects and you initiate the causes to produce the effects you’re looking for. The more you study search results, especially search results that don’t concern you, the better able you are to produce effects.

Today you’re hunting kittens. Tomorrow you’ll be hunting whole prides of lions.

That’s SERPology, the psychology of search optimization.

{ 0 comments… add one now }