Chasing alligators and other SEO secretions

Shari Thurow has divided search engine optimization into two camps: Short vs. Long-term SEO. Wonderful topic. Couldn’t say it better myself.

In fact, I was planning to say something altogether different anyway. But Shari’s comment about algorithm chasers caught my eye. I understand what she’s saying and I agree with her. There are indeed many people who want to get their brand new sites into the SERPs today and then want to respond to every algorithm change. I call that kind of SEO chasing the alligator because it’s dangerous.

As a long-time algorithm chaser, I know what it’s like to see rankings bounce up and down. I hate the feeling of having to depend on the whims of some search engine programmer (who is probably only half my age and has yet to learn 1/5 of what I have forgotten about programming).

Inktomi made me do it. Actually, Altavista made me desperate for good rankings but one quick lesson in doorway design fixed my Altavista problems (until Black Monday occurred in October 1999 — now that was a painful update, but most of you wouldn’t remember it anyway). I never wanted to use doorway pages but the basic doorway design taught me a great deal about understanding how search engines look at HTML code. That lesson is as valid today as it was in 1999.

Still, it’s a huge step to go from upscaling doorway page templates so they embrace actual content (we called them “Content Rich Doorways” back then — now you call them “Landing pages”) to reverse engineering a search engine algorithm. I had to do that with Inktomi because Inktomi at one time drove more search traffic than anyone else, including Altavista. In 1999, Inktomi was a greater force in search than Google is today.

Inktomi was looking at several factors. It divided pages between two indexes (placing most of them in the Secondary Index, from which no results were served). The number of links pointing to a domain (the link popularity appeared to be passed to child pages on domains) mattered more than with Google (Google is more concerned with “quality”). Some characters could kill a title tag in Inktomi faster than 10-year-old boys learn to spit. That’s why you see so many bars (|) in title tags today — I still use them occasionally, even though they are no longer necessary or helpful. And, oh yes, somewhere in there Inktomi looked at on-page content too.

The problems with ranking in Inktomi were legion. There was no front door so you couldn’t just submit a URL to them. Oh, their “partner” sites often accepted submissions, but some partners were better than others about reporting the submissions. And it seemed like as soon as people announced one good submission page, it vanished. But the greatest challenge with Inktomi was that if you managed to sneak into the main index you didn’t stay there for long. People kept complaining about seeing Slurp grab their pages without ever seeing any results. I would get a page into Inktomi for a month and watch it drop out.

As people explained link popularity to me, I began to experiment. Doorway pages didn’t work very well with Inktomi, but for the pages I had which stayed in the main index, title tags were killer. The problem was: how do you get a page to stay in the main index? One day it struck me: a good, old-fashioned spam technique might actually work with Inktomi. (Technically, there was a lot of spam in Inktomi — they were trying vainly to get rid of it, but at the expense of keeping legitimate content out).

So in the old days spammers who created thousands of doorway pages would create “hallway” pages. A hallway just links to a lot of doorways. Well, the search engineers weren’t stupid. They read the forums. One forum even regularly reported spammers to the search engines. So the old “one-hallway-to-100-doorways” trick stopped working at some point.

I thought, “Why not create a content-rich hallway that only links to a few doorways?” Then I thought, “But I need several of them.” Okay, I created several hallway pages. I placed them on multiple domains. And now here is the real sneaky part: I linked to the hallway pages from several pages that were already in the main index. It worked. Inktomi crawled the hallway pages and indexed the pages I wanted crawled in the main index.

I actually borrowed that idea from people who would regularly run site searches in one search engine for reciprocal links in another. That is, back then, in 1999, people understood that if a crawling search engine found your page through a crawl, it seemed to treat your page better than if you had submitted the URL. Problem was, all the search engines had small indexes. They weren’t very good at crawling the Web and their databases weren’t very large.

So people would find inbound links on, say, Infoseek and then submit those pages to Altavista and vice versa. There was nothing wrong with doing that, but the submission process back then was tedious. Some of us even used submission software (that was actually useful) but I got tired of submitting one page at a time. Each time my wizard updated its database the length of submission time per page increased.

So I thought, “Why not create a crawl page for all those pages linking in to me?” Now, I wasn’t the first person to think of that idea. When I floated it in a forum, someone mentioned they were already doing it and even linked to an older thread where the idea had been discussed. So my idea was vindicated. All I did for Inktomi, later on, was switch the type of links from other people’s pages to my own.

What I was doing was putting site maps for my domains on other domains. Sounds very simple now, doesn’t it? It didn’t take Brett Tabke long to think of creating what became the first link farm after I announced I had gotten dozens of pages included in Inktomi’s main index and ranking with my crawl pages (note: there were other links to those pages that, when indexed by Inktomi, also helped). This all came about at a time when people were grumbling over the expense of paid submission (basically, Inktomi said to business site operators: “We’ll crawl and index all your cheap spammy pages but you have to pay us every month”).

If you have hated the fact that Google has been so easily manipulated by spammers through the years, you can thank Inktomi for your frustration. Inktomi made spammers jump through hoops that Google is only just now learning to put up. Which is not to say that Google hasn’t served good results through the years. Sometimes Google’s results have been excellent. Sometimes they suck. You can usually gauge how close the next real algorithm change at Google is by how much garbage you see in the search results.

It’s an unfortunate truth that search engines are algorithm chasers. As fast as the spammers develop a new algorithm, the search engines have to reverse engineer it and set up filters or other responses to it. Sorry Shari, but that sword cuts both ways. As long as someone can derive value from manipulating search engine results, the search engines will continue to be on the defensive.

Now, there is a point to the history lesson. You see, you cannot engage in effective long-term SEO unless you have some basic understanding of what the search engines do. That is, at some level, you have to rely on algorithmic behavior. It’s inescapable because algorithms make it possible for people to navigate from any one site to any other site through random links.

But where can you stop chasing the alligator? Where should you stop chasing the alligator? That’s the $1 million question and I won’t pretend to have the definitive answer, although I have a theory. I think search engine optimization is only one aspect of a robust marketing campaign. There is something wrong with a business plan that relies solely upon search engine optimization to build traffic and visibility.

Sure, you can do it solely through SEO, but why? What an agonizingly inefficient approach to Web marketing SEO truly is. I mean, you’re wasting your money if SEO is your only option for building an online empire. I have never relied solely upon SEO. I’ve bought advertising in magazines, on radio, run PPC campaigns, joined Webrings, banner exchanges, distributed press releases, been written up by major media (including TV Guide, Rolling Stone Australia, and Wired), done radio and television interviews — in short, I’ve engaged in a full, robust marketing effort.

Search engine optimization should be part of the solution, not the whole solution. Which is not to say you cannot or should not deal with specialists. You can hire a linking specialist for all I care. But if you’re struggling to get your new domain ranking for anything after a year, telling yourself, “Well, it’s the sandbox effect”, you’re doing everything wrong.

Regardless of how long it takes for your Web site to rank in any search engine, you should be driving traffic to it from every means possible. Your “Contact us” form should work just as well for people who typed in your URL after seeing it in a Superbowl commercial as for people who found it in organic search.

If you’re following all the hot SEO blogs looking for the latest list of “cheap, easy free links”, get out of the SEO business. You’ll make more money with a simple PPC campaign. People who really have good free link sources ruin them as soon as they post a “Top Ten Great Linking Sources” article or forum thread. I wouldn’t take advice from someone who does that if they were the last self-proclaimed SEO “expert” on the Web.

Sure, I used to do it too. But I’ve learned from history. It’s stupid to tell people where to get powerful free links on the Web. They don’t stay powerful for very long. Great directories don’t stay great once they are declared to be “SEO friendly”. By the time SEO Idioto Numero Uno is sharing his secret sauce on the Web, you had better be doing something else.

And that’s what so frustrating about learning search engine optimization. The people who really know what works won’t tell you what to do. People who sell their consulting services throw bones. They release teaser information that isn’t nearly as good whatever they actually know (although some consultants are better at consulting than actually doing, in my opinion).

When people ask me how to boost their search engine rankings, I ask them what they are doing to drive traffic to their sites. That question confuses more people than actually get it. After all, they think SEO is what you do to build traffic. Actually, search engine optimization is more useful for branding than for building traffic. Ask any PPC-only advertiser if they’ll ever go back to organic SEO. See what they say.

But should we give up on search engine optimization? I don’t think so. I think it plays its part in a robust marketing strategy. The search engines aren’t going away. But organic search results may one day go the way of static directory listings. Show of hands: how many of you are still getting most of your traffic from your Yahoo! directory listings? Anyone?

Things change, but you have an online business to run. Can you really afford to wait up to a year to get traffic to your Web site? If you don’t know how to get past the sandbox effect, are you just going to sit on your hands or are you going to go get some traffic?

That’s the real secretion to SEO: when whatever you’re doing doesn’t work, try something else. It just takes a long time to get the message across to a lot of people. It all comes back to experiment, evaluate, and adjust. Those three principles make alligators look more like algorithms.