• Register
• Log in

SEO Theory - SEO Theory and Analysis Blog

Algorithm analysis, Web community relationship analysis, SEO practices and techniques, industry news, etc.

• Home
• About SEO Theory
• SEO Glossary

The Searchable Web - Ecosystem and Mechanism

Posted by Michael Martinez on February 7, 2008 in Advanced SEO

The Searchable Web is a three-point ecosystem. There are Web Publishers, Web Indexers, and Web Searchers bound together in an inextricable symbiotic relationship with each other.

You cannot have content without indexing, you cannot have searching without indexing, you cannot index without content, and there is no point in creating content or indexing it if no one is interested in the content.

These symbiotic groups don’t feed on each other. Instead, they are all cannibalistic: they only feed on their own kind. Searchers help other searchers find content. Indexers absorb and integrate other indexes. Publishers republish content, absorb other publishers, or copy other publishers’ ideas. That is, we grow as searchers, indexers, and publishers by interacting with other people doing similar things.

Within each group there are specialists: Web Publishers include content creators, content promoters, and content killers. Indexers include content reviewers, index managers, and index killers. Searchers include content finders, content consumers, and content dismissers.

Each group ultimately has two powers: a positive, nurturing power to create, make visible, or share content and a negative, destructive power to destroy, hide, or denigrate content.

Of course, Matt Cutts falls into all three groups but he is only in a single state when he writes on his blog, he is only in a single state when he helps manage or improve Google’s index, and he is only in a single state when he searches the Web for neat stuff. We are all like Matt Cutts. In this Searchable Web Ecosystem model, we are all Matt Cutts.

If I create a Web site, I am a publisher. If I create a site search tool — even using nothing more than an embeddable search box provided by Ask, Google, Live, or Yahoo! — I am an indexer. If I use those search engines — or my search tool — to look for my content or someone else’s content, I am a searcher. I AM the Searchable Web Ecosystem. So are you. We all are.

Ecosystems follow natural laws. They seek to be self-sustaining. An ecosystem that cannot perpetuate itself devolves into chaos and ceases to exist. Ecosystems evolve to conform to the restraints of their environments. That is why what is now India once was the bottom of an ocean and what is now the Sahara Desert once was a lush and fertile savannah. Poland has a small desert region that was created by deforestation in the Middle Ages, and there are creatures that have — in only a few hundred years — learn to thrive in that desert region.

The Searchable Web follows similar natural laws. As the environment of the Web changes, publishers, indexers, and searchers all evolve. The Web environment is defined by the technology that creates it: software and hardware that allow us to share thoughts and ideas across thousands of miles without ever seeing each other is constantly being updated and replaced. Each day, in subtle, small evolutionary steps, our Web environment gradually changes.

A year from now we’ll be creating content we never conceived of before. We’ll be indexing content in ways we never thought to. We’ll be looking for content we never knew existed or could exist.

As Ecosystems change — especially as their environments change — some members of the Ecosystems fall behind the rest and “die off”. We’ve had mass extinctions even on the Web. Remebmer the dot-com meltdown of 2000-2001? A lot of Web sites that once existed are now gone. A lot of search engines that once existed are now gone. Even many Web searchers have long since left the Web, some because they lost acces, some because they passed away, and some because they got tired of it.

And as Ecosystems grow they develop new members, sub-species of species, isolated communities that break away from the rest and form new micro-Ecosystems, etc. We have evolved as content publishers, indexers, and searchers. Today people help each other search in ways we never imagined ten years ago through user-generated content sites, social bookmarking sites, and other user-oriented tools.

When you look at the Searchable Web as a living Ecosystem it becomes less predictable, more chaotic, more unfocused through the narrow lense of search engine optimization. The Searchable Web is larger than any conjecture about why your search results just tanked or zoomed. The Searchable Web is only as understandable and as managable as the theory behind your principles.

For example, you can describe the Searchable Web as being a mechanism comprised of four components:

1. The content you publish
2. The content other people publish
3. The search engines that index content
4. The searches people perform to find content

While that may sound familiar, it’s a slightly different way of looking at the only universal explanation for why your search traffic changes. When you view the Searchable Web as a mechanism with four primary components, you can look at how the mechanism operates by identifying the channels and connection points between the four components.

Your content may have more connections to searches than to indexes (you’ve optimized for queries but cannot get indexed).

Your content may have more connections to other people’s content than to indexes (you’ve built linking relationships but you cannot get indexed).

Your content may have more connections to indexes than to other people’s content (you’re indexed but no one links to you).

You can start to look at the mechanism in this way to determine if there are broken connections (or channels), inefficient connections or channels, or superslick, high-performance connections. The Searchable Web is a huge mechanism and some parts of it can break down quietly amid all the noise of activity. Your content may be in one of those parts that breaks down.

Breakdowns inside the mechanism don’t usually last long but their durations can seem interminable to people whose content has become lost in the broken gears and cables.

Whether you think of the Searchable Web as an Ecosystem or a mechanism, there is a ceaseless movement toward establishing natural balance. The more you understand your role in the larger system, the less vulnerable you feel to unexpected changes in the environment around you. Those changes affect everyone around you, and they even affect people thousands of miles away.

We are constantly working to heal or repair the damage that the Ecosystem mechanism has suffered through loss and change. We are constantly working to adapt to the new environment, to the changing Ecosystem mechanism, and to the changes we experience in our content, in our indexing, and in our searching.

Some people might call this outlook the basis of Holistic Search Engine Optimization, in which your goal should be to find the right place in which to integrate new content and the right ways to inspire new searches. Relevance can be created through new initiatives, but most SEOs spend relatively little time creating relevance.

We create relevant content but Relevance consists of three components: content, indexing, and queries. That is, if you create content about Snarple Bargers and get it indexed, you have no relevance until people start searching for Snarple Bargers. Once the content, indexing, and queries come together Relevance has been established. From that point onward, the Ecosystem mechanism strives to improve relevance until the relevance is no longer useful.