Webmasters today spend some time optimizing their websites for search engines. Books were written on search engines and a kind of industry has developed to offer Search Engine Optimization (SEO) services to potential customers. But how did the SEO world in which we live today start?
A student at McGill University named Alan Emtage developed the first search engine for the Internet in 1990. The search engine called “Archie” was designed to archive documents available on the Internet at that time. A year later, Gopher, another search engine to Archie, was developed at the University of Minnesota. Both search engines just triggered the birth of what we use today as search engines.
In 1993, Matthew Gray developed the very first search engine robot – the World Wide Web Wanderer. However, it was not until 1994 that search engines as we know them today were born. Lycos, Yahoo! and Galaxy. Two of them, Lycos and Yahoo! are still around today.
In 1994, some companies have begun experimenting with the concept of SEO. The focus was solely on the submission process at that time. Within 12 months, the first automatic submission software packages were released. Of course, it did not take long until the concept of spamming search engines was “invented”. Some webmasters quickly realized they could manipulate and swamp the search result pages by over submission of their sites. But – search engines acted fast against this preventing it from occurring.
Soon, the SEO and search engines began to play a kind of “cat and mouse” game. The moment SE optimizers found a way to manipulate the search engines, they took advantage of this situation. The search engines subsequently revised and improved their ranking algorithms to respond to these strategies. Very soon, it became clear that a small group of webmasters was abusing the search engine algorithms to gain competitive advantage. Black Hat Search Engine Optimization is born. The manner contrary to the ethics of manipulating search engine resulted in faster responses from search engines. Search engines try to keep the search results clean of SPAM to provide the best customer service.
The search engine industry quickly realized that SEO as an industry would not be eliminated and to maintain useful indices, they should at least accept the industry. Search engines now partially work with the SEO industry but are always keen to sort out the spammers who try to manipulate the results.
When Google.com started to be the search engine of choice of Internet users, it was very visible to anyone in the industry that the search engine spamming had reached a new dimension. Google.com is much more important for the success of a website .Many webmasters solely concentrated on optimizing their sites for Google only because its payment was worth the effort. Again – Black Hat SEO took place, pushing down the honest webmaster and their sites in the search results provided. Google started fighting back. Several major updates to Google’s algorithms forced all webmasters to adapt to new strategies. Black Hat SE-optimizers suddenly saw something different happens. Instead of being just pushed into the search results their websites were suddenly completely removed from the search index.
And then there was something called “Google Sandbox” to show up in the discussions. Websites either disappeared into the sandbox or new websites never made it the index and thus were put into the Google Sandbox. The sandbox seems to be the place where Google ‘park’ websites either considered SPAMMY or does not conform to the policies of Google (duplicate websites under different domain names, etc.). The Google Sandbox so far has not been confirmed or denied by Google, and many webmasters consider it as a myth.
In late 2004 Google announced to have 8 billion pages / sites in the search index. The difference between Google and two competitors, MSN and Yahoo! seems to increase. However – in 2005 MSN and Yahoo! started fighting back to life in the search engine war. MSN and Yahoo seems to be gaining grounds in delivering better and cleaner results compared to Google. In July 2005, Yahoo! announced having more than 20 billion pages / sites in the search index – leaving Google far behind. No search engine has won the war yet. The three major search engines however are eagerly fighting for market share and a mistake could change the fortunes of a search engine.