Remember that there are many factors that contribute to how search engines work, and some of them are closely-guarded secrets. Google, the most popular search engine, keeps parts of their operation under wraps to prevent fraud and dishonesty.
Web Spiders and Bots
In the old days of the Internet, a website would often have to be submitted directly to a search engine in order for listing on that search engine. This was often the only way to gain a listing on a site like Yahoo! because directory listing sites relied on human editors to select the topics and contents of directory listings. Some search engines had rudimentary ways to search pages on the Internet. The problem is that search engines hadn’t yet developed the ability to filter results very well. By using a search engine, the user couldn’t be assured that he or she would even come upon pages related to the keywords typed into the search engine.
Today, however, no submission is necessary in order to be listed in search engines. Because technology has evolved, search engine listings are much more refined and certainly more user-friendly. Today’s search engines spend lots of time, money and research in order to provide a searching service that will give the user the best web experience possible. In fact, the search engine business model relies upon this fact. The more people that visit a search engine, the more web traffic that site will receive. The more web traffic, the more people will see advertising on that search engine. Since search engines frequently rely upon advertising to support the site, this works out for everyone. The user gets better search results and the search engine gets more users to their site.
Search engines are an important part of the growing online economy, mainly because they are becoming an important part of how people use the Internet. Just 10 short years ago, Internet users weren’t using search engines very often, but favored index sites like Yahoo!, and for good reason. Index sites generally had human editors that chose websites based on topic and posted them in a hierarchical way where there were main categories and sub categories that got more specific. For example, under the heading “Arts and Humanities,” were more subtopics like “Artists, Art History, Visual Arts” and more. When users clicked on any of these subtopics, they would see even more subtopics. The directory of sites by topic allowed the user to find the exact topics they wanted and verity, with the help of human editors, that those pages contained good information. The sites listed were usually reliably accurate in regards to what the user was seeking and still exist to this day.
On the other hand, search engine technology was far behind today’s standards ten years ago. By using older search engines, users would likely come up with search results unrelated to their keywords. Back then, the technology hadn’t evolved to accurately find and filter search engine results. By simply using some shady techniques, a webmaster could get his or her site displayed for totally unrelated keywords. For example, a site selling herbal supplements might be able to get their site to show up in search engines under search terms unrelated like “education” or “home mortgage.” The result was that search engines were fairly unreliable and not as many people used them. Today more than 85% of web users use search engines to find the pages they want.
By far, the most popular search engine is Google. There are several reasons for this. First of all, Google has invested a lot of money in developing algorithm technology that will give users the listings they want. Google has also developed a unique style, including text-only advertising. This makes for pages that load quickly and don’t contain offensive or obtrusive pictures. Because customers like this type of search experience, they’ve begun to overwhelmingly prefer Google as a search engine of choice.
There are a few search engines that are based on Google. These sites use the Google technology in order to provide their searches. Some examples of these are AOL Search and MySpace search. These search engines work in almost the same way as Google at retrieving information, even if they don’t look the same.
In the United States, Google takes up almost half of the search engine market. That means, for every search done on a search engine in the United States, nearly one in two is done with Google. Following Google is Yahoo! Search with about 24% of searches, and MSN with about 10%. AOL rounds out this list with about 6% of searches. Worldwide, Google takes an even bigger share of the market with up to 56% of worldwide searches.