Webmasters and content providers began optimizing websites for search engines in the mid-1990s, as the first search engines were cataloging the early
Web. Initially, all webmasters only needed to submit the address of a page, or
URL, to the various engines, which would send a
web crawler to crawl that page, extract links to other pages from it, and return information found on the page to be
indexed.
[5] The process involves a search engine spider downloading a page and storing it on the search engine's own server. A second program, known as an
indexer, extracts information about the page, such as the words it contains, where they are located, and any weight for specific words, as well as all links the page contains. All of this information is then placed into a scheduler for crawling at a later date.
Website owners recognized the value of a high
ranking and visibility in search engine results,
[6] creating an opportunity for both
white hat and
black hat SEO practitioners. According to industry analyst
Danny Sullivan, the phrase "search engine optimization" probably came into use in 1997. Sullivan credits Bruce Clay as one of the first people to popularize the term.
[7]Early versions of search
algorithms relied on webmaster-provided information such as the keyword
meta tag or index files in engines like
ALIWEB. Meta tags provide a guide to each page's content. Using metadata to index pages was found to be less than reliable, however, because the webmaster's choice of keywords in the meta tag could potentially be an inaccurate representation of the site's actual content. Flawed data in meta tags, such as those that were not accurate, complete, or falsely attributes, created the potential for pages to be mischaracterized in irrelevant searches.
[8][
dubious β
discuss] Web content providers also manipulated some attributes within the
HTML source of a page in an attempt to rank well in search engines.
[9]