It is the SQL of WWW. Or in other words, it is the web tool that helps the users in locating the required information on the WWW. Various top search engines are like Google, MSN, and Yahoo. And the information that is gathered by the spider is used for creating the searchable index for the content on the web. This spider crawls according to the search engine algorithm and that is known as web crawling.
