Captain Jean-Luc Picard 1 Posted August 2, 2003 Who is Googlebot Spider? They have no profile and a member seems to think it could be an AI. Anyone know about this? Share this post Link to post Share on other sites
Takara_Soong 4 Posted August 2, 2003 In the Upgrade thread in this forum VBG explained what a spider is as follows: The Spider is an automated "robot" of sorts that search engines send out to index the web. This Spider is going through the postings and making an index so that when people search their search engine for something, like the name Quark they can give that person different instances of Quark online. We may come up in that search because of "Quarks Bar". The Spiders have been indexing our site for about a week or so now. The Google Bot spider is the spider sent by the Google search engine. Share this post Link to post Share on other sites
Data 0 Posted August 2, 2003 Since VBG got the software upgrade, we are now seeing the spiders from the search engines. Share this post Link to post Share on other sites
VaBeachGuy 12 Posted August 2, 2003 Since VBG got the software upgrade, we are now seeing the spiders from the search engines. Actually we've had quite a bit of Spider activity from HotBot and now Google. Before the upgrade the Spiders came on and for the most part just wee viewing the main index (Portal page). Now with the upgrade the spiders are doing what's called a "Deep Crawl". The Google Bot has gone quite deep into topics and user profiles. Here's the Log of Spider activity since the upgrade, this shows the number of visits by each. I also have a page where I can see the specific page they visited but it's not possible to post that. Share this post Link to post Share on other sites
