haidong 发表于 2011-12-30 10:25:14

搜索引擎的spider蜘蛛总汇


1.Sosospider+(+http://help.soso.com/webspider.htm)腾讯搜搜
2.Mozilla/5.0 (compatible; YodaoBot/1.0; http://www.yodao.com/help/webmaster/spider/; )有道搜索1
3.Sogou web spider/4.0(+http://www.sogou.com/docs/help/webmasters.htm#07)搜狗搜索1
4.Baiduspider+(+http://www.baidu.com/search/spider.htm)百度搜索
5.Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; QihooBot 1.0 qihoobot@qihoo.net)奇虎搜索1
6.Mozilla/5.0 (Twiceler-0.9 http://www.cuill.com/twiceler/robot.html)cuil搜索1
7.Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)谷歌搜索
8.Mozilla/5.0 (compatible;YodaoBot-Image/1.0;http://www.yodao.com/help/webmaster/spider/;)有道搜索2
9.Mozilla/5.0 (compatible; YoudaoBot/1.0; http://www.youdao.com/help/webmaster/spider/; )新有道搜索
10.Sogou web spider/3.0(+http://www.sogou.com/docs/help/webmasters.htm#07)搜狗搜索2
11.Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1)
12.Mediapartners-Google
13.Mozilla/5.0 (compatible;YodaoBot-Image/1.0;http://www.youdao.com/help/webmaster/spider/;)有道搜索3
14.Mozilla/4.0 (compatible; MSIE 5.5; Windows NT 5.0)
15.Mozilla/5.0 (compatible; Yahoo! Slurp China; http://misc.yahoo.com.cn/help.html)yahoo搜索
16.Mozilla/5.0 (Twiceler-0.9 http://www.cuil.com/twiceler/robot.html)cuil搜索2
17.Mozilla/5.0 (compatible; ia_archiver +http://www.greaterera.com/)
18.Yeti/1.0 (+http://help.naver.com/robots/)naver搜索
19.Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1)
20.Mozilla/5.0 (compatible; MJ12bot/v1.2.1; http://www.majestic12.co.uk/bot.php?+)majestic12
21.psbot/0.1 (+http://www.picsearch.com/bot.html)picsearch
22.Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 1.1.4322)
23.Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; QQDownload 1.7)
24.Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; CollapsarWEB qihoobot@qihoo.net)奇虎搜索2
25.Mozilla/3.0 (compatible; Indy Library)
26.sogou in spider(+http://www.sogou.com/docs/help/webmasters.htm#07)搜狗搜索3
27.Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 2.0.50727)
28.Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 1.1.4322; .NET CLR 2.0.50727)
29.Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)
30.Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.16) Gecko/20080702 Firefox/2.0.0.16
31.Mozilla/5.0 (compatible; MJ12bot/v1.2.3; http://www.majestic12.co.uk/bot.php?+)majestic12
32.Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp)yahoo搜索
33.Sogou head spider/3.0(+http://www.sogou.com/docs/help/webmasters.htm#07)搜狗搜索4




http://bbs.hh010.com/xwb/images/bgimg/icon_logo.png 该贴已经同步到 haidong的微博

微博评论 发表于 2011-12-30 11:05:41

不明智、不健康、不正直地生活是不可能活得愉快的；同样，活得不愉快也就不可能活得明智、健康和正直 [啊]2-00-_00个_8_-8块 http://t.cn/SxQv2y [强吻]

http://bbs.hh010.com/xwb/images/bgimg/icon_logo.png 来自 鱼岩9 的新浪微博

云中之舟 发表于 2012-1-7 15:58:23

{:6_290:}
页: [1]
查看完整版本: 搜索引擎的spider蜘蛛总汇