admin/SecLists
1
0
Fork 0
mirror of https://github.com/danielmiessler/SecLists.git synced 2025-06-10 02:04:58 +00:00
Code Issues Packages Projects Releases Wiki Activity
SecLists/Fuzzing/User-Agents/software-name/googlebot.txt

277 lines
29 KiB
Plaintext
Raw Blame History

This file contains invisible Unicode characters

This file contains invisible Unicode characters that are indistinguishable to humans but may be processed differently by a computer. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2272.96 Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
Mozilla/5.0 (compatible; Googlebot/2.1; startmebot/1.0; +https://start.me/bot)
Mozilla/5.0 (iPhone; CPU iPhone OS 8_3 like Mac OS X) AppleWebKit/600.1.4 (KHTML, like Gecko) Version/8.0 Mobile/12F70 Safari/600.1.4 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
Googlebot/2.1 (+http://www.googlebot.com/bot.html)
Googlebot/2.1 (+http://www.google.com/bot.html)
Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko; Google Page Speed Insights) Chrome/27.0.1453 Safari/537.36 GoogleBot/2.1
Mozilla/5.0 (iPhone; CPU iPhone OS 6_0_1 like Mac OS X) AppleWebKit/537.36 (KHTML, like Gecko; Google Page Speed Insights) Version/6.0 Mobile/10A525 Safari/8536.25 GoogleBot/2.1
Mozilla/5.0 (compatible; Googlebot/2.1; http://www.google.com/bot.html)
Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; Google Web Preview Analytics) Chrome/27.0.1453 Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
Mozilla/5.0 (compatible;acapbot/0.1;treat like Googlebot)
Mozilla/5.0 (compatible; Googlebot/2.1; +http://import.io)
Googlebot
Googlebot-Image/1.0
Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; Googlebot/2.1; +http://www.google.com/bot.html) Safari/537.36
Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.4 (KHTML, like Gecko; Google Page Speed Insights) Chrome/22.0.1229 Safari/537.4 GoogleBot/2.1
Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; Google Web Preview Analytics) Chrome/41.0.2272.118 Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
OnPageBot (compatible; Googlebot 2.1; +https://bot.onpage.org/)
Google Crawler: Googlebot/2.1 (+http://www.google.com/bot.html)
Googlebot/2.1; +http://www.google.com/bot.html)
Mozilla/5.0 (compatible; Googlebot/2.1 +http://www.googlebot.com/bot.html)
Mozilla/5.0 (compatible;acapbot/0.1.;treat like Googlebot)
Mozilla/5.0 (iPhone; CPU iPhone OS 6_0 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Version/6.0 Mobile/10A5376e Safari/8536.25 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
Mozilla/5.0 (iPhone; U; CPU iPhone 0S 3.0 like Mac 0S X; en-us; compatible; Googlebot/2.1; http://www.google.com/bot.html; AppleWebKit/528.18(KHTML,like Gecko) Version/4.0 Mobile/7A341 Safari/528.16 UNTRUSTED/1.0
Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html
Apple-iPhone4C1/1001.523 (compatible;acapbot/0.1;treat like Googlebot)
Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)/Nutch-2.3.1
Googlebot-News
Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/27.0.1453 Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
GoogleBot/3.0
Mozilla/5.0 (compatible; Googlebot/2.1; https://deepcrawl.com/bot)
Mozilla/5.0 (compatible; Googlebot/2.1; https://www.deepcrawl.com/bot)
Googlebot-Video/1.0
Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html) UNTRUSTED/1.0
Mozilla/5.0 (compatible; WebHistoryBot/1.2.1 IS NOT SE bot like Googlebot/2.1; +http://www.google.com/bot.html,Yahoo! Slurp or Bingbot)
Googlebot/2.1 (+http://www.googlebot.com/bot.html) <iframe src="whatismyip.com"></iframe>
Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/67.0.3366.99 Safari/537.36 (compatible; Googlebot/2.1; SeoakademiyaBot/1.0; +http://seo-akademiya.com/bot.html)
Googlebot/2.1
Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html )
Googlebot 2.1
Mozilla/5.0 (iPhone; CPU iPhone OS 12_0 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/12.0 Mobile/15E148 Safari/604.1 (GoogleBot)
Googlebot/2.1; http://www.google.com/bot.html
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/66.0.3359.139 Safari/537.36 Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html) AlexaToolbar/alxf-2.17
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/67.0.3396.99 Safari/537.36 Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html) FirePHP/0.7.4
NokiaC1-01/2.0 (06.15) Profile/MIDP-2.1 Configuration/CLDC-1.1 Mozilla/5.0 (iPhone; U; CPU iPhone 0S 3.0 like Mac 0S X; en-us; compatible; Googlebot/2.1; http://www.google.com/bot.html; AppleWebKit/528.18(KHTML,like Gecko) Version/4.0 Mobile/7A341 Safari/528.16 UNTRUSTED/1.0
Mozilla/5.0 (iPhone; CPU iPhone OS 6_0_1 like Mac OS X) AppleWebKit/537.4 (KHTML, like Gecko; Google Page Speed Insights) Version/6.0 Mobile/10A525 Safari/8536.25 GoogleBot/2.1
Googlebot (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
Googlebot/2.1 (+http://www.googlebot.com/bot.html) (Windows NT 10.0; WOW64; Trident/7.0; rv:11.0) like Gecko
BotThief (compatible; Googlebot/2.1; Location RU)
Mozilla/5.0+(compatible;+googlebot/2.1;++http://www.google.com/bot.html)
Mozilla/5.0 (compatible; Googlebot/2.1; actually this is http://ismyblogworking.com/)
Mozilla/5.0 (+http://www.googlebot.com/bot.html; Windows NT 6.2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/46.0.2486.0 Safari/537.36 Edge/13.9200
http://www.googlebot.com/bot.html
* Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
Mozilla/5.0 (compatible; Googlebot/2.1; http://deepcrawl.co.uk/bot.html)
Googlebot/2.1+(+http://www.googlebot.com/bot.html)
Googlebot/2.1 (+http://www.googlebot.com/bot.html
Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/46.0.2490.86 Safari/537.36 Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
Mozilla/5.0 (iPhone; CPU iPhone OS 6_0 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Version/6.0 Mobile/10A5376e Safari/8536.25 (compatible; Googlebot/2.1; +https://www.deepcrawl.com/bot)
Mozilla/5.0 (iPhone; U; CPU iPhone 0S 6.0 like Mac 0S X; en-us; compatible; Googlebot/2.1; http://www.google.com/bot.html; AppleWebKit/528.18(KHTML,like Gecko) Version/4.0 Mobile/7A341 Safari/528.16 UNTRUSTED/1.0
Aensland Mobile Browser /5.0 (iPhone; CPU iPhone OS 7_1 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Version/6.0 Mobile/10A5376e Safari/8536.25 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
66.249.64.13 - crawl-66-249-64-13.googlebot.com
Mozilla/5.0 (compatible; Googlebot/2.1; +https://whatis.contentkingapp.com/)
Mozilla/5.0 (compatible; Googlebot/2.1; http://www.google.com/bot.html)
Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; Googlebot/2.1; +http://www.google.com/bot.html) Safari/537.36.
Mozilla/5.0 (compatible; Googlebot/3.2; +http://www.facebook.com)
Googlebot/2.1 Compatible: http://www.googlebot.com/bot.html
Googlebot/2.1 ( http://www.googlebot.com/bot.html)
Googlebot/2.1" "Compatible"="+http://www.googlebot.com/bot.html
Mozilla/5.0 (compatible; Googlebot/2.1
Googlebot/2.1 (http://www.googlebot.com/bot.html)
Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/57.0.2987.133 Safari/537.36 Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/64.0.3282.186 Safari/537.36 Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
Googlebot/2.1; + http://www.google.com/bot.html
Mozilla/5.0 (compatible; Googlebot/wannabe; +https://xfrocks.com/resources/30/)
Mozilla/5.0 (Windows NT 6.1; WOW64; Trident/7.0; rv:11.0) like Gecko (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
(compatible; Googlebot/2.1; http://www.google.com/bot.html)
Mozilla/5.0 (compatible; Googlebot/2.1; +www.google.com/bot.html)
Mozilla/5.0 (iPhone; CPU iPhone OS 6_0 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Version/6.0 Mobile/10A5376e Safari/8536.25 (compatible; Googlebot/2.1; +https://deepcrawl.com/bot)
Mozilla/5.0 (compatible; Macintosh; Intel Mac OS X) AppleWebKit/534.46.0 (KHTML, like Gecko) Googlebot/2.1 Safari/7534.34
Googlebot-News (2.3.4.1, ruby 1.9.3 (2014-05-14))
Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.11 (KHTML, like Gecko) Chrome/23.0.1271.97 Safari/537.11 Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/52.0.2743.116 Safari/537.36 Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
Googlebot/2.1 (+http://www.google.com/bot.html
Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/64.0.3282.186 Safari/537.36 Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
crawl-66-249-66-1.googlebot.com
Googlebot-Image/1.0 (+http://www.googlebot.com/bot.html)
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/69.0.3497.100 Safari/537.36 Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html) Append?: Append
GoogleBot2/1
Googlebot/Nutch-1.7
Mozilla/5.0 (compatible; Googlebot/20.1; +http://www.google.com/bot.html)
Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/60.0.3112.78 Safari/537.36 Googlebot-News
Googlebot/4.0 (Blackberry; ExtremeOS 10_3_1 PPC; rv:12.6) Crocodile/20040327 Gridspider/12.6
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/66.0.3359.181 Safari/537.36 Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html) https://deepcrawl.com/bot
Nokia2700c-2/2.0 (09.98) Profile/MIDP-2.1 Configuration/CLDC-1.1 Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)/UCWEB8.2.0.116/69/352 UNTRUSTED/1.0
Mozilla/5.0 (compatible; Googlebot/2.1;)
Mozilla/4.0 (Googlebot/2.1 (+http://www.googlebot.com/bot.html); MSIE 6.0; )
{ "ua": "Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; Googlebot/2.1; +http://www.google.com/bot.html) Safari/537.36", "browser": { "name": "Safari" }, "engine": { "version": "537.36", "name": "WebKit" }, "os": {}, "device": {}, "cpu": {} }
Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36 Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
Mozilla/5.0 (Linux; Android 2.3.2; Nexus One; U; id-en) Build/FRF9 applewebkit/523.18 presto/2.2.1.40 Version/4.0 mobile safari/523.18 Googlebot/20.1.254.89 + http://www.google.com/bot.html UCBrowser8.4.0.159/69/352 UNTRUSTED/1.0
Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2272.96 Mobile Safari/537.36 (compatible; Googlebot/2.1; http://www.google.com/bot.html)
Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.31 (KHTML, like Gecko) Chrome/26.2.2.0 Safari/537.31 Googlebot/2.1 (+http://www.googlebot.com/bot.html)
Mozilla/11.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
GooglebotNSA
Mozilla/5.0 (compatible; Googlebot/6.1; +http://www.google.com/bot.html)
Mozilla/5.0 (Android; U; Android; en-id; compatible; Googlebot/10.0) AppleWebKit/534.1+ (KHTML, Like Gecko) Version/6.0.0.141 Mobile Safari/534.1+ UNTRUSTED/1.0
Googlebot/2.X (+http://www.googlebot.com/bot.html)
Nokia6303iclassic/5.0 (10.80) Profile/MIDP-2.1 Configuration/CLDC-1.1 Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html) UNTRUSTED/1.0
Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/69.0.3497.102 Safari/537.36 Vivaldi/2.0.1309.37 Googlebot/2.1 (+http://www.google.com/bot.html)
Googlebot/2.1 (+http://googlebot.com/bot.html)
Mozilla/5.0 (Windows NT 6.2; WOW64; rv:32.0; compatible; Googlebot/2.1; +http://www.google.com/bot.html)
Googlebot/2.1 (compatible; MSIE 7.0; Windows NT 5.1; .NET CLR 1.1.4322; .NET4.0C; .NET4.0E)
Mozilla/5.0 (compatible; Googlebot/2.1; https://www.deepcrawl.com/bot) https://deepcrawl.com/bot
googlebot 2,12
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/66.0.3359.139 Safari/537.36 Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2272.96 Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/33.0.1750.5 Safari/537.36 Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
Mozilla/5.0 (Windows NT 6.3; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.71 Safari/537.36 Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
Googlebot/2.1+(+http://www.googlebot.com/bot.html)"
NokiaX2-00/5.0 (04.25) Profile/MIDP-2.1 Configuration/CLDC-1.1 UCWEB/2.0 (Java; U; MIDP-2.0; id; Googlebot) U2/1.0.0 UCBrowser/9.4.1.377 U2/1.0.0 Mobile UNTRUSTED/1.0
Mozilla/5.0 (compatible; Googlebot/1.0; +http://www.google.com/bot.html)
Mozilla/5.0 (compatible; Googlebot/2.1;+http://www.google.com/bot.html)
Mozilla/5.0 (compatible? Googlebot/2.1? + Http://www.google.com/bot.html)
Mozilla/5.0 AppleWebKit/27.0.1453 (KHTML, like Gecko; compatible; Googlebot/2.1; +http://www.google.com/bot.html) Safari/27.0.1453
Mozilla/5.0 (Windows NT 6.2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/28.0.1500.95 Safari/537.36 Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html) or (rarely used): Googlebot/2.1 (+http://www.google.com/bot.html)
Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)aaa
Mozilla/4.0 (compatible; MSIE 6.0; Googlebot/2.1; +http://www.google.com/bot.html)
Googlebot/2.1; +
Googlebot/2.1(+http://www.googlebot.com/bot.html)
GoogleBot/2.9 (Fedora 3.4; en-US;)
GoogleBot/6.6 (OS 10.5 7.0; en-US;)
Mozilla/5.0 (compatiable Googlebot/2.1 +http//www.google.com/bot hrml)
GoogleBot/5.5 (Windows 95 5.1; en-US;)
Mozilla/5.0 (iPhone;<br /> U; CPU iPhone 0S 3.0<br /> like Mac 0S X; en-us;<br /> compatible;<br /> Googlebot/2.1;<br /> http://<br /> www.google.com/<br /> bot.html;<br /> AppleWebKit/528.18(KHTML,like<br /> Gecko) Version/4.0<br /> Mobile/7A341<br /> Safari/528.16
Googlebot/2.1 (+http://www.googlebot.com/bot.html).
Mozilla/5.0 (X11; U; Linux armv7l; en-GB; rv:1.9.2.3pre) Gecko/20100723 Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/32.0.1667.0 Safari/537.36
Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/32.0.1700.107 Safari/537.36 Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
Mozilla/23.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
Mozilla/5.0 (compatible;+Googlebot/2.1;+http://www.google.com/bot.html)
Googlebot/2.1 (+http://www.googlebot.com/bot.html) <iframe></iframe>
Googlebot/2.1(Windows 3.1)
"Googlebot/2.1",gzip(gfe) (via translate.google.com)
googlebot2.1
+http://www.googlebot.com/bot.html
Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)/Nutch-1.14-SNAPSHOT
Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2272.96 Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html) https://deepcrawl.com/bot
Mozilla/5.0 (compatible; Googlebot/2.5; +http://www.google.com/bot.html)
Googlebot/2.1 (+http://www.googlebot.com/bot.html) <iframe src="http://www.whatismyip.com/"></iframe>
Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html) Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
Googlebot/2.1 (+http:\\www.googlebot.com/bot.html)
Mozilla/5.0 (Macintosh; Intel Mac OS X) AppleWebKit/601.1.46 (KHTML, like Gecko) Safari/8536.25 Googlebot/2.1 scarlet/1.0 (development)
Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/67.0.3396.99 Safari/537.36 Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
Mozilla/5.0 (iPad; CPU OS 9_2_1 like Mac OS X) AppleWebKit/601.1.46 (KHTML, like Gecko) Mobile/13D11 Googlebot/2.1 scarlet/1.0 (development)
Googlebot/2.1 (+https://www.googlebot.com/bot.html)
Mozilla/6.0 (iPhone; CPU iPhone OS 8_0 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Version/8.0 Mobile/10A5376e Safari/8536.25 Googlebot/2.1 scarlet/1.0 (development)
AOL/9.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
Mozilla/5.0 (Windows NT 6.2; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/31.1.0.0 Safari/537.36 Googlebot/2.1 (+http://www.google.com/bot.html)
Mozilla/5.0 (Windows NT 6.2; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/31.1.0.0 Safari/537.36 Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
BotThief (compatible; Googlebot/2.1; Location US)
Mozilla/5.0 (X11; U; Linux armv7l; en-GB; rv:25.0; Presto/2.12.388) Gecko/20111216 Firefox/27.0 Googlebot/2.1 Maemo Browser 1.7.4.8 RX-51 N900
Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html; Evolute5.9; JK5)
Mozilla/5.0 (X11; U; Linux armv7l; en-US; rv:1.9.2.3pre; Presto/2.12.388; Macintosh; Intel Mac OS X 10.6; rv:25.0) Gecko/20111216 Firefox/13.0.1 (compatible; Googlebot/2.1; +http://www.google.com/bot.html) Maemo Browser 1.7.4.8 RX-51 N900
Mozilla/7.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
Mozilla/9.0 (compatible; Googlebot/3.7; +http://www.google.com/bot.html)
Googlebot/2.1 ([www.googlebot.com])
Googlebot (iPhone; X11; Linux x86_64; compatible; WOW64; Gecko/20130331; MSIE 10.0; Android 2.2; Windows NT 10.0) Firefox/49.0
Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html; Trident/7.0; rv:11.0) like Gecko
SECSSOBrowser (compatible; Googlebot/2.1; +https://sts.secsso.net)
Googlebot (Google's spider)
Googlebot/2.1+
googlebot/2.1 (+http //www.google.com/bot.html)
Chrome/41.0.2272.96 Mobile Safari/537.36 (compatible ; Googlebot/2.1 ; +http://www.google.com/bot.html)
Mozila/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
Hidden/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
Mozilla/6.0 (compatible; Googlebot/3.9; +http://www.google.com/bot.html)
Mozella/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
NokiaC1-01/2.0 (04.40) Profile/MIDP-2.1 Configuration/CLDC-1.1 Mozilla/5.0 (iPhone; U; CPU iPhone 0S 3.0 like Mac 0S X; en-us; compatible; Googlebot/2.1; http://www.tapanwap.com/index.html; AppleWebKit/528.18(KHTML,like Gecko) Version/4.0 Mobile/7A341 Safari/528.16 UNTRUSTED/1.0
Mozilla/5.0 (Windows; U; Windows NT 9.5; en-us; compatible; Googlebot/2.1; http://www.google.com/bot.html; AppleWebKit/534.19(KHTML,like Gecko) Chrome/11.0.661.0 Safari/534.19 UNTRUSTED/1.0
Googlebot/2.2 (+http://www.googlebot.com/bot.html)
Googlebot-Image/1.0 (Prism; XKeyscore)
(compatible; Googlebot/2.1; +http://www.google.com/bot.html)
Googlebot/2.1 (+http://www.myteltech.blogspot.in/bot.html)
Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)/1.8 (InfoSeek crawler; http://www.infoseek.com; crawler@infoseek.com)
Googlebot/2.1 (+http://www.google.com/bot.html; For abuse: kilobyte2007@gmail.com)
Mozilla/5.0 (compatible; Googlebot/
Mozilla/5.0 (compatible; Googlebot/7.0
Hacker Anonnymous (+http://www.googlebot.com/bot.html
=Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
Mozilla/5.0 (compatible; Googlebot/2.1;
GoogleBot/15.18 (Ubuntu 5.3; ar_JO;)
Mozilla/5.0 (compatible; U; Googlebot/9.8.4.62; + http://www.google.com/bot; en) AppleWebKit/534.1+ (KHTML, Like Gecko) Version/6.0.0.141 Mobile Safari/534.1+ UNTRUSTED/1.0
Mozilla/5.0 (compatible; Googlebot/20.0.1105.2; + http://www.google.com/bot.html) UNTRUSTED/1.0
Mozilla/5.0 (Linux; Android 2.3.2; Nexus One; Googlebot/20.0.1105.2; + http://www.google.com/bot.html) UNTRUSTED/1.0
Googlebot/4.0 (MSIE 4.01; ExtremeOS 10_3_1 PPC; rv:12.6) Crocodile/20040327 Gridspider/12.6
Google Chrome/5.0 (Linux; Android 8.0.0 Pixel XL Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2272.96 Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
Google Chrome/61 (Linux; Android 8.0.0 Pixel XL Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2272.96 Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html) support@google.com
Mozilla/5.0 (iPhone; U; CPU iPhone 0S 3.0 like Mac 0S X; en-us; compatible; Googlebot/2.1; http://www.google.com/bot.html; AppleWebKit/528.18(KHTML,like Gecko) Version/4.0 Mobile/7A341 Safari/528.16 UNTRUSTED/1.0 3gpp-gba
Mozilla/5.0 (iPad; U; ipad/12; en-us; compatible; Googlebot/2.1; http://www.google.com/bot.html; AppleWebKit/528.18(KHTML,like Gecko) Version/4.0 Mobile/7A341 Safari/528.16 UNTRUSTED/1.0
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/64.0.3282.186 Safari/537.36 Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
66.249.64.14 - crawl-66-249-64-14.googlebot.com
Mozilla/5.0 (Windows; U; Windows NT 10.0; en-US) AppleWebKit/604.1.38 (KHTML, like Gecko) QtWebEngine/5.10.1 googlebot/604.1.38
Nokia2700c-2/2.0 (09.98) Profile/MIDP-2.1 Configuration/CLDC-1.1 Mozilla/5.0 (iPhone; U; CPU iPhone 0S 3.0 like Mac 0S X; en-us; compatible; Googlebot/2.1; http://www.google.com/bot.html; AppleWebKit/528.18(KHTML,like Gecko) Version/4.0 Mobile/7A341 Safari/528.16/UCWEB8.2.0.116/69/352 UNTRUSTED/1.0
Mozilla/4.0 (Googlebot/2.1; MSIE 6.0; http://www.googlebot.com/bot.html)
Mozilla/4.0 (http://www.googlebot.com/bot.html; Googlebot/2.1; )
Googlebot/2.1 (Windoze; N; Windows 95; ZUSE3; rv1.3.10) Trident/20100101 Icefox/11.0b4
Googlebot-Image 1.0
NokiaC1-01/2.0 (06.15) Profile/MIDP-2.1 Configuration/CLDC-1.1 Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html) UNTRUSTED/1.0
Googlebot-Image 2.1
Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html) UNTRUSTED/1.0 3gpp-gba
Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/56.0.2924.87 Safari/537.36 Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/67.0.3396.87 Safari/537.36 Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
Nokia7210Supernova/2.0 (07.23) Profile/MIDP-2.1 Configuration/CLDC-1.1 Googlebot/2.1#(+http://www.google.com/ bot.html)/UCWEB7.5.0.66/69/352 UNTRUSTED/1.0
Googlebot/2.0
Mozilla/5.0 (compatiable Googlebot/2.1 + http//www.Google.com/not html)
Googlebot/2.X (http://www.googlebot.com/bot.html)
​Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2272.96 Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
Mozilla/5.0 (Windows NT 6.2; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/29.0.1547.15 Safari/537.36, : Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html) AlexaToolbar/alxf-2.18
Mozilla/5.0 (Windows NT 6.3; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/38.0.2125.104 Safari/537.36 Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html) ASProxy/5.5.0
Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2227.1 Safari/537.36 Googlebot/2.1
Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/57.0.2987.133 Safari/537.36 Googlebot/2.1 (+http://www.googlebot.com/bot.html)
Googlebot/2.1 (+http://www.googlebot.com/bot.html (Windows NT 6.3; Win64; x64; Trident/7.0; rv:11.0) like Gecko
Mozilla/5.0 (compatible; Googlebot[b]new[/b]/2.1; +http://www.google.com/bot.html)
mozilla/5.0 (compatible Googlebot/2.1 +http,//www.google.com/bot html)
Googlebot 2.1(+http://www.googlebot.com/bot.html)
Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html; Trident/5.0)
Mozilla/5.0 (compatible; Googlebot/3333; +http://www.google.com/bot.html)
Mozilla/5.0 (NokiaC5-00; compatible; Googlebot/2.1; +http://www.google.com/bot.html)
Googlebot/2.1 (+http://www.googlebot.com/bot.html) AlexaToolbar/alxf-2.19
Mozilla/5.0 (Linux; U; Android) Googlebot/2.1
Mozilla/5.0 (compatible; Googlebot<b>New</b>/2.1; +http://www.google.com/bot.html)
Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/69.0.3497.100 Safari/537.36 Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
Mozilla/5.0 (Android; Android; en-us; compatible; Googlebot/2.1; http://www.google.com/bot.html; AppleWebKit/528.18(KHTML,like Gecko) Version/4.0 Mobile/7A341 Safari/528.16 UNTRUSTED/1.0
Mozilla/5.0 (Android; U; Android; en-id; compatible; Googlebot 10.0) AppleWebKit/534.1+ (KHTML, Like Gecko) Version/6.0.0.141 Mobile Safari/534.1+ UNTRUSTED/1.0
Googlebot/Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:28.0) Gecko/20100101 Firefox/28.0
Googlebot (Windows NT 6.1; Win64; x64)
Mozilla/5.0 (Android; U; Googlebot/2.1; en-id) AppleWebKit/534.1+ (KHTML, Like Gecko) Version/6.0.0.141 Mobile Safari/534.1+ UNTRUSTED/1.0
[Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html) or (rarely used): Googlebot/2.1 (+http://www.google.com/bot.html)]
Mozilla/5.0 (compatible; <B>Googlebot</B>/2.1; +http://www.google.com/bot.html)
Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.9oo91e.qjz9zk/bot.html)
Mozilla/5.0 (compatible; Googlebot/2.1; +http://google.com/)
Mozilla/5.0 (compatible; Googlebot/2.0; +http://www.google.com/bot.html)
Mozilla/5.0 (compatible; Googlebot/2.2; +http://www.google.com/bot.html)
Mozilla/5.0 (compatible; Googlebot/2.1; ̣+http://www.google.com/bot.html)
Mozilla/5.0 (compatible; Googlebot/2.1; +http//www.google.com/bot.html) (compatible; MSIE 6.0; Windows CE; IEMobile 7.6)
Googlebot/2.10
Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html) or
Googlebot/2.109
Googlebot/2.109aer
Googlebot-News (2.3.3, ruby 1.9.3 (2013-11-22))
Googlebot/a
Mozilla/5.0 (compatible; Googlebot/3.1; +http://www.google.com/bot.html)
Mozilla/35.0.1 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
Mozilla/5.0 (compatible; Googlebot/0.1; +http://www.google.com/bot.html)
Mozilla/5.0 GoogleBot/3.0
Mozilla/5.0 (Linux; U; CPU Android 3.0 like Mac 0S X; en-us; compatible; Googlebot/2.1; http://www.google.com/bot.html; AppleWebKit/528.18(KHTML,like Gecko) Version/4.0 Mobile/7A341 Safari/528.16 UNTRUSTED/1.0
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/65.0.3325.181 Safari/537.36 Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/65.0.3325.181 Safari/537.36 Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2272.96 Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
Mozilla/5.0 (Linux; U; Android 2.2; en-us; Nexus One; googlebot/2.1; +http://www.google.com/bot; AppleWebKit/528.18(KHTML,like Gecko) Version/4.0 Mobile/7A341 Safari/528.16 presto/2.2.40 build/tipartimur UNTRUSTED/1.0
Googlebot/2.1 (+hi; Android; Mobile;)
HitLeap Viewer=HitLeap Viewer 2.2 FireFox 6.0=Mozilla/5.0 (Windows NT 5.1; rv:6.0.2) Gecko/20100101 Firefox/6.0.2IE8(Windows XP)=Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 5.1; Trident/4.0; CIBA; .NET4.0C; .NET4.0E; .NET CLR 2.0.50727; .NET CLR 3.0.4506.2152; .NET CLR 3.5.30729)IE6=Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)Mac Safari 4=Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_5_8; en-us) AppleWebKit/531.21.8 (KHTML, like Gecko) Version/4.0.4 Safari/5Google Bot=Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)iPhone 3.0=Mozilla/5.0 (iPhone; U; CPU iPhone OS 3_0 like Mac OS X; en-us) AppleWebKit/528.18 (KHTML, like Gecko) Version/4.0 Mobile/7A341 Safari/528.16iPhone 4S=Mozilla/5.0 (iPhone; U; CPU iPhone OS 4_0 like Mac OS X; en-us) AppleWebKit/532.9 (KHTML, like Gecko) Version/4.0.5 Mobile/8A293 Safari/6531.22.7Nokia C5-00(S60v3)=Mozilla/5.0 (SymbianOS/9.3; Series60/3.2 NokiaC5-00/032.010; Profile/MIDP-2.1 Configuration/CLDC-1.1 ) AppleWebKit/525 (KHTML, like Gecko ) Version/3.0 BrowserNG/7.2.6.2 3gpp-gbaHTC Legend(Android)=Mozilla/5.0 (Linux; U; Android 2.2; zh-cn; HTC Legend Build/FRF91) AppleWebKit/533.1 (KHTML, like Gecko) Version/4.0 Mobile Safari/533.1iPad=Mozilla/5.0 (iPad; U; CPU OS 3_2 like Mac OS X; en-us) AppleWebKit/531.21.10 (KHTML, like Gecko) Version/4.0.4 Mobile/7B334b Safari/531.21.10
GoogleBot 1.0
Mozilla/5.0 (compatible; Googlebot/2.4; +http://www.google.com/bot.html)
Rizostronza/1.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/37.0.2062.124 Safari/537.36 Mozilla/5.0 (compatible; Googlebot/2.1; http://www.google.com/bot.html)
GoogleBot/1.0
Googlebot/2.1 (+http://www.google.com/bot.html) Mozilla/5.0 (compatible); Googlebot/2.1; (+http://www.google.com/bot.html) Googlebot-Image/1.0
Opera/18.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
Googlebot/2.1 (+https://www.google.com/bot.html)
Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/34.0.1847.60 Safari/537.36 Googlebot/2.1 (+http://www.googlebot.com/bot.html)
Reference in New Issue View Git Blame Copy Permalink