User Agent | Count |
Sogou web spider/4.0(+http://www.sogou.com/docs/help/webmasters.htm#07) | 2119 |
Mozilla/5.0 (compatible; MJ12bot/v2.0.2; http://mj12bot.com/) | 195 |
Mozilla/5.0 (compatible; AhrefsBot/7.0; +http://ahrefs.com/robot/) | 179 |
Mozilla/5.0 (compatible; SemrushBot/7~bl; +http://www.semrush.com/bot.html) | 108 |
Mozilla/5.0 (compatible) | 93 |
Expanse, a Palo Alto Networks company, searches across the global IPv4 space multiple times per day to identify customers' presences on the Internet. If you would like to be excluded from our scans, please send IP addresses/domains to: scaninfo@paloaltonetworks.com | 69 |
Mozilla/5.0 (compatible; MJ12bot/v1.4.8; http://mj12bot.com/) | 51 |
Embarcadero URI Client/1.0 | 51 |
ZoominfoBot (zoominfobot at zoominfo dot com) | 50 |
python-requests/2.32.3 | 32 |
Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm) Chrome/116.0.1938.76 Safari/537.36 | 30 |
Mozilla/5.0 (compatible; Dataprovider.com) | 27 |
Mozilla/5.0 (compatible; MJ12bot/v2.0.0; http://mj12bot.com/) | 25 |
Mozilla/5.0 (compatible; CensysInspect/1.1; +https://about.censys.io/) | 22 |
Go-http-client/1.1 | 20 |
Mozilla/5.0 (compatible; SemrushBot-BA; +http://www.semrush.com/bot.html) | 19 |
Mozilla/5.0 (compatible; Barkrowler/0.9; +https://babbar.tech/crawler) | 18 |
Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; Amazonbot/0.1; +https://developer.amazon.com/support/amazonbot) Chrome/119.0.6045.214 Safari/537.36 | 15 |
Mozilla/5.0 | 14 |
Mozilla/5.0 (compatible; InternetMeasurement/1.0; +https://internet-measurement.com/) | 14 |
Mozilla/5.0 (compatible; DataForSeoBot/1.0; +https://dataforseo.com/dataforseo-bot) | 14 |
Mozilla/5.0 (compatible; YandexBot/3.0; +http://yandex.com/bots) | 13 |
Apache-HttpClient/4.5.2 (Java/1.8.0_161) | 13 |
CheckMarkNetwork/1.0 (+http://www.checkmarknetwork.com/spider.html) | 11 |
Mozilla/5.0 (compatible; NetcraftSurveyAgent/1.0; +info@netcraft.com) | 11 |
Scrapy/2.11.2 (+https://scrapy.org) | 11 |
Mozilla/5.0 (compatible; DotBot/1.2; +https://opensiteexplorer.org/dotbot; help@moz.com) | 11 |
Mozilla/5.0 (compatible; SB Intuitions Corp./0.1) | 10 |
CCBot/2.0 (https://commoncrawl.org/faq/) | 10 |
SEMrushBot | 10 |
Mozilla/5.0 (compatible; BitSightBot/1.0) | 8 |
Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; GPTBot/1.2; +https://openai.com/gptbot) | 8 |
Mozilla/5.0 (Android 14; Mobile; rv:123.0) Gecko/123.0 Firefox/123 | 8 |
Mozilla/5.0 (compatible; BLEXBot/1.0; +https://help.seranking.com/en/blex-crawler) | 8 |
python-requests/2.27.1 | 8 |
Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; ClaudeBot/1.0; +claudebot@anthropic.com) | 7 |
Mozilla/5.0 (compatible; Baiduspider/2.0; +http://www.baidu.com/search/spider.html) | 7 |
2ip bot/1.1 (+http://2ip.io) | 7 |
Apache-HttpClient/4.5.2 (Java/1.8.0_151) | 7 |
serpstatbot/2.1 (advanced backlink tracking bot; https://serpstatbot.com/; abuse@serpstatbot.com) | 7 |
panscient.com | 6 |
curl/7.88.1 | 6 |
Mozilla/5.0 (compatible; MJ12bot/v2.0.1; http://mj12bot.com/) | 6 |
Pandalytics/2.0 (https://domainsbot.com/pandalytics/) | 5 |
Mozilla/5.0 (compatible; CMS-Checker/1.0; +https://example.com) | 5 |
Python/3.6 aiohttp/3.8.6 | 4 |
Mozilla/5.0 Firefox/35.0 | 4 |
Mozilla/5.0 (compatible; SEOkicks; +https://www.seokicks.de/robot.html) | 4 |
ALittle Client | 4 |
Python/3.8 aiohttp/3.8.4 | 4 |
Mozilla/5.0 (compatible; SeekportBot; +https://bot.seekport.com) | 3 |
Mozilla/5.0 (compatible; BLEXBot/1.0; +http://webmeup-crawler.com/) | 3 |
Mozilla/5.0 (compatible; wpbot/1.1; +https://forms.gle/ajBaxygz9jSR8p8G9) | 3 |
Mozilla/5.0 (compatible; ips-agent) | 3 |
Screaming Frog SEO Spider/20.2 | 3 |
Mozilla/5.0 (compatible; MixrankBot; crawler@mixrank.com) | 3 |
YisouSpider | 3 |
python-requests/2.22.0 | 2 |
python-urllib3/2.2.3 | 2 |
curl/7.76.1 | 2 |
libwww-perl/6.43 | 2 |
Screaming Frog SEO Spider/21.4 | 2 |
Chrome | 2 |
Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; GeedoProductSearch; +http://www.geedo.com/product-search.html) Chrome/79.0.3945.88 Safari/537.36 | 2 |
curl/7.29.0 | 2 |
meta-externalagent/1.1 (+https://developers.facebook.com/docs/sharing/webmasters/crawler) | 2 |
Mozilla/5.0 (compatible; t3versionsBot/1.0; +https://www.t3versions.com/bot) | 2 |
Mozilla/5.0 (compatible; DotBot/1.1; http://www.opensiteexplorer.org/dotbot, help@moz.com) | 2 |
fasthttp | 2 |
Mozilla/5.0 (compatible; AwarioBot/1.0; +https://awario.com/bots.html) | 2 |
Mozilla/5.0 (compatible; wpbot/1.2; +https://forms.gle/ajBaxygz9jSR8p8G9) | 2 |
CyotekWebCopy/1.9 CyotekHTTP/6.4 | 1 |
DnBCrawler-Analytics | 1 |
Mozilla/5.0 (X11; CrOS i686 3912.101.0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/27.0.1453.116 Safari/537.36 | 1 |
Mozilla/5.0 (X11; CrOS x86_64 14541.0.0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/115.0.0.0 Safari/537.36 | 1 |
Opera/9.80 (S60; SymbOS; Opera Mobi/499; U; ru) Presto/2.4.18 Version/10.00 | 1 |
Python/3.10 aiohttp/3.11.11 | 1 |
Mozilla/5.0 (compatible; MJ12bot/v2.0.0; http://mj12bot.com/; ident=m08fNQhyTiR7) | 1 |
CATExplorador/1.0beta (sistemes at domini dot cat; https://domini.cat/catexplorador/) | 1 |
curl/7.79.1 | 1 |
python-httpx/0.27.2 | 1 |
w3m/0.5.3+git20230121 | 1 |
Python/3.8 aiohttp/3.9.5 | 1 |
curl/8.10.0 | 1 |
curl/8.4.0 | 1 |
Scrapy/2.13.0 (+https://scrapy.org) | 1 |
python-requests/2.31.0 | 1 |
Scrapy/2.12.0 (+https://scrapy.org) | 1 |
Mozilla/5.0 (X11; CrOS x86_64 14541.0.0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/121.0.0.0 Safari/537.3 | 1 |
SEC-SGHX210/1.0 UP.Link/6.3.1.13.0 | 1 |
curl/7.61.1 | 1 |
Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko) LookoutBot/1.0; Chrome/133.0.0.0 Safari/537.36 | 1 |
BaiduSpider | 1 |
python-requests/2.25.1 | 1 |
Mozilla/5.0 (compatible; archive.org_bot +http://archive.org/details/archive.org_bot) Zeno/6508505 warc/v0.8.71 | 1 |
Mozilla/5.0 (X11; CrOS x86_64 8172.45.0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.64 Safari/537.36 | 1 |
合計 | 3462 |
---|