User Agent | Count |
Sogou web spider/4.0(+http://www.sogou.com/docs/help/webmasters.htm#07) | 2103 |
Mozilla/5.0 (compatible; MJ12bot/v2.0.2; http://mj12bot.com/) | 197 |
Mozilla/5.0 (compatible; AhrefsBot/7.0; +http://ahrefs.com/robot/) | 179 |
Mozilla/5.0 (compatible; SemrushBot/7~bl; +http://www.semrush.com/bot.html) | 108 |
Mozilla/5.0 (compatible) | 93 |
Expanse, a Palo Alto Networks company, searches across the global IPv4 space multiple times per day to identify customers' presences on the Internet. If you would like to be excluded from our scans, please send IP addresses/domains to: scaninfo@paloaltonetworks.com | 68 |
Embarcadero URI Client/1.0 | 51 |
ZoominfoBot (zoominfobot at zoominfo dot com) | 50 |
Mozilla/5.0 (compatible; MJ12bot/v1.4.8; http://mj12bot.com/) | 50 |
python-requests/2.32.3 | 31 |
Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm) Chrome/116.0.1938.76 Safari/537.36 | 30 |
Mozilla/5.0 (compatible; Dataprovider.com) | 27 |
Mozilla/5.0 (compatible; MJ12bot/v2.0.0; http://mj12bot.com/) | 25 |
Mozilla/5.0 (compatible; CensysInspect/1.1; +https://about.censys.io/) | 22 |
Go-http-client/1.1 | 21 |
Mozilla/5.0 (compatible; SemrushBot-BA; +http://www.semrush.com/bot.html) | 19 |
Mozilla/5.0 (compatible; Barkrowler/0.9; +https://babbar.tech/crawler) | 18 |
Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; Amazonbot/0.1; +https://developer.amazon.com/support/amazonbot) Chrome/119.0.6045.214 Safari/537.36 | 15 |
Mozilla/5.0 | 14 |
Mozilla/5.0 (compatible; InternetMeasurement/1.0; +https://internet-measurement.com/) | 14 |
Mozilla/5.0 (compatible; DataForSeoBot/1.0; +https://dataforseo.com/dataforseo-bot) | 14 |
Mozilla/5.0 (compatible; YandexBot/3.0; +http://yandex.com/bots) | 13 |
Apache-HttpClient/4.5.2 (Java/1.8.0_161) | 13 |
CheckMarkNetwork/1.0 (+http://www.checkmarknetwork.com/spider.html) | 11 |
Scrapy/2.11.2 (+https://scrapy.org) | 11 |
Mozilla/5.0 (compatible; DotBot/1.2; +https://opensiteexplorer.org/dotbot; help@moz.com) | 11 |
Mozilla/5.0 (compatible; SB Intuitions Corp./0.1) | 10 |
CCBot/2.0 (https://commoncrawl.org/faq/) | 10 |
Mozilla/5.0 (compatible; NetcraftSurveyAgent/1.0; +info@netcraft.com) | 10 |
SEMrushBot | 10 |
Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; ClaudeBot/1.0; +claudebot@anthropic.com) | 8 |
Mozilla/5.0 (compatible; BitSightBot/1.0) | 8 |
Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; GPTBot/1.2; +https://openai.com/gptbot) | 8 |
Mozilla/5.0 (Android 14; Mobile; rv:123.0) Gecko/123.0 Firefox/123 | 8 |
Mozilla/5.0 (compatible; BLEXBot/1.0; +https://help.seranking.com/en/blex-crawler) | 8 |
python-requests/2.27.1 | 8 |
Mozilla/5.0 (compatible; Baiduspider/2.0; +http://www.baidu.com/search/spider.html) | 7 |
2ip bot/1.1 (+http://2ip.io) | 7 |
Apache-HttpClient/4.5.2 (Java/1.8.0_151) | 7 |
serpstatbot/2.1 (advanced backlink tracking bot; https://serpstatbot.com/; abuse@serpstatbot.com) | 7 |
panscient.com | 6 |
curl/7.88.1 | 6 |
Mozilla/5.0 (compatible; MJ12bot/v2.0.1; http://mj12bot.com/) | 6 |
Pandalytics/2.0 (https://domainsbot.com/pandalytics/) | 5 |
Mozilla/5.0 (compatible; CMS-Checker/1.0; +https://example.com) | 5 |
Python/3.6 aiohttp/3.8.6 | 4 |
Mozilla/5.0 Firefox/35.0 | 4 |
Mozilla/5.0 (compatible; SEOkicks; +https://www.seokicks.de/robot.html) | 4 |
ALittle Client | 4 |
Python/3.8 aiohttp/3.8.4 | 4 |
Mozilla/5.0 (compatible; SeekportBot; +https://bot.seekport.com) | 3 |
Mozilla/5.0 (compatible; BLEXBot/1.0; +http://webmeup-crawler.com/) | 3 |
Mozilla/5.0 (compatible; wpbot/1.1; +https://forms.gle/ajBaxygz9jSR8p8G9) | 3 |
Mozilla/5.0 (compatible; ips-agent) | 3 |
Screaming Frog SEO Spider/20.2 | 3 |
Mozilla/5.0 (compatible; MixrankBot; crawler@mixrank.com) | 3 |
YisouSpider | 3 |
python-requests/2.22.0 | 2 |
python-urllib3/2.2.3 | 2 |
curl/7.76.1 | 2 |
libwww-perl/6.43 | 2 |
Screaming Frog SEO Spider/21.4 | 2 |
Chrome | 2 |
Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; GeedoProductSearch; +http://www.geedo.com/product-search.html) Chrome/79.0.3945.88 Safari/537.36 | 2 |
curl/7.29.0 | 2 |
meta-externalagent/1.1 (+https://developers.facebook.com/docs/sharing/webmasters/crawler) | 2 |
Mozilla/5.0 (compatible; t3versionsBot/1.0; +https://www.t3versions.com/bot) | 2 |
Mozilla/5.0 (X11; CrOS x86_64 14541.0.0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/121.0.0.0 Safari/537.3 | 2 |
Mozilla/5.0 (compatible; DotBot/1.1; http://www.opensiteexplorer.org/dotbot, help@moz.com) | 2 |
fasthttp | 2 |
Mozilla/5.0 (compatible; AwarioBot/1.0; +https://awario.com/bots.html) | 2 |
Mozilla/5.0 (compatible; wpbot/1.2; +https://forms.gle/ajBaxygz9jSR8p8G9) | 2 |
CyotekWebCopy/1.9 CyotekHTTP/6.4 | 1 |
DnBCrawler-Analytics | 1 |
Mozilla/5.0 (X11; CrOS i686 3912.101.0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/27.0.1453.116 Safari/537.36 | 1 |
Mozilla/5.0 (X11; CrOS x86_64 14541.0.0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/115.0.0.0 Safari/537.36 | 1 |
Opera/9.80 (S60; SymbOS; Opera Mobi/499; U; ru) Presto/2.4.18 Version/10.00 | 1 |
Python/3.10 aiohttp/3.11.11 | 1 |
Mozilla/5.0 (compatible; MJ12bot/v2.0.0; http://mj12bot.com/; ident=m08fNQhyTiR7) | 1 |
CATExplorador/1.0beta (sistemes at domini dot cat; https://domini.cat/catexplorador/) | 1 |
curl/7.79.1 | 1 |
python-httpx/0.27.2 | 1 |
w3m/0.5.3+git20230121 | 1 |
Python/3.8 aiohttp/3.9.5 | 1 |
curl/8.10.0 | 1 |
curl/8.4.0 | 1 |
Scrapy/2.13.0 (+https://scrapy.org) | 1 |
python-requests/2.31.0 | 1 |
Scrapy/2.12.0 (+https://scrapy.org) | 1 |
Mozilla/5.0 (compatible; GoParserBot/1.0) | 1 |
SEC-SGHX210/1.0 UP.Link/6.3.1.13.0 | 1 |
curl/7.61.1 | 1 |
Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko) LookoutBot/1.0; Chrome/133.0.0.0 Safari/537.36 | 1 |
BaiduSpider | 1 |
python-requests/2.25.1 | 1 |
Mozilla/5.0 (compatible; archive.org_bot +http://archive.org/details/archive.org_bot) Zeno/6508505 warc/v0.8.71 | 1 |
Mozilla/5.0 (X11; CrOS x86_64 8172.45.0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.64 Safari/537.36 | 1 |
合計 | 3448 |
---|