- A+
所属分类：Web技巧
最近整理了一下各平台爬虫的信息，记录一下（有异议的地方请指正）。有些个人的恶意IP并没统计。以下整理的爬虫IP都是根据对应的 User-Agent排序的，所以可能统一号段的IP分属不同UA（当然UA也是可以随意伪造的），加上网上搜集的一些但是UA不确定所以都放到每个分组的最后边。
百度爬虫IP列表
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 |
[crayon-65c4e51392a1b246648617 inline="true" ]爬虫Agent：Mozilla/5.0 (Windows NT 6.1; WOW64; Baidu Transcoder) AppleWebKit/536.11 (KHTML, like Gecko) Chrome/20.0.1132.57 Safari/536.11 61.135.169.75 61.135.169.55 61.135.169.22 220.181.51.40 220.181.51.39 111.13.13.74 111.13.13.73 111.13.13.72 111.13.13.71 111.13.13.6 111.13.13.5 111.13.13.4 111.13.13.3 101.254.184.206 1.95.9.244 爬虫Agent：Mozilla/5.0 (compatible; heritrix/3.1.1 +http://www.baidu.com) 59.50.71.83 211.89.227.50 211.89.227.16 211.89.227.15 183.224.87.36 124.119.30.90 117.25.173.119 116.113.28.179 爬虫Agent：Mozilla/5.0 (compatible; Baiduspider/2.0; +http://www.baidu.com/search/spider.html) 61.191.191.87 61.184.241.206 61.183.41.95 59.49.46.165 58.222.20.226 58.218.204.136 45.113.253.53 27.151.30.176 27.151.28.24 222.216.28.45 222.186.3.229 220.181.51.81 220.181.51.76 220.181.51.75 220.181.51.74 220.181.51.110 220.181.51.109 220.181.51.107 220.181.108.97 220.181.108.96 220.181.108.87 220.181.108.86 220.181.108.84 220.181.108.83 220.181.108.81 220.181.108.80 220.181.108.79 220.181.108.78 220.181.108.77 220.181.108.76 220.181.108.75 220.181.108.186 220.181.108.184 220.181.108.181 220.181.108.179 220.181.108.178 220.181.108.177 220.181.108.175 220.181.108.174 220.181.108.173 220.181.108.172 220.181.108.171 220.181.108.170 220.181.108.168 220.181.108.166 220.181.108.165 220.181.108.161 220.181.108.158 220.181.108.157 220.181.108.156 220.181.108.155 220.181.108.153 220.181.108.152 220.181.108.151 220.181.108.150 220.181.108.149 220.181.108.147 220.181.108.146 220.181.108.145 220.181.108.144 220.181.108.143 220.181.108.142 220.181.108.141 220.181.108.140 220.181.108.139 220.181.108.123 220.181.108.122 220.181.108.120 220.181.108.119 220.181.108.117 220.181.108.116 220.181.108.115 220.181.108.114 220.181.108.113 220.181.108.112 220.181.108.111 220.181.108.110 220.181.108.109 220.181.108.108 220.181.108.107 220.181.108.106 220.181.108.105 220.181.108.104 220.181.108.103 220.181.108.101 220.181.108.100 219.153.68.137 219.138.135.236 202.46.52.201 202.46.51.156 202.46.51.132 183.61.171.152 183.61.171.137 183.2.242.150 180.97.35.165 180.76.15.9 180.76.15.8 180.76.15.7 180.76.15.6 180.76.15.5 180.76.15.34 180.76.15.33 180.76.15.32 180.76.15.31 180.76.15.30 180.76.15.29 180.76.15.28 180.76.15.26 180.76.15.24 180.76.15.23 180.76.15.22 180.76.15.21 180.76.15.19 180.76.15.18 180.76.15.17 180.76.15.163 180.76.15.162 180.76.15.161 180.76.15.158 180.76.15.157 180.76.15.156 180.76.15.155 180.76.15.154 180.76.15.153 180.76.15.152 180.76.15.151 180.76.15.150 180.76.15.15 180.76.15.149 180.76.15.148 180.76.15.147 180.76.15.145 180.76.15.144 180.76.15.143 180.76.15.142 180.76.15.141 180.76.15.14 180.76.15.138 180.76.15.137 180.76.15.136 180.76.15.13 180.76.15.12 180.76.15.11 180.76.15.10 14.29.49.206 124.237.78.217 124.232.151.230 124.228.91.179 123.125.71.98 123.125.71.94 123.125.71.92 123.125.71.91 123.125.71.90 123.125.71.89 123.125.71.87 123.125.71.86 123.125.71.85 123.125.71.84 123.125.71.83 123.125.71.82 123.125.71.81 123.125.71.78 123.125.71.74 123.125.71.73 123.125.71.70 123.125.71.69 123.125.71.60 123.125.71.58 123.125.71.57 123.125.71.56 123.125.71.55 123.125.71.53 123.125.71.52 123.125.71.50 123.125.71.49 123.125.71.47 123.125.71.45 123.125.71.44 123.125.71.43 123.125.71.42 123.125.71.41 123.125.71.39 123.125.71.38 123.125.71.34 123.125.71.33 123.125.71.32 123.125.71.31 123.125.71.30 123.125.71.29 123.125.71.28 123.125.71.27 123.125.71.26 123.125.71.23 123.125.71.22 123.125.71.20 123.125.71.19 123.125.71.18 123.125.71.17 123.125.71.15 123.125.71.14 123.125.71.13 123.125.71.12 123.125.71.116 123.125.71.115 123.125.71.114 123.125.71.113 123.125.71.111 123.125.71.110 123.125.71.109 123.125.71.108 123.125.71.107 123.125.71.106 123.125.71.105 123.125.71.103 123.125.71.102 123.125.71.100 123.125.67.152 123.125.67.148 122.10.89.22 120.39.244.111 117.34.115.181 117.25.139.66 117.25.139.42 117.25.139.35 117.25.139.34 116.9.156.37 116.55.241.24 115.47.26.143 115.231.26.31 113.113.120.28 112.124.57.11 爬虫Agent：Baiduspider+(+http://www.baidu.com/search/spider.htm) 61.146.178.41 61.146.178.186 爬虫Agent：Baidu-YunGuance-VSBot(ce.baidu.com) 111.206.36.17 |
[/crayon]
|
1 2 3 |
[crayon-65c4e51392a20330449234 inline="true" ]#网络搜集 180.149.130.* 180.76.5.66 |
[/crayon]
百度云观测IP
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 |
[crayon-65c4e51392a24270164867 inline="true" ]安全检测服务User-Agent： User-Agent:Baidu-YunGuanCe-ScanBot(ce.baidu.com） 可用性检测 其他服务User-Agent： User-Agent:Baidu-YunGuanCe-SLABot(ce.baidu.com） 可用性检测 User-Agent:Baidu-YunGuance-PerfBot(ce.baidu.com) 速度检测 User-Agent:Baidu-YunGuance-VSBot(ce.baidu.com) 认证服务 User-Agent:Baidu-YunGuance-RSBot(ce.baidu.com) 即时速度检测 180.97.106.36 180.97.106.37 180.97.106.161 180.97.106.162 115.239.212.7 115.239.212.8 115.239.212.9 115.239.212.10 115.239.212.11 115.239.212.6 115.239.212.4 115.239.212.5 115.239.212.65 115.239.212.66 115.239.212.67 115.239.212.68 115.239.212.69 115.239.212.70 115.239.212.71 115.239.212.72 115.239.212.134 115.239.212.135 115.239.212.136 115.239.212.137 115.239.212.138 115.239.212.139 115.239.212.132 115.239.212.133 115.239.212.193 115.239.212.194 115.239.212.195 115.239.212.196 115.239.212.197 115.239.212.198 115.239.212.199 115.239.212.200 |
[/crayon]
google爬虫列表
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 |
[crayon-65c4e51392a28723413355 inline="true" ]爬虫Agent：Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko; Google Web Preview) Chrome/27.0.1453 Safari/537.36 45.79.76.220 爬虫Agent：Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko; Google Search Console) Chrome/27.0.1453 Safari/537.36 66.249.84.239 66.249.84.233 爬虫Agent：Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2272.96 Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html) 66.249.79.26 66.249.79.142 66.249.79.138 66.249.75.7 66.249.75.23 66.249.75.15 66.249.69.48 66.249.69.15 66.249.65.55 66.249.65.162 203.208.60.158 203.208.60.152 爬虫Agent：Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html) 66.249.79.69 66.249.79.64 66.249.79.25 66.249.79.24 66.249.79.231 66.249.79.210 66.249.79.194 66.249.79.187 66.249.79.180 66.249.79.174 66.249.79.171 66.249.79.168 66.249.79.146 66.249.79.127 66.249.75.8 66.249.75.24 66.249.75.16 66.249.69.254 66.249.69.213 66.249.69.209 66.249.69.206 66.249.69.197 66.249.69.192 66.249.69.19 66.249.69.188 66.249.69.176 66.249.69.140 66.249.69.137 66.249.69.134 66.249.69.128 66.249.69.11 66.249.66.67 66.249.66.64 66.249.66.254 66.249.66.197 66.249.66.186 66.249.66.183 66.249.66.180 66.249.66.168 66.249.66.166 66.249.66.162 66.249.66.158 66.249.66.154 66.249.66.150 66.249.66.142 66.249.66.139 66.249.66.138 66.249.66.135 66.249.66.134 66.249.66.126 66.249.65.61 66.249.65.60 66.249.65.59 66.249.65.57 66.249.65.56 66.249.65.186 66.249.65.183 66.249.65.180 66.249.65.168 66.249.65.166 66.249.64.50 66.249.64.140 66.249.64.137 66.249.64.134 54.159.17.214 203.208.60.157 203.208.60.156 203.208.60.155 203.208.60.154 203.208.60.153 203.208.60.151 203.208.60.150 203.208.60.149 122.155.168.168 爬虫Agent：Mozilla/5.0 (compatible; Google-Site-Verification/1.0) 66.249.82.245 66.249.82.242 66.249.82.239 66.249.82.208 66.249.82.205 爬虫Agent：DoCoMo/2.0 N905i(c100;TB;W24H16) (compatible; Googlebot-Mobile/2.1; +http://www.google.com/bot.html) 66.249.66.192 |
[/crayon]
|
1 2 3 4 5 6 7 |
[crayon-65c4e51392a2d632208167 inline="true" ]#网上搜集 67.221.235.* 66.249.68.* 66.249.67.* 203.208.60.* 66.249.72.* 66.249.71.* |
[/crayon]
360爬虫列表
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 |
[crayon-65c4e51392a30878985322 inline="true" ]爬虫Agent：Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; Trident/5.0); 360Spider 42.236.49.171 42.236.10.* 爬虫Agent：Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; Trident/5.0); 360Spider(compatible; HaosouSpider; http://www.haosou.com/help/help_3_2.html) 182.118.25.* 182.118.22.* 182.118.21.* 182.118.20.* 爬虫Agent：Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; Trident/5.0); 360Spider 180.153.187.* 180.153.186.* 180.153.185.* 爬虫Agent：Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.1 (KHTML, like Gecko) Chrome/21.0.1180.89 Safari/537.1; 360Spider(compatible; HaosouSpider; http://www.haosou.com/help/help_3_2.html) 101.226.169.* 101.226.168.* 101.226.167.* 101.226.166.* |
[/crayon]
360爬虫这个倒是挺规矩的，基本上都是统一号段的IP都用一个UA。
必应爬虫列表
|
1 2 3 4 5 6 7 8 9 10 11 12 13 |
[crayon-65c4e51392a35848002914 inline="true" ]爬虫Agent：Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm) 40.77.167.95 40.77.167.39 40.77.167.19 207.46.13.183 207.46.13.151 207.46.13.64 207.46.13.13 157.55.39.212 157.55.39.203 157.55.39.188 157.55.39.169 157.55.39.131 |
[/crayon]
|
1 2 3 4 5 6 7 8 |
[crayon-65c4e51392a38019721119 inline="true" ]#网上搜集 65.52.108.142 74.125.75.3 74.125.64.81 74.125.44.82 74.125.158.86 74.125.156.82 65.52.108.146 |
[/crayon]
soso爬虫列表
|
1 2 3 |
[crayon-65c4e51392a3c934710886 inline="true" ]#网上搜集 124.115.4.* 124.115.0.* |
[/crayon]
sogou爬虫列表
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 |
[crayon-65c4e51392a3f070916545 inline="true" ]#网上搜集 220.181.94.231 220.181.94.229 220.181.94.223 220.181.125.71 220.181.125.69 220.181.125.45 123.126.50.76 123.126.50.70 220.181.94.237 220.181.94.235 220.181.94.233 220.181.94.225 220.181.94.224 220.181.94.213 220.181.125.43 220.181.125.162 220.181.125.108 202.85.214.159 123.126.50.81 123.126.50.78 123.126.50.77 123.126.50.75 123.126.50.74 123.126.50.72 123.126.50.71 123.126.50.69 123.126.50.68 123.126.50.66 |
[/crayon]
yahoo爬虫列表
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 |
[crayon-65c4e51392a43488418626 inline="true" ]#网络搜集 72.30.142.223 67.195.37.168 67.195.37.154 202.160.180.198 202.160.180.176 202.160.180.118 202.160.179.16 202.160.179.127 202.160.178.70 110.75.176.30 110.75.176.29 110.75.176.28 110.75.176.27 110.75.176.26 110.75.176.25 110.75.173.176 110.75.173.175 110.75.173.174 110.75.173.173 110.75.173.172 110.75.173.171 74.6.18.249 203.209.252.21 202.160.189.241 202.160.189.234 202.160.188.215 202.160.184.15 202.160.182.11 202.160.181.190 202.160.180.8 202.160.180.73 202.160.180.66 202.160.180.59 202.160.180.53 202.160.180.45 202.160.180.39 202.160.180.32 202.160.180.27 202.160.180.191 202.160.180.187 202.160.180.17 202.160.180.165 202.160.180.163 202.160.180.16 202.160.180.158 202.160.180.155 202.160.180.154 202.160.180.148 202.160.180.138 202.160.180.136 |
[/crayon]
有道爬虫列表
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 |
[crayon-65c4e51392a47057671317 inline="true" ]#网络搜集 61.135.217.27 61.135.249.9 61.135.249.89 61.135.249.78 61.135.249.75 61.135.249.62 61.135.249.6 61.135.249.59 61.135.249.37 61.135.249.32 61.135.249.249 61.135.249.239 61.135.249.236 61.135.249.232 61.135.249.231 61.135.249.229 61.135.249.220 61.135.249.216 61.135.249.213 61.135.249.210 61.135.249.191 61.135.249.187 61.135.249.175 61.135.249.171 61.135.249.146 61.135.249.144 61.135.249.143 61.135.249.137 61.135.249.122 61.135.249.112 61.135.248.228 61.135.248.226 61.135.248.219 61.135.248.210 61.135.248.203 61.135.220.9 61.135.220.3 |
[/crayon]
msn爬虫列表
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 |
[crayon-65c4e51392a4b986751683 inline="true" ]#网络搜集 207.46.204.38 207.46.204.37 207.46.204.35 207.46.204.128 207.46.199.244 207.46.199.242 207.46.199.213 207.46.194.95 207.46.194.91 207.46.194.88 207.46.194.85 207.46.194.78 207.46.194.67 207.46.194.55 207.46.194.140 207.46.194.130 207.46.194.129 207.46.204.44 207.46.204.43 207.46.204.42 207.46.204.40 207.46.204.39 207.46.204.34 207.46.204.31 207.46.204.30 207.46.204.138 207.46.204.137 207.46.204.133 207.46.204.129 207.46.199.249 207.46.199.246 207.46.199.240 207.46.199.238 207.46.199.229 207.46.199.218 207.46.199.216 207.46.199.215 207.46.195.167 207.46.194.96 207.46.194.94 207.46.194.90 207.46.194.89 207.46.194.87 207.46.194.83 207.46.194.82 207.46.194.81 207.46.194.77 207.46.194.75 207.46.194.74 207.46.194.73 |
[/crayon]
针对爬虫IP，至于是喷杀虫药还是放进来自己喂，就看各位施主了。
此篇文章，可以理解为我闲的纯记录所用，不过目测会有人用到。
...
送给每个来过的人，不管你是刚开始学习的小白，还是独领风骚的老司机：不是每个人都能认同你，即使是你身边的人。人嘛，都这样，谁说不是呢？要一直相信在以后的日子里一定还会有来自各方诸多的刁难、抱怨，你不可能把所有人的嘴巴都堵上，也不可能让所有人的观点都和你一致。你要做的就是尽自己最大的努力，用实际行动去回敬那些曾反对、刁难过你的人。
我可以对一个人无限的好，前提是值得。 ——慕冬雪
好帅的混血狼（拿来镇楼）！
最近整理了一下各平台爬虫的信息，记录一下（有异议的地方请指正）。有些个人的恶意IP并没统计。以下整理的爬虫IP都是根据对应的 User-Agent排序的，所以可能统一号段的IP分属不同UA（当然UA也是可以随意伪造的），加上网上搜集的一些但是UA不确定所以都放到每个分组的最后边。
百度爬虫IP列表
[/crayon]
[/crayon]
百度云观测IP
[/crayon]
google爬虫列表
[/crayon]
[/crayon]
360爬虫列表
[/crayon]
360爬虫这个倒是挺规矩的，基本上都是统一号段的IP都用一个UA。必应爬虫列表
[/crayon]
[/crayon]
soso爬虫列表
[/crayon]
sogou爬虫列表
[/crayon]
yahoo爬虫列表
[/crayon]
有道爬虫列表
[/crayon]
msn爬虫列表
[/crayon]
针对爬虫IP，至于是喷杀虫药还是放进来自己喂，就看各位施主了。
此篇文章，可以理解为我闲的纯记录所用，不过目测会有人用到。
...送给每个来过的人，不管你是刚开始学习的小白，还是独领风骚的老司机：不是每个人都能认同你，即使是你身边的人。人嘛，都这样，谁说不是呢？要一直相信在以后的日子里一定还会有来自各方诸多的刁难、抱怨，你不可能把所有人的嘴巴都堵上，也不可能让所有人的观点都和你一致。你要做的就是尽自己最大的努力，用实际行动去回敬那些曾反对、刁难过你的人。
我可以对一个人无限的好，前提是值得。
