我说我吧百度屏蔽了他还收录
116.179.33.81 - - [05/Aug/2022 :00:06:54 +0800] "GET / HTTP/1.1" 200 49098 "-" "Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/48.0.2564.116 Safari/537.36"
220.181.51.85 - - [05/Aug/2022 :00:06:54 +0800] "GET / HTTP/1.1" 200 49098 "-" "Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/48.0.2564.116 Safari/537.36"
116.179.33.20 - - [05/Aug/2022 :00:11:45 +0800] "GET / HTTP/1.1" 200 49098 "-" "Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/48.0.2564.116 Safari/537.36"
116.179.33.15 - - [05/Aug/2022 :00:16:36 +0800] "GET / HTTP/1.1" 301 162 "-" "Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/48.0.2564.116 Safari/537.36"
116.179.33.78 - - [05/Aug/2022 :00:16:36 +0800] "GET / HTTP/1.1" 200 49098 "-" "Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/48.0.2564.116 Safari/537.36"
看看UA里面居然没有baidu 我说我在nginx 屏蔽UA 百度 都吧网页收录了。。。
反查了 不是百度,但是下面就有蜘蛛爬行的 所以这基本就是百度的IP。、
应该是百度先用其他的IP来检测下 好像是win模拟访问你的网页 在蜘蛛来爬行 可能怕你是ZP网站。
220.181.51.85 - - [05/Aug/2022 :00:06:54 +0800] "GET / HTTP/1.1" 200 49098 "-" "Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/48.0.2564.116 Safari/537.36"
116.179.33.20 - - [05/Aug/2022 :00:11:45 +0800] "GET / HTTP/1.1" 200 49098 "-" "Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/48.0.2564.116 Safari/537.36"
116.179.33.15 - - [05/Aug/2022 :00:16:36 +0800] "GET / HTTP/1.1" 301 162 "-" "Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/48.0.2564.116 Safari/537.36"
116.179.33.78 - - [05/Aug/2022 :00:16:36 +0800] "GET / HTTP/1.1" 200 49098 "-" "Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/48.0.2564.116 Safari/537.36"
看看UA里面居然没有baidu 我说我在nginx 屏蔽UA 百度 都吧网页收录了。。。
反查了 不是百度,但是下面就有蜘蛛爬行的 所以这基本就是百度的IP。、
应该是百度先用其他的IP来检测下 好像是win模拟访问你的网页 在蜘蛛来爬行 可能怕你是ZP网站。