排查过程
打开blackbox_exporter后台 http://127.0.0.1:9115/
找到监控项对应的 Debug Logs
点开查看日志:
Logs for the probe:
ts=2024-11-05T06:38:25.081157873Z caller=main.go:304 module=http_2xx target=http://img.xxx.cn/test.jpg level=info msg="Beginning probe" probe=http timeout_seconds=9.5
ts=2024-11-05T06:38:25.081232991Z caller=http.go:318 module=http_2xx target=http://img.xxx.cn/test.jpg level=info msg="Resolving target address" ip_protocol=ip6
ts=2024-11-05T06:38:25.144654104Z caller=http.go:318 module=http_2xx target=http://img.xxx.cn/test.jpg level=info msg="Resolved target address" ip=0e:924:1:1::14:d29
ts=2024-11-05T07:09:55.086648267Z caller=client.go:250 module=http_2xx target=http://img.xxx.cn/test.jpg level=info msg="Making HTTP request" url=http://[0e:924:1:1::14:d29]/test.jpg host=img.xxx.cn
ts=2024-11-05T07:09:55.086753304Z caller=main.go:119 module=http_2xx target=http://img.xxx.cn/test.jpg level=error msg="Error for HTTP request" err="Get http://[0e:924:1:1::14:d29]/test.jpg: dial tcp [0e:924:1:1::14:d29]:80: connect: network is unreachable"
ts=2024-11-05T07:09:55.086776385Z caller=main.go:119 module=http_2xx target=http://img.xxx.cn/test.jpg level=info msg="Response timings for roundtrip" roundtrip=0 start=2024-11-05T15:09:55.086709464+08:00 dnsDone=2024-11-05T15:09:55.086709464+08:00 connectDone=2024-11-05T15:09:55.086729233+08:00 gotConn=0001-01-01T00:00:00Z responseStart=0001-01-01T00:00:00Z end=0001-01-01T00:00:00Z
ts=2024-11-05T07:09:55.086798521Z caller=main.go:304 module=http_2xx target=http://img.xxx.cn/test.jpg level=error msg="Probe failed" duration_seconds=0.00569331
可以看到是域名解析到ipv6地址
而我监控的域名并没有ipv6解析,从而导致超时。
# dig -6 img.xxx.cn
; <<>> DiG 9.11.4-P2-RedHat-9.11.4-26.P2.el7_9.9 <<>> -6 img.xxx.cn
;; global options: +cmd
;; connection timed out; no servers could be reached
解决方案A:
域名增加ipv6解析,并测试验证。
解决方案B:
修改blackbox.yml,指定使用ipv4解析地址:
modules:http_2xx:prober: httphttp:preferred_ip_protocol: "ip4"