【开源系列】Loki 与 Promtail 实现轻量级日志管理系统

news/2024/11/14 11:25:25/文章来源:https://www.cnblogs.com/o-O-oO/p/18537516

1、简介

Grafana Loki 是一个用于日志聚合和分析的开源工具，专为云原生环境设计，与 Grafana 无缝集成，能够高效地收集、存储和查询日志。以下是如何使用 Grafana Loki 收集日志的详细步骤。

2、环境准备

确保你已经安装了以下组件：

Docker（用于容器化部署）Grafana（用于可视化日志数据）Loki（存储日志）Promtail（收集日志）

3、安装 Loki

可以通过 Docker 轻松安装 Loki。运行以下命令：

[root@mast01 istio]# vim /etc/loki/local-config.yamlauth_enabled: falseserver:
http_port: 3100distributor:
ring:kvstore:store: inmemoryingester:
chunk_idle_period: 5m
chunk_block_size: 720kb
max_chunk_age: 1h
lifecycler:heartbeat_timeout: 1mstorage_config:
boltdb:directory: /loki/indexschema_config:
configs:- from: 2020-10-22store: boltdbschema: v11index:prefix: loki_index_period: 168hcompactor:
working_directory: /loki/compactor

[root@mast01 istio]# docker run -d --name=loki -p 3100:3100 grafana/loki:latest -config.file=/etc/loki/local-config.yaml

4、安装 Promtail

[root@mast01 istio]# vim /etc/promtail/config.yamlserver:
http_listen_port: 9080positions:
filename: /tmp/positions.yamlclients:- url: http://192.168.58.116:3100/loki/api/v1/pushscrape_configs:- job_name: varlogsstatic_configs:- targets:- localhostlabels:job: varlogs__path__: /var/log/*.log

[root@mast01 istio]# docker run -d --name=promtail -v /var/log:/var/log -v $(pwd)/promtail-config.yaml:/etc/promtail/config.yaml grafana/promtail:latest -config.file=/etc/promtail/config.yaml

5、安装 Grafana

[root@mast01 ~]# vim grafana.yamlapiVersion: v1
kind: ConfigMap
metadata:
name: grafana-config
data:
grafana.ini: |[server]root_url = %(protocol)s://%(domain)s:%(http_port)s/serve_from_sub_path = true[auth]disable_login_form = falsedisable_signout_menu = false[users]allow_sign_up = true---
apiVersion: apps/v1
kind: Deployment
metadata:
name: grafana
spec:
replicas: 1
selector:matchLabels:app: grafana
template:metadata:labels:app: grafanaspec:containers:- name: grafanaimage: registry.cn-shanghai.aliyuncs.com/study-03/grafana:9.0.1ports:- containerPort: 3000env:- name: GF_SECURITY_ADMIN_PASSWORDvalue: "123456"  # 设置管理员密码volumeMounts:- name: grafana-config-volumemountPath: /etc/grafana/grafana.inisubPath: grafana.inivolumes:- name: grafana-config-volumeconfigMap:name: grafana-config---
apiVersion: v1
kind: Service
metadata:
labels:# For use as a Cluster add-on (https://github.com/kubernetes/kubernetes/tree/master/cluster/addons)# If you are NOT using this as an addon, you should comment out this line.kubernetes.io/cluster-service: 'true'kubernetes.io/name: monitoring-grafana
name: monitoring-grafana
spec:# In a production setup, we recommend accessing Grafana through an external Loadbalancer# or through a public IP.# type: LoadBalancer# You could also use NodePort to expose the service at a randomly-generated port# type: NodePort
ports:- port: 80targetPort: 3000
selector:app: grafana
type: NodePort

[root@mast01 ~]# kubectl apply -f gra.yaml
configmap/grafana-config created
deployment.apps/grafana created
service/monitoring-grafana created

[root@mast01 ~]# kubectl get pod
NAME                       READY   STATUS   RESTARTS   AGE
grafana-5f996b9995-skg9n   1/1     Running   0         22s

6、在 Grafana 中配置 Loki 数据源

登录 Grafana 控制台。进入 “Configuration” > “Data Sources”。点击 “Add data source” 并选择 “Loki”。在 URL 中输入 Loki 的地址（例如 http://192.168.58.116:3100）。点击 “Save & Test” 以确认连接成功。

7、验证效果

点击左侧工具栏，选择Explore在查询框中输入日志查询，例如：

#使用 Grafana 提供的强大查询语言，可以进行更复杂的日志查询。例如，你可以使用 `|~` 进行正则匹配，或使用 `|=` 进行字符串过滤，构建适合你的查询。
{job="varlogs"} |= "error"

[root@mast01 log]# cat vmware-vmsvc-root.log |grep 2024-11-07|grep error
[2024-11-07T07:22:37.812Z] [ warning] [vmsvc] SimpleSock: failed to connect (1023 => 976), error 110: 连接超时