[日志分析篇]-利用ELK分析jumpserver日志-日志拆分篇

news/2024/10/24 17:38:31/文章来源:https://www.cnblogs.com/amsilence/p/18500021

需要通过elk日志分析平台接收jumpserver日志,对日志进行过滤和拆分。并通过Grafana进行企业微信告警推送和大屏展示

1.系统介绍

名称 软件版本
jumpserver jumpserver-3.10.13-tls
elasticsearch elasticsearch-8.12.2
kibana kibana-8.12.2
logstash logstash-8.12.2
granfa Grafana v11.0.0

2.jumpserver配置syslog

此处参考飞致云syslog配置文档:https://kb.fit2cloud.com/?p=123#heading-11
在/opt/jumpserver/config/config.txt配置文件钟添加syslog配置

# 加入 syslog 相关设置
SYSLOG_ENABLE=True
SYSLOG_ADDR=10.22.3.12:5149
  • 重启jumpserver
/opt/jumpserver-offline-release-v3.10.13-amd64/jmsctl.sh restart

3.jumpserver的日志类型

jumpserver日志一共有:登录日志 、上传文件日志、下载文件日志、操作日志、改密日志、会话日志、命令日志这几种类型

4.logstash拆分jumpserver日志

# 定义接收jumpserver syslog的端口
input {udp {port => 5149}
}# 定义拆分jumpserver syslog的规则
filter {
# 判断日志类型是否为session_command_log,特定grok规则来解决linux机器记录w和top等命令日志无法解析情况
# 对于非session_command_log日志,将采用通用grok规则if [message] =~ /session_command_log/ {grok {match => { "message" => "<14>jumpserver: session_command_log - %{GREEDYDATA:reallogs}\u0000" }add_field => { "logtype" => "session_command_log" }}} else {grok {match => { "message" => "<%{NUMBER:priority}>%{GREEDYDATA:logsouce}: %{GREEDYDATA:logtype} - %{GREEDYDATA:reallogs}\u0000" }}}# 利用json拆分实际记录的日志json {source => "reallogs"target => "manage"}# 判断日志类型是否为login_log,用来获取用户账户名和用户账户显示名称if [logtype] != "login_log" {mutate {gsub => ["[manage][user]", "\)$", ""]split => { "[manage][user]" => "(" }}mutate {add_field => {"[manage][user_name]" => "%{[manage][user][0]}""[manage][user_account]" => "%{[manage][user][1]}"}}}# 移除不需要的日志mutate {remove_field => ["reallogs", "@version", "event", "logsouce", "priority", "[manage][user]", "message"]}# 解析login_log日志,用日志内的真实时间来替换@timestamp,保证日志时间的真确性if [logtype] == "login_log" {date {match => ["[manage][datetime]", "yyyy/MM/dd HH:mm:ss Z"]target => "@timestamp"}}# 解析ftp_log,拆分资产名字和资产IP,拆分资产连接账号和资产账户显示名称,用日志内的真实时间来替换@timestamp,保证日志时间的真确性if [logtype] == "ftp_log" {mutate {gsub => ["[manage][asset]", "\)$", "","[manage][account]", "\)$", ""]split => {"[manage][asset]" => "(""[manage][account]" => "("}}mutate {add_field => {"[manage][asset_name]" => "%{[manage][asset][0]}""[manage][asset_ip]" => "%{[manage][asset][1]}""[manage][asset_account_name]" => "%{[manage][account][0]}""[manage][asset_account_user]" => "%{[manage][account][1]}"}remove_field => [ "[manage][asset]","[manage][account]" ]}date {match => ["[manage][date_start]", "yyyy/MM/dd HH:mm:ss Z"]target => "@timestamp"}}# 解析operation_log日志,用日志内的真实时间来替换@timestamp,保证日志时间的真确性if [logtype] == "operation_log" {date {match => ["[manage][datetime]", "yyyy/MM/dd HH:mm:ss Z"]target => "@timestamp"}}# 解析password_change_log,操作人员名称和显示名,用日志内的真实时间来替换@timestamp,保证日志时间的真确性if [logtype] == "password_change_log" {mutate {gsub => [ "[manage][change_by]", "\)$", "" ]split => { "[manage][change_by]" => "(" }}mutate {add_field => {"[manage][changeby_user]" => "%{[manage][change_by][0]}""[manage][changeby_account]" => "%{[manage][change_by][1]}"}remove_field => [ "[manage][change_by]" ]}date {match => ["[manage][datetime]", "yyyy/MM/dd HH:mm:ss Z"]target => "@timestamp"}}# 解析host_session_log,拆分资产名字和资产IP,拆分资产连接账号和资产账户显示名称,用日志内的真实时间来替换@timestamp,保证日志时间的真确性if [logtype] == "host_session_log" {mutate {gsub => ["[manage][asset]", "\)$", "","[manage][account]", "\)$", ""]split => {"[manage][asset]" => "(""[manage][account]" => "("}}mutate {add_field => {"[manage][asset_name]" => "%{[manage][asset][0]}""[manage][asset_ip]" => "%{[manage][asset][1]}""[manage][asset_account_name]" => "%{[manage][account][0]}""[manage][asset_account_user]" => "%{[manage][account][1]}"}remove_field => [ "[manage][asset]","[manage][account]" ]}if [manage][date_end] {mutate {add_field => { "connect-time" => "%{[manage][duration]}" }}date {match => ["[manage][date_end]", "yyyy/MM/dd HH:mm:ss Z"]target => "@timestamp"}} else if [manage][date_start] {date {match => ["[manage][date_start]", "yyyy/MM/dd HH:mm:ss Z"]target => "@timestamp"}}}# 解析session_command_log,拆分资产名字和资产IP,拆分资产连接账号和资产账户显示名称,用日志内的真实时间来替换@timestamp,保证日志时间的真确性if [logtype] == "session_command_log" {mutate {gsub => ["[manage][asset]", "\)$", "","[manage][account]", "\)$", ""]split => {"[manage][asset]" => "(""[manage][account]" => "("}}mutate {add_field => {"[manage][asset_name]" => "%{[manage][asset][0]}""[manage][asset_ip]" => "%{[manage][asset][1]}""[manage][asset_account_name]" => "%{[manage][account][0]}""[manage][asset_account_user]" => "%{[manage][account][1]}"}remove_field => [ "[manage][asset]","[manage][account]" ]}date {match => ["[manage][timestamp_display]", "yyyy/MM/dd HH:mm:ss Z"]target => "@timestamp"}}# 日志拆分完毕后,删除不需要的日志mutate {remove_field => ["[manage][id]","[manage][asset_id]","[manage][account_id]","[manage][org_id]","[manage][terminal_display]","[manage][terminal][id]","[manage][terminal][name]","[manage][user_id]","[manage][timestamp]", "[manage][session]"]}}# 创建索引
output {elasticsearch {hosts => ["http://localhost:9200"]index => "sh-blj-%{+YYYY.MM.dd}"action => "create"user => "elastic"password => "password"}stdout {codec => "rubydebug"}
}

5.解析后日志的显示

本文来自互联网用户投稿,该文观点仅代表作者本人,不代表本站立场。本站仅提供信息存储空间服务,不拥有所有权,不承担相关法律责任。如若转载,请注明出处:http://www.hqwc.cn/news/821266.html

如若内容造成侵权/违法违规/事实不符,请联系编程知识网进行投诉反馈email:809451989@qq.com,一经查实,立即删除!

相关文章

IDEA运行不了代码

同样的代码在eclipse可以正常运行,创建的方法也保持一样,为什么在IDEA就报错,本人是小白看不懂下面的报错

扩展被恢复分区挡住的 C 盘

之前装系统的时候想着以后要不要装个 Ubuntu 以作备用,所以给 SSD 分区的时候留了一小部分,没有全分给 C 盘。结果后来用 WSL 用得乐不思蜀了,觉得剩下的空间留着没必要,于是想把剩下的空间扩容给 C 盘。结果操作的时候发现 C 盘后面跟了一个恢复分区!无法给 C 盘扩容了。…

若依开启注册功能

若依开启用户注册功能 1、修改数据库,如下:2、修改前端:参考鸣谢: https://blog.csdn.net/weixin_43684214/article/details/121609310

OCR视图识别(Tess4J)

1.概述 图片文字识别 OCR (Optical Character Recognition,光学字符识别)是指电子设备(例如扫描仪或数码相机)检查纸上打印的字符,通过检测暗、亮的模式确定其形状,然后用字符识别方法将形状翻译成计算机文字的过程 2.Tess4j快速入门 1.导入依赖<dependencies><…

『模拟赛』多校A层冲刺NOIP2024模拟赛12

『模拟赛记录』多校A层冲刺NOIP2024模拟赛12Rank 挂了不少,还行A. Alice 和璀璨花 签。 一眼最长上升子序列,昨天在 AT 专题里刚见过,不过赛时没想到离散化之后树状数组,所以打的动态开点,结果细节挂了 30pts。 和最长上升子序列思路基本一致,直接区间查询 \([1,a_i-1]\)…

分享一些利用商品详情数据挖掘潜在需求的成功案例

以下是一些利用商品详情数据挖掘潜在需求的成功案例: 一、亚马逊的个性化推荐系统:案例背景:亚马逊是全球知名的电商平台,拥有海量的商品和庞大的用户群体。为了提高用户的购物体验和增加销售额,亚马逊投入大量资源开发个性化推荐系统。 数据挖掘过程:亚马逊通过分析用户…

抖音2024推文副业,欢迎来咨询

不管钱多钱少,俗话说:苍蝇再少也是肉 这个道理希望大家明白,想做就一起探讨,嫌少的也不用来了哈(只能做朋友,不能做副友),想做的欢迎来咨询想做收益多的可看这边文章(仅供参考) https://mp.weixin.qq.com/s?__biz=MzkxNTg0NDI4OA==&mid=2247483656&idx=1&am…

Zabbix添加企业微信机器人告警

环境查看 系统环境# cat /etc/redhat-release CentOS Stream release 9 # uname -a Linux CentOSStream9Zabbix203 5.14.0-391.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Nov 28 20:35:49 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux软件环境 # zabbix_server --version zabbix_se…

DirectX Repair(DirectX修复工具)V4.3增强版

DirectX修复工具(DirectX Repair)是一款系统级工具软件,简便易用。 本程序的主要功能是检测当前系统的DirectX状态,如果发现异常则进行修复。程序主要针对0xc000007b问题设计,可以完美修复该问题。本程序中包含了最新版的DirectX redist(Jun2010),并且全部DX文件都有Micros…

问题

dea输入字母时和正常的不一样,出现半角字符liaozk_c于 2024-01-06 16:36:32 发布 阅读量997 收藏 19 点赞数 16分类专栏: 日常异常信息处理 文章标签: java ide版权日常异常信息处理专栏收录该内容 8 篇文章1 订阅 订阅专栏报错内容 idea输入字母异常,出现半角输入 复现首先…