开源大数据集群部署(九)Ranger审计日志集成(solr)

作者:櫰木

1、下载solr安装包并解压包

tar -xzvf solr-8.11.2.gz
cd solr-8.11.2
执行安装脚本

./bin/install_solr_service.sh /opt/solr-8.11.2.tgz

安装后,会在/etc/default/ 下生成solr.in.sh文件。

2、在rangeradmin下生成solr相关配置

cd /opt/ranger-2.3.0-admin/contrib/solr_for_audit_setup/

编辑install.properties

# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements.  See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License.  You may obtain a copy of the License at
#
#     http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.#Note:
#1. This file is sourced from setup.sh, so make sure there are no spaces after the "="
#2. For variable with file path, please provide full path#!/bin/bash#JAVA_HOME to be used by Solr. Solr only support JDK 1.7 and above. If JAVA_HOME is not set
#in the env, then please set it here
#JAVA_HOME=#The operating system (linux) user used by Solr process. You need to run Solr as the below user and group
SOLR_USER=solr
SOLR_GROUP=solr
#How long to keep the audit logs. Please note, audit records grows very rapidly. Make sure to
#allocate enough memory and disk space to the server running Solr.
MAX_AUDIT_RETENTION_DAYS=90#If you want this script to install Solr, set the value to true. If it is already installed, then set this to false
#If it is true, then it will download and install it.
#NOTE: If you want the script to install Solr, then this script needs to be executed as root.
SOLR_INSTALL=false### BEGIN: if SOLR_INSTALL==true ###
#Location to download Solr. If SOLR_INSTALL is true, then SOLR_DOWNLOAD_URL is mandatory#For open source version, pick a mirror from below. Recommended versions are Apache Solr 5.2.1 or above
#http://lucene.apache.org/solr/mirrors-solr-latest-redir.html#Note: If possible, use the link from one of the mirror site
#SOLR_DOWNLOAD_URL=http://archive.apache.org/dist/lucene/solr/5.2.1/solr-5.2.1.tgz
SOLR_DOWNLOAD_URL=http://172.16.104.165:8999/warehouse/solr/solr-8.11.2.tgz### END: if SOLR_INSTALL==true ###
#The folder where Solr is installed. If SOLR_INSTALL=false, then Solr need to be preinstalled, else the setup will
#install at the below location
#Note: If you are using RPM from LucidWorks in HDP, then Solr is by default installed in the following location:
#SOLR_INSTALL_FOLDER=/opt/lucidworks-hdpsearch/solr
SOLR_INSTALL_FOLDER=/opt/solr#The location for the Solr configuration for Ranger. This script copies required configuration and
#startup scripts to the $SOLR_RANGER_HOME folder.
#NOTE: In SolrCloud mode, the data folders are under this folder. So make sure this is on seperate drive
#      with enough disk space. Have 1TB free disk space on this volume. Also regularly monitor available disk space
#      for this volume
#SOLR_RANGER_HOME=/opt/solr/ranger_audit_server
SOLR_RANGER_HOME=/opt/solr/ranger_audit_server#Port for Solr instance to be used by Ranger.
SOLR_RANGER_PORT=8983#Standalone or SolrCloud. Valid values are "standalone" or "solrcloud"
SOLR_DEPLOYMENT=solrcloud
#### BEGIN: if SOLR_DEPLOYMENT=standalone ##########################
#Location for the data files. Make sure it has enough disk space. Since audits records can grow dramatically,
#please have 1TB free disk space for the data folder. Also regularly monitor available disk space for this volume
SOLR_RANGER_DATA_FOLDER=/opt/solr/ranger_audit_server/data
#### END: if SOLR_DEPLOYMENT=standalone ############################## BEGIN: if SOLR_DEPLOYMENT=solrcloud ##########################
#Comma seperated list of of zookeeper host and path. Give fully qualified domain name for the host
#SOLR_ZK=localhost:2181/ranger_audits
SOLR_ZK=hd1:2181,hd2:2181,hd3:2181/ranger_audits
#Base URL of the Solr. Used for creating collections
SOLR_HOST_URL=http://`hostname -f`:${SOLR_RANGER_PORT}
#Number of shards
SOLR_SHARDS=1
#Number of replication
SOLR_REPLICATION=1
#### END: if SOLR_DEPLOYMENT=solrcloud ###########################Location for the log file. Please note that "solr" or the process owner should have write permission
#to log folder
#SOLR_LOG_FOLDER=logs
SOLR_LOG_FOLDER=/var/log/solr/ranger_auditsSOLR_RANGER_COLLECTION=ranger_audits#Memory for Solr. Both min and max memory to the java process are set to this value.
#Note: In production, please assign enough memory. It is recommended to have at least 2GB RAM.
#      Higher the RAM, the better. Solr core can take upto 32GB. For dev test you can use 512m
#SOLR_MAX_MEM=2g
#SOLR_MAX_MEM=512m
SOLR_MAX_MEM=2g

配置说明

  • /opt/solr/ranger_audit_server #生成配置文件的路径
  • SOLR_INSTALL_FOLDER=/opt/solr #solr安装路径
  • SOLR_DEPLOYMENT=SolrCloud #solr模式为分布式模式
  • SOLR_ZK=hd1:2181,hd2:2181,hd3:2181/ranger_audits #zk地址及solrznode
    执行生成脚本
./setup.sh
cd /opt/solr/ranger_audit_server/scripts/
可以查看到对应配置

在这里插入图片描述
./setup.sh
cd /opt/solr/ranger_audit_server/scripts/
可以查看到对应配置

cp solr.in.sh /etc/default/

在zk中生成solr的znode

bash add_ranger_audits_conf_to_zk.sh

启动solr

service solr start

默认日志路径

/var/log/solr/ranger_audits

查看端口是否启动成功

ss -tunlp | grep 8983

在这里插入图片描述
查看页面
http://hd1.dtstack.com:8983/solr/#/
在这里插入图片描述
创建audit 索引
cd /opt/solr/ranger_audit_server/scripts
修改create_ranger_audits_collection.sh脚本,添加端口8983
在这里插入图片描述
bash create_ranger_audits_collection.sh
成功后,可以在页面看到对应名称
在这里插入图片描述
修改rangeradmin的audit的配置
cd /opt/ranger-2.3.0-admin/
修改install.properties
在这里插入图片描述

audit_store=solr
# * audit_solr_url URL to Solr. E.g. http://<solr_host>:6083/solr/ranger_audits
audit_solr_urls=http://hd1.dtstack.com:8983/solr/ranger_audits
audit_solr_user=
audit_solr_password=
audit_solr_zookeepers=hd1:2181,hd2:2181,hd3:2181/ranger_auditsaudit_solr_collection_name=ranger_audits
#solr Properties for cloud mode
audit_solr_config_name=ranger_audits
audit_solr_configset_location=
audit_solr_no_shards=1
audit_solr_no_replica=1
audit_solr_max_shards_per_node=1
audit_solr_acl_user_list_sasl=solr,infra-solr
audit_solr_bootstrap_enabled=true

执行./setup.sh ,生成配置,重启rangeradmin
ranger-admin restart
如果一切正常,则在页面可以看到审计数据
在这里插入图片描述
更多技术信息请查看云掣官网https://yunche.pro/?t=yrgw

本文来自互联网用户投稿,该文观点仅代表作者本人,不代表本站立场。本站仅提供信息存储空间服务,不拥有所有权,不承担相关法律责任。如若转载,请注明出处:http://www.hqwc.cn/news/449180.html

如若内容造成侵权/违法违规/事实不符,请联系编程知识网进行投诉反馈email:809451989@qq.com,一经查实,立即删除!

相关文章

el-table点击某一行选中改变背景色且执行方法

elementUI table表格点击某一行选中并且改变背景色 使用:row-style"rowStyle"及row-click“selectRow”&#xff1a; 其中 selectRow 方法中&#xff1a; row 输出&#xff1a;当前行的内容 column 输出&#xff1a;当前列的信息 event 输出&#xff1a;当前事件 …

BUUCTF-Real-[Flask]SSTI

目录 漏洞描述 模板注入漏洞如何产生&#xff1f; 漏洞检测 漏洞利用 get flag ​编辑 漏洞描述 Flask框架&#xff08;jinja2&#xff09;服务端模板注入漏洞分析&#xff08;SSTI&#xff09; Flask 是一个 web 框架。也就是说 Flask 为您提供工具、库和技术来允许您构…

三、数据背后的二进制

文章目录 数据背后的二进制1.1 整数的二进制表示与位运算1.1.1 正整数的二进制表示1.1.2 负整数的二进制表示 1.2 原码、反码、补码1.2.1 机器数和机器数的真值1.2.2 原码, 反码, 补码的基础概念和计算方法1.2.3 为何要使用原码、反码和补码1.2.4 补码计算原理 1.3 小数的二进制…

2024美赛数学建模C题思路分析 - 网球的动量

1 赛题 问题C&#xff1a;网球的动量 在2023年温布尔登绅士队的决赛中&#xff0c;20岁的西班牙新星卡洛斯阿尔卡拉兹击败了36岁的诺瓦克德约科维奇。这是德约科维奇自2013年以来首次在温布尔登公开赛失利&#xff0c;并结束了他在大满贯赛事中历史上最伟大的球员之一的非凡表…

Vue实现公告循环横向播报组件

一、代码组件 注意&#xff1a;当公告字数很少时会固定不动&#xff0c;当字数达到最大宽度时&#xff0c;则会循环播报 <template><div class"TopCard"><!-- 小喇叭 --><div style"width: 70px"><notify style"width: 2…

【Qt+MSVC2017_64bit +Cmake新建项目编译出错】

项目场景&#xff1a; 提示&#xff1a;这里简述项目相关背景&#xff1a; 项目新电脑环境配置 QtMSVC2017_64bit Cmake新建项目编译出错 问题描述 提示&#xff1a;这里描述项目中遇到的问题&#xff1a; QtMSVC2017_64bit Cmake新建项目编译出错 Running C:\Program Fil…

【Boost】:parser代码的基本结构(二)

parser代码的基本结构 一.总体概述二. EumeFile的实现三.ParserHtml的实现四.SaveHtml实现五.完整源代码 打开parser.cc,用vscode或者vim都行。 一.总体概述 首先递归式的把文件名和路径读入一个数组内&#xff0c;接着把数组内的每一个数据按照一定的格式进行划分&#xff0c;…

Flask框架开发学习笔记《5》简易服务器代码

Flask框架开发学习笔记《5》 Flask是使用python的后端&#xff0c;由于小程序需要后端开发&#xff0c;遂学习一下后端开发。 简易服务器代码 接口解析那一块很关键&#xff0c;学后端服务器这一块&#xff0c;感觉主要就是学习相应地址的接口怎么处理。 然后写清楚每个地址…

springboot 文件下载

前言 文件下载: 将服务器某个资源文件下载到用户本地计算机过程称之为文件下载 用户通过浏览器访问页面&#xff0c;点击链接之后&#xff0c;就能从服务器下载本地中。 具体思路&#xff1a; a.确定项目中哪些资源可以被下载 aa.txt 用户须知.doc … b.将可以被下载资源放入服…

MySQL篇----第三篇

系列文章目录 文章目录 系列文章目录前言一、InnoDB与MyISAM的区别二、索引三、常见索引原则有前言 前些天发现了一个巨牛的人工智能学习网站,通俗易懂,风趣幽默,忍不住分享一下给大家。点击跳转到网站,这篇文章男女通用,看懂了就去分享给你的码吧。 一、InnoDB与MyISAM…

上海市七普分乡、镇、街道数据, Shp、excel格式,多年龄段人口分布数据均有

基本信息. 数据名称: 上海市七普分乡、镇、街道数据 数据格式: Shp、excel 数据几何类型: 面 数据坐标系: WGS84 数据时间&#xff1a;2020年 数据来源&#xff1a;网络公开数据 数据字段&#xff1a; 序号字段名称字段说明1zrks总人口数&#xff08;人&#xff09;…

【类和对象】4

日期类的拓展 c语言中的printf函数只能打印内置类型&#xff0c;为了弥补这一不足&#xff0c;c利用运算符重载可以打印自定义类型。 void operator<<(ostream&out);//声明在date.h中void Date::operator<<(ostream& out)//定义在date.cpp中 {out<<…