uniq
作用：去重，一般与sort搭配使用，单用uniq，是合并相邻两行相同内容
参数： -c #统计重复行的次数
eg：[root@oldboyedu ~]# cat uniq.txt
oldboy
lidao
lidao
lidao
lidao
oldboy
oldboy
lidao
oldboy
pldboy
[root@oldboyedu ~]# uniq uniq.txt
oldboy
lidao
oldboy
lidao
oldboy
pldboy
[root@oldboyedu ~]# uniq -c uniq.txt1 oldboy4 lidao2 oldboy1 lidao1 oldboy1 pldboy
[root@oldboyedu ~]# sort uniq.txt | uniq -c 5 lidao4 oldboy1 pldboy#因其只能合并相邻两行的，故常与sort搭配使用
awk
作用：取列
eg：[root@oldboyedu ~]# cat awk.txt
oldboy 18 192.168.1.1
oldboy1 19 196.168.1.2
oldboy2 15 196.168.1.3
oldboy3 15 196.168.1.3
oldboy4 15 196.168.1.3
oldboy5 15 196.168.1.3
oldboy6 15 196.168.1.3
oldboy7 15 196.168.1.3
#取出第一列
[root@oldboyedu ~]# awk '{print $1}' awk.txt
oldboy
oldboy1
oldboy2
oldboy3
oldboy4
oldboy5
oldboy6
oldboy7
#取出第二列
[root@oldboyedu ~]# awk '{print $2}' awk.txt
18
19
15
15
15
15
15
15
#取出最后一列，去倒数几列就是NF-（n-1），切记要加小括号
[root@oldboyedu ~]# awk '{print $NF}' awk.txt
192.168.1.1
196.168.1.2
196.168.1.3
196.168.1.3
196.168.1.3
196.168.1.3
196.168.1.3
196.168.1.3secure日志分析
1.过滤出failed password，失败登录，取行，按页查看
[root@oldboyedu ~]# grep -i 'failed password' secure-20161219 | less
Dec 11 03:49:25 localhost sshd[27087]: Failed password for invalid user support from 123.31.34.190 port 55493 ssh2
Dec 11 03:49:27 localhost sshd[27092]: Failed password for invalid user admin from 123.31.34.190 port 56193 ssh2
Dec 11 03:49:30 localhost sshd[27094]: Failed password for invalid user uucp from 123.31.34.190 port 56223 ssh2
Dec 11 04:37:36 localhost sshd[28081]: Fail:
2.提取出这行中的ip的列，取列，按页查看
[root@oldboyedu ~]# grep -i 'failed password' secure-20161219 | awk '{print $(NF-3)}' | less
123.31.34.190
123.31.34.190
123.31.34.190
221.126.233.134
221.126.233.134
221.126.233.134
110.45.145.222
110.45.145.222
110.45.145.222
3.对IP进行排序，相同IP在一起，按页查看
[root@oldboyedu ~]# grep -i 'failed password' secure-20161219 | awk '{print $(NF-3)}'| sort |less
103.237.144.68
103.27.238.252
103.27.238.252
103.27.238.252
103.27.238.252
103.27.238.252
103.27.238.252
103.27.238.252
103.27.238.252
4.去掉重复统计次数。去重并统计次数。uniq -c 按页查看
[root@oldboyedu ~]# grep -i 'failed password' secure-20161219 | awk '{print $(NF-3)}'| sort | uniq -c | less1 103.237.144.6812 103.27.238.2529 110.45.145.2223206 111.73.46.15621 111.73.46.16215 111.73.46.3618065 112.85.42.10317163 112.85.42.10717163 112.85.42.124
5.对次数排序并取出前5个
[root@oldboyedu ~]# grep -i 'failed password' secure-20161219 | awk '{print $(NF-3)}'| sort | uniq -c |sort -rnk1 | head -568652 218.65.30.2534326 218.65.30.5321201 218.87.109.15418065 112.85.42.10317164 112.85.42.99
access日志分析
1.提取出这行中的ip的列，取列，按页查看
[root@oldboyedu ~]# awk '{print $1}' access.log | less
101.226.61.184
101.226.61.184
27.154.190.158
101.226.61.184
114.94.29.165
101.226.61.184
218.79.64.76
101.226.61.184
218.79.64.76
2.对IP进行排序，相同IP在一起，按页查看
[root@oldboyedu ~]# awk '{print $1}' access.log | sort | less
101.199.108.120
101.199.108.120
101.199.108.120
101.199.108.120
101.199.108.120
101.199.108.120
101.199.108.120
101.199.108.120
101.199.108.120
3.去掉重复统计次数。去重并统计次数。uniq -c 按页查看
[root@oldboyedu ~]# awk '{print $1}' access.log | sort | uniq -c | less20 101.199.108.12056 101.199.108.5021 101.199.108.5120 101.199.108.5256 101.199.108.5620 101.199.112.5344 101.199.112.54160 101.224.50.244501 101.224.5.39
4.对次数排序并取出前5个
[root@oldboyedu ~]# awk '{print $1}' access.log | sort | uniq -c | sort -rnk1 | head -512049 58.220.223.6210856 112.64.171.981982 114.83.184.1391662 117.136.66.101318 115.29.245.13
打包压缩
| 压缩工具 | 说明 |
|---|---|
| tar | 使用最多 |
| gzip | 一般与管道命令搭配 |
| zip/unzip | window，Linux直接传输（Linux，windows默认都支持的格式） |
tar
| tar选项 | 说明 |
|---|---|
| zcvf简写zcf | c create打包 z gzip压缩 v verbose显示过程 f指定压缩包（f放最后） |
| ztvf简写tf | t list 显示压缩包内容 |
| zxvf简写xf | x extract解压 |
| -C | 解压到指定目录 |
eg：创建压缩包，压缩/etc/目录，存放到/backup/下先检查/backup是否存在，存在后再去创建[root@oldboyedu ~]# ls /backup
etc etc.tar.gz etc.zip
[root@oldboyedu ~]# tar zcvf /backup/etc.tar.gz /etc
eg：查看压缩包内容
[root@oldboyedu ~]# tar ztvf /backup/etc.tar.gz
eg:解压压缩包（默认解压到当前目录）
[root@oldboyedu ~]# tar zxvf /backup/etc.tar.gz
eg：解压到/backup-new/
[root@oldboyedu ~]# tar xf /backup/etc.tar.gz -C /backup-new
创建压缩包提示
[root@oldboyedu ~]# tar zcvf /backup/etc.tar.gz /etc
tar: 从成员名中删除开头的“/”
/etc/
/etc/mtab
/etc/fstab
/etc/crypttab
/etc/resolv.conf
...
#出现这个提示是因为tar命令的安全机制，把绝对路径转换为相对路径，解压到当前目录，避免解压成绝对路径，覆盖对应文件目录，造成损失
gzip
作用：原地压缩，不常用
eg：[root@oldboyedu ~]# touch wang.txt
[root@oldboyedu ~]# ls
02-【awk练习文件】access-服务的访问日志-v3.zip
access.log
akw.txt
anaconda-ks.cfg
awk.txt
etc
initial-setup-ks.cfg
secure-20161219
secure-20161219.zip
secure.big
sort
uniq.txt
wang.txt
#压缩
[root@oldboyedu ~]# gzip wang.txt
[root@oldboyedu ~]# ls
02-【awk练习文件】access-服务的访问日志-v3.zip
access.log
akw.txt
anaconda-ks.cfg
awk.txt
etc
initial-setup-ks.cfg
secure-20161219
secure-20161219.zip
secure.big
sort
uniq.txt
wang.txt.gz
#解压
[root@oldboyedu ~]# gzip -d wang.txt.gz
[root@oldboyedu ~]# ls
02-【awk练习文件】access-服务的访问日志-v3.zip
access.log
akw.txt
anaconda-ks.cfg
awk.txt
etc
initial-setup-ks.cfg
secure-20161219
secure-20161219.zip
secure.big
sort
uniq.txt
wang.txt
zip/unzip
主要掌握解压即可。unzip
#压缩 -r是递归打包 -q是不显示打包的详细信息
[root@oldboyedu ~]# zip -rq /backup/etc.zip /etc
#解压unzip
[root@oldboyedu ~]# unzip etc.zip
Linux常见核心文件
| 文件 | 说明 |
|---|---|
| /etc/hostname | 主机名 |
| /etc/os-release | 系统版本信息 |
面试题：你拿到一个Linux后一般做什么？检查，优化
查看与修改主机名
#查看主机信息
[root@oldboyedu ~]# hostnamectl Static hostname: oldboyeduIcon name: computer-vmChassis: vmMachine ID: a96de4ce7d974781baf086e28bea12f6Boot ID: 39af812bab274fc9aff157a40c7a2a2cVirtualization: vmwareOperating System: Kylin Linux Advanced Server V10 (Lance)Kernel: Linux 4.19.90-52.22.v2207.ky10.x86_64Architecture: x86-64
#修改主机名
[root@oldboyedu ~]# hostnamectl set-hostname oldboyedu-hard-stone
#需要查看主机名，不显示其他内容，用hostname
IP
ip a #addr address
ifconfig
版本信息
cat /etc/os-release
内核版本
uname -a 查看所有
uname -r 查看内核
