CentOS7 部署ElastiFlow网络流量分析平台

(图片可点击放大查看)

(图片可点击放大查看)

(图片可点击放大查看)

本文参考如下链接完成

https://docs.elastiflow.com/docs/install_linux
https://cloud.tencent.com/developer/article/1648854
https://blog.csdn.net/weixin_43838503/article/details/122432963
https://blog.51cto.com/coolsky/3190806

(图片可点击放大查看)

条件准备

• 1、host-sflow agent

https://github.com/sflow/host-sflow/releases/download/v2.0.25-3/hsflowd-centos7-2.0.25-3.x86_64.rpm

• 2、ELK的安装包

elasticsearch-7.17.2-x86_64.rpm
kibana-7.17.2-x86_64.rpm
logstash-7.17.2-x86_64.rpm

具体安装部署步骤如下

一、调整相关的内核参数并关闭防火墙

sed -i 's/enable/disabled/g' /etc/selinux/config
setenforce 0
hostnamectl set-hostname elastiflow
echo "vm.max_map_count=262144" | sudo tee /etc/sysctl.d/70-elasticsearch.conf > /dev/null
echo -e "net.core.netdev_max_backlog=4096nnet.core.rmem_default=262144nnet.core.rmem_max=67108864nnet.ipv4.udp_rmem_min=131072nnet.ipv4.udp_mem=2097152 4194304 8388608" |  tee /etc/sysctl.d/60-net.conf > /dev/null
 sysctl -w vm.max_map_count=262144 &&   sysctl -w net.core.netdev_max_backlog=4096 &&   sysctl -w net.core.rmem_default=262144 &&    sysctl -w net.core.rmem_max=67108864 &&    sysctl -w net.ipv4.udp_rmem_min=131072 &&   sysctl -w net.ipv4.udp_mem='2097152 4194304 8388608'
 systemctl stop firewalld.service &&  systemctl disable firewalld.service

(图片可点击放大查看)

(图片可点击放大查看)

二、安装JDK环境并安装ELK

 yum install java-openjdk-devel java-openjdk

(图片可点击放大查看)

(图片可点击放大查看)

 rpm -ivh logstash-7.17.2-x86_64.rpm 
 rpm -ivh elasticsearch-7.17.2-x86_64.rpm 
 rpm -ivh kibana-7.17.2-x86_64.rpm

(图片可点击放大查看)

 systemctl daemon-reload
 systemctl enable elasticsearch.service
 systemctl enable kibana.service
 systemctl enable logstash.service

(图片可点击放大查看)

三、Elasticsearch和Kibana配置文件修改

 vim /etc/elasticsearch/elasticsearch.yml 

(图片可点击放大查看)

 vim /etc/elasticsearch/jvm.options

(图片可点击放大查看)

 vim /etc/kibana/kibana.yml

(图片可点击放大查看)

(图片可点击放大查看)

四、elastiflow安装包解压

https://codeload.github.com/robcowart/elastiflow

(图片可点击放大查看)

mv elastiflow-master elastiflow
cp -a elastiflow/logstash/elastiflow/. /etc/logstash/elastiflow/
cp -a elastiflow/logstash.service.d/. /etc/systemd/system/logstash.service.d/

(图片可点击放大查看)

vim /etc/logstash/pipelines.yml 
 修改添加成如下
#- pipeline.id: main
#  path.config: "/etc/logstash/conf.d/*.conf"
- pipeline.id: elastiflow
  path.config: "/etc/logstash/elastiflow/conf.d/*.conf"
  pipeline.workers: 4
  
  

(图片可点击放大查看)

(图片可点击放大查看)

 chown -R logstash:logstash /etc/logstash/elastiflow
 chown -R logstash:logstash /etc/logstash/pipelines.yml

 vi /etc/logstash/jvm.options 
 vim /etc/logstash/startup.options
 

(图片可点击放大查看)

(图片可点击放大查看)

(图片可点击放大查看)

/usr/share/logstash/bin/logstash-plugin install logstash-codec-sflow 
/usr/share/logstash/bin/logstash-plugin install logstash-codec-netflow 
/usr/share/logstash/bin/logstash-plugin install logstash-input-udp 
/usr/share/logstash/bin/logstash-plugin install logstash-input-tcp 
/usr/share/logstash/bin/logstash-plugin install logstash-filter-dns 
/usr/share/logstash/bin/logstash-plugin install logstash-filter-geoip 
/usr/share/logstash/bin/logstash-plugin install logstash-filter-translate

(图片可点击放大查看)

(图片可点击放大查看)

/usr/share/logstash/bin/system-install

(图片可点击放大查看)

五、启动服务

systemctl restart elasticsearch.service 
systemctl restart kibana.service
systemctl restart logstash.service

(图片可点击放大查看)

六、Kibana配置

导入elastiflow.kibana.7.8.x.ndjson到kibana 

(图片可点击放大查看)

(图片可点击放大查看)

修改如下kibana中的配置 

(图片可点击放大查看)

(图片可点击放大查看)

七、Linux服务器安装及配置hsflow

rpm -ivh hsflowd-centos7-2.0.25-3.x86_64.rpm 

vim /etc/hsflowd.conf
添加如下配置
    sampling = 1
    #截取包大小为256
    headerBytes = 256
    #设置收集器地址和端口
    collector { ip=192.168.31.189 udpport=6343 }
    #设置采样的网卡
    pcap { dev = ens33 }

systemctl enable hsflowd
systemctl start hsflowd
systemctl status hsflowd

(图片可点击放大查看)

(图片可点击放大查看)

(图片可点击放大查看)

(图片可点击放大查看)

(图片可点击放大查看)

(图片可点击放大查看)

(图片可点击放大查看)

九、Tips

1、交换机配置sflow暂未进行测试 

后续有环境会可以进行测试

2、若logstash启动失败 

需要/var/log/logstash/logstash-plain.log 和journalctl -xe -u logstash进行排错

3、上面的kibana与ES未配置安全认证 后续有时间再做调整