The previous blog guided you through installing, configuring, and running Suricata as an Intrusion Detection and Intrusion Prevention System. Suricata is a high performance, open-source network analysis and threat detection software. If you have not already read Part 1, we would recommend starting there:
Now that we have Suricata setup, it’s time to install & configure the Elastic Stack so we can visualize and search the Suricata logs in a more efficient way. So, boot up your other Ubuntu machine to begin the installation steps of Elasticsearch.
Download and install the public signing key:
$ wget -qO - https://artifacts.elastic.co/GPG-KEY-elasticsearch | sudo gpg --dearmor -o /usr/share/keyrings/elasticsearch-keyring.gpg
Save the repository definition to /etc/apt/sources.list.d/elastic-8.x.list:
$ echo "deb [signed-by=/usr/share/keyrings/elasticsearch-keyring.gpg] https://artifacts.elastic.co/packages/8.x/apt stable main" | sudo tee /etc/apt/sources.list.d/elastic-8.x.listInstall the Elasticsearch Debian package:
$ sudo apt-get update && sudo apt-get install elasticsearchElasticsearch is installed, so we can begin configuring it. Just like Suricata, Elasticsearch has a yaml file that we will be editing.
$ sudo nano /etc/elasticsearch/elasticsearch.ymlEnter ^W network.host and uncomment the line and type the ip address of your Elastic Stack VM. If you are following the lab's topology it will be 192.168.1.71.
Also, uncomment the http.port line.
Scroll to the Security Auto Configuration section and disable xpack.security.enabled and xpack.security.enrollment.enabled. Disabling the security configurations will allow us to send an HTTP request to make sure Elasticsearch is running.
xpack.security.enabled: false
xpack.security.enrollment.enabled: false
Save and close the file.
Enable the Elasticsearch service.
$ sudo systemctl start elasticsearch.serviceSend a curl request to our Elasticsearch Host with the following command:
$ curl -X GET “192.168.1.71:9200”
Nice! We can see Elasticsearch version 8.6.1 is up and running. Note that you could be on an updated version of Elasticsearch when following this blog.
You can go back into your elasticsearch.yml file and enable the xpack.security.
Add the following line to the xpack.security.http.ssl & change xpack.security.http.ssl setting.
verification_mode: none
The verification_mode setting will not check for a certificate.
Save and close the elasticsearch.yml file. Restart the elasticsearch.service:
$ sudo systemctl restart elasticsearch.serviceNow that we have enabled xpack.security, we need to generate passwords for elastic and kibana_system. We will do this by using the reset-password command. Make sure to copy down the passwords.
$ cd /usr/share/elasticsearch/bin
$ sudo ./elasticsearch-reset-password -u elastic
Do the same for the kibana_system user.
$ sudo ./elasticsearch-reset-password -u kibana_system
Remember the passwords for elastic and kibana_system users as we will be using those later. The kibana_system user will be used for configuring Kibana while elastic user will be used for configuring Filebeat, Auditbeat, and logging into Kibana. If you forget your password, you can use the above command to reset the password to an autogenerated password.
Now that we have our passwords and Elasticsearch setup we can now start configuring Kibana. Kibana should only be installed after installing Elasticsearch. Since we already installed the Elasticsearch package source, we can install the remaining components of the Elastic Stack using apt. For a reminder, we will be installing Kibana on the same machine as Elasticsearch.
Installing & Configuring Kibana
$ sudo apt install kibana
Generate the xpack.security settings and encryptions keys that will be used to store session data in Kibana.
$ cd /usr/share/kibana/bin/
$ sudo ./kibana-encryption-keys generate -q
Copy the output somewhere secure. We will add them to the kibana.yml configuration file.
$ sudo nano /etc/kibana/kibana.ymlGo to the end of the nano file and paste the xpack.security encryption keys.
Keep the kibana.yml open as we will now setup up the networking configurations.
Enter ^/ and type 6. This will take you to line 6 in the file.
Delete the comment for server.port: 5601
Delete the comment for server.host: and replace “localhost” with your Elastic Stack VM ip address. If you followed this lab’s topology, then you would put 192.168.1.71.
Enter ^W and type “elasticsearch.hosts:”. This will take you to the elasticsearch.host line. Uncomment the line and enter the ip address of your Elastic Stack VM.
Enter ^W and type “elasticsearch.username:” This will take you to the lines where we will type that kibana_system password you made. Remember to uncomment the lines. Leave elasticsearch.username default and change elasticsearch.password to the password you created.
Enter ^W and type “elasticsearch.ssl.verificationMode:”. Uncomment the line and type ‘none.’ This will skip the hostname verification when Kibana makes an outbound SSL/TSL connection to Elasticsearch.
Save the file and close.
Start the Kibana service.
$ sudo systemctl start kibana.service
Check your Kibana.service status.
$ sudo systemctl status kibana.service
Now, go to your favorite web browser and check that Kibana is running properly. I will enter 192.168.1.71:5601 to access Kibana.
You can log in with elastic as the username and the password you created for elastic.
Once logged in, we will create a new user account, so we do not have to use the built-in elastic superuser account.
To create roles and users to grant access to Kibana open the main menu, then click Stack Management > Users.
Create a new user with the button in the top right corner and assign the kibana_admin, kibana_system, monitoring_user, & editor roles.
Log out of the current profile and log in with the newly made user account. Currently, we do not have any data to view in Kibana because we have not set up Filebeat & Auditbeat on our Suricata host. Filebeat will be used to ingest the Suricata logs into Kibana and Auditbeat will be used to audit the activities of users and processes our Suricata Host.
Installing & Configuring Filebeat & Auditbeat
Jump back over to your Suricata host to download and install the Public Signing key:
$ wget -qO - https://artifacts.elastic.co/GPG-KEY-elasticsearch | sudo apt-key add -Save the repository definition to /etc/apt/sources.list.d/elastic-8.x.list:
$ echo "deb https://artifacts.elastic.co/packages/8.x/apt stable main" | sudo tee -a /etc/apt/sources.list.d/elastic-8.x.listInstall Filebeat. Information on Filebeat can be found here.
$ sudo apt-get update && sudo apt-get install filebeat
Next, we will configure Filebeat to connect to Elasticsearch and Kibana.
$ sudo nano /etc/filebeat/filebeat.ymlEnter ^W and setup.kibana. Uncomment the line and type your Elastic Stack VM ip address with the Kibana port number (5601). This will connect Filebeat to Kibana to create various SIEM dashboards and processing pipelines in Elasticsearch to organize our Suricata logs.
Scroll down to the Elasticsearch Output section and type in the Elastic Stack VM ip address with the elasticsearch port number. Uncomment the #protocol line since we have https enabled on Elasticsearch. Scroll down to the Elasticsearch username and password and fill it in with the password you created and uncomment both lines. Change the hosts address to include https://192.168.1.71:9200. Lastly, add ssl.verifcation_mode: ‘none.’
Do not close and save the file yet. Filebeat has built-in Suricata modules that we will enable. Filebeat modules simplify the collection, parsing, and visualization of common log formats. You can learn more about all the Filebeat modules here. With Elasticsearch 8.x, there is a bug with importing modules so we will need to import the Suricata module through the .yml file. Here is a thread on the bug.
Scroll up and find the Filebeat modules section or enter ^W and type “Filebeat modules.” We will manually import the Suricata module. The .yml file is very sensitive to spacing and indentation, so make sure to check for extra spaces and tabs when editing the file.
Comment out everything under the Filebeat modules section.
Add the following lines below #reload.period: 10s
- module: suricata
eve:
enabled: true
var.paths: ["/var/log/suricata/eve.json"]
The var.paths is where your logs from Suricata are stored. Filebeat then ships those logs to Kibana for search and data visualizations.
Save and close the file.
Next, we will run the following command to force the Suricata module to run.
$ sudo filebeat setup --pipelines --modules suricata -M "suricata.eve.enabled=true" -c /etc/filebeat/filebeat.yml
Next, we will run the filebeat setup command. The command setups the initial environment, including the index template, ILM policy and write alias, Kibana dashboards, and machine learning jobs. The setup process should take a couple of minutes.
$ sudo filebeat setup -eMake sure to check at the end of the text output that you see suricata-eve pipelines. This verifies that you will now be successfully shipping your Suricata eve.json logs to Kibana once we start the Filebeat service.
Start the Filebeat service. Filebeat will begin sending logs to Elasticsearch that will be organized in Kibana. We are very close to installing and seeing our logs in Kibana after setting up Auditbeat.
$ sudo systemctl start filebeat.service
Now, we will install Auditbeat on our Suricata host. Since we already have the beats repository downloaded, we just need to install Auditbeat with apt. Information on Auditbeat can be found here.
$ sudo apt-get update && sudo apt-get install auditbeat
Open the auditbeat.yml configuration file.
$ sudo nano /etc/auditbeat/auditbeat.ymlThe setup of Auditbeat is the same as Filebeat. Enter ^W and type setup.kibana to get to the right section in the file. Uncomment the line and type in your Elastic Stack VM ip address with the port number.
Scroll down to the Elasticsearch output section. Do the exact same setup that we did for Filebeat.
Save and close the file.
Start Auditbeat. Now any audited changes will be shipped to Kibana for viewing. Auditbeat is good for checking authentication logs.
$ sudo systemctl start auditbeat.service 
Viewing Logs in Kibana
Log back into Kibana using the account you created. Visit 192.168.1.71:5601.
Once logged into Kibana we can explore the Suricata dashboards. In the top search field, type ‘Suricata Events Overview.’ Click on the Events Overview. This will show a summary of all the logged Suricata events.
Success!! We can now visualize our Suricata logs with Kibana. This shows all the current Suricata events in the last 15 minutes. There is a search bar that you can use to specifically query logs if you want to see specific events.
Click on the Alerts text beside the Suricata logo in the top left corner. This will show us any malicious traffic alerts.
You should see the GPL_ATTACK_RESPONSE alert signature which is the alert we generated using the http://testmynids.org/uid/index.html.
Let’s find the Auditbeat logs in Kibana. Click the hamburger menu in the left corner > Security > Explore > Hosts > localhost.
Here you can find authentication logs for successes and failures for your Suricata host. You can test this by typing in a wrong password when logging into your Suricata host.
Kibana has many different features and tools for visualizing your logs. Now, you can explore Kibana and create custom dashboards, rulesets, graphs, and filtering to fit your lab environment.
Conclusion
In this 2-part blog, you’ve learned how to protect your network by shipping Suricata logs to the Elastic Stack SIEM for visualizing, correlating, indexing, and organizing your Suricata logs.
The lab has endless opportunities to expand and gain valuable skills in cybersecurity. It's your turn to customize the lab to your interests and needs. The trend for skilled cybersecurity professionals will keep growing exponentially with the yearly rise in online security threats such as ransomware so it is always worth the time to learn how to secure and protect your data.
Still have questions or want to discuss your Suricata environment reach out to us at CDA. We’d love to discuss how we can help you with your cybersecurity needs!