Installation guide for Elasticsearch, Filebeat, and Kibana

Our Goal
The goal is to set up Filebeat to gather syslogs of server, and set up Kibana to visualize the gathered
logs.

Elasticsearch: Stores all of the logs

Kibana: Web interface for searching and visualizing logs
Filebeat: Installed on client servers that will send their logs to Elasticsearch, Filebeat serves as
a log shipping agent that utilizes the lumberjack networking protocol to communicate with
Elasticsearch

Prerequisites
To complete this you will require sudo access on an Ubuntu 16.04 server or desktop,
If you would prefer to use CentOS instead, check out this tutorial: How To Install ELK on CentOS 7.
The amount of CPU, RAM, and storage that your ELK Server will require depends on the volume of
logs that you intend to gather. For this we will be using with the following specs for our ELK Server:

OS: Ubuntu 16.04

RAM: 4GB
CPU: 2

Install Java 8
Elasticsearch and Filebeat require Java, so we will install that now. We will install a recent version of
Oracle Java 8 because that is what Elasticsearch recommends. It should, however, work fine with
OpenJDK, if you decide to go that route.

Add the Oracle Java PPA to apt:

sudo add-apt-repository -y ppa:webupd8team/java

Update your apt package database:

sudo apt-get update

Install the latest stable version of Oracle Java 8 with this command (and accept the license agreement
that pops up):
sudo apt-get -y install oracle-java8-installer

Now that Java 8 is installed, let's install ElasticSearch.

Install Elasticsearch with .tar.gz
Download and install the .tar.gz package
The .tar.gz archive for Elasticsearch v5.6.3 can be downloaded and installed as follows:
wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-
5.6.3.tar.gz
tar -xzf elasticsearch-5.6.3.tar.gz
cd elasticsearch-5.6.3/

Elasticsearch is now installed. Let's edit the configuration:

sudo nano elasticsearch/config/elasticsearch.yml

network.host: localhost

Save and exit elasticsearch.yml.

Running Elasticsearch from the command line

Elasticsearch can be started from the command line as follows:

./bin/elasticsearch

Install Kibana with .tar.gz

Download and install the Linux 64-bit package

The 64-bit Linux archive for Kibana v5.6.3 can be downloaded and installed as follows:
wget https://artifacts.elastic.co/downloads/kibana/kibana-5.6.3-linux-x86_64.tar.gz

tar -xzf kibana-5.6.3-linux-x86_64.tar.gz

cd kibana-5.6.3-linux-x86_64/

Running Kibana from the command line

Kibana can be started from the command line as follows:

./bin/kibana
Install Filebeat with .tar.gz
Download and install the Linux 64-bit package

The 64-bit Linux archive for filebeat-5.0.2 can be downloaded and installed as follows:

curl -L -O https://artifacts.elastic.co/downloads/beats/filebeat/filebeat-5.0.2-
darwin-x86_64.tar.gz

tar xzvf filebeat-5.0.2-darwin-x86_64.tar.gz

Filebeat is now installed. Let's edit the configuration:

sudo nano filebeat/filebeat.yml

- input_type: log
# Paths that should be crawled and fetched. Glob based paths.
paths:
- /var/log/*.log
output.elasticsearch:
# Array of hosts to connect to.
hosts: ["localhost:9200"]

Running Filebeat from the command line

filebeat can be started from the command line as follows:

./bin/filebeat filebeat.yml