ELK getting started notes

Getting started notes on ELK (Elasticsearch, Logstash, Kibana).

Logstash - collect and filter logs

Installation:

  1. logstash-xxx.tar.gz or logstash-xxx.zip
    • extract the package
    • prepare a xxx.conf file
    • cd logstash-xxx
      • bin/logstash agent -f xxx.conf
      • nohup bin/logstash -f xxx.conf &
      • bin/logstash -e ‘input { stdin {} } output { stdout { codec => rubydebug } }’
  2. deb/rpm package or apt-get/yum install => recommended
    • prepare xxx.conf under /etc/logstash/conf.d
      • service logstash start/stop/restart/configtest/status
      • /etc/init.d/logstash start/stop/restart/status

Conf file

  input {
    xxx {}
    ...
  }

  filter {
    yyy {}
    ...
  }

  output {
    zzz {}
    ...
  }

Common plugins

Plugin management

  • cd logstash-xxx
  • bin/plugin install/uninstall/update/list
  • bin/plugin install xxx (from rubygems)
  • bin/plugin install /path/to/xxx-plugin.gem (from local gem)

Logging

  • /var/log/logstash/logstash.err
  • /var/log/logstash/logstash.log
  • /var/log/logstash/logstash.stdout

Tips

By default logstash is running under ‘logstash’ user and it will encounter permission issues when accessing some system files such as syslog:

  $ usermod -a -G adm logstash

or

  $ setfacl -m u:logstash:r /var/log/syslog

Docs

DSL

  • Logstash::Event
  • Reference field: [field], [field][field], …
  • Data value type:
    • bool: true or false
    • string: “hello,logstash”
    • number: 514
    • array: [“input”, “output”]
    • hash: {“codec” => “rubydebug”}
  • Conditionals and expression:
    • ==,!=,<,>,<=,>=
    • =~ (regex match), !~ (regex nonmatch)
    • in, not in
    • and, or, nand, xor
    • (), !()

Elasticsearch - index and store logs

Installation

  1. elasticsearch-xxx.tar.gz or elasticsearch-xxx.zip
    • extract the package
    • cd elasticsearch-xxx && bin/elasticsearch (-d for daemon process)
    • curl -X GET http://localhost:9200
  2. deb/rpm package or apt-get/yum install => recommended
    • edit config if necessary: vi /etc/elasticsearch/elasticsearch.yml
    • service elasticsearch start/stop/restart/status

Logging

  • /var/log/elasticsearch/elasticsearch.log

Docs

Kibana - view and search logs

Installation

  • download kibana-xxx.tar.gz or kibana-xxx.zip
  • extract the package
  • cd kibana-xxx/config/ && vi kibana.yml
  • set “elasticsearch.url” (e.g. http://localhost:9200)
  • cd kibana-xxx/ && nohup bin/kibana &
  • open browser to: http://localhost:5601

Logging

  • kibana-xxx/nohup.out

Docs

Reference links

devops logstash elasticsearch kibana