Monday, August 5, 2019

Installing logstash-filter-tld plugin for newer Logstash versions


The logstash tld plugin (logstash-filter-tld) is used to extract the individual labels of a fqdn. This is great for analysis of e.g. domains from a Domain Generation Algorithm (DGA). Unfortunately this plugin does not work on newer logstash versions. (See https://github.com/logstash-plugins/logstash-filter-tld/pull/10)
This is caused by java version requirements. To make it work, all that is needed is changing that requirement, then (re)building the gem. This is a few simple steps (done on Debian below - substitute with your distros package manager).
On the development machine:
The finalized plugin is now saved as logstash-filter-tld-3.0.3.gem. Copy that to your logstash instances, then install it using:
  • /usr/share/logstash/bin/logstash-plugin install --no-verify ./logstash-filter-tld-3.0.3.gem

For the logstash .conf-file(s) here's an example using the tld plugin:
filter {        

   tld {

        source => "computer_name"

    }


    mutate {

        rename => { "[tld][domain]" => "highest_registered_domain" }

        rename => { "[tld][trd]" => "sub_domain" }

        rename => { "[tld][tld]" => "top_level_domain" }

        rename => { "[tld][sld]" => "parent_domain" }

    }

}
 
 
Take a look at these logstash configurations too: https://github.com/HASecuritySolutions/Logstash

This post was first published on Peerlyst: https://www.peerlyst.com/posts/installing-logstash-filter-tld-plugin-for-newer-logstash-versions-martin-boller

No comments: