Logstash azure plugin

Question

I am very new to logstash, and am kind of stuck how to do this part.

I copied logs from azure blob container, and was able to parse via logstash and put the records in elastic search on my local machine. Now I want to change the input so that instead of file, it gets the logs directly from the azure blob storage. I looked for plugins, but did not find anything for azure.

I am hoping I can do something like this in the input. Any ideas how to get this?

input {
  azureblob {
    storage_name => "abcd"
    storage_key => "key"
    container => "logLocation"
  }
}

Answer 1

After playing around, and reading a bunch of stuff, I got it to work on my local windows machine.

Install ruby. Install jruby. command line -> gem install bundle.

Above is needed to build gems

Then I got these files. https://github.com/juliusl/logstash-input-azurewadtable/tree/0.9.2

Replaced lib/logstash/inputs/azurewadtable.rb with https://github.com/mspnp/semantic-logging/blob/elk/ELK/logstash-extension/inputs/azureblob.rb

Replaced all places with azurewadtable with azureblob in all files.

Via command line gem install "localtion_Of_gemSpec" This will give you the gem file.

Logstash\bin\plugin install "location_of_gem_file" Logstash\bin\plugin install --no-verify

Verify a couple of things: Logstash\gemfile has this entry and Logstash\vendor\local_gems has the files

Setup config:

input {
  azureblob {
    storage_account_name => "STORAGE ACCOUNT NAME"
    storage_access_key => "STORAGE ACCESS KEY"
    container => "CONTAINER NAME"
  }

}

Logstash/bin/logstash -f "location_of_conf_File"

And it works :)