Outbound Proxy settings

So I have a few problems with the AWS Plugin and configuring an outbound proxy.

I have upgraded to GrayLog 2.2.3 (from 2.1.3) with AWS plugin 1.3.2 and have enabled the proxy setting in the configuration that read the server.conf file line http_proxy_uri.

After some figuring out of the actual URLs the AWS plugin is using and the permission required I seem to be connecting to the Kineses flowlog streams I created from the AWS Plugin documentation.

Now I can see in the server.log that the plugin is encountering a parse error when trying to ingest the flowlogs from an AWS ELB:

[27]: index [graylog_5], type [message], id [8e95caa1-28e6-11e7-a325-06831b6fd7b7], message [MapperParsingException[failed to parse [protocol]]; nested: NumberFormatException[For input string: "TCP"];]
[28]: index [graylog_5], type [message], id [8e95caa2-28e6-11e7-a325-06831b6fd7b7], message [MapperParsingException[failed to parse [protocol]]; nested: NumberFormatException[For input string: "ICMP"];]

I haven’t modified the ELB logging format so I would have expected the Plugin to be able to parse that data. Has anyone got an tips that might help resolve this issue?

You have to make sure that the protocol message field is always a string.

You can use a custom Elasticsearch index mapping for this purpose: http://docs.graylog.org/en/2.2/pages/configuration/elasticsearch.html#custom-index-mappings

Hi @jochen,

Thanks for that so “current write-active index is graylog_5” and if I do this:

  "template": "graylog_*",
  "mappings" : {
    "message" : {
       "properties" : {
        "protocol" : {
          "type" : "string"

curl -X PUT -d @'graylog-custom-mapping.json' 'http://localhost:9200/_template/graylog-custom-mapping?pretty'

So from the documentation that looks like it will only effect new indexes and not the current index. When I look at the mapping for ‘graylog_5’ it has not changed so how to move forward?

This is correct.

You can manually trigger an index rotation in the web interface on the System → Index Sets page.

Thanks @jochen - rolling the index did indeed pick up the new mapping plus some extra aws properties.

@jochen - just as an after thought do we have a way of finding out exactly which AWS permissions the plugin actually needs?

I have added some very permissive IAMs policies but would like to tighten them. I could do it the old fashioned way and reduce to nil then add them up as the log complains about them but…

The required IAM permissions are listed in the plugin’s README file:

Thanks @jochen - I hadn’t got that far down the guide.