Allow empty fields in graylog

Graylog Central (peer support)
1

Is it possible to configure graylog 3.0 to allow adding empty fields to the graylog index?

We are storing output of tlog-rec-session to the elasticsearch using graylog (as part of normal log stream). But unfortunately this software requires “in_txt” field to be stored into the index. This field is in most cases empty (in_txt=""). It is also present in incoming raw message. But it is not shown either in graylog nor in direct elasticsearch query…

Dynamic template for graylog fields should allow empty fields because they are not analyzed anyway, shouldn’t it?

“store_generic” : {
“match” : “*”,
“mapping” : {
“index” : “not_analyzed”
}
}

Any advise is greatly appreciated!

2

create a processing pipeline that writes a 0 into the field if no value is given would be my idea …

3

Thank you, but its important to have this field as it is. Even if it is empty.

4

you need to create a custom mapping … including a null_value that would solve your issue:

https://www.elastic.co/guide/en/elasticsearch/reference/6.8/null-value.html

http://docs.graylog.org/en/3.0/pages/configuration/elasticsearch.html#custom-index-mappings

5

First of all null_value parameter is not supported for field types “string” / “text”.
Secong - if I put empty string directly to the elasticsearch using curl:

curl -XPOST -H “Content-Type: application/json” “https://test:3_secure@test-elk-01.inatec.local:9200/tlog_0/message” -d “{ “in_txt” : “” }”

then I am able to query it from elasticsearch:

curl -XPOST "https://test:3_secure@test-elk-01.inatec.local:9200/tlog_0/_search?pretty"
{
  "took" : 4,
  "timed_out" : false,
  "_shards" : {
    "total" : 3,
    "successful" : 3,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : 1,
    "max_score" : 1.0,
    "hits" : [
      {
        "_index" : "tlog_0",
        "_type" : "message",
        "_id" : "AWv6rPGJJPr8Y22RTGH_",
        "_score" : 1.0,
        "_source" : {
          "in_txt" : ""
        }
      }
    ]
  }
}

But if I am trying insert empty string via graylog using pipeline using, for example set_field(“in_txt”, “”) it just ignores it. I think it even does not send it to elastic. So the question is still there - is graylog able to put empty strings to elasticsearch?

6

But if I am trying insert empty string via graylog using pipeline using, for example set_field(“in_txt”, “”) it just ignores it. I think it even does not send it to elastic. So the question is still there - is graylog able to put empty strings to elasticsearch?

I guess that Graylog can’t - could you please open a feature request for that over in Github.

7

Could you provide me with link to the github project? Is it graylog2-server?

8

yes

9

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.