Numerical converter on extractor not working?

Graylog Central (peer support)
1

Apologies, I am new and I have tried to search this info out, hoping someone can point me in the right direction as I’m a bit stuck.

New graylog server 4.3.3, elastic/mongodb on the same box.

Creating a dashboard, things are working well, except im having issues with a data extractor and converter I have created. One of my syslog inputs is receiving data from a network device. All im trying to do is create a bytes received/bytes sent graph.

I have a extractor set up using regular expression, on the source field rcvdbyte using ([0-9]+) called Bytes_Received. There is a numeric value converter on it. I also have a second extractor on source field sentbyte using ([0-9]+) called Bytes_Sent. This also has a numeric value converter.

I have a stream set up with rules for this network device only. This stream is using a index I created, shows it has messages under management.

In the dashboard I have created a area chart. Under the metrics I can add Bytes_Received as a Sum (over time as a row) and this works without issue. However if I try to use Bytes_Sent as a Sum (or any mathematical function on it), it fails with a error:

Elasticsearch exception [type=illegal_argument_exception, reason=Field [Bytes_Sent] of type [keyword] is not supported for aggregation [sum]].

Ive tried recreating the extractor and creating it with different options. The extractors regular expression should only recognize numerical values so it shouldnt be grabbing anything else and my understanding is that the converter should “convert” it to a value that is useable by that function.

The fact that one of them works makes me think ive done part of this right or atleast have the right idea.

The graylog server log doesnt give me any additional info (since this appears to be a type issue from elastic that makes sense though), im just not sure where to go from here as im new to this area. Elasticsearch logs dont tell me anything either, so im stuck with the error I have.

Ive done some google searching but havent found anything that has resolved the issue yet.

My best guess based on what I have found is this has something to do with the indices and I may have to go through some of that but I wasnt sure yet and again hoping someone can put me on the right track before I go down the wrong one. (or help me figure out how to get more info)

2

What I read is that you are trying to do something with the source field, but that should contain a IP addres or hostname where data is coming from. You should extract your data from the message.

Maybe putting a complete message here could make help people on this.

3

Hello Arie,

Maybe im misunderstanding, I thought I was extracting the data from the message when I use the extractor in the input?

From the input, I click “Manage Extractors” and then click “Get Started”, I select a message, then select the field I want (in this case sentbyte) and create a regular expression on that field. The window displayed shows that sentbyte is the field selected and I give it a regex, a name and a numerical converter?

4

hello,

Something similar happened like this with another member, The Index template used, there field was set as type [keyword] not for a Numerical field. Check you Index setting for the field Bytes_Sent
This link may shed some light on what i am talking about here…

2 Likes
5

Elastic decided, that the field “Bytes_Sent” is of type “keyword” and not Integer. Therefor you can not aggregate a sum. You will need to change the type of the field in Elastic, as shown in the post linked above.
Afterwards you need to rotate your index and then you will be able to aggregate a sum for new messages :slight_smile:

6

Hello gsmith, ihe,

Thanks for the feedback and the links. This is exactly what I needed

:smiley:

1 Like
7

awesome-yes-will-ferrell

8

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.