Help Needed with Graylog Pipeline - Conversion Issue

Graylog Central (peer support)
1

Hi everyone,

I’m encountering an issue in Graylog with a pipeline that has two rules:

Stage 6

rule "rtpengine mos"
when
  contains(
  value: to_string($message."message"),
  search: "Average MOS"
)
then
  let gl2_fragment_grok_results = grok(
  pattern: "%{SYSLOGTIMESTAMP} %{SYSLOGHOST} %{SYSLOGPROG}: %{WORD:loglevel}: \\[%{DATA:call_id}(@%{DATA:cloud_ip}(:%{POSINT})?)?\\]:( \\[%{WORD}?\\])? ------ Average MOS %{BASE16FLOAT:average_mos}, lowest MOS %{BASE16FLOAT:lowest_mos} \\(at %{SECOND:start}\\), highest MOS %{BASE16FLOAT:highest_mos} \\(at %{SECOND:start2}\\)",
  value: to_string($message."message")
);
set_fields(
  fields: gl2_fragment_grok_results
);
end

Stage 7

rule "convert average_mos to numeric"
when
  has_field("average_mos")
then
  let averageMosString = to_string($message.average_mos);
  let averageMosNumeric = to_double(averageMosString);
  set_field("average_mos_num", averageMosNumeric);
end

Summary:

In Stage 6, the variable average_mos is created as a string. In Stage 7, average_mos is copied to a new field average_mos_num, which is then supposed to be converted to a double.

However, the issue I’m facing is that average_mos_num remains a string instead of being converted to a double.

Has anyone experienced a similar issue or have any suggestions on how to resolve this? Your help would be greatly appreciated!

Thank you!

2

Sounds like an issue with dynamic field types.
If you rotate the index, does the problem persist?

3

Under “Configure Filebeat Field Types,” I select the field name and change the type, and I, of course, check the box for immediate rotation. However, when I search for the field name under /search/Fields, it still shows the type as STRING!

4

If I change the field name from:average_mos_num -> averagemosnum
,then the type is recognized correctly.

5

Changing the name is another way of dealing with wrong type mapping.
Since the name is new, the type is newly assigned.

6

If I create a new variable with the name ‘average_num’, the type remains an integer. If I create the variable ‘averagenum’, then the type can be selected.

7

On the left edge of the search screen is a Button that is labeled “X1”. Click it to see all the currently known field names and their type.
My guess is that average_num was already used. Double-check that you are using a previously unknown name.
I’d be very surprised if there was an issue with the underscore in a name, since it is so widely used.

image
image609×638 66.9 KB

8

I have now created a completely new variable and now it seems to be working, very strange, but thanks :slight_smile: