Rexgex Extractor - for pfsense devices

Graylog Central (peer support)
1

Hi, all, i want to make and regex extrator for Pfsense device. This device give me a raw message like this:

“<134>Mar 7 18:57:53 filterlog: 79,16777216,1469649672,em0,match,pass,out,4,0x0,62,13847,0,DF,6,tcp,60,1.1.1.1,2.2.2.2,54787,80,0,S,3818924332,5840,mss;sackOK;TS;nop;wscale”

For example: i want to get : 2.2.2.2 value (position number: 20 of the message) in the csv string.

Is possible?
Thanks.

2

Hey @monchito,

take a look at the CSV To Fields converter.
Steps:

  1. Go to the Extractor page of your Input, click Get Started.
  2. Create an Copy Input extractor on your message field, that copies to itself.
  3. At the bottom add and configure a CSV To Fields converter.

Hint: You maybe want to use a GROK Extractor first to filter out <134>Mar 7 18:57:53 filterlog: before your actual csv. Something like filterlog: ${GREEDYDATA:csvString}

If you only need some specific fields, you could write an GROK Extractor, that marks fields you don’t need as UNWANTED. Look here for more details.

Greetings - Phil

3

Hi Phillipp ! Thanks for reply.

I followed this step:

take a look at the CSV To Fields converter.
Steps:

  1. Go to the Extractor page of your Input, click Get Started.
  2. Create an Copy Input extractor on your message field, that copies to itself.
  3. At the bottom add and configure a CSV To Fields converter.

But i get only the same message. I cant extract anything.

I cant found how extract a message like this, neither Rexgex or Grok because it not have labels
"<134>Mar 7 18:57:53 filterlog: 79,16777216,1469649672,em0,match,pass,out,4,0x0,62,13847,0,DF,6,tcp,60,1.1.1.1,2.2.2.2,"

If were like this (filterlog:79 with label case):
"<134>Mar 7 18:57:53 filterlog: 79,**a:**16777216,**c:**1469649672,**d:**em0,**e:**match,**f:**pass,**g:**out,**h:**4,**i:**0x0,62,13847,0,DF,6,tcp,60,**r:**1.1.1.1,**s:**2.2.2.2,"

regex_value: s:?([0-9.]+)                    result= 2.2.2.2

I think, that i have to do something to count between comas.

Please, tell me if i’m not clear, or maybe i missing something.

Thanks!

4

Hey,

here are two solutions:

Solution 1: Getting all fields:

Two Extractors, first a GROK to remove syslog-header (actually, why is it in there?), second a Copy-Input with csv-converter.

ss (2017-03-09 at 10.29.29).png1678×522 22.8 KB

The GROK Extractor:

The Copy-Input with CSV To Fields converter
Note: In the field names attribute, there are 29 fields needed, since the csv is 29 columns wide. Use appropriate names if they are known for better readability.

This is the result:

Solution 2: Only 20th (n th) field.

Use a Split and Index Extractor, that splits on “,” (comma) and set the target index to 20.

ss (2017-03-09 at 10.37.34).png1079×558 6.04 KB

The result:
ss (2017-03-09 at 10.39.48).png1211×226 12.9 KB

Greetings - Phil

PS: For your convenience: value1,value2,value3,value4,value5,value6,value7,value8,value9,value10,value11,value12,value13,value14,value15,value16,value17,value18,value19,value20,value21,value22,value23,value24,value25,value26,value27,value28,value29

2 Likes
5

Hi Philipp, i’m coming too late i just configured it ! it works like charm !

Thanks for all !