pfSense Pipeline

[branden]

I am new to Graylog, and thought i understood how this was going to work before attempting to implement, but something is not working… or i misunderstood its use case.

I have a message
filterlog:215,,,1463078993,ovpns1,match,pass,in,4,0x0,,128,6315,0,none,17,tcp,63,192.168.1.22,192.168.1.96,63466,53,43

And i am trying to parse it through a pipeline, that i got from here https://github.com/trunet/graylog-pfsense-pipeline/blob/master/rule_pfsense_filterlog_ipv4_tcp.txt

rule "pfSense filterlog: IPv4 TCP"
when
  regex("^.*,(in|out),4,.*,(?i)tcp,.*$", to_string($message.message)).matches == true
then
  // needs ,0 to ensure trailing strings not being splitted if empty
  let msg = concat(to_string($message.message), ",0");
  let m = split(",", msg);
  set_field("RuleNumber", m[0]);
  set_field("SubRuleNumber", m[1]);
  set_field("Anchor", m[2]);
  set_field("Tracker", m[3]);
  set_field("Interface", m[4]);
  set_field("Reason", m[5]);
  set_field("Action", m[6]);
  set_field("Direction", m[7]);
  set_field("IPVersion", m[8]);
  set_field("TOS", m[9]);
  set_field("ECN", m[10]);
  set_field("TTL", m[11]);
  set_field("ID", m[12]);
  set_field("Offset", m[13]);
  set_field("Flags", m[14]);
  set_field("ProtocolID", m[15]);
  set_field("Protocol", m[16]);
  set_field("Length", m[17]);
  set_field("SourceIP", m[18]);
  set_field("DestIP", m[19]);
  set_field("SourcePort", m[20]);
  set_field("DestPort", m[21]);
  set_field("DataLength", m[22]);
  set_field("TCPFlags", m[23]);
  set_field("Sequence", m[24]);
  set_field("ACK", m[25]);
  set_field("Window", m[26]);
  set_field("URG", m[27]);
  set_field("Options", m[28]);
end

I thought what that would do is create fields that i could then use… but its not working. Should i be using an extractor instead?

[jan (Jan Doberstein)]

what issue are you facing?

[branden]

Sorry, I guess that would have made sense to put in the original post. I have the stage 0 pipeline, with the rule for TCP applied to the stream. However, when I look at the stream the pipeline doesn’t seem to be creating these fields. I still only have the one message field.

My goal is to parse the CSV into usable fields and make the message more readable