Netflow input and pipeline simulator

Graylog Central (peer support)
1

In what format should I paste sample messages in order to test using a netflow input?
We don’t have the original message content as with the syslog input. I tried pasting in netflow data from wireshark capture but without success.

Advice much appreciated.

2

Hey @wdarcey

NETFLOW is not equal SYSLOG and Wireshark PCAP is not NETFLOW.
A test with NETFLOW version 5 is a good idea and works perfectly out of the box.

I use “pmacct” to transfer NETFLOWs. Link: http://www.pmacct.net/

3

Thanks. I will take a look. I’ve since solved the rule error but it will be good to have a method of capturing netflow messages in a format I can paste in to the GL rules simulator.

4

It won’t work like that. Netflow has its own protocol header that you need information about. Put simply, a NETFLOW packet is an IP packet without an IP packet Payload but with status information. You need to create a FLOW probe (pmacct) and send it to a collector (GL NF INPUT). Or you use a flow proxy like nProbe to adjust the flow information.

5

My input is receiving messages OK. I wish to develop some rules to act upon it.

I guess therefore I am looking for the appropriate message format to paste in the ‘Raw message’ field of the Graylog pipeline simulator when testing a netflow input.

I don’t need to capture real netflow messages per se. Just a sample, that I can then edit ad hoc, paste in to the ‘Raw message’ box, and test my pipeline. An example would be very handy!
Or is it the case the simulator cannot be used with netflow inputs.

6

I’ll give you an example from my GL: (It’s IPFIX, the evolution of NETFLOW v9), You see that the messages field won’t get you anywhere. As written, with NETFLOW most of it happens in the header.

{
  "ipDiffServCodePoint": 0,
  "dst_ip": "x.x.x.x",
  "src_ip": "x.x.x.x",
  "src_ip_threat_indicated": false,
  "egressInterface": 0,
  "ingressInterface": 3,
  "dot1qCustomerVlanId": 40,
  "gl2_message_id": "01H3W7EMF20NGDG47SZAAATT8B",
  "octetDeltaCount": 108,
  "message": "Ipfix [x.x.x.x]:161 <> [x.x.x.x]:51425 proto:17 pkts:1 bytes:108",
  "dot1qVlanId": 40,
  "gl2_remote_port": 54951,
  "normalize": "success",
  "gl2_remote_ip": "x.x.x.x",
  "streams": [
    "63ff42747abffb4b188b8649"
  ],
  "dst_ip_is_internal": true,
  "gl2_source_node": "f762ecee-287f-48f2-a1d2-2665444e3296",
  "real_timestamp": "2023-06-26 18:09:09",
  "timestamp": "2023-06-26T16:09:09.000Z",
  "flowStartNanoseconds": "2093-06-25T15:59:10.449744735Z",
  "gl2_source_input": "63ff16ec7ddffb4b188b1e5a",
  "protocol": "UDP",
  "dst_ip_internal_location": "DMZ1",
  "_id": "c86fa511-143b-11ee-9614-2a8df24f1fad",
  "source": "x.x.x.x",
  "dst_port": 51425,
  "src_port": 161,
  "flowEndNanoseconds": "2093-06-25T15:59:10.502675912Z",
  "gl2_accounted_message_size": 581,
  "packetDeltaCount": 1
}

Maybe this will help you?

7

@H077E Many thanks for the example and link.
With the GL syslog input, there’s the option to Store the full original syslog message as full_message. Which can then be used in the simulator.
However there’s no equivalent option in the netflow input config, and if I understand you right, formatting a netflow payload for use in the simulator UI may not be viable.
It’s my first outing with netflow and just exploring at the moment. Thanks again.

8

You’re welcome
Syslog is just a message field with ASCII characters. Other network protocols are more specific.
Much happens in learning! :+1:

9

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.