IPFIX from Sophos UTM

Graylog Central (peer support)
14

Created the files below based on what I’m seeing in the pcap, using this (https://github.com/elastic/beats/blob/cd4907841d0f5e740cbb08eacb759dda3469911d/x-pack/filebeat/input/netflow/decoder/fields/zfields_assorted.go) from the Elastic Beats repo for reference:

pen21373.json:

{
        "enterprise_number": 21373,
        "information_elements": [
                {
                        "element_id": 4,
                        "name": "mark",
                        "data_type": "unsigned32"
                },
                {
                        "element_id": 6,
                        "name": "conntrack_id",
                        "data_type": "unsigned32"
                }
        ]
}

pen9789.json:

{
        "enterprise_number": 9789,
        "information_elements": [
                {
                        "element_id": 1,
                        "name": "afcProtocol",
                        "data_type": "unsigned16"
                },
                {
                        "element_id": 2,
                        "name": "afcProtocolName",
                        "data_type": "string"
                }
                {
                        "element_id": 4,
                        "name": "flowDirection",
                        "data_type": "unsigned8"
                }

        ]
}

Added both of those to the input (figured out the text box could take multiple entries):

image
image696×113 8.29 KB

And after several minutes, I started seeing messages. But, some appear to be getting dropped due to this error in server.log:

2020-02-07T18:03:11.648-05:00 ERROR [DecodingProcessor] Unable to decode raw message RawMessage{id=0370daf0-49fe-11ea-808a-000c29404c16, journalOffset=394098681, codec=ipfix, payloadSize=807, timestamp=2020-02-07T23:03:11.647Z, remoteAddress=/192.168.0.1:37828} on input <5e38c5e829ccde06888b7552>.
2020-02-07T18:03:11.649-05:00 ERROR [DecodingProcessor] Error processing message RawMessage{id=0370daf0-49fe-11ea-808a-000c29404c16, journalOffset=394098681, codec=ipfix, payloadSize=807, timestamp=2020-02-07T23:03:11.647Z, remoteAddress=/192.168.0.1:37828}
java.lang.IndexOutOfBoundsException: readerIndex(126) + length(4) exceeds writerIndex(128): UnpooledHeapByteBuf(ridx: 126, widx: 128, cap: 128/128)
        at io.netty.buffer.AbstractByteBuf.checkReadableBytes0(AbstractByteBuf.java:1477) ~[graylog.jar:?]
        at io.netty.buffer.AbstractByteBuf.checkReadableBytes(AbstractByteBuf.java:1463) ~[graylog.jar:?]
        at io.netty.buffer.AbstractByteBuf.readBytes(AbstractByteBuf.java:896) ~[graylog.jar:?]
        at io.netty.buffer.AbstractByteBuf.readBytes(AbstractByteBuf.java:904) ~[graylog.jar:?]
        at org.graylog.integrations.ipfix.IpfixParser.parseDataSet(IpfixParser.java:430) ~[?:?]
        at org.graylog.integrations.ipfix.codecs.IpfixCodec.lambda$decodeMessages$3(IpfixCodec.java:206) ~[?:?]
        at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193) ~[?:1.8.0_242]
        at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1382) ~[?:1.8.0_242]
        at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:482) ~[?:1.8.0_242]
        at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:472) ~[?:1.8.0_242]
        at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708) ~[?:1.8.0_242]
        at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) ~[?:1.8.0_242]
        at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:566) ~[?:1.8.0_242]
        at org.graylog.integrations.ipfix.codecs.IpfixCodec.decodeMessages(IpfixCodec.java:212) ~[?:?]
        at org.graylog2.shared.buffers.processors.DecodingProcessor.processMessage(DecodingProcessor.java:148) ~[graylog.jar:?]
        at org.graylog2.shared.buffers.processors.DecodingProcessor.onEvent(DecodingProcessor.java:91) [graylog.jar:?]
        at org.graylog2.shared.buffers.processors.ProcessBufferProcessor.onEvent(ProcessBufferProcessor.java:90) [graylog.jar:?]
        at org.graylog2.shared.buffers.processors.ProcessBufferProcessor.onEvent(ProcessBufferProcessor.java:47) [graylog.jar:?]
        at com.lmax.disruptor.WorkProcessor.run(WorkProcessor.java:143) [graylog.jar:?]
        at com.codahale.metrics.InstrumentedThreadFactory$InstrumentedRunnable.run(InstrumentedThreadFactory.java:66) [graylog.jar:?]
        at java.lang.Thread.run(Thread.java:748) [?:1.8.0_242]

Thanks for all the help. -Steve