Right after that Graylog started processing logs but they didn’t show up in dashboard. I found there were some errors in logs while trying to parse data:
org.graylog.integrations.ipfix.IpfixException: Missing information element definitions for private enterprise number 29305
org.graylog.integrations.ipfix.IpfixException: Missing information element definitions for private enterprise number 45346
Then I included json files for translating incoming logs for both velocloud (45346 ) and ipfix (29305):
At this moment I am encountering the following error in graylog.log:
2021-05-31T07:25:01.846Z ERROR [DecodingProcessor] Unable to decode raw message RawMessage{id=4ff8a630-c1e1-11eb-a4f5-005056919081, journalOffset=44413432, codec=ipfix, payloadSize=1817, timestamp=2021-05-31T07:25:01.843Z, remoteAddress=/172.23.9.132:54112} on input <60af6f3b3f1dd3671d48e2fc>.
2021-05-31T07:25:01.846Z ERROR [DecodingProcessor] Error processing message RawMessage{id=4ff8a630-c1e1-11eb-a4f5-005056919081, journalOffset=44413432, codec=ipfix, payloadSize=1817, timestamp=2021-05-31T07:25:01.843Z, remoteAddress=/172.23.9.132:54112}
java.lang.NullPointerException: null
at org.graylog.integrations.ipfix.IpfixParser.parseDataSet(IpfixParser.java:338) ~[?:?]
at org.graylog.integrations.ipfix.codecs.IpfixCodec.lambda$decodeMessages$3(IpfixCodec.java:206) ~[?:?]
at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193) ~[?:1.8.0_282]
at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1384) ~[?:1.8.0_282]
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:482) ~[?:1.8.0_282]
at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:472) ~[?:1.8.0_282]
at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708) ~[?:1.8.0_282]
at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) ~[?:1.8.0_282]
at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:566) ~[?:1.8.0_282]
at org.graylog.integrations.ipfix.codecs.IpfixCodec.decodeMessages(IpfixCodec.java:212) ~[?:?]
at org.graylog2.shared.buffers.processors.DecodingProcessor.processMessage(DecodingProcessor.java:147) ~[graylog.jar:?]
at org.graylog2.shared.buffers.processors.DecodingProcessor.onEvent(DecodingProcessor.java:90) [graylog.jar:?]
at org.graylog2.shared.buffers.processors.ProcessBufferProcessor.onEvent(ProcessBufferProcessor.java:90) [graylog.jar:?]
at org.graylog2.shared.buffers.processors.ProcessBufferProcessor.onEvent(ProcessBufferProcessor.java:47) [graylog.jar:?]
at com.lmax.disruptor.WorkProcessor.run(WorkProcessor.java:143) [graylog.jar:?]
at com.codahale.metrics.InstrumentedThreadFactory$InstrumentedRunnable.run(InstrumentedThreadFactory.java:66) [graylog.jar:?]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_282]
It looks like your question has remained unanswered for a few days. Let’s move it to “Daily Challenges” where it’s more likely to get noticed in the community.
In the meanwhile, you may want to check to ensure that you’re configured to send data to the correct port
Sorry for that late responce. I dont have much knoweldge of this type of IPFIX input. What I would like to do is establish where the problem is at (i.e. Dashboard, configuration issue, firewall, permission issue, etc…). I need to ask some question to understand what is going on.
Do you see log/s from the input when you click “Show received messages”?
If you do not see messages on that input, did you try setting the input to default, meaning without
a definition configured? If so, when adding your ipfix_definition_path configuration have you use just one definition see if that helps?
When the Input is changed to without any definitions paths I get the following error in graylog logs:
Missing information element definitions for private enterprise number 45346
Missing information element definitions for private enterprise number 29305
which requires file definitions for both velocloud and ipfix.
When both definition files are in place I get
2021-06-09T07:31:57.281Z ERROR [DecodingProcessor] Unable to decode raw message RawMessage{id=c54e98d0-c8f4-11eb-a11d-005056919081, journalOffset=54629489, codec=ipfix, payloadSize=1742, timestamp=2021-06-09T07:31:57.277Z, remoteAddress=/172.23.9.132:46012} on input <60af6f3b3f1dd3671d48e2fc>.
2021-06-09T07:31:57.281Z ERROR [DecodingProcessor] Error processing message RawMessage{id=c54e98d0-c8f4-11eb-a11d-005056919081, journalOffset=54629489, codec=ipfix, payloadSize=1742, timestamp=2021-06-09T07:31:57.277Z, remoteAddress=/172.23.9.132:46012}
java.lang.NullPointerException: null
I tried to use accept messages on RAW/Plaintext UDP" Input. Messages were there but they looked as follows:
This is the only one input where I experience issues. All others work fine. Time is correct on both sides.
I’ve also tried Netflow Input as well but it’s not compatible with IPFIX 10 version:
2021-06-10T07:04:30.445Z ERROR [NetFlowCodec] Error parsing NetFlow packet <1a20f680-c9ba-11eb-a11d-005056919081> received from <172.23.9.132:51639>
org.graylog.plugins.netflow.flows.InvalidFlowVersionException: Invalid NetFlow version 10
Hello,
Maybe I overlooked the past post, but have you tried just one definition?
I know you said you put two definition for velocloud and ipfix. Are these two different devices or one?
If these are two different devices try testing just one, like IPFIX. Trying to eliminate all possibilities of why this is not working for you.
At this point I’m only offering suggestion. I was looking around for a device that sends IPFIX logs here to test those configurations, but I could not find one.
Sorry I can’t be more help
This is the one device but somehow keeps sending packets including both velocloud and ipfix. I’ve tried to use the only one definition for IPFIX, no luck. Thank you for your help anyway.
