Greetings,
I use the open version of graylog.
I configured graylog for IPFIX. In the input I see messages received correctly, however pressing on “show received messages” shows nothing. I didn’t understand how to insert the json.
But I have a doubt. Do I need the paid version to configure IPFIX?
OS: ubuntu 22.04
Graylog Version: 4.3.8
Hello && Welcome @marpino
It’s is not a paid version, If you seen messages coming in on the INPUT I would check out time zone on Graylog and/or remote clients. If you expand the Search result from 5 minutes to “All” are you able to see any messages? If not I would check out the log files. If you do not see anything in there then you may need more help which would entail configuration files, log files, etc… to see if the community can spot your issue.
Meaning you should post the details on the input as well as and relevant log events… happy to look…
Greetings,
thanks to both of you for the reply. Yesterday before answering, I tried to search again to be independent, however I did not find anything.
In the graylog log I got this error:
2022-11-23T07:34:22.965Z ERROR [DecodingProcessor] Unable to decode raw message RawMessage{id=3fe7ab40-6b01-11ed-bf96-005056b7a11f, messageQueueId=783102474, codec=ipfix, payloadSize=11253, timestamp=2022-11-23T07:34:22.964Z, remoteAddress=/xxx.xxx.xxx.xxx:YYYY} on input <6358e47a8445bc549707c76e>.
2022-11-23T07:34:22.966Z ERROR [DecodingProcessor] Error processing message RawMessage{id=3fe7ab40-6b01-11ed-bf96-005056b7a11f, messageQueueId=783102474, codec=ipfix, payloadSize=11253, timestamp=2022-11-23T07:34:22.964Z, remoteAddress=/xxx.xxx.xxx.xxx:YYYY}
org.graylog.integrations.ipfix.IpfixException: Missing information element definitions for private enterprise number 5951
at org.graylog.integrations.ipfix.InformationElementDefinitions.getDefinition(InformationElementDefinitions.java:86) ~[?:?]
at org.graylog.integrations.ipfix.IpfixParser.parseDataSet(IpfixParser.java:337) ~[?:?]
at org.graylog.integrations.ipfix.codecs.IpfixCodec.lambda$decodeMessages$3(IpfixCodec.java:206) ~[?:?]
at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:195) ~[?:?]
at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1655) ~[?:?]
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484) ~[?:?]
at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474) ~[?:?]
at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:913) ~[?:?]
at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) ~[?:?]
at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:578) ~[?:?]
at org.graylog.integrations.ipfix.codecs.IpfixCodec.decodeMessages(IpfixCodec.java:212) ~[?:?]
at org.graylog2.shared.buffers.processors.DecodingProcessor.processMessage(DecodingProcessor.java:154) ~[graylog.jar:?]
at org.graylog2.shared.buffers.processors.DecodingProcessor.onEvent(DecodingProcessor.java:94) [graylog.jar:?]
at org.graylog2.shared.buffers.processors.ProcessBufferProcessor.onEvent(ProcessBufferProcessor.java:95) [graylog.jar:?]
at org.graylog2.shared.buffers.processors.ProcessBufferProcessor.onEvent(ProcessBufferProcessor.java:49) [graylog.jar:?]
at com.lmax.disruptor.WorkProcessor.run(WorkProcessor.java:143) [graylog.jar:?]
at com.codahale.metrics.InstrumentedThreadFactory$InstrumentedRunnable.run(InstrumentedThreadFactory.java:66) [graylog.jar:?]
at java.lang.Thread.run(Thread.java:829) [?:?]I am convinced that the configuration is missing:
I am convinced, as an ignoramus on the subject, that the problem is due to the lack of the json in the “IPFIX field definitions” field, but I didn’t understand how to load it.
At the end the data is present.
However even if you set “all time” no messages are displayed.
Tell me if you need anything else
Greetings,
thanks to both of you for the reply. Yesterday before answering, I tried to search again to be independent, however I did not find anything.
In the graylog log I got this error:
2022-11-23T07:34:22.965Z ERROR [DecodingProcessor] Unable to decode raw message RawMessage{id=3fe7ab40-6b01-11ed-bf96-005056b7a11f, messageQueueId=783102474, codec=ipfix, payloadSize=11253, timestamp=2022-11-23T07:34:22.964Z, remoteAddress=/xxx.xxx.xxx.xxx:YYYY} on input <6358e47a8445bc549707c76e>.
2022-11-23T07:34:22.966Z ERROR [DecodingProcessor] Error processing message RawMessage{id=3fe7ab40-6b01-11ed-bf96-005056b7a11f, messageQueueId=783102474, codec=ipfix, payloadSize=11253, timestamp=2022-11-23T07:34:22.964Z, remoteAddress=/xxx.xxx.xxx.xxx:YYYY}
org.graylog.integrations.ipfix.IpfixException: Missing information element definitions for private enterprise number 5951
at org.graylog.integrations.ipfix.InformationElementDefinitions.getDefinition(InformationElementDefinitions.java:86) ~[?:?]
at org.graylog.integrations.ipfix.IpfixParser.parseDataSet(IpfixParser.java:337) ~[?:?]
at org.graylog.integrations.ipfix.codecs.IpfixCodec.lambda$decodeMessages$3(IpfixCodec.java:206) ~[?:?]
at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:195) ~[?:?]
at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1655) ~[?:?]
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484) ~[?:?]
at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474) ~[?:?]
at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:913) ~[?:?]
at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) ~[?:?]
at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:578) ~[?:?]
at org.graylog.integrations.ipfix.codecs.IpfixCodec.decodeMessages(IpfixCodec.java:212) ~[?:?]
at org.graylog2.shared.buffers.processors.DecodingProcessor.processMessage(DecodingProcessor.java:154) ~[graylog.jar:?]
at org.graylog2.shared.buffers.processors.DecodingProcessor.onEvent(DecodingProcessor.java:94) [graylog.jar:?]
at org.graylog2.shared.buffers.processors.ProcessBufferProcessor.onEvent(ProcessBufferProcessor.java:95) [graylog.jar:?]
at org.graylog2.shared.buffers.processors.ProcessBufferProcessor.onEvent(ProcessBufferProcessor.java:49) [graylog.jar:?]
at com.lmax.disruptor.WorkProcessor.run(WorkProcessor.java:143) [graylog.jar:?]
at com.codahale.metrics.InstrumentedThreadFactory$InstrumentedRunnable.run(InstrumentedThreadFactory.java:66) [graylog.jar:?]
at java.lang.Thread.run(Thread.java:829) [?:?]I am convinced that the configuration is missing:
I am convinced, as an ignoramus on the subject, that the problem is due to the lack of the json in the “IPFIX field definitions” field, but I didn’t understand how to load it.
At the end the data is present
However even if you set “all time” no messages are displayed.
Yesterday I wrote a post but the antispam system blocked it.
I try again to write avoiding to put the screenshot.
The error is this:
2022-11-23T07:34:22.965Z ERROR [DecodingProcessor] Unable to decode raw message RawMessage{id=3fe7ab40-6b01-11ed-bf96-005056b7a11f, messageQueueId=783102474, codec=ipfix, payloadSize=11253, timestamp=2022-11-23T07:34:22.964Z, remoteAddress=/xxx.xxx.xxx.xxx:31886} on input <6358e47a8445bc549707c76e>.
2022-11-23T07:34:22.966Z ERROR [DecodingProcessor] Error processing message RawMessage{id=3fe7ab40-6b01-11ed-bf96-005056b7a11f, messageQueueId=783102474, codec=ipfix, payloadSize=11253, timestamp=2022-11-23T07:34:22.964Z, remoteAddress=/xxx.xxx.xxx.xxx:31886}
org.graylog.integrations.ipfix.IpfixException: Missing information element definitions for private enterprise number 5951
at org.graylog.integrations.ipfix.InformationElementDefinitions.getDefinition(InformationElementDefinitions.java:86) ~[?:?]
at org.graylog.integrations.ipfix.IpfixParser.parseDataSet(IpfixParser.java:337) ~[?:?]
at org.graylog.integrations.ipfix.codecs.IpfixCodec.lambda$decodeMessages$3(IpfixCodec.java:206) ~[?:?]
at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:195) ~[?:?]
at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1655) ~[?:?]
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484) ~[?:?]
at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474) ~[?:?]
at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:913) ~[?:?]
at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) ~[?:?]
at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:578) ~[?:?]
at org.graylog.integrations.ipfix.codecs.IpfixCodec.decodeMessages(IpfixCodec.java:212) ~[?:?]
at org.graylog2.shared.buffers.processors.DecodingProcessor.processMessage(DecodingProcessor.java:154) ~[graylog.jar:?]
at org.graylog2.shared.buffers.processors.DecodingProcessor.onEvent(DecodingProcessor.java:94) [graylog.jar:?]
at org.graylog2.shared.buffers.processors.ProcessBufferProcessor.onEvent(ProcessBufferProcessor.java:95) [graylog.jar:?]
at org.graylog2.shared.buffers.processors.ProcessBufferProcessor.onEvent(ProcessBufferProcessor.java:49) [graylog.jar:?]
at com.lmax.disruptor.WorkProcessor.run(WorkProcessor.java:143) [graylog.jar:?]
at com.codahale.metrics.InstrumentedThreadFactory$InstrumentedRunnable.run(InstrumentedThreadFactory.java:66) [graylog.jar:?]
at java.lang.Thread.run(Thread.java:829) [?:?]I am convinced that the problem is the lack of configuration, I see that a json must be inserted and configured, but I could not figure out how to load it.
Even setting “all time” no message is seen.
Tell me if you need anything else.
I wrote three messages, however they were blocked by the spam filter. I try this other way.
Selecting “all time” does not display the messages. Inside the logs I find this error:
I am convinced that the error is due to missing configuration (the json) but I could not figure out how to upload, even trying.
Hey @marpino
So what I got out of the logs were.
Unable to decode raw message RawMessage, Missing information element definitions for private enterprise number 5951
So the logs show up on the input but they cannot be ingest because its missing information in the logs needed. Ran into something before while back but it was not resolved.
This is what I found during my googling.
Have you tried or tested other INPUT’s like Raw Plaintext/Syslog to see if the message will appear?
Unfortunately I’m not to familiar with IPFIX. Perhaps someone else here has a better idea, if not you can post on GitHub for this issue.
Thanks a lot for the answer. They were links that I had already viewed. It will definitely be my problem, but I can’t figure out how to load the config file, I think that’s just the problem.
I didn’t understand if it should be loaded via the graylog interface (where?) or loaded directly into the system.
Maybe it’s written there but I don’t see it.
I had to install IPFIX precisely because the machine sends in that format. Otherwise I would have saved myself the trouble of using and configuring filebeat, but I can’t install anything.
hey, @marpino
Maybe I can shed some light on this.
Once the INPUT is created you can upload it to the correct input my using this…
1.Created INPUT IPFIX-UDP
2.Clicked on “Manage extractors” on the IPFIX input.
3.Upper right corner clicked on “Action” and choose “Import extractor”
4.Copy -N- Paste JSON template
Results:
I completely understand, I use Netflow v9. That’s about All I know, like i said I haven’t worked with IPFIX devices, but if you do get it to work, it would be appreciate posting here with the results
This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.
