Duplicate Messages in one Index


I’ve just installed the first graylog-instance.

I’m ingesting Squid-Proxy-Logs - one UDP-Syslog Input is defined and used by 2 Proxys. an Extractor is connected to the input, I’ve defined 2 Streams to seperate each Proxy-Log in it’s own index.

Everything works as intended - except that each message is stored twice. One copy contains the original (raw) incoming message, the second one has the extracted fields. The only other difference is the graylog-timestamp and the ID (both are stored in the same index), the raw message is stored about 1 sec before the extracted one.

So the message is not sent twice, it has been removed from the all_messages index and is stored two times in the same index.

I’d like to get rid of the original message, can somebody shed some light on that problem ?

Thank you

Did you really checked if you ingest the messages only once? How did you ingest the messages exactly? What is your configuration for that?

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.