Single log getting repeated multiple times on graylog2


(Rakesh Bhatt) #1

Single log getting repeated multiple times on graylog2

I have setup graylog2 on ubuntu server. Currently i am sending logs using GELF UDP. I had created streams to separate out logs. Graylog is receiving logs at input but same logs are getting repeated multiple times when i see on search and streams. While on original log files the log has occurred only once.

  • Graylog Version: Graylog v2.2.3+7adc951
  • Elasticsearch Version: 2.3.1
  • MongoDB Version: 3.4.3
  • Operating System: 16.04

I had implemented https://marketplace.graylog.org/addons/bb434928-f9bb-445e-b8fe-6ac1030ddbd7
to send logs to graylog UDP input. Stream has no output configured

Most logs are send by cron scheduler and most of task run asynchronously or in parallel in threads
i am using APSchedular for this purpose.

I am very new to graylog. I had searched the issue but didn’t get more information. No idea how i should procceed


(Jochen) #2

Check your client code. It probably sends the same message multiple times.


(Rakesh Bhatt) #3

Yes jochen

I had from my client code i changed logging library. PyGelf provides support for TCP input. I tried it but still i am getting multiple logs. I think it is because of APSchedular python library that is making graylog go crazy in terms of logs. Normal logs that are not drive by APSchedular are not getting multiplied.
Need to look for python logging more to solve it.

Thanks,


(Rakesh Bhatt) #4

Solved problem. I was re initiating logger with below lines every time a scheduler initiate process task very silly mistake

logger = logger.get_logger(name)
handler = graypy.GELFHandler('52.66.71.150', 12201)
logger.addHandler(handler)

Thanks for help