Certain logs aren't being indexed


(Ryan Jones) #1

I just noticed that some of my logs aren’t being indexed even though it is received by the graylog host… however when I do a search in graylog there are no results…

I’m taking in syslog from cisco devices on udp 514 using the authbind package.


(Jan Doberstein) #2

Hej Ryan,

you might want to check this community - currently many Cisco devices did not send valid syslog messages.

To get your problem solved, change the input to RAW and extract the Data yourself. Your Logfiles should show some errors.


(Ryan Jones) #3

Thanks,

I’m alrrady logging thousands of cisco devices including others of the same typ4. it seems seems that it’s just a few devices that are exhibiting this. I did a tcp dump and generated logs from one of those devices and saw the log come in. I will say one other thing I’ve seen some notifications that say " Uncommited messages deleted from journal" and " Journal utilization is too high" I assume those errors are specific to Elasticsearch ?


(Jan Doberstein) #4

Hej @eyeball,

the problem with the syslog dialect Cisco is speaking changes from device to device and from firmware to firmware. Maybe you can find a way to configure the few devices in some different ways?

The “uncommitted messages deleted” Messages will be shown when your Elasticsearch Cluster/Server is not fast enough to get the messages ingested. You need to configure and tune Graylog and special Elasticsearch for the messages you are pushing into your cluster.


(Ryan Jones) #5

I’ve switched to Raw input but I noticed I lost the ability to do reverse lookups which is import due to the large number of devices. As for the host itself it’s a 24 core 64G of RAM and 16T 10K RPM SAS drives.


(Jan Doberstein) #6

you should contact Cisco and fill a bug report that not all devices sending rfc syslog.

Additional in the upcoming version Lookup Tables are added and that can be extended with a reverse dns lookup. But that will not be ready in the next days.


(system) #7

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.