Can some one help me to figure it out below scenario.
I have configured graylog (version 2.4.5) and now I am trying to get logs from windows hosts.
I have installed SIdecar Graylog collector tool with filebeat and winlogbeat and I am able to get event logs to the server.
Now I need to get IIS logs as well, I have followed the many option such as nxlogs and all are, as per the community topics but not working.
Can any one help me how we can get IIS logs to gray log ? is there any specific inputs need to configure in gray log server get IIS logs. Is sidecar filebeat is capable to push the logs to Graylog ? or do we need to install any dependency to it.
Kindly help to figure this out.
The main question is, how did you get what kind of events already into Graylog?
Should you use winlogbeat configured via collector-sidecar, then just configure filebeat to collect the logfile and push it to the same input you already do.
Yes, I was using winlogbeat. Now i tried by filebeat conf. There was some issue in inputconf file log path in graylog server side. I corrected it and now i am able to get the logs.
Now I am with another query , is there any size limit for filebeat upload logs to the gray log server, since i may need to process 15 GB in total from different 6 iis servers. How can we do this in efficient manner ? do i need to install the filebeat in all the servers? will it affect the performance of the iis servers ? or can i install filebeat in a centralized system with all logs available there in a common share mount path ?
Thanks Jan for your response.
Is that recommended setup from Graylog to install agent in all the servers ?
Also I could see the source and collector node id is same if we do it from a centralized system, Is there any way we can setup source from log message or any other way for segregation to know the logs are from which server?
Is that recommended setup from Graylog to install agent in all the servers ?
I guess that everyone would install some kind of logforder on the fleed of systems where he needs to get data from.
Also I could see the source and collector node id is same if we do it from a centralized system, Is there any way we can setup source from log message or any other way for segregation to know the logs are from which server?
You have installed the collector-sidecar? Than you might want to change the settings you can change in the configuration on the server where the sidecar is installed.
Depending on the collector you use, the messages will contain a field with the information from what host the messages are ingested.
Thanks Jan. I am using sidecar and I was just checking the option if we use a centralized system to push the log via sidecar to gray log server, how we will split the node based the logs. But I think there is no option since node id is taking from Sidecar configuration.
Now i am getting an error in joornal contains unprocessed message ( the number is increasing) , my elastic search cluster health is green and with 1 node and active 8 shards.
My heap size info is below,
curl -sS -XGET “http://localhost:9200/_cat/nodes?h=heap*&v”
heap.current heap.percent heap.max
214.7mb 21 990.7mb
How can I increase it ?
Will that fix this issue or any thing else we need to look in to, my server resource utilization is normal
its only taking 40% memory and 60 % CPU while processing
Thanks Jan. I am using sidecar and I was just checking the option if we use a centralized system to push the log via sidecar to gray log server, how we will split the node based the logs. But I think there is no option since node id is taking from Sidecar configuration.
you would seperate by hostname and location the log is fetched from - for exampel.
Now i am getting an error in joornal contains unprocessed message ( the number is increasing) , my elastic search cluster health is green and with 1 node and active 8 shards
you should look at your graylog server.log - this might give you one idea what is wrong with the connection between Graylog and Elasticsearch.
“you would separate by hostname and location the log is fetched from - for exampel.” Can you please elaborate little more on this…
I can’t because I didn’t know how you messages arrive at Graylog and what fields are present.
When you use beats (winlog or filebeat) you will have a field that gives you the option to identify the server where this log is comming from and another field that gives you the information where this is collected from.
