Graylog Application-wise Logs

Graylog Central (peer support)
1

Hi Team, we have microservices architecture and we are using Graylog 4.x version for container logs, it is a single server on which we have Elastic, Graylog and Mondo DB installed and have around 2 TB of disk attached.

Our Applications are generating lots and lots of logs and it is piling up the entire 2 TB disk in 1 week(7 days) and post that we clean up the indexes and it works fine, hence we are not able to retain the logs for a longer period.

We are planning the optimize the logs that the server receives and we are not sure which of the application is generating huge logs.

Is there any way we can find out which input is generating how much amount of logs? This will help us to identify the logs pattern and the projects to optimize.

we are expecting the below details for each input(project)

  1. Current Logs Size (in MB)
  2. Avg Growth size per day (In MB)
2

You can curl the information you need directly from Elasticsearch:

curl -X GET "elastic-server:9200/_cat/indices/*?v&s=index&pretty"

That will tell you what you need to know about each index, size as well as number of messages it contains.

1 Like
3

Thanks, but this gives me the data for the entire inputs, I want the data application(input) ise, is it possible.

Thanks,
Abhishek

4

Actually it gives you the data from the entire index, you could have multiple inputs placing data in the same index… That being said, Graylog doesn’t separate out size of data of a particular group of messages…that I know of. With some research/googling with Elasticsearch, you might be able to cobble together a curl command that would separate out that data. Sorry - I don’t know curl/elastic it to that depth!

5

Thanks for your response @tmacgbay

I will lookout for the way in elasticsearch

6

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.