hello everyone, i have some basic queries which i need to clear for graylog 3.x
first query is what will be the hardware requirement in the below two scenarios
for UAT server we are getting 2000 requests per hour
2.for production server we are getting 20000 request per hour
so in both the above scanarios what will be the hardware specification for the greylog and we are also installing mongodb and elastic search on the same server.
My second query is how does this indices works. i am finding very difficult to understand the indices concept. my requiremnt is to store the logs for atleast minimum one year,then it should automatically get deleted. so how can we configure index sets to store the logs for one year.
You maybe think for indices. You have to set Time based rotation, New index after 1 day, delete after 365 days.
Indices contain shards. There is no limit about that, but the suggestion to keep the size of shards between 20 and 40 GB.
You also have to calculate with elastic memory. If you want to search in the one year logs, you need about 2% memory for elasticsearch of the RAW log data. Eg. If you get 480k log/day, 1kb avarage size, 480mb/day -> 175Gb/year 3,5 GB elasticsearch heap.
