Hi
We are trying out the Graylog to see if this is something we can use. We have a setup with graylog in docker running graylog 4.2.3. We have only setup 2 systems to forward logs to graylog. And we where surprised how much data graylog says that have been logged. If we go to the page System/Overview and looking at the section: Graylog cluster - Outgoing traffic the overview says that
we have logged:
oct. 9th: 6.896G
oct. 8th: 4.543G
oct. 7th: 4.668G
But looking at the indecies there are much less data.
1 index have a retention period of one week. Total 245 MiB
1 index with a retention period of one day.
index for oct. 9.: Contains messages from a day ago up to 5 hours ago (1.0GiB / 3,141,974 messages)
index for oct. 8.: Contains messages from 2 days ago up to a day ago (670.7MiB / 2,092,509 messages)
index for oct. 7.: Contains messages from 3 days ago up to 2 days ago (571.1MiB / 1,757,799 messages)
Using the REST API (System/ClusterTraffic : Cluster traffic stats) I get these traffic stats:
| Date | Incomming | Decoded | Outgoing |
|---|---|---|---|
| 07-10-2022 | 50.444.158.011 | 48.888.754.197 | 4.670.274.864 |
| 08-10-2022 | 36.922.528.760 | 35.787.946.659 | 4.545.440.223 |
| 09-10-2022 | 38.958.743.534 | 37.749.931.628 | 7.128.138.631 |
| 10-10-2022 | 8.950.238.443 | 8.672.468.000 | 1.611.908.984 |
There are messages which we drops in a pipeline, which explains why only about 20% of the messages are outgoing. So why are the overview reporting 6 times as much data??
Which data/traffic counter is the license depending on. Data in the indexes or the outgoing traffic from the system overview, which reports 6x as much data???
Are data counts as new traffic in every stage in the pipelines?
kind regards
Jens M. Kofoed