Working with non-timestamped data?

Development
1

I have a question related to more static data for analytics and if anyone has any ideas for doing it with Graylog.

Let’s say I have a property like total System RAM that I want to pop into Graylog.

I’m not sure how I could do this and still have the data in Graylog be useful for analytics. I could import the data just once but that may be an issue if the index fills up and rotates, it would also be an issue if the data ever has to change.

I could just put a timestamp on it, import it at some frequency, and treat it like a normal log entry – but it doesn’t seem that I’ve got a way to query and say, “Give me only the most recent entry (or the entry that is closest to this time) that matches this query for each source” (something I may want to do if I wanted to know the average max ram across my fleet). This would give me a timelapse for a specific machine but doesn’t seem like it would be useful for analytics across the fleet.

I also cannot apply a time range to an entry to say this is the value for this range of time – I could then just write a query that says what is the value of the max ram in the log entries whose time ranges cover a specific time.

Is Graylog not the right tool for this? Is there anyway to get some of this functionality across a fleet versus just a timelapse for a specific endpoint?

Bill

2

hej @strawgate

for me it looks like you want to have metrics stored somewhere and not a log management/message tool.

You could for example use metricbeat to report the metrics of the systems continuously into graylog.

with kind regards
Jan

1 Like
3

You want a CMDB, not a log-management tool.

1 Like
4

@Jochen – I certainly didn’t mean to imply that Graylog should perform CMDB duties – so that was definitely a poor example.

But if I wanted to use something like MetricBeat to report CPU Usage I can see how I would be able to graph average CPU usage during the day for a specific host – I would make sure the CPU Usage was a field, I would graph the value of the field and filter onto a specific host.

However, I’m not as sure how I would graph average CPU usage throughout the day or average cpu usage at a specific time of day across my fleet of devices. I could limit my search to a 1 minute timespan but I’d have to hope that I have an event for all my devices within that one minute (and that I have no devices that have reported two events).

5

I would recommend using a metrics database for your use case. An example would be Influxdb. Easy to achieve what you describe.

6

I was wondering if there is an easy way to output messages from Graylog to InfluxDB so we can achieve this easily, any ideas?

7

There’s no InfluxDB message output as far as I know, but you could use the Graylog Metrics plugin as a base for your own plugin: