Graylog capabilities

Graylog Central (peer support)
1

Hello,

I’m new on Graylog usage and despite of reading documentation, I’m unable to know if Graylog can help me for my use case.

I wanted to analyse logs from my application in order to see if my items are well managed.
In applicative logs, these items are referenced by unique id and log lines gave information about my item lifecycle.
To make it simple, my logs look like that (I simplified the log lines):

timestamp, itemID=67937364907, msg=“Begin - Item code generation”
timestamp, itemID=67937364907, msg=“End - Item code generation”
timestamp, itemID=67937364907, msg=“Begin - Request files”
timestamp, itemID=67937364907, msg=“End - Request files”
timestamp, itemID=67937364907, msg=“Sending message to partner A”
timestamp, itemID=56489827672, msg=“Begin - Item code generation”
timestamp, itemID=67937364907, msg=“Sending message to partner B”
timestamp, itemID=67937364907, msg=“Receiving response from partner B”
timestamp, itemID=67937364907, msg=“Receiving response from partner A”

All logs lines to find for a specific itemID are well knowed to tell that an item was well managed by the system.

Does Graylog is able to provide an information like that: (I plan to use a dashboard with datatable). If yes what feature I have to look for?

ItemId | Status
56489827672 | On going
67937364907 | Completed

Thanks for your help.

2

Maybe lookup table is what you are looking for:

Create CSV file with ItemId and Status, and assign new field Status based on ItemID using pipeline rule. Finally use this field in datatable in dashboard.

3

Hello shoothub,

Thanks for this idea, that could be the good one.
I worked on it but I’m facing a Graylog limitation on updating CSV Lookup Tables using pipeline rules functions:
https://docs.graylog.org/en/3.3/pages/pipelines/functions.html#lookup
So, I can’t update the status value in the lookup table.

I’m starting to try with MongoDB Lookup Tables, will see…

4

Why do you wan to update CSV lookup table using pipeline rule? Lookup table only read data from CSV file from disk and lookup for values in CSV based on key.

MongoDB lookup works only with enterprise license.

5

Because I need to change the status of my items along time.

When Graylog receive the first message, I need to add a new line in my lookup table: ‘56489827672’,‘ONGOING’

When Graylog receive the last message for this itemID, I have to pass it to:
‘56489827672’,‘COMPLETED’

Maybe I didn’t catch the way you intend using lookup tables for my need…

6

Sorry, I probably didn’t corectly understand your goal. CSV lookup table is read-only. I thought, that you wanted to only translate ItemID to status.
I didn’t play with mongo lookup, sorry.

7

Ok, no problem :wink:

8

he @MazeOfFate

you can modify those entries or create entries in the processing pipelines and a mongodb backend. you could even read/modify that via Graylog API from within any other application.

So you could decide to delete the entries when you have reached “completed” or you check all completed entries from outside and delete them after some time. This way you could verify later if those are completed or not.

9

Hello @jan

Thank you, I will try that.
Currently, I’m waiting the Enterprise license…

10

Hello,

I succeded to create a pipeline with rules to maintain a MongoDB lookup table that reflects the real time state of my items.

Now, I wanted to display this lookup table content by using a Graylog dashboard but I’m afraid that is not possible…
Could you confirm?

I also added a new field named “state” to my log messages but this state can only be considered as right if no new message with a same ItemId was received in the meanwhile (that’s why @shoothub suggested to me to use lookup tables)

Second question, does a “group by” feature exists on Graylog dashboards to be able to count only one time an ItemId with an “ONGOING” state?

Thanks

11

he @MazeOfFate

the Graylog Dashboard can’t display the content of a lookup table. That would be a feature request over at Github.

12

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.