How to search for the logs of a specific application?
Logs of this specific application (can be a DB table, OS logs or else) are coming to Graylog from multiple sources using GELF TCP Input.
The idea is to search for these application logs only and not to mix the result with logs of other applications that may come to Graylog.
In other SIEMSs this is usually done by making a dedicated index for each application (log type). As far as I have seen to ingest logs to a specific Graylog index a stream is needed, whose rules require the presence of a field inside GELF content to distinguish between different applications (log types).
Does this mean that the best solution is to add an application dedicated field in GELF, kind of:
I might be able to help. As for your first question.
Depends on how you setup your environment for searching a particular application.
This can be done either with a unique field or tag to sort through the ingest logs. This can be done through a pipeline, extractor or from the source (log Shipper).
Example from the source. This is a remote server in my environment. I was extracting logs from a application call NextCloud using NXlog. I created a input called the same name as my application
Depending on what type of log shipper your using this is possible.
You could use a pipeline to grab what you need and route-to-stream that way also. I guess the main subject here is you may need the correct field first.
Are you are saying that in my case the best thing to do is to add a new user-created field to the GELF, kind of dataplus_app=my_app, and then have every search start with dataplus_app:my_app?
If yes, yours is the answer.
ps. GELF docs says nothing about field SourceModuleName. I have Graylog 3.3.2
No, this would be created using GELF input and the source is sending GELF format. So no need to add that field it will be created for you. This depend on what type of log shipper and/or type of logs that are being ingested. Can you give us the configuration you have for you INPUT and log shipper and what kind of fields generated already that you have. Maybe we can give you some suggestion on a HowTo for you.
This will give mea idea for a mockup in my lab.
What does the work is the Oracle package UTL_TCP - which send data through TCP, and as such event to Graylog TCP Input. Scheduler is just to automate the procedure that iterates through table records and sends them to Graylog using UTL_TCP.
Ummmm. Correct me if I’m wrong but you stated your using GELF/TCP INPUT and your sending logs using Oracle package UTL_TCP.
I didn’t know that Oracle formatted there log files in a GELF format. Looking at the UTL_TCP documentation, it’s clearly a plain-text “raw” connection.
If you have a field /w your application name, then you would use that field.
Example, I created a field called application_name and under that field I have the name of my application.
This means you either have this field or you need to create it. From your other post was explained on how to go about doing that.
no problem at all @gsmith
I didn’t clarified in the main post that (1) I was not using any log shipper and (2) that I was aiming for a content pack. furthermore this topic is very similar (but not equal) to topic 21810, which adds more confusion.
I will go for adding application_name=“my_app” to my Oracle-sent GELF, and that first search I will use to filter only my events in my content pack, or any related search.
All rest - index, strean, pipeline … can wait.