Search logs of a specific application

[altink (Altin Karaulli)]

How to search for the logs of a specific application?

Logs of this specific application (can be a DB table, OS logs or else) are coming to Graylog from multiple sources using GELF TCP Input.

The idea is to search for these application logs only and not to mix the result with logs of other applications that may come to Graylog.

In other SIEMSs this is usually done by making a dedicated index for each application (log type). As far as I have seen to ingest logs to a specific Graylog index a stream is needed, whose rules require the presence of a field inside GELF content to distinguish between different applications (log types).

Does this mean that the best solution is to add an application dedicated field in GELF, kind of:
_application_brand_name=“my_app_name”
?

any other idea?

best regards
Altin

[gsmith]

Hello,

I might be able to help. As for your first question.

@altink

How to search for the logs of a specific application?

Depends on how you setup your environment for searching a particular application.

@altink

Logs of this specific application (can be a DB table, OS logs or else) are coming to Graylog from multiple sources using GELF TCP Input.

This can be done either with a unique field or tag to sort through the ingest logs. This can be done through a pipeline, extractor or from the source (log Shipper).

Example from the source. This is a remote server in my environment. I was extracting logs from a application call NextCloud using NXlog. I created a input called the same name as my application
“nextcloud”.

<Input nextcloud>
   Module       im_file
   FILE         "/mnt/nextcloud-data1/nextcloud.log"
   SavePos       TRUE
   ReadFromLast  TRUE
   PollInterval  1
   Exec  $Message = $raw_event;
</Input>

Now here is what I receive on my GL server.

I just use the field called SourceModuleName to route my messages to stream. Therefor, I do not need to create a extractor/pipeline for a new field.

As you can see I can have multiple ones. As shown on my widget below.

Depending on what type of log shipper your using this is possible.
You could use a pipeline to grab what you need and route-to-stream that way also. I guess the main subject here is you may need the correct field first.

Hope that helps
EDIT: is this the same post?

GELF TCP Input Data to specific index

I need to put all data ingested by a specific TCP Input entry into a specific index user made. For this in the UI of Graylog I created a Stream on this index. Checked “Remove matches from ‘All messages’ stream”. When create/edit the TCP Input I cannot find any place for Stream binding. I created a single stream rule “Rule always matches” for the stream created above. Question: How do I connect my specific TCP Input to this Stream (and thus index)? I want other TCP Inputs to keep storing rec…

[tellistone (Tully Elliston)]

The default application_name field should contain the source application name.

[altink (Altin Karaulli)]

thank you @tellistone

I would like it this way, but in the GELF documentation field application_name is not mentioned anywhere. I have Graylog 3.3.2

https://docs.graylog.org/docs/gelf

[altink (Altin Karaulli)]

thank you @gsmith

Are you are saying that in my case the best thing to do is to add a new user-created field to the GELF, kind of dataplus_app=my_app, and then have every search start with dataplus_app:my_app?
If yes, yours is the answer.

ps. GELF docs says nothing about field SourceModuleName. I have Graylog 3.3.2

[gsmith]

@altink

Are you are saying that in my case the best thing to do is to add a new user-created field to the GELF,

No, this would be created using GELF input and the source is sending GELF format. So no need to add that field it will be created for you. This depend on what type of log shipper and/or type of logs that are being ingested. Can you give us the configuration you have for you INPUT and log shipper and what kind of fields generated already that you have. Maybe we can give you some suggestion on a HowTo for you.
This will give mea idea for a mockup in my lab.

[altink (Altin Karaulli)]

I am sending the records of an Oracle table using an Oracle job Scheduler calling a procedure. The later uses UTL_TCP to send data to Graylog. I guess I have no “log shipper”

[gsmith]

@altink

Oracle job Scheduler

I’m unfamiliar with that type of application.
Can Oracle job Scheduler send logs in GELF format? if so how did you configure it?

If Oracle job Scheduler cant then I would suggest using something like Syslog TCP/UDP for an input. You would then need to create a pipeline or extractors to create the needed field.

[altink (Altin Karaulli)]

What does the work is the Oracle package UTL_TCP - which send data through TCP, and as such event to Graylog TCP Input. Scheduler is just to automate the procedure that iterates through table records and sends them to Graylog using UTL_TCP.

[gsmith]

Ummmm. Correct me if I’m wrong but you stated your using GELF/TCP INPUT and your sending logs using Oracle package UTL_TCP.
I didn’t know that Oracle formatted there log files in a GELF format. Looking at the UTL_TCP documentation, it’s clearly a plain-text “raw” connection.

[tellistone (Tully Elliston)]

Hi Altink,

May I ask which software are you using to collect logs and send them to Graylog in GELF format (Filebeats? NxLog?) so I can attempt to reproduce?

It seems desirable that your GELF format uses the same field name for application name as all other inputs eg. application_name , so you can sort by this field within the Graylog UI.

[altink (Altin Karaulli)]

I am using no software. I am sending the events (table records) straight from Oracle, using UTL_TCP on Oracle side and GELF TC P Input on Graylog side.

https://docs.oracle.com/en/database/oracle/oracle-database/12.2/arpls/UTL_TCP.html

The operational logic is embedded inside an Oracle procedure, which its is called by an Oracle Scheduler Job.

[altink (Altin Karaulli)]

I see many people talking about this field - application_name - but I cannot find it in the Graylog documentation.

Do I risk something going wrong if I use it on Graylog 3.3.2 (my version)? What about later versions?

best regards
Altin

[gsmith]

@altink

I see many people talking about this field - application_name - but I cannot find it in the Graylog documentation.

If you have a field /w your application name, then you would use that field.

Example, I created a field called application_name and under that field I have the name of my application.
This means you either have this field or you need to create it. From your other post was explained on how to go about doing that.

[altink (Altin Karaulli)]

thank you very much @gsmith

[gsmith]

Of course, and I apologies for misunderstand you post.

[altink (Altin Karaulli)]

no problem at all @gsmith
I didn’t clarified in the main post that (1) I was not using any log shipper and (2) that I was aiming for a content pack. furthermore this topic is very similar (but not equal) to topic 21810, which adds more confusion.
I will go for adding application_name=“my_app” to my Oracle-sent GELF, and that first search I will use to filter only my events in my content pack, or any related search.
All rest - index, strean, pipeline … can wait.

thank you very much for your support @gsmith