Ngnix Error log content Pack for Graylog 4

[ajay.negi (Ajay)]

Hi Team,

Any idea on NginxError log for Graylog 4 with UDP input.If anyone has content pack for this please let me know.

Regards
Ajay Negi

[gsmith]

Hello @ajay.negi

Is this a specific Nginx content pack or do you need any Nginx content pack. There are a couple I see here so I assume this may not be what you wanted?

[ajay.negi (Ajay)]

Hi @gsmith ,
Thanks for the update…Actually we only need Nginx Error log content pack…as there is some date parsing error coming when we received the Nginx Error log to Graylog 4.2.7.
Nginx Access log we already install content pack and working fine…Only concern on error log.

[ajay.negi (Ajay)]

Also like Error log message which we received has 8 hours difference…
gl2_processing_error
Error evaluating action for rule <test to change/62bac83953dc79608e0b6dca> (pipeline <testpipeline/62bc119b53dc79608e0cdef0>) - In call to function ‘parse_date’ at 6:21 an exception was thrown: Invalid format: “2022-06-30T09:53:21.000+08:00” is malformed at “.000+08:00”

Pipeline rule is:-
rule “test to change”
when
has_field(“timestamp”)
then
// the following date format assumes there’s no time zone in the string
let newtimestamp = parse_date(to_string($message.timestamp), “yyyy-MM-dd’T’HH:mm:ss,SSS”);
set_field(“timestamp”, newtimestamp);
remove_field(“timestamp”);
end
any idea how we can parse the time of Error log…

[gsmith]

I see one of your problems, there is a issue with using the field name.
Meaning you need something like this. Other wise your removing the same field you adjusted.

Pipeline rule is:-
rule “test to change”
 when
   has_field(“timestamp”)
 then
   // the following date format assumes there’s no time zone in the string
    let newtimestamp = parse_date(to_string($message.timestamp), “yyyy-MM-dd’T’HH:mm:ss,SSS”);
    set_field(“new_timestamp”, newtimestamp);
    remove_field(“timestamp”);
end

There are tons of pipelines/ Extractors Date/Time conversions in this forum, perhaps this post may help to give you an idea.

• Replaced invalid timestamp value in message - #9 by gsmith

[gsmith]

@ajay.negi

Also like Error log message which we received has 8 hours difference

For a time difference I would check the Date/time on the device that nginx is on, you may need to use NTP or something similar and check your Graylog server under System/Overview Time configuration

[ajay.negi (Ajay)]

My Nginx Logs are something like this :- <187>Jul 1 06:03:45 nginx: 2022/07/01 06:03:45 [error] 1966872#0: *17257159…so on that can we take nginx as a filed.
Timestamp
2022-06-30 22:03:45.000
timestamp
2022-06-30 22:03:45.000 +00:00

Time configuration

Dealing with timezones can be confusing. Here you can see the timezone applied to different components of your system. You can check timezone settings of specific graylog-server nodes on their respective detail page.

User admin:
2022-07-01 04:10:40 +00:00
Your web browser:
2022-07-01 09:40:40 +05:30
Graylog server: 2022-07-01 12:11:11 +08:00

message
nginx: 2022/07/01 06:03:45 [error] 1966872#0: *17257159 could not be resolved (3: Host not found), client: 172.28.213.20, server: Servername, request: “GET /version/ HTTP/1.1”, host: “hostname”

[gsmith]

I see

2022-07-01 04:10:40 +00:00
2022-07-01 09:40:40 +05:30
2022-07-01 12:11:11 +08:00

Man, I m not sure what going on. I see 4 different timestamps and the Nginx logs or 8 hours off I’m not sure if this is the category for these post, perhaps Graylog Central

[ajay.negi (Ajay)]

any help on this…as i created a pipeline rule i.e rule “replace timestamp”
when
true
then
let result = regex(“([0-9-T.:]+)”, to_string($message.timestamp));
let new_date = parse_date(to_string(result[“0”]), “yyyy-MM-dd’T’HH:mm:ss.SSS”);
set_field(“timestamp”, new_date);
end

and its working fine no error but messages not shown on the stream…confused on that part now…

[gsmith]

@ajay.negi

and its working fine no error but messages not shown on the stream…confused on that part now…

How do you know its working fine? Use Debug() in your pipeline and tail -f Graylogs log file. This will give you and idea what’s going on.

EDIT:
Example of what I was referring to.

then
  let new_date = parse_date(to_string(result[“0”]), “yyyy-MM-dd’T’HH:mm:ss.SSS”);
  set_field(“timestamp”, new_date);
  debug (new_date ); <----------ADD THIS INTO YOUR PIPEINE
end

Then

root# tail -f /var/log/graylog-server/server.log