Plesk nginx log_access gelf json uncompressed

Development
1

Plesk is one of the most using hosting panel and we like to add logs of all shared hosting server they use plesk to get logs centralized! Only the transfer of the nginx logs crashed why something is not so nice as on other linux servers, a problem of nginx log sending via UDP to the graylog server!

Can been one of them a workarround?

a) Find a solution to filter the first 50 bytes of the input to trash - the rest is then the nice input uncompressed and I thing so graylog can handle?
b) Find a solution, that nginx in plesk create the correct file for UDP with compression in gzip? I dont know that possible with compression the nginx produce in the beginning the 50 bytes, we and I thing so graylog not need?

More about the problem:

  1. Nginx works for many domains in conjunction with the apache daemon. The log files for apache comes via gelf udp perfect, but the client-ips are every time the ip of the server. I thing so the mod_log_gelf.so use the wrong files (in conjuction with nginx).

  2. About this I set up a 2nd stream for the nginx logs to an other UDP-input! I set in the nginx.conf a log_format with escape=json and then I send via access_log to the server and I have check with tcpdump the transfered datas:

2.1 In first the nginx in plesk will not compress the datas. A setting on “gzip on” in ngnix.conf brings errors!
2.2 In second the nginx in plesk produce something in the first 50 bytes, when the access_log send to the graylog server. When the access_log send the same to a local file, then the 50 bytes are not their - i show here:

    0x0000:  9ad6 b45e df83 2899 3a99 a2f5 0800 4500  ...^..(.:.....E.
    0x0010:  02c5 744c 4000 3a11 8bf7 2e69 5322 b01f  ..tL@.:....iS"..
    0x0020:  0c3a 88b7 3071 02b1 a0c7 3c31 3930 3e41  .:..0q....<190>A
    0x0030:  7567 2032 3120 3038 3a31 393a 3138 2070  ug.21.08:19:18.p
    0x0040:  6c31 7365 7276 6572 206e 6769 6e78 3a20  l1server.nginx:.
    0x0050:  7b20 2274 696d 6573 7461 6d70 223a 2022  {."timestamp":."
    0x0060:  3230 3230 2d30 382d 3231 5430 383a 3139  2020-08-21T08:19
    0x0070:  3a31 382b 3032 3a30 3022 2c20 2272 656d  :18+02:00",."rem
    0x0080:  6f74 655f 6164 6472 223a 2022 3135 372e  ote_addr":."157.
    0x0090:  3535 2e33 392e 3437 222c 2022 636f 6e6e  55.39.47",."conn
    0x00a0:  6563 7469 6f6e 223a 2022 3238 3022 2c20  ection":."280",.
    0x00b0:  2263 6f6e 6e65 6374 696f 6e5f 7265 7175  "connection_requ
    0x00c0:  6573 7473 223a 2031 2c20 2270 6970 6522  ests":.1,."pipe"

It looks like a header in the file and the problem is the byte with the char “<” for graylog! Graylog stop with this warning:

com.fasterxml.jackson.core.JsonParseException: Unexpected character (’<’ (code 60)): expected a valid value (number, String, array, object, ‘true’, ‘false’ or ‘null’)
at [Source: (String)"<190>Aug 21 08:45:16 abcserver nginx: { “timestamp”: “2020-08-21T08:45:16+02:00”, “remote_addr”: “256.256.256.256”, “connection”: “643”, “connection_requests”: 1, “pipe”: “.”, “body_bytes_sent”: 1326, “request_length”: 198, “request_time”: 0.000, “response_status”: 200, “request”: “GET /robots.txt HTTP/1.1”, “request_method”: “GET”, “host”: “www.zeitzug.com”, “upstream_cache_status”: “”, “upstream_addr”: “”, “http_x_forwarded_for”: “”, “http_referrer”: “”, “http_user_agent”: “Mozilla/5.0 (compatible”[truncated 187 chars]; line: 1, column: 2]

2

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.