Graylog + Nginx Load Balanced Syslog - Source is the Load Balancer

Graylog Central (peer support)
1

We’ve got 3 graylog servers, and I added an Nginx load balancer in front of them today. We’re load balancing GELF, syslog TCP and UDP, and the web interface itself.

One thing I noticed on the UDP syslog input is that the source is coming into Graylog as the load balancer’s DNS name - not the source of the actual server sending the logs. This isn’t the end of the world, but it also isn’t ideal.

Anything slick we can do with either Nginx or Graylog to keep the source as the original source?

Here’s the relevant piece of my nginx.conf file:

    upstream gludpsyslog {
         server popglweb1.pop.portptld.com:1514;
         server popglweb2.pop.portptld.com:1514;
         server popglweb3.pop.portptld.com:1514;
    }

    server {
         listen 1514 udp;
         listen 514 udp;
         proxy_pass gludpsyslog;
         proxy_responses 0;
    }

And here is the full message:

<167>2017-04-26T01:45:16.820Z vmhost10.pop.portptld.com Hostd: verbose hostd[E081B70] [Originator@6876 sub=SoapAdapter.HTTPService] HTTP Response: Auto-completing at 129/129 bytes

Screenshot of the message in graylog:.

Capture.PNG1521×394 30.5 KB

2

Which version of Graylog are you using? Graylog 2.2.3 should correctly parse the syslog message you’ve provided.

Also, what kind of client does create these syslog messages?

Output of a quick test case using Graylog’s syslog parser:

fields = {HashMap@3161}  size = 7
 0 = {HashMap$Node@3166} "level" -> "7"
 1 = {HashMap$Node@3167} "full_message" -> "<167>2017-04-26T01:45:16.820Z vmhost10.pop.portptld.com Hostd: verbose hostd[E081B70] [Originator@6876 sub=SoapAdapter.HTTPService] HTTP Response: Auto-completing at 129/129 bytes"
 2 = {HashMap$Node@3168} "_id" -> "117f97c0-2a59-11e7-80da-56ae9a9f0542"
 3 = {HashMap$Node@3169} "source" -> "vmhost10.pop.portptld.com"
 4 = {HashMap$Node@3170} "message" -> "vmhost10.pop.portptld.com Hostd: verbose hostd[E081B70] [Originator@6876 sub=SoapAdapter.HTTPService] HTTP Response: Auto-completing at 129/129 bytes"
 5 = {HashMap$Node@3171} "facility" -> "local4"
 6 = {HashMap$Node@3172} "timestamp" -> "2017-04-26T01:45:16.820+00:00"
3

Graylog 2.2.3+7adc951.

This is coming from a VMWare VMHost.