Graylog input doesn't send HTTP response

Good morning.

I recently started using Graylog and have been stuck on a problem. I spent 2 days searching for an answer but unsucessful so far.

My use case is the following:

  • We use Azure Devops for development purposes. I have setup some outbound webhooks that trigger on certain events (builds, deploys, etc). This sends a HTTP Post to my Graylog cluster.
  • The endpoint Azure Devops is posting to, is a load balancer machine with a Nginx proxy server. I have configured the proxy as HTTP server and set a rule to pass through all incoming traffic on a certain port to my 2 Graylog nodes.
  • On Graylog, I have configured a Raw/Plaintext TCP input with a JSON extractor to parse these inbound webhooks (as well as a stream and an elasticsearch index to redirect and store them).

Now, my problem is:

  • At first, the HTTP posts do arrive at my Graylog, I can see the documents in there.This proves the concept and configuration, and suggests that it should work just fine.
  • However, Azure Devops thinks that the information did not arrive at the destination, as the webhook reports a timeout on the HTTP response, and hence a “Failed” status every single time. After X fails, Azure Devops automatically puts my webhook on probation (where the triggers are inhibited for a few minutes), and after Y fails disables it forever (which breaks my entire application logic). As far as I have searched, there is no way to change this behavior from Azure’s side.
  • The core of my problem seems to be that apparently, Graylog does not give a HTTP response code to the incoming webhook, atleast on Raw/Plaintext TCP input type. Is this correct?? I have searched the internet far and wide, and read the Graylog documentation, but this bit of information seems to be amiss.
  • If the Raw/Plaintext TCP does send HTTP response codes, what might be preventing these responses from reaching my Azure Devops?
  • Mind you that I cannot format my webhook body, so I can’t meet “GELF” input types required formatting; and the “JSON path from HTTP API” input type requires Graylog to initiate the webhook which is the opposite of what I need.

I can provide you extra information if needed.
Thank you very much for any help you are able to provide me.

Best regards,
Andre

Good afternoon.

Anyone has any idea to enlighten me about this? I haven’t been able to confirm my suspicion yet.

Thank you.

The core of my problem seems to be that apparently, Graylog does not give a HTTP response code to the incoming webhook , atleast on Raw/Plaintext TCP input type. Is this correct?? I have searched the internet far and wide, and read the Graylog documentation, but this bit of information seems to be amiss.

a RAW Input does not give any HTTP response codes - as it is not a webhook. RAW Inputs are like running netcat on a specific port.

You might need to write yourself a little daemon that accepts the message and write that in a way to Graylog that works.

1 Like

Thanks a lot jan, that does confirm my suspicion.

I got this to work, but had to resort to an unefficient workaround to do it, which is having a script re-enabling my azure devops webhooks through API every few seconds. Ugly but seems to get the job done.

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.