Graylog server can not receive log from F5 BIG-IP

Graylog Central (peer support)
1

Hello everyone!

I have installed Graylog v3.0.2 server to store log from F5 BIG-IP. The F5 is running with service CGNAT. When I configured on F5 to send logs to local-syslog(F5 itself), the server can receive CGNAT logs of F5, but when I configured on F5 to send logs directly via F5 LSN CGNAT publisher, the server can not receive the logs from F5. As a result of troubleshooting using tcpdump on F5, i see that F5 sending the logs out, but when i run tcpdump on the server, I don’t see any message receive from F5.

Thank you for your reading and help advise in advance.

2

hi
I guess the timing of the Graylog server is not right with the timing F5

3

Sorry, but I can’t understand you.
As your post you don’t send the logs to graylog.
It seems F5 or network issue. Solve it, and continue this topic if you got the messages on the graylog server, and you don’t see the logs.

4

Hi bahram, thank for your comment. Let me check and verify, thanks.

5

Hi macko003, Thank for your feedback. Please double read my post. Anyway,I have checked with F5 support team, they told no problem with F5 due they recommend to run tcpdump command on F5 and see logs send out. They said problem is with Graylog server.
Sorry if i have bad English made you not understand.Thanks.

6

I read it again. It’s not a graylog issue.

7

That’s still can not solve, would you have any more advice, please? thanks

8

Please post your configuration of graylog (Input, port, tcp/udp?) and configuration of F5 side.

9

Hi, Thank for your comment. Please find below information:

  • On Graylog i use udp port 514 with configuration below:
  • allow_override_date: true
  • bind_address:0.0.0.0
  • expand_structured_data: false
  • force_rdns:false
  • number_worker_threads:4
  • override_source:
  • port:1514
  • recv_buffer_size:262144
  • store_full_message:false

Thank for your more advice.

10

Hi,

you post, that you use udp port 514 on graylog, but in Input i see port 1514.

So my advice:

  1. Check your configuration on both side, check on side F5 if it use same port as on graylog Input
  2. Graylog by default could listen on port 514 (because it’s running as normal user), so better is to use port higher than 1024, so 1514 is fine.
  3. Check on F5 side, that you setup destination port 1514 on pool of members for remote logging:
    https://techdocs.f5.com/kb/en-us/products/big-ip_ltm/manuals/product/cgn-implementations-11-5-0/11.html
11

Thank for your advice. Well you see port 1514 due to I use default port 514 which is lower than port 1024 and since beginning i added the below command on Graylog server to redirect port 514 to 1514
following the document Graylog: http://docs.graylog.org/en/3.0/pages/faq.html
iptables -t nat -A PREROUTING -p tcp --dport 514 -j REDIRECT --to 1514
iptables -t nat -A PREROUTING -p udp --dport 514 -j REDIRECT --to 1514

So , Please kindly advise more, thanks.

12

I have bad experience with forwarding port 514 through firewall, some devices had problem with it. Try to rather use direct port 1514 (or another) without iptables and setup F5 to send remote syslog to this port, if it helps.

13

I tried use another udp port 1515, but still not help, :frowning:

14

  1. Try to debug connection from F5 to graylog, if is working correctly:
    myF5

  2. Send test message from F5 to graylog:

echo ‘<0>Testing LTM for connectivity’ | nc -w 1 -u graylog_server_ip 1515

  1. Check if message received to graylog with tcpdump:

sudo tcpdump -A -n -vv -i ens160 port 1515

  1. If you see tcpdump message received in graylog, but not shown in graylog, it’s probably problem with timestamp. Try to use Absolute search timerange, and setup interval from yesterday to tomorrow (future) if it find something. If there is a problem with timestamps, messages can be saved to graylog correctly, but not shown, because it’s from future.

  2. If nothing received in tcpdump, check connection between F5 and graylog, at least iptables firewall on graylog server, if you allowed port 1515 on it.

15

Dear shoothub,

Thank for your more information of troubleshooting, I have followed your advice and let me update as below:

  1. Try to debug connection from F5 to graylog, if is working correctly:
    –> YES, I confirm, it’s working correctly.

  2. Send test message from F5 to graylog:
    echo ‘<0>Testing LTM for connectivity’ | nc -w 1 -u graylog_server_ip 1515
    –> YES, It’s successfully. Graylog received the test message and the message also appeared as screenshot below:
    image

  3. Check if message received to graylog with tcpdump:
    —> YES! Graylog can receive message on both tcpdump and Graylog Web Interface in case option#2 above.
    —> NO! Graylog can not receive anything both on tcpdump and Graylog Web Interface when I tried to send from F5 LSN CGNAT Publisher. But I would like to confirm that when I tried to send from F5 LSN CGNAT publisher as in option#1, F5 really sent to Graylog via the port 1515. I still wondering why it can not receive, is it because of amount of LSN CGNAT logs is very huge message? or What? Hmm…

Appreciated for your more advices.

Thank you.

16

If you didn’t see any output on tcpdump running on graylog box, it means nothing come to graylog box at all. It doesn’t matter, if there is a graylog running or not. Anyway you should definetely see sended data from F5 on tcpdump output running on graylog box. If not, there is a problem with connection from F5 to graylog box (linux box), not graylog service alone. Check your network and firewall settings once again…

Try to check, if F5 doesn’t still use port 514 (tcpdump will listen for ports 514 and 1515):

sudo tcpdump -nnAs0 -i ens160 port 514 or port 1515

17

Thank for your more information. If the problem in the middle between F5 and Graylog, why i tested the option#2 successfully ?

18

Maybe F5 uses another destination port, protocol or source IP, which is not allowed to connect to graylog box. Try to run tcpdump on F5 LSN CGNAT Publisher and check source ip, destination ip and port and protocol UDP/TCP. If you are able, post output of tcpdump here.

19

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.