Hi Graylog Community,
I have a Graylog instance deployed inside an AWS VPC on an EC2 instance. It is in our “private” subnet and I have recently implemented a Squid proxy to replace the NAT gateway.
I’ve got outbound internet traffic working okay from the Graylog host with http_proxy and https_proxy set. I’ve also got Graylog traffic going to the internet via the http_proxy_uri setting in the server.conf file.
However, I don’t seem to be able to set up a Kinesis input through Graylog. It just times out and I see no activity in the Squid access log. I’ve checked the “use proxy” checkmark in the configuration of the AWS plugin but can’t see what else I may be missing?
Any advice greatly appreciated!
Can you ping Graylog from the device sending log via Kinesis? Do you see the device using tcpdump on graylog?
Thanks for your reply. There’s not really a “device” as such at the other end. I’m trying to get CloudWatch Logs (for VPC Flow Logs, AWS WAF and Network Firewall logs) into Graylog ultimately.
It worked fine when the Graylog host was hitting the internet through an AWS NAT Gateway but introducing Squid seems to have stopped it from even being able to create a new Kinesis stream.
For what it’s worth, the Squid proxy seems to be working with no issues for other hosts/accessing other sites etc and I’ve allow-listed .amazonaws.com so Kinesis isn’t being “blocked”. As I say, I don’t even see the Graylog host attempting to go through the proxy when attempting this (but do see other successful connections outbound from it).
Just “re-upping this”. I ran a wireshark capture while attempting to add the AWS Kinesis input and I can see the attempt being made directly to the Kinesis IP address, as opposed to hitting the proxy. So this suggests to me that the AWS plugin config is ignoring the proxy setting (or the proxy setting isn’t correct for it).
I’ve got the “Use HTTP Proxy” check box checked on the plugin config and I’ve got my http_proxy_uri specified correctly in the server.conf - what else am I missing?
Sorry for the late responce Ive been working on some other open source software.
Can you enlighting me on what plugin this is?
You have something like this?
http_bind_address = my_domain.com9000
http_publish_uri = https://my_domain.com:9000/
Have you tried using External Graylog URI?
No need to apologise! Appreciate your time
The plugin is part of Graylog itself to my knowledge. Here’s a screenshot (from a different implementation because on my AWS one the plugin box is checked)
I have the following in my conf file:
http_publish_uri = http://awsgraylog.domain.local:9000/
http_bind_address = <host IP address>:9000
http_proxy_uri = http://<squid proxy IP address>:3128
Ok i see now. Unfortantly I havent used that plugin yet. I assume after the configurations made you reatrted the service ( GL)? Did you check firewall for port 3128 is open?
What version of Graylog are you using? The reseason i ask is i seen this GitHub - Graylog2/graylog-plugin-aws: Several bundled Graylog plugins to integrate with different AWS services like CloudTrail and FlowLogs.
I’m on version 5.0.8. The AWS network firewall is in place (the proxy routes traffic outbound through it) but outbound traffic is open (hence why we have the proxy for egress filtering).
I presume that link is the AWS plugin that is in place in my Graylog install even though it’s obviously quite an old version of the plugin (maybe that’s the most recent).
I wonder if this is a bug with the plugin, just not respecting the “use proxy” checkbox.
Have added an “issue” to the github page for that plugin. Not sure how active the development is but will see if it gets picked up.
Just bumping this to stop auto-close and in case anyone else is having (or has fixed) similar issue.
This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.