I have a Graylog instance deployed inside an AWS VPC on an EC2 instance. It is in our “private” subnet and I have recently implemented a Squid proxy to replace the NAT gateway.
I’ve got outbound internet traffic working okay from the Graylog host with http_proxy and https_proxy set. I’ve also got Graylog traffic going to the internet via the http_proxy_uri setting in the server.conf file.
However, I don’t seem to be able to set up a Kinesis input through Graylog. It just times out and I see no activity in the Squid access log. I’ve checked the “use proxy” checkmark in the configuration of the AWS plugin but can’t see what else I may be missing?
Thanks for your reply. There’s not really a “device” as such at the other end. I’m trying to get CloudWatch Logs (for VPC Flow Logs, AWS WAF and Network Firewall logs) into Graylog ultimately.
It worked fine when the Graylog host was hitting the internet through an AWS NAT Gateway but introducing Squid seems to have stopped it from even being able to create a new Kinesis stream.
For what it’s worth, the Squid proxy seems to be working with no issues for other hosts/accessing other sites etc and I’ve allow-listed .amazonaws.com so Kinesis isn’t being “blocked”. As I say, I don’t even see the Graylog host attempting to go through the proxy when attempting this (but do see other successful connections outbound from it).
Just “re-upping this”. I ran a wireshark capture while attempting to add the AWS Kinesis input and I can see the attempt being made directly to the Kinesis IP address, as opposed to hitting the proxy. So this suggests to me that the AWS plugin config is ignoring the proxy setting (or the proxy setting isn’t correct for it).
I’ve got the “Use HTTP Proxy” check box checked on the plugin config and I’ve got my http_proxy_uri specified correctly in the server.conf - what else am I missing?
The plugin is part of Graylog itself to my knowledge. Here’s a screenshot (from a different implementation because on my AWS one the plugin box is checked)
Ok i see now. Unfortantly I havent used that plugin yet. I assume after the configurations made you reatrted the service ( GL)? Did you check firewall for port 3128 is open?
I’m on version 5.0.8. The AWS network firewall is in place (the proxy routes traffic outbound through it) but outbound traffic is open (hence why we have the proxy for egress filtering).
I presume that link is the AWS plugin that is in place in my Graylog install even though it’s obviously quite an old version of the plugin (maybe that’s the most recent).
I wonder if this is a bug with the plugin, just not respecting the “use proxy” checkbox.
