So I have two instances in AWS, one is a Windows server running IIS with NXLog configured and running and the other is a Ubuntu instance with Graylog installed. I can’t for the life of me figure out how to ship the logs from the Windows Machine to the Ubuntu Graylog instance…
I might have configured my Graylog server wrong since I can’t seem to figure out where it wants the AWS Public IP and AWS Private IP. Here is my server.conf HTTP Settings on the Ubuntu box.
###############
HTTP settings
###############
HTTP bind address
The network interface used by the Graylog HTTP interface.
This network interface must be accessible by all Graylog nodes in the cluster and by all clients
using the Graylog web interface.
If the port is omitted, Graylog will use port 9000 by default.
Default: 127.0.0.1:9000
http_bind_address = AWSPRIVATEIP:9000
#http_bind_address = [2001:db8::1]:9000
HTTP publish URI
The HTTP URI of this Graylog node which is used to communicate with the other Graylog nodes in the cluster and by all
clients using the Graylog web interface.
The URI will be published in the cluster discovery APIs, so that other Graylog nodes will be able to find and connect to this Graylog node.
This configuration setting has to be used if this Graylog node is available on another network interface than $http_bind_address,
for example if the machine has multiple network interfaces or is behind a NAT gateway.
If $http_bind_address contains a wildcard IPv4 address (0.0.0.0), the first non-loopback IPv4 address of this machine will be used.
This configuration setting must not contain a wildcard address!
Default: http://$http_bind_address/
#http_publish_uri = http://AWSPRIVATEIP:9000/
External Graylog URI
The public URI of Graylog which will be used by the Graylog web interface to communicate with the Graylog REST API.
The external Graylog URI usually has to be specified, if Graylog is running behind a reverse proxy or load-balancer
and it will be used to generate URLs addressing entities in the Graylog REST API (see $http_bind_address).
When using Graylog Collector, this URI will be used to receive heartbeat messages and must be accessible for all collectors.
This setting can be overriden on a per-request basis with the “X-Graylog-Server-URL” HTTP request header.
Default: $http_publish_uri
http_external_uri = http://AWSPUBLICIP:9000/
Enable CORS headers for HTTP interface
This is necessary for JS-clients accessing the server directly.
If these are disabled, modern browsers will not be able to retrieve resources from the server.
This is enabled by default. Uncomment the next line to disable it.
#http_enable_cors = false
Enable GZIP support for HTTP interface
This compresses API responses and therefore helps to reduce
overall round trip times. This is enabled by default. Uncomment the next line to disable it.
#http_enable_gzip = false
The maximum size of the HTTP request headers in bytes.
#http_max_header_size = 8192
The size of the thread pool used exclusively for serving the HTTP interface.
#http_thread_pool_size = 16
Here is my nxlog.conf on the Windows machine. As you can see I set the AWS Private IP as the Graylog host. Is this correct?
Both servers are within the same VPC and are allowed to talk to eachother on ALL ports via a security group. Can someone maybe point me in the right direction? Thanks!
Panic Soft
#NoFreeOnExit TRUE
define ROOT C:\Program Files (x86)\nxlog
define CERTDIR %ROOT%\cert
define CONFDIR %ROOT%\conf
define LOGDIR %ROOT%\data
define LOGFILE %LOGDIR%\nxlog.log
LogFile %LOGFILE%
Moduledir %ROOT%\modules
CacheDir %ROOT%\data
Pidfile %ROOT%\data\nxlog.pid
SpoolDir %ROOT%\data
#######################################################################
IIS EXTENTIONS
#######################################################################
Module xm_gelf Module xm_json#######################################################################
IIS NXLOG
#######################################################################
Module xm_csv Fields $date, $time, $s_ip, $cs_method, $cs_uri_stem, $cs_uri_query, $s_port, $cs_username, $c_ip, $cs_User_Agent, $cs_Referer, $sc_status, $sc_substatus, $sc_win32_status, $time_taken FieldTypes string, string, string, string, string, string, string, string, string, string, string, string, string, string, string Delimiter ' ' QuoteChar '"' EscapeControl FALSE UndefValue - Module im_file File "C:\inetpub\logs\LogFiles\W3SVC1\u_ex*" SavePos TRUEExec if $raw_event =~/^#/ drop();\
else\
{\
w3c->parse_csv();\
$EventTime = parsedate($date + " " + $time);\
$EventTime = parsedate($date + " " + $time + "Z");\
$SourceName = "IIS";\
$raw_event = to_json();\
}
Exec $Hostname = hostname_fqdn();
Exec $raw_event =$Hostname + ' IIS-NXLOG ' + $raw_event;
#Use the following line for debugging (uncomment the fileop extension above as well)
#Exec file_write("C:\\Program Files (x86)\\nxlog\\data\\nxlog_output.log", $raw_event);
#######################################################################
/IIS NXLOG
#######################################################################
Module xm_syslog Module xm_charconv AutodetectCharsets iso8859-2, utf-8, utf-16, utf-32 Module xm_exec Module xm_fileop# Check the size of our log file hourly, rotate if larger than 5MB
<Schedule>
Every 1 hour
Exec if (file_exists('%LOGFILE%') and \
(file_size('%LOGFILE%') >= 5M)) \
file_cycle('%LOGFILE%', 8);
</Schedule>
# Rotate our log file every week on Sunday at midnight
<Schedule>
When @weekly
Exec if file_exists('%LOGFILE%') file_cycle('%LOGFILE%', 8);
</Schedule>