How to import apache logs to graylog


(gany) #1

Hello everyone, I was asked to configure graylog for centralized log monitoring. I have configured graylog on Centos 7. I have exported rsyslog to graylog but now I need to export apache logs to graylog. Please advise how it can accomplished.

Regards,

Hussain Gany


(Jan Doberstein) #2

He Hussain,

you would need to ship your Apache logs to Graylog. That can be done by using shipper, like filebeat your rsyslog or nxlog (or any other you like).

What fits your environment you need to decide yourself. Personal I would you filebeat. Managed manually or by the collector-sidecar.


#3

@hussain

We use filebeat with collectore-sidecar, it works well.

Tip:
We use an own json logformat, like

#secweb_extended version JSON format
LogFormat "{ \"host\":\"%h\",\
\"GEOIP_COUNTRY_CODE\":\" %{GEOIP_COUNTRY_CODE}e\",\
\"User\":\"%u\",\
\"timestamp\":%{%s}t,\
.....
 \"short_message\":\"%r\" }" secweb_extended_json

This makes it easier to parse


(gany) #4

Hi Jan,

I have installed graylog 1.2.2 on Centos 7. What version of collector-sidecar should be installed? I have installed “collector-sidecar-0.1.5-1.x86_64” and nxlog in client machine but I am unable to start the collector-sidecar service as it says Unrecognized service.


(Jan Doberstein) #5

that historical version of Graylog does not support Collector Sidecar - it does also not have a beats input working with filebeat.

You should use a stable supported version of Graylog which is 2.4.5 at the time of writing.


(gany) #6

Thanks Jan for your suggestion. Tomorrow morning let me start with installation of graylog 2.4.5.


(gany) #7

Hi Jan,

I was able to export Rsyslog to Graylog when I was using graylog 1.2.2. After successful installation of Graylog 2.4.5, I am facing difficulties in exporting rsyslog of remote machine.Please advise.

vi /etc/rsyslog.conf

$ModLoad imudp
$UDPServerRun 1514

   *.* @10.6.14.82:1514;RSYSLOG_SyslogProtocol23Format

(Jan Doberstein) #8

Did you have on the IP 10.6.14.82 on Port 1514 your UDP Syslog Input running at the Graylog server?


(gany) #9

image


(Jan Doberstein) #10

you have the port 1514 on some IP open accepting connections - ok.

But is the IP you use in the rsyslog configuration available at the Graylog server? Did you check if no firewall in between is blocking? Can the sending server reach the Graylog server?


(gany) #11

I am getting above error in graylog.log file.


(gany) #12

Open port 1514 has to checked in Client server or graylog server (10.6.14.82). Please advise.


(Jochen) #13

If Graylog is unable to connect to Elasticsearch, it won’t be able to index new messages.

Check the logs of your Elasticsearch node(s): http://docs.graylog.org/en/2.4/pages/configuration/file_location.html


(gany) #14

Please find the above log from Elasticsearch.


(Jochen) #15

Are Elasticsearch and Graylog running on the same machine?

If not, using the loopback interface (https://en.wikipedia.org/wiki/Loopback#Virtual_loopback_interface) won’t work.


(gany) #16

Yes both are running in the same machine.


(Jochen) #17

What’s the output of the following command on the machine running Graylog?

# curl -i 'http://127.0.0.1:9200/?pretty'

(gany) #18

image


(Jochen) #19

Then Graylog should be able to communicate with Elasticsearch at http://127.0.0.1:9200.

Try restarting Graylog and watch the logs of both, Graylog and Elasticsearch.


(gany) #20

Hi Jochen,

I have configured graylog 2.4.5 on another machine but now I am able to collect windows logs. Still I am unable to collect Syslog from Linux machine. Please correct me where I have made mistake?