I am absolutely newbee to graylog. I have downloaded the latest 2.2.x OVA appliance of graylog, installed it.
I have also installed side car to server 2012 r2 x64 bit machine
I have 2 questions.
How do i push windows extended logs (microsoft-printservice-operational) logs (specific events, 307 and 805, xml data) into gray log server.
How do i push csv files on windows share into graylog server.
any blogpost, pointers, help will be great.
Thanks
Nav
Hej Nav,
maybe this blog posting will give you an idea how to get the different event logs.
The filebeat Input is what you are looking for to get some files, line by line into Graylog.
You might find the documentation useful.
/jd
I tried to follow the post, it’s difficult to follow as too much [basic building block] info is on the external links.
I currently focusing on getting CSV file data [generated everday] into the gray. Then i will look into the windows log which are more tricky, at least at my level.
I have managed to install sidecar and the service graylog collector is running now.
I don’t see default filebeat.yml file under generated folder and error is logged
I have searched for a sample filebeat.yml file and configured it to get started. After i restart the service, i get the following.
Exiting: error initializing publisher: No outputs are defined. Please define one under the output section.
The folder contains csv files in 2 subfolders, like csv/FolderA/ and csv/FolderB/
filebeat.yml
prospectors:
- C:\csv**
encoding: utf-8
exclude_lines: ["^#"]
exclude_files: [".zip"]
ignore_older: 240h
registry_file: “C:/ProgramData/filebeat/registry”
Regards,
Navdeep
Configure the output.logstash section of your filebeat.yml to send data to your graylog server IP address. Setup a beats input on your Graylog server. Ensure that you use the same port on both. Beats typically uses TCP port 5044 by default.
I have updated the filebeat.yml with the following
filebeat:
prospectors:
output:
logstash:
hosts: [“172.30.48.145:5044”]
and now i get this in the logs … last few lines
Exiting: Error in initing prospector: Invalid input type: log - C:\csv\DocumentPrinted*
Exiting: Error in initing prospector: Invalid input type: log - C:\csv\DocumentPrinted**
Exiting: Error in initing prospector: Invalid input type: log - C:\csv\DocumentPrinted**
Exiting: Error in initing prospector: Invalid input type: log - C:\csv\DocumentPrinted**
Exiting: Error in initing prospector: Invalid input type: log - C:\csv\DocumentPrinted**
Exiting: error loading config file: yaml: line 4: did not find expected
Exiting: error loading config file: yaml: line 4: did not find expected
Exiting: error loading config file: yaml: line 4: did not find expected
Exiting: error loading config file: yaml: line 4: did not find expected
Exiting: error loading config file: yaml: line 3: did not find expected key
Exiting: error loading config file: yaml: line 3: did not find expected key
looks like there was some whitespace inthe filebeat.yml file, now the config looks like
filebeat:
prospectors:
output:
logstash:
hosts: [“172.30.48.145:5044”]
compression_level: 3
and i don’t get any error in the logs
so what is the next step from here on. I still don’t see anything on the graylog server. I have configure filebeat collector running on port 5044 on graylog server.
On graylog server, under collectors, i see graylog-sidecar-collector with status failing
It says
Status: No configuration found for configured tags!
on client, collector_sidecar.log states
time=“2017-04-25T15:49:41+08:00” level=info msg="[RequestConfiguration] No configuration found for configured tags!"
looks i am missing some configuration here.
Is your beats input in Graylog started?
If you look in your filebeat mybeat logfile on the server shipping the logs, do you see any events being sent (should see that x registry updated) xxxx events sent or something along those lines.
Can you see the connection between the two systems using netstat on the graylog server using something like netstat.
netstat -antp | grep 5044
I checked from the webinterface, it seems to be started
It looks like it has received 183.5kb so far. Can you see the data show up under All Messages? Also try clicking on the Show Received Messages.
It’s empty, do i need to configure collector configurations, i suspect problem is somewhere here.
I tried to follow this article, http://docs.graylog.org/en/2.2/pages/collector_sidecar.html#sidecar-step-by-step
however i am not sure if i have got output part right.
I haven’t used SideCar Collector so I can’t say, but you should be able to get filebeat sending to Graylog without Collector even being involved. Try to minimize the number of variables involved to just get it working, Once it is working, add collector so you can manage filebeat on your server remotely.
By default, filebeat will send its logs in Windows to C:\ProgramData<beat-name>\Logs. Check your mybeat file to see if it shows anything being sent.
i think i am near to the fix but not so near. Do you know how can we specify multiple paths in input under configuration
I tried this but getting error.
[‘C:\csv\DocumentPrinted*.csv’], [‘C:\csv\RenderJobDiag*.csv’]
i checked that, but now the filebeat.yml file is being updated via graylog server through webinterface.
i suspect the problem lies somwhere between inputs and outputs.
does this filebeat.yml content looks fine to you?
filebeat:
prospectors:
it’s auto updated.
do i need to create system>outputs type gelf ?
I have create inputs and outputs under system>collector> manage configuration.
output is set to filebeat with graylog server ip and 5044 port
input is set to filebeat with output create above
No, an output would be if you wanted to send data from Graylog to somewhere else. You are working on getting an input working. Can you try to post a chunk of your filebeat log file. Please encapsulate in ``` code ``` so it gets rendered properly. Your filebeat logs might be under C:\Program Files\graylog\collector-sidecar\logs
File beat log is empty, i was getting too many errors when troubleshooting, so i cleared them after success. I haven’t seen any logs populating in the logs file after that. I have restarted the service few times.
Is that the only log you have for filebeat?