Graylog sidecar with nxlog

symbol-c · April 23, 2019, 2:45pm

Hi, i’m new to graylog and trying to configure the sidecar to send locally stored log files to the graylog-server. below is my sidecar config for nxlog.
please help

define ROOT C:\Program Files (x86)\nxlog

Moduledir %ROOT%\modules
CacheDir %ROOT%\data
Pidfile %ROOT%\data\nxlog.pid
SpoolDir %ROOT%\data
LogFile %ROOT%\data\nxlog.log
LogLevel INFO

<Extension logrotate>
    Module  xm_fileop
    <Schedule>
        When    @daily
        Exec    file_cycle('%ROOT%\data\nxlog.log', 7);
     </Schedule>
</Extension>


<Extension gelfExt>
  Module xm_gelf
  # Avoid truncation of the short_message field to 64 characters.
  ShortMessageLength 65536
</Extension>

<Input eventlog>
        Module im_msvistalog
        PollInterval 1
        SavePos True
        ReadFromLast True
        
        #Channel System
        #<QueryXML>
        #  <QueryList>
        #   <Query Id='1'>
        #    <Select Path='Security'>*[System/Level=4]</Select>
        #    </Query>
        #  </QueryList>
        #</QueryXML>
</Input>

<Input error.log>
        Module   im_file
        File    'C:\GRAYLOG'
</Input>

<Input file>
	Module im_file
	File  C:\GRAYLOG\log.txt
	PollInterval 1
	SavePos	True
	ReadFromLast True
	Recursive False
	RenameCheck False
	Exec $FileName = file_name(); # Send file name with each message
</Input>

<Input in>
    Module  im_file
    File    C:\GRAYLOG\log.txt
</Input>


<Output gelf>
	Module om_udp
	Host 192.168.3.44
	Port 5044
	OutputType  GELF
	<Exec>
	  # These fields are needed for Graylog
	  $gl2_source_collector = '${sidecar.nodeId}';
	  $collector_node_id = '${sidecar.nodeName}';
	</Exec>
</Output>


<Route route-1>
  Path eventlog => gelf
</Route>
<Route route-2>
  Path file => gelf
</Route>

tmacgbay · April 23, 2019, 3:02pm

What else do you see (that we can’t) Log files? Errors? are you accessing the file twice? You have to give a LOT of detail…

symbol-c · April 24, 2019, 8:04am

Hi thanks
below are the logs from the sidecar
ime=“2019-04-23T15:17:03+01:00” level=error msg="[UpdateRegistration] Failed to report collector status to server: Put http://192.168.3.44:9000/api/sidecars/6acea415-ac5f-4796-b094-e206fd92f916: dial tcp 192.168.3.44:9000: connectex: No connection could be made because the target machine actively refused it."
time=“2019-04-23T15:17:14+01:00” level=error msg="[UpdateRegistration] Failed to report collector status to server: Put http://192.168.3.44:9000/api/sidecars/6acea415-ac5f-4796-b094-e206fd92f916: dial tcp 192.168.3.44:9000: connectex: No connection could be made because the target machine actively refused it."
time=“2019-04-23T15:17:26+01:00” level=error msg="[UpdateRegistration] Failed to report collector status to server: Put http://192.168.3.44:9000/api/sidecars/6acea415-ac5f-4796-b094-e206fd92f916: dial tcp 192.168.3.44:9000: connectex: No connection could be made because the target machine actively refused it."
time=“2019-04-24T08:38:21+01:00” level=error msg="[UpdateRegistration] Failed to report collector status to server: Put http://192.168.3.44:9000/api/sidecars/6acea415-ac5f-4796-b094-e206fd92f916: dial tcp 192.168.3.44:9000: connectex: A socket operation was attempted to an unreachable host."
time=“2019-04-24T08:50:06+01:00” level=info msg=“Stopping signal distributor”
time=“2019-04-24T08:50:06+01:00” level=info msg="[nxlog] Stopping"
time=“2019-04-24T08:50:09+01:00” level=info msg=“Starting signal distributor”
time=“2019-04-24T08:50:19+01:00” level=info msg=“Adding process runner for: nxlog”
time=“2019-04-24T08:50:19+01:00” level=info msg="[nxlog] Configuration change detected, rewriting configuration file."
time=“2019-04-24T08:50:19+01:00” level=info msg="[nxlog] Stopping"
time=“2019-04-24T08:50:21+01:00” level=info msg="[nxlog] Starting (svc driver)"
time=“2019-04-24T08:57:27+01:00” level=info msg=“Stopping signal distributor”
time=“2019-04-24T08:57:27+01:00” level=info msg="[nxlog] Stopping"
time=“2019-04-24T08:57:29+01:00” level=info msg=“Starting signal distributor”
time=“2019-04-24T08:57:39+01:00” level=info msg=“Adding process runner for: nxlog”
time=“2019-04-24T08:57:39+01:00” level=info msg="[nxlog] Configuration change detected, rewriting configuration file."
time=“2019-04-24T08:57:39+01:00” level=info msg="[nxlog] Stopping"
time=“2019-04-24T08:57:40+01:00” level=info msg="[nxlog] Starting (svc driver)"

And the logs from nxlog

2019-04-24 08:50:21 ERROR couldn't parse expression at line 48, character 9 in C:\Program Files\Graylog\sidecar\generated\nxlog.conf; invalid character: ':' (0x3a)
2019-04-24 08:50:21 ERROR invalid expression in 'File', string type required at C:\Program Files\Graylog\sidecar\generated\nxlog.conf:48
2019-04-24 08:50:21 ERROR couldn't parse expression at line 59, character 14 in C:\Program Files\Graylog\sidecar\generated\nxlog.conf; invalid character: ':' (0x3a)
2019-04-24 08:50:21 ERROR invalid expression in 'File', string type required at C:\Program Files\Graylog\sidecar\generated\nxlog.conf:59
2019-04-24 08:50:21 ERROR module 'file' has configuration errors, not adding to route 'route-2' at C:\Program Files\Graylog\sidecar\generated\nxlog.conf:80
2019-04-24 08:50:21 ERROR route route-2 is not functional without input modules, ignored at C:\Program Files\Graylog\sidecar\generated\nxlog.conf:80
2019-04-24 08:50:21 WARNING not starting unused module error.log
2019-04-24 08:50:21 WARNING not starting unused module file
2019-04-24 08:50:21 WARNING not starting unused module in
2019-04-24 08:50:21 INFO nxlog-ce-2.10.2150 started
2019-04-24 08:50:21 INFO reconnecting in 1 seconds
2019-04-24 08:50:21 ERROR ### ASSERTION FAILED at line 849 in module.c/resume_senders(): "j > 0" ###
2019-04-24 08:50:22 WARNING Due to a limitation in the Windows EventLog subsystem, a query cannot contain more than 256 sources.
2019-04-24 08:50:22 WARNING The following sources are omitted to avoid exceeding the limit in the generated query:  Microsoft-Windows-SMBServer/Connectivity Microsoft-Windows-SMBServer/Operational Microsoft-Windows-SMBServer/Security Microsoft-Windows-SMBWitnessClient/Admin Microsoft-Windows-SMBWitnessClient/Informational Microsoft-Windows-StateRepository/Operational Microsoft-Windows-StateRepository/Restricted Microsoft-Windows-Storage-ClassPnP/Operational Microsoft-Windows-Storage-Storport/Health Microsoft-Windows-Storage-Storport/Operational Microsoft-Windows-Storage-Tiering/Admin Microsoft-Windows-StorageManagement/Operational Microsoft-Windows-StorageSpaces-Driver/Diagnostic Microsoft-Windows-StorageSpaces-Driver/Operational Microsoft-Windows-StorageSpaces-ManagementAgent/WHC Microsoft-Windows-StorageSpaces-SpaceManager/Diagnostic Microsoft-Windows-StorageSpaces-SpaceManager/Operational Microsoft-Windows-Store/Operational Microsoft-Windows-SystemSettingsThreshold/Operational Microsoft-Windows-TaskSche
2019-04-24 08:50:22 INFO reconnecting in 1 seconds
2019-04-24 08:50:22 ERROR ### ASSERTION FAILED at line 849 in module.c/resume_senders(): "j > 0" ###
2019-04-24 08:50:23 INFO reconnecting in 1 seconds
2019-04-24 08:50:23 ERROR ### ASSERTION FAILED at line 849 in module.c/resume_senders(): "j > 0" ###
2019-04-24 08:50:24 INFO reconnecting in 1 seconds
2019-04-24 08:50:24 ERROR ### ASSERTION FAILED at line 849 in module.c/resume_senders(): "j > 0" ###

Not sure exactly what to do now

symbol-c · April 24, 2019, 8:51am

this is what i have now from nxlog

2019-04-24 09:29:30 ERROR invalid expression in 'File', string type required at C:\Program Files\Graylog\sidecar\generated\nxlog.conf:44
2019-04-24 09:29:31 WARNING stopping nxlog service
2019-04-24 09:29:31 WARNING nxlog-ce received a termination request signal, exiting...
2019-04-24 09:29:31 INFO reconnecting in 1 seconds
2019-04-24 09:29:31 ERROR ### ASSERTION FAILED at line 849 in module.c/resume_senders(): "j > 0" ###
2019-04-24 09:29:31 ERROR couldn't parse expression at line 44, character 9 in C:\Program Files\Graylog\sidecar\generated\nxlog.conf; invalid character: ':' (0x3a)
2019-04-24 09:29:31 ERROR invalid expression in 'File', string type required at C:\Program Files\Graylog\sidecar\generated\nxlog.conf:44
2019-04-24 09:29:31 ERROR couldn't parse expression at line 55, character 14 in C:\Program Files\Graylog\sidecar\generated\nxlog.conf; invalid character: ':' (0x3a)
2019-04-24 09:29:31 ERROR invalid expression in 'File', string type required at C:\Program Files\Graylog\sidecar\generated\nxlog.conf:55
2019-04-24 09:29:31 ERROR module 'file' has configuration errors, not adding to route 'route-2' at C:\Program Files\Graylog\sidecar\generated\nxlog.conf:76
2019-04-24 09:29:31 ERROR route route-2 is not functional without input modules, ignored at C:\Program Files\Graylog\sidecar\generated\nxlog.conf:76
2019-04-24 09:29:31 WARNING not starting unused module file
2019-04-24 09:29:31 WARNING not starting unused module in
2019-04-24 09:29:31 INFO nxlog-ce-2.10.2150 started
2019-04-24 09:29:31 ERROR couldn't connect to udp socket on 0.0.0.0:5044; The requested address is not valid in its context.  
2019-04-24 09:29:32 WARNING Due to a limitation in the Windows EventLog subsystem, a query cannot contain more than 256 sources.
2019-04-24 09:29:32 WARNING The following sources are omitted to avoid exceeding the limit in the generated query:  Microsoft-Windows-SMBServer/Connectivity Microsoft-Windows-SMBServer/Operational Microsoft-Windows-SMBServer/Security Microsoft-Windows-SMBWitnessClient/Admin Microsoft-Windows-SMBWitnessClient/Informational Microsoft-Windows-StateRepository/Operational Microsoft-Windows-StateRepository/Restricted Microsoft-Windows-Storage-ClassPnP/Operational Microsoft-Windows-Storage-Storport/Health Microsoft-Windows-Storage-Storport/Operational Microsoft-Windows-Storage-Tiering/Admin Microsoft-Windows-StorageManagement/Operational Microsoft-Windows-StorageSpaces-Driver/Diagnostic Microsoft-Windows-StorageSpaces-Driver/Operational Microsoft-Windows-StorageSpaces-ManagementAgent/WHC Microsoft-Windows-StorageSpaces-SpaceManager/Diagnostic Microsoft-Windows-StorageSpaces-SpaceManager/Operational Microsoft-Windows-Store/Operational Microsoft-Windows-SystemSettingsThreshold/Operational Microsoft-Windows-TaskSche

and the sidedcar

t ime=“2019-04-24T09:28:51+01:00” level=info msg="[nxlog] Starting (svc driver)"
time=“2019-04-24T09:29:30+01:00” level=info msg="[nxlog] Configuration change detected, rewriting configuration file."
time=“2019-04-24T09:29:31+01:00” level=info msg="[nxlog] Stopping"
time=“2019-04-24T09:29:31+01:00” level=info msg="[nxlog] Starting (svc driver)"

Karlis · April 24, 2019, 10:11am

self explanatory

It seems this should be double quoted\

<Input file>
	Module im_file
	File  "C:\GRAYLOG\log.txt"

The same

<Input in>
    Module  im_file
    File    "C:\GRAYLOG\log.txt"
</Input>

system · May 8, 2019, 10:20am

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.

Topic		Replies	Views
Nxlog file input Graylog Central (peer support) sidecar , nxlog , nodatanx	2	2594	April 12, 2018
Graylog Sidecar and static windows server .log files Graylog Central (peer support) sidecar , nxlog , nodatanx	2	1415	June 29, 2017
Configuring graylog sidecar log collection Graylog Central (peer support) sidecar	7	2407	October 14, 2020
Nxlog event consistency Graylog Central (peer support) sidecar , windows , nxlog , docker	9	735	August 9, 2023
Feed XML files into Graylog Graylog Central (peer support) sidecar , nxlog , nodatanx	2	1614	June 27, 2019

Graylog sidecar with nxlog

Related topics