Don't show any activity in Dashboard

thewsbarreto · December 1, 2021, 4:34pm

That’s my first time using Graylog system and I don’t have any idea what’s happen with my server or my workstation.

I installed the last version of graylog-server and agent version 1.1.0

Server: Ubuntu 20.04 LTS
Workstation: Windows 10 Pro 64x (only for tests)
System: Graylog Open
Sidecar | Winlogbeat

C:\Program Files\Graylog\sidecar\logs

time="2021-12-01T11:56:07-03:00" level=info msg="Stopping signal distributor" 
time="2021-12-01T11:56:07-03:00" level=info msg="Starting signal distributor" 
time="2021-12-01T11:56:17-03:00" level=info msg="No configurations assigned to this instance. Skipping configuration request."

/var/log/graylog-sidecar

time="2021-12-01T14:15:30Z" level=info msg="Stopping signal distributor"
time="2021-12-01T14:19:18Z" level=info msg="Starting signal distributor"
time="2021-12-01T14:19:29Z" level=error msg="[UpdateRegistration] Failed to report collector status to server: Put \"http://10.0.12.68:9000/api/sidecars/HIDDEN-TOKEN-API\": dial tcp 10.0.12.68:9000: connect: connection refused"
time="2021-12-01T14:19:39Z" level=error msg="[UpdateRegistration] Failed to report collector status to server: Put \"http://10.0.12.68:9000/api/sidecars/HIDDEN-TOKEN-API\": dial tcp 10.0.12.68:9000: connect: connection refused"
time="2021-12-01T14:19:49Z" level=error msg="[UpdateRegistration] Failed to report collector status to server: Put \"http://10.0.12.68:9000/api/sidecars/HIDDEN-TOKEN-API\": dial tcp 10.0.12.68:9000: connect: connection refused"
time="2021-12-01T14:19:59Z" level=error msg="[UpdateRegistration] Failed to report collector status to server: Put \"http://10.0.12.68:9000/api/sidecars/HIDDEN-TOKEN-API\": dial tcp 10.0.12.68:9000: connect: connection refused"
time="2021-12-01T14:20:09Z" level=error msg="[UpdateRegistration] Failed to report collector status to server: Put \"http://10.0.12.68:9000/api/sidecars/HIDDEN-TOKEN-API\": dial tcp 10.0.12.68:9000: connect: connection refused"
time="2021-12-01T14:20:19Z" level=error msg="[UpdateRegistration] Failed to report collector status to server: Put \"http://10.0.12.68:9000/api/sidecars/HIDDEN-TOKEN-API\": dial tcp 10.0.12.68:9000: connect: connection refused"
time="2021-12-01T14:20:29Z" level=error msg="[UpdateRegistration] Failed to report collector status to server: Put \"http://10.0.12.68:9000/api/sidecars/HIDDEN-TOKEN-API\": dial tcp 10.0.12.68:9000: connect: connection refused"
time="2021-12-01T14:20:39Z" level=error msg="[UpdateRegistration] Failed to report collector status to server: Put \"http://10.0.12.68:9000/api/sidecars/HIDDEN-TOKEN-API\": dial tcp 10.0.12.68:9000: connect: connection refused"
time="2021-12-01T14:20:51Z" level=info msg="No configurations assigned to this instance. Skipping configuration request."
time="2021-12-01T14:30:23Z" level=error msg="[UpdateRegistration] Failed to report collector status to server: Put \"http://10.0.12.68:9000/api/sidecars/HIDDEN-TOKEN-API\": EOF"
time="2021-12-01T14:30:33Z" level=error msg="[UpdateRegistration] Failed to report collector status to server: Put \"http://10.0.12.68:9000/api/sidecars/HIDDEN-TOKEN-API\": dial tcp 10.0.12.68:9000: connect: connection refused"

/var/log/graylog-server

2021-12-01T14:30:34.414Z ERROR [MongoAuditLogPeriodical] Not running cleanup for auditlog entries in MongoDB because there is no valid license.
2021-12-01T14:30:34.416Z INFO  [Periodicals] Starting [org.graylog.plugins.files.CleanupPeriodical] periodical in [0s], polling every [86400s].
2021-12-01T14:30:34.430Z INFO  [LookupTableService] Data Adapter watchlist-mongo/61a67925ea6be542db0dfb66 [@31e6e907] STARTING
2021-12-01T14:30:34.454Z INFO  [LookupTableService] Data Adapter watchlist-mongo/61a67925ea6be542db0dfb66 [@31e6e907] RUNNING
2021-12-01T14:30:34.583Z INFO  [LookupTableService] Cache watchlist-cache/61a67925ea6be542db0dfb64 [@63e768ee] STARTING
2021-12-01T14:30:34.584Z INFO  [LookupTableService] Cache watchlist-cache/61a67925ea6be542db0dfb64 [@63e768ee] RUNNING
2021-12-01T14:30:34.590Z INFO  [LookupTableService] Starting lookup table watchlist/61a67925ea6be542db0dfb68 [@47232629] using cache watchlist-cache/61a67925ea6be542db0dfb64 [@63e768ee], data adapter watchlist-mongo/61a67925ea6be542db0dfb66 [@31e6e907]
2021-12-01T14:30:40.471Z INFO  [NetworkListener] Started listener bound to [10.0.12.68:9000]
2021-12-01T14:30:40.472Z INFO  [HttpServer] [HttpServer] Started.
2021-12-01T14:30:40.472Z INFO  [JerseyService] Started REST API at <10.0.12.68:9000>
2021-12-01T14:30:40.472Z INFO  [ServiceManagerListener] Services are healthy
2021-12-01T14:30:40.473Z INFO  [ServerBootstrap] Services started, startup times in ms: {FailureHandlingService [RUNNING]=202, DevelopmentDirectoryObserverService [RUNNING]=208, PrometheusExporter [RUNNING]=208, JobSchedulerService [RUNNING]=208, OutputSetupService [RUNNING]=209, UrlWhitelistService [RUNNING]=210, LocalKafkaMessageQueueWriter [RUNNING]=213, LocalKafkaMessageQueueReader [RUNNING]=213, InputSetupService [RUNNING]=214, GracefulShutdownService [RUNNING]=214, BufferSynchronizerService [RUNNING]=236, LocalKafkaJournal [RUNNING]=242, ConfigurationEtagService [RUNNING]=262, EtagService [RUNNING]=263, ProcessingConfigurationManager [RUNNING]=269, UserSessionTerminationService [RUNNING]=281, MongoDBProcessingStatusRecorderService [RUNNING]=287, StreamCacheService [RUNNING]=395, PeriodicalsService [RUNNING]=449, LookupTableService [RUNNING]=618, JerseyService [RUNNING]=6504}
2021-12-01T14:30:40.480Z INFO  [ServerBootstrap] Graylog server up and running.
2021-12-01T14:30:40.481Z ERROR [AuditLogger] Unable to write audit log entry because there is no valid license.
2021-12-01T14:30:40.482Z INFO  [InputSetupService] Triggering launching persisted inputs, node transitioned from Uninitialized [LB:DEAD] to Running [LB:ALIVE]
2021-12-01T14:30:40.523Z INFO  [InputStateListener] Input [Beats/61a689209d3c440b193d4395] is now STARTING
2021-12-01T14:30:40.609Z INFO  [InputStateListener] Input [Beats/61a689209d3c440b193d4395] is now RUNNING
2021-12-01T14:30:40.625Z WARN  [AbstractTcpTransport] receiveBufferSize (SO_RCVBUF) for input Beats2Input{title=sidecar, type=org.graylog.plugins.beats.Beats2Input, nodeId=null} (channel [id: 0x292e6264, L:/0:0:0:0:0:0:0:0%0:5044]) should be >= 1048576 but is 425984.
2021-12-01T14:30:55.356Z ERROR [AuditLogger] Unable to write audit log entry because there is no valid license.
2021-12-01T14:30:56.174Z ERROR [AuditLogger] Unable to write audit log entry because there is no valid license.
2021-12-01T14:32:09.643Z ERROR [AuditLogger] Unable to write audit log entry 
2021-12-01T15:30:34.414Z ERROR [MongoAuditLogPeriodical] Not running cleanup for auditlog entries in MongoDB because there is no valid license.
2021-12-01T16:30:34.413Z ERROR [MongoAuditLogPeriodical] Not running cleanup for auditlog entries in MongoDB because there is no valid license.

Please, help me guys!

gsmith · December 3, 2021, 1:29am

Hello,

Could we see the following configurations?

Graylog Sidecar
Winlogbeat

I scanned over you logs and this is what I got out of it.

level=info msg="No configurations assigned to this instance. Skipping configuration request."

So your Web UI has no configuration attached to the sidecar.

level=error msg="[UpdateRegistration] Failed to report collector status to server: Put \"http://10.0.12.68:9000/api/sidecars/HIDDEN-TOKEN-API\": dial tcp 10.0.12.68:9000: connect: connection refused"

This probably because you didn’t configuration the sidecar properly or you forgot to attach a configuration on the Web UI. This is located System/Sidecar → Configuration. Once your node appears on the Web UI then attach Winlogbeat configuration to it.

ERROR [MongoAuditLogPeriodical] Not running cleanup for auditlog entries in MongoDB because there is no valid license.
ERROR [AuditLogger] Unable to write audit log entry because there is no valid license.

This is because you have the Enterprise plugin installed a with no license. No major issue here. Either remove the enterprise plugins or get a license will resolve that.

Please read over this post.

GRAYLOG SIDECAR INSTALLATION

Here is some basic steps you may need.

Create a Beats INPUT
Install Graylog sidecar on windows
Configure Graylog sidecar to reach your Graylog Server.
These settings to get your sidecar started.

    server_url: 
    server_api_token:
    node_name: 
    send_status: true

Insure you can see you node on the Web UI
Create a configuration located under Log Collectors.
Attach configuration to your node from the Web UI

You should have something like this.

Hope that helps

thewsbarreto · December 6, 2021, 2:36pm

Hey @gsmith thanks for your reply!

So I followed your steps and the documentation but still doesn’t work it.

Look printscreen below:

C:\Program Files\Graylog\sidecar\logs\sidecar

time="2021-12-06T11:29:35-03:00" level=info msg="Starting signal distributor" 
time="2021-12-06T11:29:45-03:00" level=info msg="Adding process runner for: winlogbeat" 
time="2021-12-06T11:29:45-03:00" level=info msg="[winlogbeat] Configuration change detected, rewriting configuration file." 
time="2021-12-06T11:29:48-03:00" level=info msg="[winlogbeat] Starting (svc driver)"

/var/log/graylog-sidecar/sidecar.log

time="2021-12-01T17:32:00Z" level=info msg="Starting signal distributor"
time="2021-12-01T17:32:10Z" level=info msg="No configurations assigned to this instance. Skipping configuration request."
time="2021-12-06T13:08:05Z" level=error msg="[UpdateRegistration] Failed to report collector status to server: Put \"http://10.0.12.68:9000/api/sidecars/47c5412e-f645-48f8-bf1a-57e82e2a55c9\": read tcp 10.0.12.68:47292->10.0.12.68:9000: read: connection reset by peer"

/var/log/graylog-server/server.log

[127]: index [graylog_0], type [_doc], id [f7f88c30-56a0-11ec-8dc0-842b2b7ab3af], message [ElasticsearchException[Elasticsearch exception [type=mapper_parsing_exception, reason=failed to parse field [winlogbeat_winlog_event_data_param1] of type [date] in document with id 'f7f88c30-56a0-11ec-8dc0-842b2b7ab3af'. Preview of field's value: 'Video.UI']]; nested: ElasticsearchException[Elasticsearch exception [type=illegal_argument_exception, reason=failed to parse date field [Video.UI] with format [strict_date_optional_time||epoch_millis]]]; nested: ElasticsearchException[Elasticsearch exception [type=date_time_parse_exception, reason=Failed to parse with all enclosed parsers]];]
2021-12-06T14:31:08.071Z ERROR [AbstractTcpTransport] Error in Input [Beats/61a689209d3c440b193d4395] (channel [id: 0xbcbf41ad, L:/10.0.12.68:5044 ! R:/10.0.32.42:50374]) (cause io.netty.channel.unix.Errors$NativeIoException: readAddress(..) failed: Connection reset by peer)

gsmith · December 6, 2021, 10:19pm

Hello,

Overview looks good since the sidecar is shown. But how about the administration section?
I did a quick mockup of what you should see.

It would help showing your configurations ( i.e. sidecar and winlogbeat)

thewsbarreto · December 7, 2021, 12:45pm

thewsbarreto · December 7, 2021, 12:47pm

This IP 10.0.12.68:5044 (of my server). I also changed to the workstation IP and nothing works.

tmacgbay · December 7, 2021, 4:18pm

Do you have a beats Input set up under System/Inputs? Post the configuration for that?

thewsbarreto · December 7, 2021, 4:40pm

tmacgbay · December 7, 2021, 6:49pm

When you say “Nothing works” Do you simply mean that you are not receiving messages in Graylog? If yes, is that all that is an issue - be sure to list all the specific issues you have…

For the windows machine you are getting an error on a field type that may cause failures - unless you have created a specific elasticsearch template or an extractor for “winlogbeat_winlog_event_data_param1” you should be able to rotate the Graylog_* index and Elasticsearch will pick up the new type. error →
..reason=failed to parse field [winlogbeat_winlog_event_data_param1] of type [date] in document with id 'f7f88c30-56a0-11ec-8dc0-842b2b7ab3af'. Preview of field's value: 'Video.UI']]; nested: ElasticsearchException[Elasticsearch exception [type=illegal_argument_exception, reason=failed to parse date field [Video.UI] with format [strict_date_optional_time||epoch_millis]]]; ...

The second Graylog error shows a connection reset at the peer/client, which could be a firewall, could be related that your two machines are on different subnets. Assuming you are a /24, Graylog is in 10.0.12.x and the other machine is in 10.0.32.x
Error in Input [Beats/61a689209d3c440b193d4395] (channel [id: 0xbcbf41ad, L:/10.0.12.68:5044 ! R:/10.0.32.42:50374]) (cause io.netty.channel.unix.Errors$NativeIoException: readAddress(..) failed: Connection reset by peer)

With the Linux test machine, have you created a separate beats configuration and applied it? Your pictures don’t show one created or applied.

Please post which versions of Elasticsearch and MongoDB you are on, that can make a difference. When you are posting a configuration, please post the test and use the forum tools (</>) so they are readable - while pictures are helpful, the direct code is better.

gsmith · December 7, 2021, 10:38pm

Hello,
@tmacgbay good catch I over looked that error.

ERROR [AbstractTcpTransport] Error in Input [Beats/61a689209d3c440b193d4395] (channel [id: 0xbcbf41ad, L:/10.0.12.68:5044 ! R:/10.0.32.42:50374]) (cause io.netty.channel.unix.Errors$NativeIoException: readAddress(…) failed: Connection reset by peer)

@thewsbarreto I would highly recommend @tmacgbay suggested first.

If you are still having issues, here are some troubleshooting tips.

Try setting your Beats Input to Global.

Your Winlogbeat configuration file should point to your Graylog Server /w Port. Make sure you have static IP Addresses for both your Devices.

I’m not sure if your familiar with tcpdump but here are a couple ways to find out if anything is sent.

Filtering by Host

sudo tcpdump -n host 192.168.1.185 (Windows Device IP)

Filtering by Port

sudo tcpdump -n port 5044

Hope that helps

system · December 21, 2021, 10:38pm

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.

Topic		Replies	Views
New Graylog Sidecar Running But Not Sending Data Graylog Central (peer support) sidecar , winlogbeat	2	1526	July 29, 2019
Failed to report collector status to server Graylog Central (peer support) sidecar , filebeat-linux , filebeat-windows	9	2221	April 12, 2021
Graylog Sidecar doesn´t send logs Graylog Central (peer support) sidecar	4	44	November 11, 2024
Unable to get Windows Agent Logs Graylog Central (peer support) sidecar , filebeat-windows , winlogbeat	10	1129	July 26, 2018
Graylog Collector Sidecar Linux Filebeat Failing Graylog Central (peer support) sidecar , filebeat-linux , filebeat-windows	2	2107	June 15, 2017

Don't show any activity in Dashboard

Related topics