Unknown source in GUI

sata11 · July 9, 2019, 8:26am

Hi,
I install Graylog 3 on CentOS and local sidecar with filebeat but in source I have “unknown”. I try add to filebeat configuration “fields.source: ${sidecar.nodeName}” but in source I have unknown.

Needed for Graylog

fields_under_root: true
fields.collector_node_id: {sidecar.nodeName} fields.gl2_source_collector: {sidecar.nodeId}
fields.source: ${sidecar.nodeName}

filebeat.inputs:

input_type: log
paths:
- /var/log/messages
- /var/log/audit/audit.log
- /var/log/cron
- /var/log/maillog
  type: log
  output.logstash:
  hosts: [“10.161.12.220:5044”]
  path:
  data: /var/lib/graylog-sidecar/collectors/filebeat/data
  logs: /var/lib/graylog-sidecar/collectors/filebeat/log

I read other post and users suggest to change timezone in filebeat but I don`t know what can I do that.

jan · July 9, 2019, 9:00am

what filebeat version did you install?

sata11 · July 9, 2019, 11:10am

filebeat-7.2.0-1.x86_64

sata11 · July 10, 2019, 6:47am

I read logs and I have

2019-07-09T09:55:36.798+02:00 ERROR [ConfigurationService] Failed to render template:
freemarker.core.ParseException: Syntax error in template "972c360e-a008-4d08-8e38- 
cd15b9c231a0" in line 5, column 26:
Encountered "}", but was expecting one of:
"false"
"true"
<LESS_THAN>
<LESS_THAN_EQUALS>
<ESCAPED_GT>
<ESCAPED_GTE>
"*"
"**"
"in"
"as"
"using"
<ID>
    at freemarker.core.FMParser.generateParseException(FMParser.java:5768) ~[graylog.jar:?]
    at freemarker.core.FMParser.jj_consume_token(FMParser.java:5627) ~[graylog.jar:?]
    at freemarker.core.FMParser.DotVariable(FMParser.java:1335) ~[graylog.jar:?]
    at freemarker.core.FMParser.PrimaryExpression(FMParser.java:582) ~[graylog.jar:?]
    at freemarker.core.FMParser.UnaryExpression(FMParser.java:706) ~[graylog.jar:?]
    at freemarker.core.FMParser.MultiplicativeExpression(FMParser.java:821) ~[graylog.jar:?]
    at freemarker.core.FMParser.AdditiveExpression(FMParser.java:773) ~[graylog.jar:?]
    at freemarker.core.FMParser.RangeExpression(FMParser.java:953) ~[graylog.jar:?]
    at freemarker.core.FMParser.RelationalExpression(FMParser.java:901) ~[graylog.jar:?]
    at freemarker.core.FMParser.EqualityExpression(FMParser.java:864) ~[graylog.jar:?]
    at freemarker.core.FMParser.AndExpression(FMParser.java:1020) ~[graylog.jar:?]
    at freemarker.core.FMParser.OrExpression(FMParser.java:1042) ~[graylog.jar:?]
    at freemarker.core.FMParser.Expression(FMParser.java:551) ~[graylog.jar:?]
    at freemarker.core.FMParser.StringOutput(FMParser.java:1545) ~[graylog.jar:?]
    at freemarker.core.FMParser.MixedContentElements(FMParser.java:3761) ~[graylog.jar:?]
    at freemarker.core.FMParser.Root(FMParser.java:4458) ~[graylog.jar:?]
    at freemarker.template.Template.<init>(Template.java:253) ~[graylog.jar:?]
    at freemarker.cache.TemplateCache.loadTemplate(TemplateCache.java:549) ~[graylog.jar:?]
    at freemarker.cache.TemplateCache.getTemplateInternal(TemplateCache.java:439) ~[graylog.jar:?]
    at freemarker.cache.TemplateCache.getTemplate(TemplateCache.java:292) ~[graylog.jar:?]
    at freemarker.template.Configuration.getTemplate(Configuration.java:2750) ~[graylog.jar:?]
    at freemarker.template.Configuration.getTemplate(Configuration.java:2599) ~[graylog.jar:?]
    at org.graylog.plugins.sidecar.services.ConfigurationService.renderTemplate(ConfigurationService.java:206) [graylog.jar:?]
    at org.graylog.plugins.sidecar.services.ConfigurationService.renderPreview(ConfigurationService.java:183) [graylog.jar:?]
    at org.graylog.plugins.sidecar.rest.resources.ConfigurationResource.validate(ConfigurationResource.java:388) [graylog.jar:?]
    at org.graylog.plugins.sidecar.rest.resources.ConfigurationResource.validateConfiguration(ConfigurationResource.java:195) [graylog.jar:?]
    at sun.reflect.GeneratedMethodAccessor284.invoke(Unknown Source) ~[?:?]
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_212]
    at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_212]
    at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory$1.invoke(ResourceMethodInvocationHandlerFactory.java:81) [graylog.jar:?]
    at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:144) [graylog.jar:?]
    at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:161) [graylog.jar:?]
    at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$TypeOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:205) [graylog.jar:?]
    at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:99) [graylog.jar:?]
    at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:389) [graylog.jar:?]
    at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:347) [graylog.jar:?]
    at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:102) [graylog.jar:?]
    at org.glassfish.jersey.server.ServerRuntime$2.run(ServerRuntime.java:326) [graylog.jar:?]
    at org.glassfish.jersey.internal.Errors$1.call(Errors.java:271) [graylog.jar:?]
    at org.glassfish.jersey.internal.Errors$1.call(Errors.java:267) [graylog.jar:?]
    at org.glassfish.jersey.internal.Errors.process(Errors.java:315) [graylog.jar:?]
    at org.glassfish.jersey.internal.Errors.process(Errors.java:297) [graylog.jar:?]
    at org.glassfish.jersey.internal.Errors.process(Errors.java:267) [graylog.jar:?]
    at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:317) [graylog.jar:?]
    at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:305) [graylog.jar:?]
    at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:1154) [graylog.jar:?]
    at org.glassfish.jersey.grizzly2.httpserver.GrizzlyHttpContainer.service(GrizzlyHttpContainer.java:384) [graylog.jar:?]
    at org.glassfish.grizzly.http.server.HttpHandler$1.run(HttpHandler.java:224) [graylog.jar:?]
    at com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:181) [graylog.jar:?]
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_212]
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_212]
    at java.lang.Thread.run(Thread.java:748) [?:1.8.0_212]

My config is:

# Needed for Graylog
fields_under_root: true
fields.collector_node_id: ${sidecar.nodeName}
fields.gl2_source_collector: ${sidecar.nodeId}
fields.source: ${sidecar.nodeName}

filebeat.inputs:
- input_type: log
  paths:
    - /var/log/messages
    - /var/log/audit/audit.log
    - /var/log/cron
    - /var/log/maillog
  type: log
output.logstash:
   hosts: ["10.161.12.220:5044"]
path:
  data: /var/lib/graylog-sidecar/collectors/filebeat/data
  logs: /var/lib/graylog-sidecar/collectors/filebeat/log

So “fields.source: ${sidecar.nodeName}” is wrong. I don`t know why. Any suggestion?

jan · July 10, 2019, 9:29am

did you have a field filebeat_source or filebeat_host or similar?

sata11 · July 10, 2019, 10:26am

I try “fields.source: {sidecar.nodeName}" but have error like above. I no try `filebeat_source` or `filebeat_host\' Should I change for "filebeat_source: {sidecar.nodeName}” ?

jan · July 10, 2019, 12:06pm

Sorry for not speak clearly.

if you look in the Graylog UI for the messages, do you have a field called filebeat_source or filebeat_host or any similar that is actually having the hostname you want to be on source?

sata11 · July 10, 2019, 12:32pm

Yes I have name in filebeat_host_name and filebeat_agent_hostname
What can I copy this field to source?

jan · July 11, 2019, 5:50am

IMHO the best solution would be use the processing pipelines for that.

sata11 · July 11, 2019, 1:26pm

I thought I had set up something wrong. I configure nxlog and now see souurce. I try grok., next try pippelines. Thanks for help.

system · July 25, 2019, 1:26pm

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.

Topic		Replies	Views
Filebeat linux source: unknown Graylog Central (peer support) sidecar , filebeat-linux , nosendlogfblx	6	5685	July 1, 2019
Unknown source with filebeat Graylog Central (peer support) sidecar , filebeat-windows , winlogbeat	3	2735	January 22, 2020
Problem with sidecar configuration Graylog Central (peer support) sidecar	1	408	January 22, 2020
Graylog logs are not receiving log with filebeat Graylog Central (peer support) sidecar , filebeat-linux	4	666	May 9, 2023
Filebeat unknown source field Graylog Central (peer support)	9	1712	March 17, 2022

Unknown source in GUI

Needed for Graylog

Related Topics