Hi all,
For some reason my Graylog’s server.log file isn’t updating. I’ve been trying to troubleshoot it but the log file hasn’t been updated in months for some reason. I deleted the old log file and recreated it and tried to restart Graylog’s service. Apparently the restart worked, but still nothing in the log file. Any ideas?
/var/log/graylog-server/server.log is the path to my log.
did you tried to configure something with the log4j.xml? Like creating API Access Logs or similar?
Hi Jan,
Here’s my XML file. I haven’t edited it except to change the xml version which for some reason was previously “0.0”.
<?xml version="1.0" encoding="UTF-8"?> <Configuration packages="org.graylog2.log4j" shutdownHook="disable"> <Appenders> <RollingFile name="rolling-file" fileName="/var/log/upstart/graylog-server.log" filePattern="/var/log/upstart/graylog-server,log.%i.gz"> <PatternLayout pattern="%d{yyyy-MM-dd'T'HH:mm:ss.SSSXXX} %-5p [%c{1}] %m%n"/> <Policies> <SizeBasedTriggeringPolicy size="2GB"/> </Policies> <DefaultRolloverStrategy max="5" fileIndex="min"/> </RollingFile> <!-- Internal Graylog log appender. Please do not disable. This makes internal log messages available via REST calls. --> <Memory name="graylog-internal-logs" bufferSize="500"/> </Appenders> <Loggers> <!-- Application Loggers --> <Logger name="org.graylog2" level="info"/> <Logger name="com.github.joschi.jadconfig" level="warn"/> <!-- This emits a harmless warning for ActiveDirectory every time which we can't work around :( --> <Logger name="org.apache.directory.api.ldap.model.message.BindRequestImpl" level="error"/> <!-- Prevent DEBUG message about Lucene Expressions not found. --> <Logger name="org.elasticsearch.script" level="warn"/> <!-- Disable messages from the version check --> <Logger name="org.graylog2.periodical.VersionCheckThread" level="off"/> <!-- Suppress crazy byte array dump of Drools --> <Logger name="org.drools.compiler.kie.builder.impl.KieRepositoryImpl" level="warn"/> <!-- Silence chatty natty --> <Logger name="com.joestelmach.natty.Parser" level="warn"/> <!-- Silence Kafka log chatter --> <Logger name="kafka.log.Log" level="warn"/> <Logger name="kafka.log.OffsetIndex" level="warn"/> <!-- Silence useless session validation messages --> <Logger name="org.apache.shiro.session.mgt.AbstractValidatingSessionManager" level="warn"/> <Root level="warn"> <AppenderRef ref="rolling-file"/> <AppenderRef ref="graylog-internal-logs"/> </Root> </Loggers> </Configuration>
It appears that your log file is configured to be: /var/log/upstart/graylog-server.log
I would check that location and possibly change it so that the logs are stored where you expect to find them 
Thank you Ponet. So I looked at that log and when I view it in real time using tail, I keep seeing these messages:
2019-01-09 08:10:13,331 main ERROR Unable to invoke factory method in class org.apache.logging.log4j.core.appender.RollingFileAppender for element RollingFile: java.lang.IllegalStateException: No factory method found for class org.apache.logging.log4j.core.appender.RollingFileAppender java.lang.IllegalStateException: No factory method found for class org.apache.logging.log4j.core.appender.RollingFileAppender
at org.apache.logging.log4j.core.config.plugins.util.PluginBuilder.findFactoryMethod(PluginBuilder.java:229)
at org.apache.logging.log4j.core.config.plugins.util.PluginBuilder.build(PluginBuilder.java:134)
at org.apache.logging.log4j.core.config.AbstractConfiguration.createPluginObject(AbstractConfiguration.java:958)
at org.apache.logging.log4j.core.config.AbstractConfiguration.createConfiguration(AbstractConfiguration.java:898)
at org.apache.logging.log4j.core.config.AbstractConfiguration.createConfiguration(AbstractConfiguration.java:890)
at org.apache.logging.log4j.core.config.AbstractConfiguration.doConfigure(AbstractConfiguration.java:513)
at org.apache.logging.log4j.core.config.AbstractConfiguration.initialize(AbstractConfiguration.java:237)
at org.apache.logging.log4j.core.config.AbstractConfiguration.start(AbstractConfiguration.java:249)
at org.apache.logging.log4j.core.LoggerContext.setConfiguration(LoggerContext.java:545)
at org.apache.logging.log4j.core.LoggerContext.reconfigure(LoggerContext.java:617)
at org.apache.logging.log4j.core.LoggerContext.reconfigure(LoggerContext.java:634)
at org.apache.logging.log4j.core.LoggerContext.start(LoggerContext.java:229)
at org.apache.logging.log4j.core.impl.Log4jContextFactory.getContext(Log4jContextFactory.java:152)
at org.apache.logging.log4j.core.impl.Log4jContextFactory.getContext(Log4jContextFactory.java:45)
at org.apache.logging.log4j.LogManager.getContext(LogManager.java:194)
at org.apache.logging.log4j.spi.AbstractLoggerAdapter.getContext(AbstractLoggerAdapter.java:122)
at org.apache.logging.slf4j.Log4jLoggerFactory.getContext(Log4jLoggerFactory.java:43)
at org.apache.logging.log4j.spi.AbstractLoggerAdapter.getLogger(AbstractLoggerAdapter.java:46)
at org.apache.logging.slf4j.Log4jLoggerFactory.getLogger(Log4jLoggerFactory.java:29)
at org.slf4j.LoggerFactory.getLogger(LoggerFactory.java:358)
at org.slf4j.LoggerFactory.getLogger(LoggerFactory.java:383)
at org.graylog2.bootstrap.CmdLineTool.(CmdLineTool.java:88)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at com.github.rvesse.airline.parser.ParserUtil.createInstance(ParserUtil.java:39)
at com.github.rvesse.airline.DefaultCommandFactory.createInstance(DefaultCommandFactory.java:25)
at com.github.rvesse.airline.parser.ParserUtil.createInstance(ParserUtil.java:94)
at com.github.rvesse.airline.parser.ParseResult.getCommand(ParseResult.java:114)
at com.github.rvesse.airline.parser.command.CliParser.parse(CliParser.java:54)
at com.github.rvesse.airline.Cli.parse(Cli.java:107)
at com.github.rvesse.airline.Cli.parse(Cli.java:93)
at org.graylog2.bootstrap.Main.main(Main.java:43)
2019-01-09 08:10:13,339 main ERROR Null object returned for RollingFile in Appenders.
2019-01-09 08:10:13,364 main ERROR Unable to locate appender “rolling-file” for logger config “root”
Any ideas?
Thanks again.
How did you install Graylog? What version of Graylog are you running?
I’ve not come across that error while working with Graylog. That error makes it look pretty broken.
If I were you, I would do a fresh install - Is that an option for you?
I installed it a while ago but I kept some documentation about how I did it…
Installed it on a fresh Ubuntu 14.04 LTS.
wget http://packages.graylog2.org/repo/packages/graylog-2.4-repository_latest.deb
sudo dpkg –I graylog-2.4-repository_latest.deb
sudo apt-get update
sudo apt-get install graylog-serversudo service graylog-server start
OK. I’ve only attempted running on Ubuntu 16.04 and Ubuntu 18.04 but, I can’t imagine there will be any issue with that.
Have you also installed Elasticsearch and MongoDB?
Please see: http://docs.graylog.org/en/2.4/pages/installation/os/ubuntu.html
sorry dude - you fucked up your log4j configuration.
Take the default from github and you should have no problems.
@Ponet Yep, got those both running.
@jan Haha, I appreciate the bluntness. I’m not honestly too familiar though…can I use any log4j ‘default’ config, or is there one specifically for Graylog?
@KuboMD The log4j2.xml file I have that is working in my system is as below.
<?xml version="1.0" encoding="UTF-8"?>
<Configuration packages="org.graylog2.log4j" shutdownHook="disable">
<Appenders>
<RollingFile name="rolling-file" fileName="/var/log/graylog-server/server.log" filePattern="/var/log/graylog-server/server.log.%i.gz">
<PatternLayout pattern="%d{yyyy-MM-dd'T'HH:mm:ss.SSSXXX} %-5p [%c{1}] %m%n"/>
<Policies>
<SizeBasedTriggeringPolicy size="50MB"/>
</Policies>
<DefaultRolloverStrategy max="10" fileIndex="min"/>
</RollingFile>
<!-- Internal Graylog log appender. Please do not disable. This makes internal log messages available via REST calls. -->
<Memory name="graylog-internal-logs" bufferSize="500"/>
</Appenders>
<Loggers>
<!-- Application Loggers -->
<Logger name="org.graylog2" level="info"/>
<Logger name="com.github.joschi.jadconfig" level="warn"/>
<!-- This emits a harmless warning for ActiveDirectory every time which we can't work around :( -->
<Logger name="org.apache.directory.api.ldap.model.message.BindRequestImpl" level="error"/>
<!-- Prevent DEBUG message about Lucene Expressions not found. -->
<Logger name="org.elasticsearch.script" level="warn"/>
<!-- Disable messages from the version check -->
<Logger name="org.graylog2.periodical.VersionCheckThread" level="off"/>
<!-- Suppress crazy byte array dump of Drools -->
<Logger name="org.drools.compiler.kie.builder.impl.KieRepositoryImpl" level="warn"/>
<!-- Silence chatty natty -->
<Logger name="com.joestelmach.natty.Parser" level="warn"/>
<!-- Silence Kafka log chatter -->
<Logger name="kafka.log.Log" level="warn"/>
<Logger name="kafka.log.OffsetIndex" level="warn"/>
<!-- Silence useless session validation messages -->
<Logger name="org.apache.shiro.session.mgt.AbstractValidatingSessionManager" level="warn"/>
<Root level="warn">
<AppenderRef ref="rolling-file"/>
<AppenderRef ref="graylog-internal-logs"/>
</Root>
</Loggers>
</Configuration>
I checked the above configuration against the one you posted earlier and there were not many differences.
In your configuration, you have SizeBasedTriggeringPolicy size="2GB" whereas, in my configuration, I have SizeBasedTriggeringPolicy size="50MB".
The other major difference I saw was your configuration has DefaultRolloverStrategy max="5" and mine is DefaultRolloverStrategy max="10".
I believe the configuration I pasted above is default as I don’t remember ever making any changes to it. Try changing using the above.
Thanks for providing your config, Jesse. I copy-pasted yours and restarted the three services - but messages are still being logged in real-time to my /var/log/upstart/graylog-server.log. And they say the same error about the missing apache appender.
That is very strange.
Is there now a file at: /var/log/graylog-server/server.log ?
I’m not too sure how or why graylog would still be writing to /var/log/upstart/graylog-server.log if you have changed the configuration and restarted. (Maybe that is some missing knowledge on my end though.)
I would double/triple check the log4j2.xml and make sure it is definitely what you now expect it to be.
Failing that, I would just start fresh with a clean server and install from the docs, provided that is an option.
Thanks again for the reply. I guess I’ll just rebuild from scratch. For the most part Ubuntu LTS 14.04 worked fine for me, but do you recommend another version @jan?
you should take a distribution that is not end of live april this year - if you want to run this environment longer. Take the LTS 18 release and you are save for some time …
Awesome. Thanks again guys.
This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.