Change where syslog files are stored is not working

Graylog Central (peer support)
1

Before you post: Your responses to these questions will help the community help you. Please complete this template if you’re asking a support question.
Don’t forget to select tags to help index your topic!

1. Describe your incident:
I am trying to change where Graylog is storing the incoming logs for our Palo Alto firewalls. I setup Graylog a few months ago and changed Elasticsearch.yml data and log paths to the mounted iSCSI share. I have the correct permission set on the new share. Graylog seemed to be working and logs were coming in. On the 30th Graylog stopped and the firewalls started screaming. I found the local HDD was out of space. It was a syslog.1 file in /var/log/ I moved the syslog.1 file to the mounted iSCSI restarted the server and I was able to get back into Graylog. I can see incoming logs in the upper right hand corner of the web interface but I can’t search anything.

2. Describe your environment:

  • OS Information: Ubuntu 24.04.1 LTS

  • Package Version:

  • Service logs, configurations, and environment variables:

3. What steps have you already taken to try and solve the problem?
I tried restarting the server and double checking the folder permissions as well as elasticsearch.yml file.

4. How can the community help?

Helpful Posting Tips: Tips for Posting Questions that Get Answers [Hold down CTRL and link on link to open tips documents in a separate tab]

2

Sounds like elastic is not happy, check the elastic log files and see what it says (and check that the service is running etc)

Ive never seen a syslog file in /var/logs so im not sure what that is related to.

3

Just my 2 cents:

From my point of view the /var/log/syslog.xx is not directly related to graylog.
This file should only contain messages of the operating system(Linux) itself.

However, if graylog/opensearch/… generate error messages, they could also find their way to the file.

So there might be two different issues:
First:

  • the syslog file grows by errors the system put in (too many errors? you’ll have to look inside)
  • the rotation of the file is not (or not correct) configured. You’ll have to consult your operating system related configs

Second:

  • as you don’t find anything in your graylog searches, there might be a different problem. This time I would guess a misconfiguration in the elasticsearch config.
  • on my side the path of the database (however we use opensearch instead of elastic) is: /var/lib/opensearch
  • Looks like this can be configured in /etrc/opensearch/opensearch.yml:
# Path to directory where to store the data (separate multiple locations by comma):
#
path.data: /var/lib/opensearch
#

Hope, that helps in any way :slight_smile:

4

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.