I have a question regarding filebeat and graylog configuration. I have configured a windows machine that is recording siebel DB transactions in log files to send those log files from a specific path to logstash, and then logstash forwards them to graylog. This is working.
The problem is that log files on the source machine have the same names (e.g. fileA has new content appended by the siebel) name until a period of time (probably until a server restart) while the log files updated (new content appended) every time a new activity occurs by the user. This means, filebeat sends each time the new appended content to graylog. That behaviour is not the preferred one because for 1 file I may end to have more than 10 entries in my graylog which may not contain useful information for auditing.
Is there any way to force filebeat send me the files once per day (at least to configure a specific time) or append the new appended content that arrives for the same log filename to the entry that corresponds to that log file? I want to end up with one entry per file and the “message” field of graylog should contain the whole content of the file.
I know that after server restart, the log files from the source machine are archived and one solution should be to retrieve those files to graylog but this is not recommended because server restart is not scheduled.