But now I wish to import the old stored squid logs to my “squid_proxy_logs” index using this same format that I’m using to send the real time logs. Is this possible? How can I do this?
If squid is streaming it in real time, you probably can’t get it to read the stored logs. You can use an agent like filebeat or nxlog to grab them, but you would likely need to write a parser to put them in the same fields as the streamed logs.
Is there a reason you need the stored logs to be put into Graylog? Or just curiosity?
I figured the solution would pass through some parser. I just wanted to make sure that I wouldn’t be putting effort into this unnecessarily if there was an easier solution (some API feature maybe).
I just want to be able to import the old logs to have historical data for analysis and to compose the statistics on the dashboard.
It’s not wasted effort if you need the historical data, but most people just start from Day 1 and go forward. The effort required to write a parser, though educational if you need to write more of them, might be more work than it is worth in the end. YMMV.
Actually it’s not a “necessity”, it would just be interesting to import this data if it wouldn’t take too much work time. But it’s okay to “start from day 1” too. Thanks!
