Thanks Jan, your input is always appreciated. Some notes:
- 7 days (6814 messages) exports okay, the file size is 437KB.
- 14 days (13946 messages) exports okay - but takes really long, it seems to send in 400KB chunks - as I watch chrome download chunk by chunk. 1580KB in total. There has to be a better way!
- 30 days (29610 messages) causes Chrome to say “Failed - Network error”.
- Graylog-server.log shows “Could not close chunked output stream for query scroll” and Elastic-search.log shows nothing.
Note this machine is fairly powerful with 64GB of ram, only 13GB of it is being used. I am running Graylog v2.4 with Elastic Search v2.3 (we have plans to upgrade both soon but can’t rush it just to solve this current issue).
Is it possible everyone has this same CSV export problem - but it is so very rarely used so nobody is reporting the issue? Can anyone confirm they are able to export 30,000+ records to CSV?
These queries are such small amounts of data as far as I am concerned. There is not much point collecting all these logs if we can’t properly access them when they are needed.
Jan, any suggestions on how to proceed? Are there any settings I can tweak? Maybe Elastic search has a debug mode that will log more information? Any suggestions are welcomed!