Export CSV different times from 10 minutes to over an hour

My Environment


16 Cors

Graylog Version: Graylog 3.3.8+e223f85
Elasticsearch Version: 6.8.12
MongoDB Version: 4.0.20
Operating System: Ubuntu 18.04
Browser version: 81.0.2 (64-Bit) under Windows 10

Graylog Start Options

GRAYLOG_SERVER_JAVA_OPTS="-Xms4g -Xmx4g -XX:NewRatio=1 -server -XX:+ResizeTLAB -XX:+UseConcMarkSweepGC -XX:+CMSConcurrentMTEnabled -XX:+CMSClassUnloadingEnabled -XX:-OmitStackTraceInFastThrow -Djava.net.preferIPv4Stack=true"

Elasticsearch Start Options


Changes to Orginal Server.conf

allow_leading_wildcard_searches = true
output_batch_size = 4000
processbuffer_processors = 7
outputbuffer_processors = 7
ring_size = 262144
inputbuffer_ring_size = 262144

Missing in my Server.conf

Set the default scheme when connecting to Elasticsearch discovered nodes

Default: http (available options: http, https)
#elasticsearch_discovery_default_scheme = http
Configures the prefix used for graylog event indices
Default: gl-events
#default_events_index_prefix = gl-events

Configures the prefix used for graylog system event indices
Default: gl-system-events
#default_system_events_index_prefix = gl-system-events

Automatically load content packs in “content_packs_dir” on the first start of Graylog.
#content_packs_loader_enabled = false

The directory which contains content packs which should be loaded on the first start of Graylog.
#content_packs_dir = data/contentpacks

A comma-separated list of content packs (files in “content_packs_dir”) which should be applied on
the first start of Graylog.
Default: empty
#content_packs_auto_install = grok-patterns.json

The allowed TLS protocols for system wide TLS enabled servers. (e.g. message inputs, http interface)
Setting this to an empty value, leaves it up to system libraries and the used JDK to chose a default.
Default: TLSv1.2,TLSv1.3 (might be automatically adjusted to protocols supported by the JDK)
#enabled_tls_protocols= TLSv1.2,TLSv1.3


Now to my Question,

like I wrote in CSV Export Progress bar or speed , the
export of the CSV file takes some time 10 minutes and other times over an hour.
Yesterday it took me two hours.

In the manuel for 3.3.8 there is written:

9.7.4 TroubleshootingDepending on the number of messages the export may take a while.
If the download never starts or the document doesnot contain the expected result, have a look at the server.log.

Warning:Exporting results to a CSV willnotpreserve sorting because Graylog is using the virtual_docfieldto “sort”
documents for performance reasons. If you need to have the exported data ordered you will need to eithermake a
scroll query to ElasticSearch and process it after, or to download the file and post process it via other means.

I checked the server.log /var/log/graylog-server/server.log,
/var/log/elasticsearch/graylog.log and

and I can not find any entries for the export.

I’m interested to figure out why it some times takes 10 minutes and some times over an hour.

What can i do to figure that out and for what do i have to look.

Thanks for help SR.

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.