Performance impact of moving ES/OS to remote storage

Graylog Central (peer support)
, ,
1

I’m currently running 5.1 in a docker setup. I’ll be upgrading to 5.2 soon.

I want to avoid using up a lot of storage on my docker host and instead pushing that over to the NAS. Looking at the existing volumes, it appears to be the primary user is ES/OS which stores the logs. From what I can tell, archiving logs older than 30 days requires an Operations license, so instead I’m considering just moving the entire volume to a share.

How badly will this affect my performance when doing queries, etc? I assume the ingestion won’t be affected too much as it’s mostly sequential. Are there any other options to look at in order to reduce the docker storage? My setup is currently only showing 500M of daily usage in the Overview page but I expect that to grow as I make more use of the capabilities of Graylog.

2

Hi @CJRoss

I assume that E/OS and graylog are on the same server.

Bringing storage only to an external NAS will have impact due to
a lot of extra storage and network handling that comes in between.

Is there an option to move ES/OS to an external server, maybe even a cluster
to make it faster or does your NAS has capabilities to run ES/OS?

Kind greetings,
Arie

3

Yes, currently I’m using a docker compose file with all of the containers running on the same docker host.

I’ve had no performance issues so far with this setup. My concern is just the amount of storage that the logs will take up, hence wanting to move the ES/OS volume to a network share.

I could break ES/OS out onto another server, but then I would lose the easy administration of the current docker compose setup.

What I’d like to understand is how much of a performance hit will I take from just moving the data to a network share. Is it going to be somewhat slow or will it end up completely unusable?

4

Like they say, it depends.

Network and disk response should be fast enough for this because you are going to
let ES/OS going to to work with an external disk over a network. Take in mind that
there is continuous talk between the software and disk for ES/OS. It memorizes
thru indexes what is kept in shards and from that after a query disk information is
given back to the software and so the users.

So you need to optimize your network settings for this to. And use smaller
shards if your queries ar not to far in the past.

Within a fast network you need thins like this on all sides:

  net.ipv4.conf.all.arp_ignore = 1
  net.ipv4.conf.all.arp_filter = 1
  net.core.rmem_default = 1048576
  net.core.rmem_max = 2097152
  net.core.wmem_default = 1048576
  net.core.wmem_max = 2097152

And you could do some things on your nas to make it faster if it is
a linux nas. But my guess is there will be an impact.

Kind greetings
Arie

5

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.