Hello everyone,
I’m currently working on a project where we’re deploying Graylog in a hybrid cloud environment, and I’ve run into a couple of challenges that I’m hoping someone here might be able to help with.
To give some context, we’re using a combination of on-premises infrastructure and public cloud resources (AWS in this case). The idea is to centralize logs from both environments into a single Graylog instance. While Graylog’s ability to handle a variety of log sources has been great, managing logs from a hybrid cloud setup has introduced some complexity, particularly with data security and ensuring minimal latency for log collection.
Here are a few specific challenges I’m encountering:
- Log Shipping: What’s the best way to configure reliable log shipping from both on-prem servers and cloud instances? We’re considering using Filebeat for agents, but I’m curious if there’s a more efficient or scalable method for a hybrid cloud environment.
- Cluster Management: Has anyone had experience setting up Graylog in a distributed cluster, where parts of the cluster are hosted on-prem and others in the cloud? Any best practices to ensure smooth integration between the two environments?
- Data Retention & Compliance: Given that our hybrid setup spans multiple jurisdictions, how do you handle data retention policies for compliance without impacting performance?
I’d appreciate any tips, recommendations, or resources that the community could share, especially if you’ve worked on similar hybrid cloud deployments with Graylog.
Looking forward to your insights!