Mhm - So, nevermind the MySQL. I will stick with Elasticsearch as my DB.
So, to send ntopng to Graylog - I’d have to do ntopng > logstash > elasticsearch > graylog?
That sounds straight forward. Now I am wondering if I can use Graylog for logging and Kibana for visualization. So in short: Graylog would be where everything gets logged and Kibana would be used only for Dashboards. Would this be feasible?
You are sorely misunderstanding what Graylog is 
Graylog IS an alternative to Kibana! And at the same time it is a flexible receiver and aggregator of logs.
It’s NTopNG -> Graylog.
The transfer to Graylog is LogStash through Beats. And Graylog will store the incoming data in Elastic.
I know its an alternative - but I’ve seen in many places that you can do both. It is undeniable that Kibana has better visualization, so I wish to use it just for that. Surely just hooking up Kibana with the Elasticsearch will be enough, since Graylog stores the data in Elastic anyway
When you say Logstash through Beats do you mean that Filebeat and then from Filebeat to Logstash ?
I’m not quite sure what I mean 
I don’t have prior experience with LogStash, but I noticed earlier (see above) that the original LogStash input for Graylog was replaced with the standard Beats one. So perhaps NTopNG’s LogStash output can actually speak Beats? I don’t know, but it sounds like something to look into!
Basically, BEATS will send the data to Logstash and then Logstash will forward it to Kibana/ES
That’s the purpose of Logstash, to gather logs from BEATS and other sources (correct if im wrong).
I am trying to spin a test box to try these things.
Going back to my latest question about using Kibana as visualization. I think the way it’d work is:
Graylog would have all the logging data (ntopng, winevt, cisco) stored in ES. Kibana will then have access to these logs and there I will be able to create the visualization.
The data would need to be ingested via Graylog into Elasticsearch to be visible and usable in Graylog. When you then use Kibana to access the data you bypass the security/access model that Graylog provides.
1 Like
So let me get this straight.
The data would have to first go to graylog and then to elasticsearch. That’s just how it works right.
The kibana is the bit I dont understand. If I set up kibana to do the visualization, I will be bypassing the security/access model that Graylog provides. What does this mean exactly ? I won’t have any security protocol on Graylog if I utilize Kibana? could you elaborate on that? Thanks
Jan means for example:
- if you were to setup access lists and filters inside Graylog, to prevent user A from seeing sensitive data
- and user A also has access to Kibana
- Then user A can see all the sensitive data he is not supposed to see.
Graylog is the thing that puts the logs into ElasticSearch. So yes.
As I’ve said before: the $100 question is whether the LogStash output of NTopNG can speak directly to the BEATS input of Graylog. That would be A Win™. Hence why I asked whether we know whether the LogStash output of NTopNG speaks the BEATS protocol, or something else.
I see, but only admins and the network people would have access to Graylog/Kibana.
Also, surely when purchasing Kibana you can set up some sort of security protocol.
One issue I have with your posts is whenever you say “Does logstash speak beats by default” what exactly do you mean by that? Can logstash communicate with Beats agent? if so, yes.
Sorry about that, as I said I don’t have logstash experience… what I mean is: NTopNG can be configure to send data into Logstash. Which protocol does THAT use? Can we just tell it to send it to the BEATS port on Graylog directly?
If NTopNG sends to a logstash beats input then yes. If it says it’s using Lumberjack protocol, then you can send that straight to a beats input as well (since Lumberjack is the beats protocol). You could even, theoretically, do NTopNG -> Logstash -> Graylog via a Logstash Lumberjack output to a Graylog beats input.
However I haven’t managed to get that Logstash -> Graylog part working on account of certificate shenanigans since it requires TLS on Logstash’ output end to work.
1 Like
Would Filebeat be able to get the logs from Ntopng and forward to logstash?
Boh, I have no idea if NTopNG writes logs. But then you wouldn’t need to send to Logstash, Filebeat can send to Graylog (Filebeat output to a Logstash type output, point it at a Graylog Beats input).
What about sending the ntopng logs to ES and then using Filebeat to pick the logs and forward to Graylog?
That would mean the logs will be duplicated ?
im basically brainstorming here. I don’t have a test box to try this all out, so I can only hypothesize.
Why would you do that? Graylog uses ES as it’s backend storage in the first place, so you’d be duplicating load and logs all over the place. Besides, filebeat can’t read from ES.
I feel that we’re going round-and-round-and-round-and-round-and-round in circles in this thread.
- NTopNG speaks LogStash.
- LogStash is almost literally BEATS, as @benvanstaveren indicated.
- Graylog can thus probably receive the data directly from NTopNG.
- Graylog will store it in ElasticSearch
- Kibana can read it from Elastic.
But now I’m just left wondering why the heck you’re using Graylog. What’s the use-case for you? If you use Kibana for the visualization and your searches Graylog just serves as an unneeded intermediary.
I just prefer Graylog over Kibana for logging.
Also, I just ran a test and was able to get Kibana and Graylog to work together. I couldn’t test the Ntopng but will be what I try next.
1 Like
I’m going to sound like an asshole here but I wish you luck in whatever you try to do, but I’m not answering any more questions; so far you have given us the most convoluted use case imaginable, you seem to not even bother listening to what people are telling you (or asking of you, for that matter) and your mind has been made up to do things in what I would consider a hilariously wrong way.
However, you do you…
Thanks, very cool!
I have taken everything into account and I am working with it. Nothing has been 100% concrete.
If it has worked so far, why would it be considered hilarious?
And I just updated the post to say that I got it to work with kibana and thats 50% of what I wanted done, done. Now I just need to get NtopNG to work with it, but I can’t yet. I really don’t see how you felt the need to even reply to it.
The reason I would use Graylog and Kibana has been answered, because I prefer Graylog for logging and Kibana for visualization. Space isn’t an issue seeing that I get 5GB of data a month.
I appreciate your help thus far, but this reply was mediocre.
This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.