Unable to plot a data table for below query and not sure why?

[blason]

Hi Team,

I have a server where I installed DNS RPZ and have windows DNS Server. I am getting logs from logstash to graylog which is installed on DNS RPZ Server; while at the same time I installed packetbeat on the windows server and then sending logs to logstash to DNS RPZ and then to graylog from there.

192.168.5.111 → Windows DNS Server with Packetbeat
192.168.5.112 → BIND DNS RPZ with graylog/logstash

Here is the situation which I am unable to resolve. The logs for now are coming in “All Messages”

DNS RPZ

action
Bad-TLD
clientipaddr
192.168.5.111
message
21-Feb-2022 08:45:05.085 rpz: info: client @0x7f42443f56c8 192.168.5.111#57955 (key.ru): rpz QNAME Local-Data rewrite key.ru/A/IN via key.ru.block.tld
origdom
key.ru/A/IN
qdomain
key.ru
rewritten
key.ru.block.tld
timestamp
2022-02-21 08:45:06 +05:30

And here is windows Packet beat logs

message
-
packetbeat_client_ip
192.168.5.74
packetbeat_destination_ip
192.168.5.111
packetbeat_dns_question_etld_plus_one
key.ru
packetbeat_dns_question_name
key.ru
packetbeat_dns_question_type
A
source
WIN-GKS07C392EJ
timestamp
2022-02-21 08:45:04 +05:30

Now the thing is I am going to create a Data Table aggregration and wanted to chart

packetbeat_client_ip “\t” qdomain “\t” action

However this is not happening. Can someone please help?

[gsmith]

Hello,

By chance are you using the Beat Input for 192.168.5.111 → Windows DNS Server with Packetbeat and a different Input for 192.168.5.112 → BIND DNS RPZ with graylog/logstash? The reason I ask this is to make sure you getting the right fields needed for you Data table ( i.e. Widget).

To be honest some more information would help specially for what your doing with the data table.

[blason]

Yes that is right - I am using different input in Graylog. That is Beat Input for 192.168.5.111 while logstash for 192.168.5.112.

Would you please suggest what exactly needs to be done then?

[gsmith]

Hello @blason

Sure, not a problem.

@blason

Would you please suggest what exactly needs to be done then?

On the search page, left side click on aggregation Add you fields to the Widget and save.

Example:

If this is not what you want, could you explain in greater detail what your trying to achieve.

[gsmith]

If you have done this already and do not see data BUT you know those fields do exist, check your Data and Timestamp. It has been known to cause issues

[blason]

I am using 3.3.16 and have already created the dashboards or aggregations however as I said I am unable to plot those fields in a single aggregation. Here e.g.

Data does not appear when I add packetbeat fields

[gsmith]

Oh I see now,

Have to tried other configurations? Meaning changing the order of the fields but keeping clientipaddr there clientipaddr —> packetbeat_client_ip → odomain

clientipaddr is being search and for/each clientipaddr there is a filed called odomain which I seen was listed above, then you adding packetbeat_client_ip after each odomain. Perhaps each field odomain does not have a packetbeat_client_ip fileld/s. Just a guess.

My apologies, I no long have version GL 3,3 and its been awhile so I’m going off memory on what might have happened.

I did a test in my lab and I can not use a Beat INPUT with GELF TCP/TLS INPUT, results are NULL.

[blason]

Does that mean it will not work since both the data is coming from different input?

[gsmith]

Hello,

I’m sorry but it might not, I have version 4.2.6 and I’m unable to get data from both types of INPUTS.
What I get is you have two fields /w IP address from two different sources/types coming in and want to match them with a domain. Depending on how bad you want this widget to work, you can use a extractor /pipelines and create unique fields that would be able get the information you need or create a widget for each. On the other hand, perhaps posting here may help.

• Sign in to GitHub · GitHub