1. Describe your incident:
I am trying to create a data adapter whose values I can lookup by using a regex pattern.
Let’s say I have values like this in my CSV:
url,status
https://attacker-ip.com/1234,online
http://malicious-address.br/abc,online
Is there a way to search through the data adapter by using a regex like e.g. .malicious-address.br.
just to recap you problem: you have a CSV with
url,status
https://attacker-ip.com/1234,online
http://malicious-address.br/abc,online
and you have a field in one of you messages with
.malicious-address.br.
now you want to to a little regex-magic to find that domain in your CSV?
I would recommend to extract all the domains from your CSV into another CSV and search in that from your lookup table.
Then extract the exact domain name from your field and use this for the search in the new domain-only CSV.
That’s exactly what I want to do.
Okay, seems like it’s not possible then, unfortunate!
Hello,
If you can give use some more info or examples, perhaps I can lab.
Hi gsmith,
thanks for the reply.
Please have a look at the Github issue where I’ve explained it with some example more in detail:
I’m looking forward to hearing your feedback on feasibility.
Quote from the git:
Why?
Let’s assume all client DNS lookups are logged. A user clicks on a phishing link and enters his credentials.
Since the phishing site is very new, the domain is not contained in any IOC database and therefore a lookup of the domain in the Pipeline is not successful.
10 days later, the phishing domain is known and in the lookup table.
It would be awesome to be able to make a search if any client was affected in the past.
I agree, that searching through your older logs for new IOCs is a valid scenario. But I don’t see a way to put old logs through the pipeline once again. Decorators will not do, as they only process logs shown on your screen and it is not possible to filter based on results of their decorations.
So far we are helping ourselves with a litte script doing queries on the stream. Everytime we add a new IOC we scan the last days for it. This is a little python-magic outside of Graylog, but does the job. If we have findings we send them as new logs into Graylog with GELF from that script.
Exactly, I don’t see the solution in putting old logs through the pipeline again but much rather implementing a feature similar to your Python solution into Graylog, that is, being able to use lookup tables not only for decorators and pipelines but also search queries.
This feature would allow endless opportunities for log enrichment and searching.
This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.