Searching Custom JSON Fields

Graylog Central (peer support)
1

My goal is to both log legibly and make my logs searchable in Graylog. Right now I am trying the method of logging JSON strings like so

{"process_id":42, "info":"started process successfully"}

I like this because it’s a human-readable log message.
I’m using these config lines in filebeat.yml to parse the message as JSON

json.message_key: event
json.keys_under_root: true

The good news is that this does produce fields that I can then search in the Graylog search bar like this:

process_id:42

I see my log appear. It’s great that I can search by custom fields that I wrote in the message itself, but I can’t query by partial strings. So this search won’t work:

info:"started process"

I would want to be able to do this if I wanted to see every time something started, regardless of it started successfully or not. Am I missing an obvious alternative way to have the parts of my log messages be searchable with things like partial matching and regexes? Or is JSON a good approach that I am just implementing incorrectly?

2

Try using wildcards, e. g. info:"started process*" (http://docs.graylog.org/en/2.4/pages/queries.html#search-query-language).

If you want real full-text search on your custom fields, you have to create a custom Elasticsearch index mapping and analyze these fields: http://docs.graylog.org/en/2.4/pages/configuration/elasticsearch.html#custom-index-mappings

3

I tried the wildcards method, but it didn’t work on our system. Does wildcard searching work the same way in version 2.2? I only get a match if I type info:“started process successfully”. It’s all or nothing.

What would we map our fields to for better searching? Something like this?

"properties" : {
          "info" : {
            "analyzer" : "standard", # which analyzer would you suggest?
            "index" : "analyzed",
            "type" : "string"
          },
4

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.