# Top 5 quick values on msg field

**URL:** <https://community.graylog.org/t/top-5-quick-values-on-msg-field/4018>\
**Category:** Graylog Central (peer support)\
**Created:** [February 1, 2018, 11:58am UTC](https://community.graylog.org/t/top-5-quick-values-on-msg-field/4018 "2018-02-01T11:58:04Z")\
**Posts on this page:** 14\
**Page:** 1

<div class="post-metadata">

**Author:** ![D4ywa1k3r](https://avatars.discourse-cdn.com/v4/letter/d/c0e974/32.png) [@D4ywa1k3r](https://community.graylog.org/u/D4ywa1k3r)\
**Post date:** [February 1, 2018, 11:58am UTC](https://community.graylog.org/t/top-5-quick-values-on-msg-field/4018/1 "2018-02-01T11:58:04Z")

</div>

Hi Guys,

i have a correct incoming JSON transfer to my Graylog instance.  
my msg field has multiple words like “ET POLICY Vulnerable Java Version 1.7.x Detected”

i want to add quick values to my dashboard using this msg field.

But i get not the full text message as value, i get each word as value on my dasboard

If you need more informations let me know.  
I hope somebody can help me.

---

<div class="post-metadata">

**Author:** ![jochen](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/jochen/32/8_2.png) [@jochen](https://community.graylog.org/u/jochen)\
**Post date:** [February 1, 2018, 1:38pm UTC](https://community.graylog.org/t/top-5-quick-values-on-msg-field/4018/2 "2018-02-01T13:38:10Z")

</div>

How are you ingesting these messages?  
Which extractors or pipeline rules are you using to process these messages?  
Also, please provide some example messages.

---

<div class="post-metadata">

**Author:** ![D4ywa1k3r](https://avatars.discourse-cdn.com/v4/letter/d/c0e974/32.png) [@D4ywa1k3r](https://community.graylog.org/u/D4ywa1k3r)\
**Post date:** [February 1, 2018, 1:47pm UTC](https://community.graylog.org/t/top-5-quick-values-on-msg-field/4018/3 "2018-02-01T13:47:35Z")

</div>

I have an csv output of snort IDS and use a unix socket to transmit the infos to the tdagent/fluentd, which transmit the correctly JSON format to graylog (including the required field timestamp and source-host).  
Because of the csv output i have no pipelining rules in graylog. The JSON goes dedicated from my tdagent to graylog.

msg examples:  
-“ICMP test detected”  
-“Consecutive TCP small segments exceeding threshold”  
-“ICMP test detected”

There auch thousands of different messages. I want as example display the top 5 of messages. Maybe its the message “ICMP test detected”

---

<div class="post-metadata">

**Author:** ![jochen](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/jochen/32/8_2.png) [@jochen](https://community.graylog.org/u/jochen)\
**Post date:** [February 1, 2018, 2:31pm UTC](https://community.graylog.org/t/top-5-quick-values-on-msg-field/4018/4 "2018-02-01T14:31:18Z")

</div>

> [@D4ywa1k3r](#):
>
> which transmit the correctly JSON format to graylog

Do you have an example?

---

<div class="post-metadata">

**Author:** ![D4ywa1k3r](https://avatars.discourse-cdn.com/v4/letter/d/c0e974/32.png) [@D4ywa1k3r](https://community.graylog.org/u/D4ywa1k3r)\
**Post date:** [February 1, 2018, 2:34pm UTC](https://community.graylog.org/t/top-5-quick-values-on-msg-field/4018/5 "2018-02-01T14:34:44Z")

</div>

`2018-02-01 07:53:50.161515442 +0100 fluentd.var.log.snort.snort.csv: {"src":"192.168.xxx.xx","dst":"xxx.xxx.xxx.xxx","msg":"ET POLICY Vulnerable Java Version 1.7.x Detected","sport":null,"dport":null,"proto":"TCP","sig_id":"2014297","timestamp":"2018-02-01 07:53:50.161","source":"snrt001"}`

---

<div class="post-metadata">

**Author:** ![jochen](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/jochen/32/8_2.png) [@jochen](https://community.graylog.org/u/jochen)\
**Post date:** [February 1, 2018, 3:04pm UTC](https://community.graylog.org/t/top-5-quick-values-on-msg-field/4018/6 "2018-02-01T15:04:23Z")

</div>

And how are you processing that message in Graylog?

---

<div class="post-metadata">

**Author:** ![D4ywa1k3r](https://avatars.discourse-cdn.com/v4/letter/d/c0e974/32.png) [@D4ywa1k3r](https://community.graylog.org/u/D4ywa1k3r)\
**Post date:** [February 1, 2018, 3:12pm UTC](https://community.graylog.org/t/top-5-quick-values-on-msg-field/4018/7 "2018-02-01T15:12:57Z")

</div>

I have a stream input \> msg field checked -\> quick values -\> add to dashboard

Or what do you exactly mean with processing?

---

<div class="post-metadata">

**Author:** ![jochen](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/jochen/32/8_2.png) [@jochen](https://community.graylog.org/u/jochen)\
**Post date:** [February 1, 2018, 4:06pm UTC](https://community.graylog.org/t/top-5-quick-values-on-msg-field/4018/8 "2018-02-01T16:06:19Z")

</div>

If you want to use anything from that JSON payload, you either have to use a JSON extractor or a pipeline rule to parse the JSON and set the message fields accordingly.

Otherwise it’s just opaque text for Graylog.

---

<div class="post-metadata">

**Author:** ![D4ywa1k3r](https://avatars.discourse-cdn.com/v4/letter/d/c0e974/32.png) [@D4ywa1k3r](https://community.graylog.org/u/D4ywa1k3r)\
**Post date:** [February 2, 2018, 11:02am UTC](https://community.graylog.org/t/top-5-quick-values-on-msg-field/4018/9 "2018-02-02T11:02:56Z")

</div>

Is this the necessary way, if i have the message field already “set” via transmission of the mentioned JSON-string?  
Because the fields are correctly set in graylog…

---

<div class="post-metadata">

**Author:** ![jochen](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/jochen/32/8_2.png) [@jochen](https://community.graylog.org/u/jochen)\
**Post date:** [February 2, 2018, 11:11am UTC](https://community.graylog.org/t/top-5-quick-values-on-msg-field/4018/10 "2018-02-02T11:11:05Z")

</div>

You didn’t provide any details about the transmission, ingest, or the input configuration in Graylog…

---

<div class="post-metadata">

**Author:** ![D4ywa1k3r](https://avatars.discourse-cdn.com/v4/letter/d/c0e974/32.png) [@D4ywa1k3r](https://community.graylog.org/u/D4ywa1k3r)\
**Post date:** [February 2, 2018, 12:11pm UTC](https://community.graylog.org/t/top-5-quick-values-on-msg-field/4018/11 "2018-02-02T12:11:21Z")

</div>

Thats my input configuration of my Transmission client:

```auto
<source >
    rotate_wait 5

    @type tail
    path /var/log/snort/snort.csv
    pos_file /var/log/td-agent/tmp/access.log.1.pos
    tag fluentd.*
    <parse >
        @type csv
        keys src,dst,msg,sport,dport,proto,sig_id
        time_key time
        keep_time_key true
    </parse>

    enable_watch_timer True
    refresh_interval 60
</source>
    

<filter fluentd.**>
     enable_ruby true
     @type record_transformer
     <record>
     timestamp ${time.strftime("%Y-%m-%d %H:%M:%S.%L")}
     source "#{Socket.gethostname}"
     </record>
</filter> 
<match fluentd.**>

    <store >
        protocol tcp
        add_msec_time true
        @type gelf
        host graylog-gelfXYZ.com
        use_record_host true
        port 12201
    </store>
    
                    
    <store >
        output_type json
        @type stdout
        @log_level warn
    </store>
    
    @type copy
</match>

```

The JSON, that is going to graylog is that here:

```auto
{"src":"192.168.xxx.xx","dst":"xxx.xxx.xxx.xxx","msg":"ET POLICY Vulnerable Java Version 1.7.x Detected","sport":null,"dport":null,"proto":"TCP","sig_id":"2014297","timestamp":"2018-02-01 07:53:50.161","source":"snrt001"}

```

in Graylog i defined a stream with this as input rule:

```auto
^.*snrt.*$

```

I hope this information helps.

here are the fields in a message in detail…

![Bildschirmfoto%20vom%202018-02-02%2013-07-12](https://us1.discourse-cdn.com/flex016/uploads/graylog/original/2X/7/7ee99f4daacf05b4c57f52bb71fece9d1d4ae0af.png)

---

<div class="post-metadata">

**Author:** ![jochen](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/jochen/32/8_2.png) [@jochen](https://community.graylog.org/u/jochen)\
**Post date:** [February 2, 2018, 12:24pm UTC](https://community.graylog.org/t/top-5-quick-values-on-msg-field/4018/12 "2018-02-02T12:24:53Z")

</div>

The “message” field is analyzed by default, which means it’s tokenized and each token is indexed. That’s why you get “single words” when running Quick Values on that field.

See [https://www.elastic.co/guide/en/elasticsearch/guide/2.x/analysis-intro.html](https://www.elastic.co/guide/en/elasticsearch/guide/2.x/analysis-intro.html) for details.

You could copy the “message” field into another field (either by using a Copy Input extractor or using the processing pipeline) which is _not_ analyzed.

Also make sure to create a custom mapping for that field:  
[http://docs.graylog.org/en/2.4/pages/configuration/elasticsearch.html#custom-index-mappings](http://docs.graylog.org/en/2.4/pages/configuration/elasticsearch.html#custom-index-mappings)

---

<div class="post-metadata">

**Author:** ![D4ywa1k3r](https://avatars.discourse-cdn.com/v4/letter/d/c0e974/32.png) [@D4ywa1k3r](https://community.graylog.org/u/D4ywa1k3r)\
**Post date:** [February 2, 2018, 12:30pm UTC](https://community.graylog.org/t/top-5-quick-values-on-msg-field/4018/13 "2018-02-02T12:30:14Z")

</div>

Thanks a lot. I will try this…

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/flex016/uploads/graylog/original/3X/c/7/c7c09c6b5099570133d6502b83f50ba4430de5b6.png) [@system](https://community.graylog.org/u/system)\
**Post date:** [February 16, 2018, 12:30pm UTC](https://community.graylog.org/t/top-5-quick-values-on-msg-field/4018/14 "2018-02-16T12:30:33Z")

</div>

Closed on February 16, 2018, 12:30pm UTC

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.
