# Unable to parse JSON logs in Graylog

**URL:** <https://community.graylog.org/t/unable-to-parse-json-logs-in-graylog/8267>\
**Category:** Graylog Central (peer support)\
**Created:** [January 5, 2019, 3:49am UTC](https://community.graylog.org/t/unable-to-parse-json-logs-in-graylog/8267 "2019-01-05T03:49:42Z")\
**Posts on this page:** 10\
**Page:** 1

<div class="post-metadata">

**Author:** ![blason](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/blason/32/2930_2.png) [@blason](https://community.graylog.org/u/blason)\
**Post date:** [January 5, 2019, 3:49am UTC](https://community.graylog.org/t/unable-to-parse-json-logs-in-graylog/8267/1 "2019-01-05T03:49:42Z")

</div>

Hi Guys,

I am writing my logs in json file and and input filter is json directly but those are being parsed correctly in elasticsearch? Any clue what I am missing here?

Here is my json file

```
{"org_name": "google.com", "policy_spf": "pass", "org_email": "noreply-dmarc-support@google.com", "policy_dkim": "pass", "policy_pct": "100", "auth_spf_result": "pass","auth_dkim_domain": "xxx,ccc", "auth_dkim_result": "pass", "identifier_header_from": "xxx,ccc", "date_end": "2019-01-04T05:29:59", "date_start": "2019-01-03T05:30:00", "source_ip": "1,.2.3.4", "count": 1, "auth_spf_domain": "xxx,ccc", "policy_p": "none", "submitter": "unknown", "policy_disposition": "none", "policy_domain": "xxx,ccc", "id": "15325652754200102860"}

```

{“org\_name”: “[google.com](http://google.com)”, “policy\_spf”: “fail”, “org\_email”: “[noreply-dmarc-support@google.com](mailto:noreply-dmarc-support@google.com)”, “policy\_dkim”: “fail”, “policy\_pct”: “100”, “auth\_spf\_result”: “pass”, “identifier\_header\_from”: “mail.xxx,ccc”, “date\_end”: “2019-01-04T05:29:59”, “date\_start”: “2019-01-03T05:30:00”, “source\_ip”: “2.3.4.5”, “count”: 1, “auth\_spf\_domain”: “[apc01-hk2-obe.outbound.protection.outlook.com](http://apc01-hk2-obe.outbound.protection.outlook.com)”, “policy\_p”: “none”, “submitter”: “unknown”, “policy\_disposition”: “none”, “policy\_domain”: “xxx,ccc”, “id”: “15325652754200102860”}  
{“org\_name”: “[google.com](http://google.com)”, “policy\_spf”: “pass”, “org\_email”: “[noreply-dmarc-support@google.com](mailto:noreply-dmarc-support@google.com)”, “policy\_dkim”: “pass”, “policy\_pct”: “100”, “auth\_spf\_result”: “pass”, “auth\_dkim\_domain”: “xxx,ccc”, “auth\_dkim\_result”: “pass”, “identifier\_header\_from”: “xxx,ccc”, “date\_end”: “2019-01-04T05:29:59”, “date\_start”: “2019-01-03T05:30:00”, “source\_ip”: “2.2.2.2”, “count”: 1, “auth\_spf\_domain”: “xxx,ccc”, “policy\_p”: “none”, “submitter”: “unknown”, “policy\_disposition”: “none”, “policy\_domain”: “xxx,ccc”, “id”: “15325652754200102860”}

And here is my logstash config file which is not parsing the JSON logs correctly. Can someone confirm what is the issue?

> input {  
> file {  
> type =\> “json”  
> path =\> “/log/\*.json”  
> start\_position =\> “beginning”  
> }  
> }

```
filter {
  if [source_type] == "json-logs" {
    json {
      source => "message"
      tag_on_failure => ["_jsonparsefailure"]
    }
  }
    }

```

output {  
gelf {  
host =\> “localhost”  
port =\> “12202”  
protocol =\> “UDP”  
}  
}

---

<div class="post-metadata">

**Author:** ![blason](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/blason/32/2930_2.png) [@blason](https://community.graylog.org/u/blason)\
**Post date:** [January 5, 2019, 4:02am UTC](https://community.graylog.org/t/unable-to-parse-json-logs-in-graylog/8267/2 "2019-01-05T04:02:46Z")

</div>

Wondering if do I need write GROK pattern for my JSON logs? Since those are JSOn should get parsed automatically right?

---

<div class="post-metadata">

**Author:** ![benvanstaveren](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/benvanstaveren/32/3398_2.png) [@benvanstaveren](https://community.graylog.org/u/benvanstaveren)\
**Post date:** [January 5, 2019, 11:02am UTC](https://community.graylog.org/t/unable-to-parse-json-logs-in-graylog/8267/3 "2019-01-05T11:02:13Z")

</div>

It doesn’t get parsed automatically - you either need to put a JSON extractor on the input, or use a pipeline to parse the JSON for you.

---

<div class="post-metadata">

**Author:** ![blason](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/blason/32/2930_2.png) [@blason](https://community.graylog.org/u/blason)\
**Post date:** [January 5, 2019, 11:05am UTC](https://community.graylog.org/t/unable-to-parse-json-logs-in-graylog/8267/4 "2019-01-05T11:05:40Z")

</div>

Oh I was assuming it gets parsed automatically since its an JSON format. ELK does that hence thought it can be applied here as well.

So what exactly I need to do? To get those messages parsed? Apply JSON extractor.

Not sure or not very well versed with Pipeline yet hence looking for better & quicker option.

---

<div class="post-metadata">

**Author:** ![blason](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/blason/32/2930_2.png) [@blason](https://community.graylog.org/u/blason)\
**Post date:** [January 5, 2019, 1:39pm UTC](https://community.graylog.org/t/unable-to-parse-json-logs-in-graylog/8267/5 "2019-01-05T13:39:16Z")

</div>

Yes after applying it worked. How do I make that automatic? Since enabling the extractor means I’ll have to wait for message to arrive then apply the extractor.

---

<div class="post-metadata">

**Author:** ![benvanstaveren](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/benvanstaveren/32/3398_2.png) [@benvanstaveren](https://community.graylog.org/u/benvanstaveren)\
**Post date:** [January 5, 2019, 2:55pm UTC](https://community.graylog.org/t/unable-to-parse-json-logs-in-graylog/8267/6 "2019-01-05T14:55:05Z")

</div>

Uh, it is automatic. Once you’ve set up an extractor on an input, every message that comes in is processed by the extractor.

---

<div class="post-metadata">

**Author:** ![Totally\_Not\_A\_Robot](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/totally_not_a_robot/32/3353_2.png) [@Totally\_Not\_A\_Robot](https://community.graylog.org/u/Totally_Not_A_Robot)\
**Post date:** [January 6, 2019, 6:57pm UTC](https://community.graylog.org/t/unable-to-parse-json-logs-in-graylog/8267/7 "2019-01-06T18:57:37Z")

</div>

> [@benvanstaveren](#):
>
> Once you’ve set up an extractor on an input, every message that comes in is processed by the extractor.

Within certain limits, which you can define yourself…

---

<div class="post-metadata">

**Author:** ![benvanstaveren](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/benvanstaveren/32/3398_2.png) [@benvanstaveren](https://community.graylog.org/u/benvanstaveren)\
**Post date:** [January 7, 2019, 10:02am UTC](https://community.graylog.org/t/unable-to-parse-json-logs-in-graylog/8267/8 "2019-01-07T10:02:02Z")

</div>

Yeah, that… forgot to mention that 😃

---

<div class="post-metadata">

**Author:** ![blason](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/blason/32/2930_2.png) [@blason](https://community.graylog.org/u/blason)\
**Post date:** [January 7, 2019, 5:20pm UTC](https://community.graylog.org/t/unable-to-parse-json-logs-in-graylog/8267/9 "2019-01-07T17:20:08Z")

</div>

That has resolved the issue. Thanks for the help.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/flex016/uploads/graylog/original/3X/c/7/c7c09c6b5099570133d6502b83f50ba4430de5b6.png) [@system](https://community.graylog.org/u/system)\
**Post date:** [January 21, 2019, 5:20pm UTC](https://community.graylog.org/t/unable-to-parse-json-logs-in-graylog/8267/10 "2019-01-21T17:20:15Z")

</div>

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.