[Input] Generic HTTP :: JSON

Graylog Tech Challenges
1

Hello,

I would have the usecase to receive HTTP JSON messages with Graylog.
Unfortunately the only HTTP Input only supports GELF as Inputtype but not JSON.
Is there any community based HTTP JSON Input or any other way to be able to receive those kind of messages?

Thanks and regards
Joerg

2

you can use a plain RAW/Plaintext tcp and then as an extractor JSON, I guess that is what you want

3

Hi @colttt
and thanks for your answer.

Well, I’ve just created that RAW/Plaintext tcp input and I’m able to receive the information which is POSTed by the HTTP client, unfortunately this HTTP POST is now split up into 66 single messages.

Can someone please help me and guide me into the right direction in order to re-combine them and parse them?

Thanks and Regards

4

fix the application to send just one message :wink:
ok so far as I know it is not easy possible,you need pipelines and configure that, but I don’t know how.

1 Like
5

Hahaha, the application is a properitary blackbox, so I can’t really change anything there.
It seems that it is seperating each line into a single message (even the HTTP Header are stored within seperate messages).

6

I agree with @colttt that you may need a pipeline for that. I think some where in this forum there might be one. Vaguely remember someone had a multi lines of messages and they want to combine them in to one message since they pertain to the same event. I could be wrong.

7

Hey @colttt , @gsmith and everyone else,

I solved the issue with several messages per HTTP POST by activating the “Null Frame delimiter” feature
within the RAW/Plaintext Input.

Now I’m somehow struggling in removing the HTTP data from my message in order to be able to parse the JSON object.

Example message:


-------------------------------message------------------------------>
    POST / HTTP/1.1

    Host: 127.0.0.1:12201

    User-Agent: fenet

    Accept: */*

    appliance-id:someblackbox123

    customer-id:123456

    customer-name:some customer

    alert-checksum:123456789123456789123456789

    EVENT_ID:123

    EVENT_TYPE:infection-match

    EVENT_SIZE:1005

    Content-Type: application/json

    Content-Length: 1005



    {
      "product": "Web MPS", 
      "appliance-id": "123456789", 
      "appliance": "fe-appliance", 
      "alert": {
        "src": {
          "mac": "00:11:22:33:44:55", 
          "ip": "127.0.0.1", 
          "host": "IM-testing.fe-notify-examples.com", 
          "vlan": "0", 
          "port": "10"
        }, 
        "uuid": "someuuid-123456-123lkjlk12-12l3123", 
        "dst": {
          "ip": "127.0.0.20", 
          "mac": "00:11:22:33:44:55", 
          "port": "20"
        }, 
        "ack": "no", 
        "explanation": {
          "malware-detected": {
            "malware": {
              "name": "FireEye-TestEvent-SIG-IM", 
              "stype": "bot-command", 
              "sid": "30"
            }
          }, 
          "protocol": "tcp", 
          "analysis": "content"
        }, 
        "alert-url": "https://fe-appliance/event_stream/events_for_bot?ev_id=123", 
        "occurred": "2021-08-16T12:04:35Z", 
        "name": "infection-match", 
        "action": "notified", 
        "sc-version": "1190.114", 
        "id": "123", 
        "severity": "minr"
      }, 
      "version": "9.0.3.936891", 
      "msg": "normal"
    }
-------------------------------end message------------------------------>

Can someone help me or guide me into the right direction to get rid of the HTTP section?

Many thanks and regards

1 Like
8

Try to use regex to extract json data, something like this:
(?s)\{.*\}

Use regex extractor or pipeline rule.

1 Like
9

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.