Create more than one stream at once via API

[anonimus (Name)]

Before you post: Your responses to these questions will help the community help you. Please complete this template if you’re asking a support question.
Don’t forget to select tags to help index your topic!

1. Describe your incident:
Transfering few hundred streams from one graylog instance to another.
Figured out syntax for API post request
POST /streams Create a stream

put 2 jsons one after another into the window and it reads only first one and creates single stream. HEEELP
jsons in curly braces one under the other, separated by comma
{
},
{
}

2. Describe your environment:

• OS Information: Linux

• Package Version: api version: 5.0.5

• Service logs, configurations, and environment variables: I dont think its relevant, it works, but only takes first entry.

3. What steps have you already taken to try and solve the problem?
trying to put jsons in {}, but then API complains about syntax

4. How can the community help?
Anyone created multiple streams at once in API?

Helpful Posting Tips: Tips for Posting Questions that Get Answers [Hold down CTRL and link on link to open tips documents in a separate tab]

[drewmiranda-gl (Drew Miranda)]

Greetings! The model schema via the API browser can be helpful for understanding what data the API endpoint can accept. For example:

Returns the following as the schema:

{
  "index_set_id": "string",
  "matching_type": "string",
  "remove_matches_from_default_stream": "boolean",
  "description": "string",
  "rules": [
    {
      "field": "string",
      "description": "string",
      "type": "integer",
      "inverted": "boolean",
      "value": "string"
    }
  ],
  "title": "string",
  "content_pack": "string"
}

However, this doesn’t answer your question. From what i can see, this API endpoint does not support bulk requests, meaning you need to send a separate API request for each stream created. Regarding JSON,

data starting with curly brace { denotes an object, which does not support multiple entires. For example

{},
{}

Is not valid json. Hope that helps.

[anonimus (Name)]

Ended up exporting stream list in API GUI, removing fields like ID, creation date and so on, that are not needed to create new stream in API.
Then another person user postman to send POST requests to http://adress/api/streams and I tried with curl on linux server to localhost:port/api/streams, bash script was reading each json object from array
[ {}, {}, {} ] and in while loop posting them one by one to URL.
jq -c '.[]' streams.json reads object and puts it into single line

streams.json - file containing array [ {}, {}, {} ]

#!/bin/bash

GRAYLOG_SERVER="http://localhost:9100"
API_USERNAME="name"
API_PASSWORD="pass"

# Array of stream configurations

readarray streams < <(jq -c '.[]' streams.json)

# Loop through the array and create streams
for stream_data in "${streams[@]}"; do
    create_stream_url="${GRAYLOG_SERVER}/api/streams"
    response=$(curl -i --header "X-Requested-By:localhost" -u "${API_USERNAME}:${API_PASSWORD}" -H "Content-Type: application/json" -X POST -d "${stream_data}" "${create_stream_url}")

    # echo "${stream_data}" - checking variable content

    if [ "$(echo "$response" | jq -r '.code')" = "201" ]; then
        echo "Stream created successfully."
    else
        echo "Failed to create stream: $(echo "$response" | jq -r '.code'), $(echo "$response" | jq -r '.message')"
    fi
done

Response was not 201 for some reason, but streams appeared in GUI.

[Joel_Duffield]

Sometimes I have used content packs to accomplish tasks like this. Obviously you can actually export a content pack and import it, but a content pack is basically just a JSON file, so you can also edit that file with whatever automation tools you want and then trigger that content pack to be installed. This is helpful in many places where multiple actions aren’t available in a single API call.

[anonimus (Name)]

content packs, saw something like that somewhere. Is that an enterprise feature? Because we use community edition or how its called, free one.

[Joel_Duffield]

Nope, they are in all versions, they let you export and import configs.