Trying to create an extractor for input via API

[zohan (rus)]

Hi everyone,

So basically I’m getting 404 whenever I do POST request to /system/inputs/{inputID}/extractors, however, I’m 100% positive that the input is there.

This is how my payload looks like:

extractor_payload = json.loads(
‘’’{“title”:“FluentD”,“extractor_type”:“json”,“converters”:{},“order”:0,“cut_or_copy”:“copy”,“source_field”:“kubernetes”,“target_field”:"",“extractor_config”:{“list_separator”:",",“kv_separator”:":",“key_prefix”:"",“key_separator”:"",“replace_key_whitespace”:false,“key_whitespace_replacement”:""},“condition_type”:“none”,“condition_value”:""}’’’)

If I do the same from browser-api, works just fine(meaning if I copy/paste the same payload).
I’ve tried all the headers in my script. I’ve made sure that, I create an input first and then use inputs ID to create an extractor, however, still getting 404. I also tried manually passing inputs ID, still no luck.

I appreciate your time,
Thanks.

[macko003]

Could you share the full cli command how do you do it?
maybe you missed to inform GL you will send the information in json. But I think, it won’t give a 404 in this case.
// Of course, with masked ip, username, and pass.

[zohan (rus)]

So this is a command that does it:

self.url = f'http://{self.options.ip_address}:9000'
self.extractor_path = "/system/inputs/{0}/extractors"
self.headers = {'X-Requested-By': 'cli'}
self.extractor_payload = json.loads(
            '''{"title":"FluentD","extractor_type":"json","converters":{},"order":0,"cut_or_copy":"copy","source_field":"kubernetes","target_field":"","extractor_config":{"list_separator":",","kv_separator":":","key_prefix":"","key_separator":"_","replace_key_whitespace":false,"key_whitespace_replacement":"_"},"condition_type":"none","condition_value":""}''')

response = requests.post(self.url + self.extractor_path.format(input_id), headers=self.headers,
                                     json=self.extractor_payload, auth=(self.user, self.password))

Please let me know if the above doesn’t make sense, but like I’ve said, if I manually copy/paste JSON payload and paste it into browser-api, works just fine.

Also I’ve made sure that I’m generating the right URL and getting inputs ID. I know, I’m probably missing something small, but it has been driving me crazy.

Thank you.

[zohan (rus)]

Let me beatify this JSON:

{
  "title": "FluentD",
  "extractor_type": "json",
  "converters": {},
  "order": 0,
  "cut_or_copy": "copy",
  "source_field": "kubernetes",
  "target_field": "",
  "extractor_config": {
    "list_separator": ",",
    "kv_separator": ":",
    "key_prefix": "",
    "key_separator": "_",
    "replace_key_whitespace": false,
    "key_whitespace_replacement": "_"
  },
  "condition_type": "none",
  "condition_value": ""
}

[macko003]

I think you should use username an password also.
I suggest to start with curl, test it from cli, and if it’s working well.

[zohan (rus)]

I do pass username and pass here - auth=(self.user, self.password). This is just a small piece from the script. Using the same headers and credentials I have a method that creates an input first and then passes the inputs ID to another method that creates an extractor.

I don’t have any issues creating an input, however, I’m getting 404 when it comes to extractor.

I’ll do it from the cli as well, but don’t think it makes any difference.

[zohan (rus)]

I get the same response doing the API call from the cli as well:

curl -u $user -X POST http://localhost:9000/system/inputs/5e864e399c26da1d7d18d1ac/extractors -H "X-Requested-By: cli" -d "{"title":"FluentD","extractor_type":"json","converters":{},"order":0,"cut_or_copy":"copy","source_field":"kubernetes","target_field":"","extractor_config":{"list_separator":",","kv_separator":":","key_prefix":"","key_separator":"_","replace_key_whitespace":false,"key_whitespace_replacement":"_"},"condition_type":"none","condition_value":""}"
Enter host password for user '$user':
{"type":"ApiError","message":"HTTP 404 Not Found"}

However input is there, please see below attached.
Thanks.

[shoothub]

Hi @zohan,

you made a small mistake, which is why you get 404:

Use use http://localhost:900/system/input/xxxx/extractors, but you forgot to use api. So correct URL will be:

http://localhost:9000/api/system/inputs/5e864e399c26da1d7d18d1ac/extractors

I tried your example usign curl and everything worked as expected:
curl -i -X POST -u admin:password -H 'Content-Type: application/json' -H 'X-Requested-By: cli' 'http://localhost:9000/api/system/inputs/5e611e345c544d030fc8148e/extractors' -d '{"title":"FluentD","extractor_type":"json","converters":{},"order":0,"cut_or_copy":"copy","source_field":"kubernetes","target_field":"","extractor_config":{"list_separator":",","kv_separator":":","key_prefix":"","key_separator":"_","replace_key_whitespace":false,"key_whitespace_replacement":"_"},"condition_type":"none","condition_value":""}'

[macko003]

nice catch. but it is easier if he shares the full link

[zohan (rus)]

Hi @shoothub,

Thank you much for that catch. I can’t believe I’ve missed it. I knew it was something small.

Thanks again, this can be closed.