Having Issues Getting JSON Results From API


(David Merris) #1

I recently set up a three host graylog cluster on CentOS using Amazon’s EC2.
I have tested sending messages to the cluster, and they seem to be arriving, but when I attempt to use the search function, I get no results.
I attempted some troubleshooting, and determined that the messages were finding their way into Elasticsearch. I then attempted to query the data using the Graylog API.
Using:

curl -u user:password -H 'Accept: text/csv' -X GET 'http://172.16.13.138:9000/api/search/universal/absolute?query=David&from=2017-09-01T04%3A00%3A00.000Z&to=2017-10-01T04%3A00%3A00.000Z&limit=150&sort=timestamp%3Adesc&fields=message'

I get results. I can see the timestamp and the log message, as I would expect.

Using:

curl -u user:password -H 'Accept: application/json' -X GET 'http://172.16.13.138:9000/api/search/universal/absolute?query=David&from=2017-09-01T04%3A00%3A00.000Z&to=2017-10-01T04%3A00%3A00.000Z&limit=150&sort=timestamp%3Adesc&fields=message'

I get:

{"query":"David","built_query":"{\n  \"from\" : 0,\n  \"size\" : 150,\n  \"query\" : {\n    \"bool\" : {\n      \"must\" : {\n        \"query_string\" : {\n          \"query\" : \"David\",\n          \"allow_leading_wildcard\" : false\n        }\n      },\n      \"filter\" : {\n        \"bool\" : {\n          \"must\" : {\n            \"range\" : {\n              \"timestamp\" : {\n                \"from\" : \"2017-09-01 04:00:00.000\",\n                \"to\" : \"2017-10-01 04:00:00.000\",\n                \"include_lower\" : true,\n                \"include_upper\" : true\n              }\n            }\n          }\n        }\n      }\n    }\n  },\n  \"_source\" : {\n    \"includes\" : [ \"timestamp\", \"message\" ],\n    \"excludes\" : [ ]\n  },\n  \"sort\" : [ {\n    \"timestamp\" : {\n      \"order\" : \"desc\"\n    }\n  } ]\n}","used_indices":[],"messages":[],"fields":[],"time":0,"total_results":0,"from":"2017-09-01T04:00:00.000Z","to":"2017-10-01T04:00:00.000Z","decoration_stats":null}

Which I believe is telling me there are no results.

I feel like I am missing something, but at this point, I’m not feeling like I can find it, and would appreciate help.

Thanks!


(Shane) #2

Have you gone to you changed your search criteria for longer than 5 minutes? For instance change it to 24 hours to see if any results populate.


(David Merris) #3

I set it for an absolute range, from 1 Sept 2017 to 1 Oct 2017. When I posted this, that was about 25 days in the past to about 5 days in the future.

edit: I did also try setting it for past 8 hours and still do not seem to get results.


(Shane) #4

If the data is in Elasticsearch, then Graylog should be able to query it and display the data. If you go to System>Indices, is there an index showing? It almost sounds like there isn’t a connection between Graylog and Elasticsearch. Also, have you checked the graylog and elasticsearch logs?


(David Merris) #5

There is a “Default index set” with 1 index, and 59,743 documents.

I looked at the logs, and I didn’t see anything that looked like it was an issue.

I’m pretty sure that Graylog and Elasticsearch can communicate though, as I get a response, with appropriate messages, when I use curl to request the response as a CSV. It is only when I request a JSON response that I seem to get an empty result. (The lack of a valid JSON response seems to be keeping the web interface from functioning.)


(Jan Doberstein) #6

@moosetheory

  • did you check the log files?
  • did you check if a test messages is available for search?

If the communication is not possible to Elasticsearch Graylog would tell you that. Did you build your own Elasticsearch Cluster or did you use Amazon Elasticsearch Cluster?


(David Merris) #7

@jan

In the log files, I see several references to:

2017-09-26T18:54:00.852Z WARN  [Reflections] could not get type for name org.graylog.plugins.map.geoip.GeoIpResolverEngine$GeoLocationInformation from any class loader
org.reflections.ReflectionsException: could not get type for name org.graylog.plugins.map.geoip.GeoIpResolverEngine$GeoLocationInformation

and several variants of that. (I’ve seen the GeoIpResolver, pipelineprocessor, collectors, as an example.) I don’t recall seeing them previously, but I may be misremembering.
And I did check if a test message is available for search.

I am using an Elasticsearch cluster built using instructions linked in the Graylog docs at https://www.elastic.co/guide/en/elasticsearch/reference/5.4/rpm.html

I created a simple program that logged a bunch of GELF messages, with the message of "Hi David"
If I use the API to request results with the Accept header set to “text/csv” I get 151 lines of CSV. When I change the Accept header to “application/json” I get the block of JSON I posted in my first post.


(system) #8

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.