Could not execute search after new installation

[IBulti (Ian Bultitude)]

Hi jochen,

That is really strange. NO that is not the whole file.

The installation did not work correctly!

$ echo “deb https://packages.elastic.co/elasticsearch/2.x/debian stable main” | sudo tee -a /etc/apt/sources.list.d/elasticsearch-2.x.list

did not work and I had to add it manually.

sudo systemctl daemon-reload sudo systemctl enable elasticsearch.service
$ sudo systemctl restart elasticsearch.service

these do not seem to start the program. If I run

root@collective:/usr/share/elasticsearch/bin# ./elasticsearch

Then the program starts but Graylog still does not work correctly.

I am thinking I am just going to have to get someone to connect to the system and have a look.

Did the server.log file I uploaded to gist.github.com work? If so I post the Elasticsearch file. and send you the link.

Regards
Ian

[IBulti (Ian Bultitude)]

Looks like it worked last time so here is the elasticsearch config file.

https://gist.github.com/anonymous/ed4a5430545f5904eeb31618da6f17e9

Regards

Ian

mailto:ian@bultitude.org
Phone +44 1380-888-181 / +44 1380-720-579

Mobile +44-7785-532-628

VOIP 869875@draytel.org (08709-744-424)

[jochen]

@IBulti

$ echo “deb https://packages.elastic.co/elasticsearch/2.x/debian stable main” | sudo tee -a /etc/apt/sources.list.d/elasticsearch-2.x.list

did not work and I had to add it manually.

Why didn’t it work? What was the output? What exactly did you add to the /etc/apt/sources.list.d/elasticsearch-2.x.list file?

@IBulti

$ sudo systemctl daemon-reload
$ sudo systemctl enable elasticsearch.service
$ sudo systemctl restart elasticsearch.service

these do not seem to start the program.

What’s the output of these commands?

@IBulti

If I run

root@collective:/usr/share/elasticsearch/bin# ./elasticsearch

Then the program starts but Graylog still does not work correctly.

That’s a bad idea because it will create several directories and files with the wrong system user (root) and the elasticsearch system user won’t be able to access these directories and files.

Also, what’s the output of dpkg -l elasticsearch on that system?

[IBulti (Ian Bultitude)]

Hi jochen,

When the command was issued there was no output just returned to the prompt.

here (I hope) is the file

ian@collective:/etc/apt/sources.list.d$ cat elasticsearch-2.x.list

deb https://packages.elastic.co/elasticsearch/2.x/debian stable main

ian@collective:/etc/apt/sources.list.d$

I also had to add it sources.list but I cannot find where I got that information from.

Ok re running the file manually. Looks like I will need to remove or chown the files. Do you have a list?

Outputs

ian@collective:~$ sudo systemctl daemon-reload

[sudo] password for ian:

ian@collective:~$

ian@collective:~$ sudo systemctl enable elasticsearch.service

elasticsearch.service is not a native service, redirecting to systemd-sysv-install

Executing /lib/systemd/systemd-sysv-install enable elasticsearch

ian@collective:~$

ian@collective:~$ sudo systemctl restart elasticsearch.service

ian@collective:~$

ian@collective:~$ dpkg -l elasticsearch

Desired=Unknown/Install/Remove/Purge/Hold

| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend

|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)

||/ Name Version Architecture Description

[jochen]

@IBulti

ian@collective:~$ dpkg -l elasticsearch

Desired=Unknown/Install/Remove/Purge/Hold

| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend

|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)

||/ Name Version Architecture Description

This is incomplete (again).

This is how the output looks like on a healthy system:

$ dpkg -l elasticsearch
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name                                          Version                     Architecture                Description
+++-=============================================-===========================-===========================-===============================================================================================
ii  elasticsearch                                 2.4.4                       all                         Elasticsearch is a distributed RESTful search engine built for the cloud. Reference documentation can be found at https://www.elastic.co/guide/en/elasticsearch/reference/current/index.html and the 'Elasticsearch: The Definitive Guide' book can be found at https://www.elastic.co/guide/en/elasticsearch/guide/current/index.html

I’m sorry to say that, but this is also taking too long to solve this and it’s obviously a problem with your Elasticsearch installation.

Please purge (uninstall and remove all data of) Elasticsearch from your system and follow the step-by-step guide at http://docs.graylog.org/en/2.2/pages/installation/os/ubuntu.html. It has been tested multiple times on a fresh installation of Ubuntu Linux and it is working.

If you don’t want to install Graylog and all of its components manually, take a look at the virtual machine images we provide: http://docs.graylog.org/en/2.2/pages/installation/virtual_machine_appliances.html

[IBulti (Ian Bultitude)]

Ok Thanks for the help.

I do not understand why my posts are being truncated every time.

There is NO WAY I can afford the professional support so I guess I’ll just abandon this project.

Regards

Ian

[jochen]

As I said before, I’d recommend using the virtual machine appliance (OVA) to install Graylog in your environment: http://docs.graylog.org/en/2.2/pages/installation/virtual_machine_appliances.html

[DRDL]

Kindly please find my tail -f /var/log/graylog-server/server.log
file i am unable to get the logs in graylog

at org.glassfish.jersey.internal.Errors.process(Errors.java:267) [graylog.jar:?]
at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:317) [graylog.jar:?]
at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:305) [graylog.jar:?]
at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:1154) [graylog.jar:?]
at org.glassfish.jersey.grizzly2.httpserver.GrizzlyHttpContainer.service(GrizzlyHttpContainer.java:384) [graylog.jar:?]
at org.glassfish.grizzly.http.server.HttpHandler$1.run(HttpHandler.java:224) [graylog.jar:?]
at com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:176) [graylog.jar:?]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [?:1.8.0_131]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [?:1.8.0_131]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_131]
2017-06-05T11:46:31.114+05:30 ERROR [AnyExceptionClassMapper] Unhandled exception in REST resource
org.elasticsearch.cluster.block.ClusterBlockException: blocked by: [SERVICE_UNAVAILABLE/1/state not recovered / initialized];
at org.elasticsearch.cluster.block.ClusterBlocks.globalBlockedException(ClusterBlocks.java:158) ~[graylog.jar:?]
at org.elasticsearch.cluster.block.ClusterBlocks.globalBlockedRaiseException(ClusterBlocks.java:144) ~[graylog.jar:?]
at org.elasticsearch.action.search.AbstractSearchAsyncAction.(AbstractSearchAsyncAction.java:94) ~[graylog.jar:?]
at org.elasticsearch.action.search.SearchQueryThenFetchAsyncAction.(SearchQueryThenFetchAsyncAction.java:53) ~[graylog.jar:?]
at org.elasticsearch.action.search.TransportSearchAction.doExecute(TransportSearchAction.java:93) ~[graylog.jar:?]
at org.elasticsearch.action.search.TransportSearchAction.doExecute(TransportSearchAction.java:47) ~[graylog.jar:?]
at org.elasticsearch.action.support.TransportAction.doExecute(TransportAction.java:149) ~[graylog.jar:?]
at org.elasticsearch.action.support.TransportAction.execute(TransportAction.java:137) ~[graylog.jar:?]
at org.elasticsearch.action.support.TransportAction.execute(TransportAction.java:85) ~[graylog.jar:?]
at org.elasticsearch.client.node.NodeClient.doExecute(NodeClient.java:58) ~[graylog.jar:?]
at org.elasticsearch.client.support.AbstractClient.execute(AbstractClient.java:359) ~[graylog.jar:?]
at org.elasticsearch.client.FilterClient.doExecute(FilterClient.java:52) ~[graylog.jar:?]
at org.graylog2.indexer.elasticsearch.GlobalTimeoutClient.doExecute(GlobalTimeoutClient.java:58) ~[graylog.jar:?]
at org.elasticsearch.client.support.AbstractClient.execute(AbstractClient.java:359) ~[graylog.jar:?]
at org.elasticsearch.client.support.AbstractClient.execute(AbstractClient.java:348) ~[graylog.jar:?]
at org.elasticsearch.client.support.AbstractClient.search(AbstractClient.java:577) ~[graylog.jar:?]
at org.graylog2.indexer.elasticsearch.GlobalTimeoutClient.search(GlobalTimeoutClient.java:64) ~[graylog.jar:?]
at org.graylog2.indexer.searches.Searches.search(Searches.java:264) ~[graylog.jar:?]
at org.graylog2.rest.resources.search.RelativeSearchResource.searchRelative(RelativeSearchResource.java:113) ~[graylog.jar:?]
at sun.reflect.GeneratedMethodAccessor78.invoke(Unknown Source) ~[?:?]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_131]
at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_131]
at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory$1.invoke(ResourceMethodInvocationHandlerFactory.java:81) ~[graylog.jar:?]
at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:144) ~[graylog.jar:?]
at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:161) ~[graylog.jar:?]
at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$TypeOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:205) ~[graylog.jar:?]
at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:99) ~[graylog.jar:?]
at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:389) ~[graylog.jar:?]
at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:347) ~[graylog.jar:?]
at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:102) ~[graylog.jar:?]
at org.glassfish.jersey.server.ServerRuntime$2.run(ServerRuntime.java:326) [graylog.jar:?]
at org.glassfish.jersey.internal.Errors$1.call(Errors.java:271) [graylog.jar:?]
at org.glassfish.jersey.internal.Errors$1.call(Errors.java:267) [graylog.jar:?]
at org.glassfish.jersey.internal.Errors.process(Errors.java:315) [graylog.jar:?]
at org.glassfish.jersey.internal.Errors.process(Errors.java:297) [graylog.jar:?]
at org.glassfish.jersey.internal.Errors.process(Errors.java:267) [graylog.jar:?]
at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:317) [graylog.jar:?]
at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:305) [graylog.jar:?]
at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:1154) [graylog.jar:?]
at org.glassfish.jersey.grizzly2.httpserver.GrizzlyHttpContainer.service(GrizzlyHttpContainer.java:384) [graylog.jar:?]
at org.glassfish.grizzly.http.server.HttpHandler$1.run(HttpHandler.java:224) [graylog.jar:?]
at com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:176) [graylog.jar:?]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [?:1.8.0_131]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [?:1.8.0_131]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_131]
2017-06-05T11:46:35.683+05:30 ERROR [AnyExceptionClassMapper] Unhandled exception in REST resource
org.elasticsearch.cluster.block.ClusterBlockException: blocked by: [SERVICE_UNAVAILABLE/1/state not recovered / initialized];
at org.elasticsearch.cluster.block.ClusterBlocks.globalBlockedException(ClusterBlocks.java:158) ~[graylog.jar:?]
at org.elasticsearch.cluster.block.ClusterBlocks.globalBlockedRaiseException(ClusterBlocks.java:144) ~[graylog.jar:?]
at org.elasticsearch.action.search.AbstractSearchAsyncAction.(AbstractSearchAsyncAction.java:94) ~[graylog.jar:?]
at org.elasticsearch.action.search.SearchQueryThenFetchAsyncAction.(SearchQueryThenFetchAsyncAction.java:53) ~[graylog.jar:?]
at org.elasticsearch.action.search.TransportSearchAction.doExecute(TransportSearchAction.java:93) ~[graylog.jar:?]
at org.elasticsearch.action.search.TransportSearchAction.doExecute(TransportSearchAction.java:47) ~[graylog.jar:?]
at org.elasticsearch.action.support.TransportAction.doExecute(TransportAction.java:149) ~[graylog.jar:?]
at org.elasticsearch.action.support.TransportAction.execute(TransportAction.java:137) ~[graylog.jar:?]
at org.elasticsearch.action.support.TransportAction.execute(TransportAction.java:85) ~[graylog.jar:?]
at org.elasticsearch.client.node.NodeClient.doExecute(NodeClient.java:58) ~[graylog.jar:?]
at org.elasticsearch.client.support.AbstractClient.execute(AbstractClient.java:359) ~[graylog.jar:?]
at org.elasticsearch.client.FilterClient.doExecute(FilterClient.java:52) ~[graylog.jar:?]
at org.graylog2.indexer.elasticsearch.GlobalTimeoutClient.doExecute(GlobalTimeoutClient.java:58) ~[graylog.jar:?]
at org.elasticsearch.client.support.AbstractClient.execute(AbstractClient.java:359) ~[graylog.jar:?]
at org.elasticsearch.client.support.AbstractClient.execute(AbstractClient.java:348) ~[graylog.jar:?]
at org.elasticsearch.client.support.AbstractClient.search(AbstractClient.java:577) ~[graylog.jar:?]
at org.graylog2.indexer.elasticsearch.GlobalTimeoutClient.search(GlobalTimeoutClient.java:64) ~[graylog.jar:?]
at org.graylog2.indexer.searches.Searches.search(Searches.java:264) ~[graylog.jar:?]
at org.graylog2.rest.resources.search.RelativeSearchResource.searchRelative(RelativeSearchResource.java:113) ~[graylog.jar:?]
at sun.reflect.GeneratedMethodAccessor78.invoke(Unknown Source) ~[?:?]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_131]
at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_131]
at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory$1.invoke(ResourceMethodInvocationHandlerFactory.java:81) ~[graylog.jar:?]
at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:144) ~[graylog.jar:?]
at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:161) ~[graylog.jar:?]
at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$TypeOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:205) ~[graylog.jar:?]
at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:99) ~[graylog.jar:?]
at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:389) ~[graylog.jar:?]
at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:347) ~[graylog.jar:?]
at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:102) ~[graylog.jar:?]
at org.glassfish.jersey.server.ServerRuntime$2.run(ServerRuntime.java:326) [graylog.jar:?]
at org.glassfish.jersey.internal.Errors$1.call(Errors.java:271) [graylog.jar:?]
at org.glassfish.jersey.internal.Errors$1.call(Errors.java:267) [graylog.jar:?]
at org.glassfish.jersey.internal.Errors.process(Errors.java:315) [graylog.jar:?]
at org.glassfish.jersey.internal.Errors.process(Errors.java:297) [graylog.jar:?]
at org.glassfish.jersey.internal.Errors.process(Errors.java:267) [graylog.jar:?]
at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:317) [graylog.jar:?]
at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:305) [graylog.jar:?]
at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:1154) [graylog.jar:?]
at org.glassfish.jersey.grizzly2.httpserver.GrizzlyHttpContainer.service(GrizzlyHttpContainer.java:384) [graylog.jar:?]
at org.glassfish.grizzly.http.server.HttpHandler$1.run(HttpHandler.java:224) [graylog.jar:?]
at com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:176) [graylog.jar:?]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [?:1.8.0_131]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [?:1.8.0_131]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_131]
2017-06-05T11:46:37.576+05:30 INFO [IndexRetentionThread] Elasticsearch cluster not available, skipping index retention checks.
2017-06-05T11:46:40.704+05:30 ERROR [AnyExceptionClassMapper] Unhandled exception in REST resource
org.elasticsearch.cluster.block.ClusterBlockException: blocked by: [SERVICE_UNAVAILABLE/1/state not recovered / initialized];
at org.elasticsearch.cluster.block.ClusterBlocks.globalBlockedException(ClusterBlocks.java:158) ~[graylog.jar:?]
at org.elasticsearch.cluster.block.ClusterBlocks.globalBlockedRaiseException(ClusterBlocks.java:144) ~[graylog.jar:?]
at org.elasticsearch.action.search.AbstractSearchAsyncAction.(AbstractSearchAsyncAction.java:94) ~[graylog.jar:?]
at org.elasticsearch.action.search.SearchQueryThenFetchAsyncAction.(SearchQueryThenFetchAsyncAction.java:53) ~[graylog.jar:?]
at org.elasticsearch.action.search.TransportSearchAction.doExecute(TransportSearchAction.java:93) ~[graylog.jar:?]
at org.elasticsearch.action.search.TransportSearchAction.doExecute(TransportSearchAction.java:47) ~[graylog.jar:?]
at org.elasticsearch.action.support.TransportAction.doExecute(TransportAction.java:149) ~[graylog.jar:?]
at org.elasticsearch.action.support.TransportAction.execute(TransportAction.java:137) ~[graylog.jar:?]
at org.elasticsearch.action.support.TransportAction.execute(TransportAction.java:85) ~[graylog.jar:?]
at org.elasticsearch.client.node.NodeClient.doExecute(NodeClient.java:58) ~[graylog.jar:?]
at org.elasticsearch.client.support.AbstractClient.execute(AbstractClient.java:359) ~[graylog.jar:?]
at org.elasticsearch.client.FilterClient.doExecute(FilterClient.java:52) ~[graylog.jar:?]
at org.graylog2.indexer.elasticsearch.GlobalTimeoutClient.doExecute(GlobalTimeoutClient.java:58) ~[graylog.jar:?]
at org.elasticsearch.client.support.AbstractClient.execute(AbstractClient.java:359) ~[graylog.jar:?]
at org.elasticsearch.client.support.AbstractClient.execute(AbstractClient.java:348) ~[graylog.jar:?]
at org.elasticsearch.client.support.AbstractClient.search(AbstractClient.java:577) ~[graylog.jar:?]
at org.graylog2.indexer.elasticsearch.GlobalTimeoutClient.search(GlobalTimeoutClient.java:64) ~[graylog.jar:?]
at org.graylog2.indexer.searches.Searches.search(Searches.java:264) ~[graylog.jar:?]
at org.graylog2.rest.resources.search.RelativeSearchResource.searchRelative(RelativeSearchResource.java:113) ~[graylog.jar:?]
at sun.reflect.GeneratedMethodAccessor78.invoke(Unknown Source) ~[?:?]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_131]
at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_131]
at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory$1.invoke(ResourceMethodInvocationHandlerFactory.java:81) ~[graylog.jar:?]
at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:144) ~[graylog.jar:?]
at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:161) ~[graylog.jar:?]
at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$TypeOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:205) ~[graylog.jar:?]
at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:99) ~[graylog.jar:?]
at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:389) ~[graylog.jar:?]
at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:347) ~[graylog.jar:?]
at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:102) ~[graylog.jar:?]
at org.glassfish.jersey.server.ServerRuntime$2.run(ServerRuntime.java:326) [graylog.jar:?]
at org.glassfish.jersey.internal.Errors$1.call(Errors.java:271) [graylog.jar:?]
at org.glassfish.jersey.internal.Errors$1.call(Errors.java:267) [graylog.jar:?]
at org.glassfish.jersey.internal.Errors.process(Errors.java:315) [graylog.jar:?]
at org.glassfish.jersey.internal.Errors.process(Errors.java:297) [graylog.jar:?]
at org.glassfish.jersey.internal.Errors.process(Errors.java:267) [graylog.jar:?]
at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:317) [graylog.jar:?]
at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:305) [graylog.jar:?]
at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:1154) [graylog.jar:?]
at org.glassfish.jersey.grizzly2.httpserver.GrizzlyHttpContainer.service(GrizzlyHttpContainer.java:384) [graylog.jar:?]
at org.glassfish.grizzly.http.server.HttpHandler$1.run(HttpHandler.java:224) [graylog.jar:?]
at com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:176) [graylog.jar:?]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [?:1.8.0_131]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [?:1.8.0_131]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_131]
2017-06-05T11:46:41.344+05:30 WARN [V20161130141500_DefaultStreamRecalcIndexRanges] Interrupted or timed out waiting for Elasticsearch cluster, checking again.
2017-06-05T11:46:41.399+05:30 INFO [IndexerClusterCheckerThread] Indexer not fully initialized yet. Skipping periodic cluster check.
2017-06-05T11:46:45.683+05:30 ERROR [AnyExceptionClassMapper] Unhandled exception in REST resource
org.elasticsearch.cluster.block.ClusterBlockException: blocked by: [SERVICE_UNAVAILABLE/1/state not recovered / initialized];
at org.elasticsearch.cluster.block.ClusterBlocks.globalBlockedException(ClusterBlocks.java:158) ~[graylog.jar:?]
at org.elasticsearch.cluster.block.ClusterBlocks.globalBlockedRaiseException(ClusterBlocks.java:144) ~[graylog.jar:?]
at org.elasticsearch.action.search.AbstractSearchAsyncAction.(AbstractSearchAsyncAction.java:94) ~[graylog.jar:?]
at org.elasticsearch.action.search.SearchQueryThenFetchAsyncAction.(SearchQueryThenFetchAsyncAction.java:53) ~[graylog.jar:?]
at org.elasticsearch.action.search.TransportSearchAction.doExecute(TransportSearchAction.java:93) ~[graylog.jar:?]
at org.elasticsearch.action.search.TransportSearchAction.doExecute(TransportSearchAction.java:47) ~[graylog.jar:?]
at org.elasticsearch.action.support.TransportAction.doExecute(TransportAction.java:149) ~[graylog.jar:?]
at org.elasticsearch.action.support.TransportAction.execute(TransportAction.java:137) ~[graylog.jar:?]
at org.elasticsearch.action.support.TransportAction.execute(TransportAction.java:85) ~[graylog.jar:?]
at org.elasticsearch.client.node.NodeClient.doExecute(NodeClient.java:58) ~[graylog.jar:?]
at org.elasticsearch.client.support.AbstractClient.execute(AbstractClient.java:359) ~[graylog.jar:?]
at org.elasticsearch.client.FilterClient.doExecute(FilterClient.java:52) ~[graylog.jar:?]
at org.graylog2.indexer.elasticsearch.GlobalTimeoutClient.doExecute(GlobalTimeoutClient.java:58) ~[graylog.jar:?]
at org.elasticsearch.client.support.AbstractClient.execute(AbstractClient.java:359) ~[graylog.jar:?]
at org.elasticsearch.client.support.AbstractClient.execute(AbstractClient.java:348) ~[graylog.jar:?]
at org.elasticsearch.client.support.AbstractClient.search(AbstractClient.java:577) ~[graylog.jar:?]
at org.graylog2.indexer.elasticsearch.GlobalTimeoutClient.search(GlobalTimeoutClient.java:64) ~[graylog.jar:?]
at org.graylog2.indexer.searches.Searches.search(Searches.java:264) ~[graylog.jar:?]
at org.graylog2.rest.resources.search.RelativeSearchResource.searchRelative(RelativeSearchResource.java:113) ~[graylog.jar:?]
at sun.reflect.GeneratedMethodAccessor78.invoke(Unknown Source) ~[?:?]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_131]
at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_131]
at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory$1.invoke(ResourceMethodInvocationHandlerFactory.java:81) ~[graylog.jar:?]
at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:144) ~[graylog.jar:?]
at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:161) ~[graylog.jar:?]
at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$TypeOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:205) ~[graylog.jar:?]
at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:99) ~[graylog.jar:?]
at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:389) ~[graylog.jar:?]
at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:347) ~[graylog.jar:?]
at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:102) ~[graylog.jar:?]
at org.glassfish.jersey.server.ServerRuntime$2.run(ServerRuntime.java:326) [graylog.jar:?]
at org.glassfish.jersey.internal.Errors$1.call(Errors.java:271) [graylog.jar:?]
at org.glassfish.jersey.internal.Errors$1.call(Errors.java:267) [graylog.jar:?]
at org.glassfish.jersey.internal.Errors.process(Errors.java:315) [graylog.jar:?]
at org.glassfish.jersey.internal.Errors.process(Errors.java:297) [graylog.jar:?]
at org.glassfish.jersey.internal.Errors.process(Errors.java:267) [graylog.jar:?]
at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:317) [graylog.jar:?]
at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:305) [graylog.jar:?]
at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:1154) [graylog.jar:?]
at org.glassfish.jersey.grizzly2.httpserver.GrizzlyHttpContainer.service(GrizzlyHttpContainer.java:384) [graylog.jar:?]
at org.glassfish.grizzly.http.server.HttpHandler$1.run(HttpHandler.java:224) [graylog.jar:?]
at com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:176) [graylog.jar:?]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [?:1.8.0_131]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [?:1.8.0_131]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_131]
2017-06-05T11:46:50.671+05:30 ERROR [AnyExceptionClassMapper] Unhandled exception in REST resource
org.elasticsearch.cluster.block.ClusterBlockException: blocked by: [SERVICE_UNAVAILABLE/1/state not recovered / initialized];
at org.elasticsearch.cluster.block.ClusterBlocks.globalBlockedException(ClusterBlocks.java:158) ~[graylog.jar:?]
at org.elasticsearch.cluster.block.ClusterBlocks.globalBlockedRaiseException(ClusterBlocks.java:144) ~[graylog.jar:?]
at org.elasticsearch.action.search.AbstractSearchAsyncAction.(AbstractSearchAsyncAction.java:94) ~[graylog.jar:?]
at org.elasticsearch.action.search.SearchQueryThenFetchAsyncAction.(SearchQueryThenFetchAsyncAction.java:53) ~[graylog.jar:?]
at org.elasticsearch.action.search.TransportSearchAction.doExecute(TransportSearchAction.java:93) ~[graylog.jar:?]
at org.elasticsearch.action.search.TransportSearchAction.doExecute(TransportSearchAction.java:47) ~[graylog.jar:?]
at org.elasticsearch.action.support.TransportAction.doExecute(TransportAction.java:149) ~[graylog.jar:?]
at org.elasticsearch.action.support.TransportAction.execute(TransportAction.java:137) ~[graylog.jar:?]
at org.elasticsearch.action.support.TransportAction.execute(TransportAction.java:85) ~[graylog.jar:?]
at org.elasticsearch.client.node.NodeClient.doExecute(NodeClient.java:58) ~[graylog.jar:?]
at org.elasticsearch.client.support.AbstractClient.execute(AbstractClient.java:359) ~[graylog.jar:?]
at org.elasticsearch.client.FilterClient.doExecute(FilterClient.java:52) ~[graylog.jar:?]
at org.graylog2.indexer.elasticsearch.GlobalTimeoutClient.doExecute(GlobalTimeoutClient.java:58) ~[graylog.jar:?]
at org.elasticsearch.client.support.AbstractClient.execute(AbstractClient.java:359) ~[graylog.jar:?]
at org.elasticsearch.client.support.AbstractClient.execute(AbstractClient.java:348) ~[graylog.jar:?]
at org.elasticsearch.client.support.AbstractClient.search(AbstractClient.java:577) ~[graylog.jar:?]
at org.graylog2.indexer.elasticsearch.GlobalTimeoutClient.search(GlobalTimeoutClient.java:64) ~[graylog.jar:?]
at org.graylog2.indexer.searches.Searches.search(Searches.java:264) ~[graylog.jar:?]
at org.graylog2.rest.resources.search.RelativeSearchResource.searchRelative(RelativeSearchResource.java:113) ~[graylog.jar:?]
at sun.reflect.GeneratedMethodAccessor78.invoke(Unknown Source) ~[?:?]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_131]
at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_131]
at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory$1.invoke(ResourceMethodInvocationHandlerFactory.java:81) ~[graylog.jar:?]
at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:144) ~[graylog.jar:?]
at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:161) ~[graylog.jar:?]
at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$TypeOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:205) ~[graylog.jar:?]
at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:99) ~[graylog.jar:?]
at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:389) ~[graylog.jar:?]
at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:347) ~[graylog.jar:?]
at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:102) ~[graylog.jar:?]
at org.glassfish.jersey.server.ServerRuntime$2.run(ServerRuntime.java:326) [graylog.jar:?]
at org.glassfish.jersey.internal.Errors$1.call(Errors.java:271) [graylog.jar:?]
at org.glassfish.jersey.internal.Errors$1.call(Errors.java:267) [graylog.jar:?]
at org.glassfish.jersey.internal.Errors.process(Errors.java:315) [graylog.jar:?]
at org.glassfish.jersey.internal.Errors.process(Errors.java:297) [graylog.jar:?]
at org.glassfish.jersey.internal.Errors.process(Errors.java:267) [graylog.jar:?]
at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:317) [graylog.jar:?]
at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:305) [graylog.jar:?]
at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:1154) [graylog.jar:?]
at org.glassfish.jersey.grizzly2.httpserver.GrizzlyHttpContainer.service(GrizzlyHttpContainer.java:384) [graylog.jar:?]
at org.glassfish.grizzly.http.server.HttpHandler$1.run(HttpHandler.java:224) [graylog.jar:?]
at com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:176) [graylog.jar:?]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [?:1.8.0_131]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [?:1.8.0_131]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_131]
2017-06-05T11:46:55.991+05:30 ERROR [AnyExceptionClassMapper] Unhandled exception in REST resource
org.elasticsearch.cluster.block.ClusterBlockException: blocked by: [SERVICE_UNAVAILABLE/1/state not recovered / initialized];
at org.elasticsearch.cluster.block.ClusterBlocks.globalBlockedException(ClusterBlocks.java:158) ~[graylog.jar:?]
at org.elasticsearch.cluster.block.ClusterBlocks.globalBlockedRaiseException(ClusterBlocks.java:144) ~[graylog.jar:?]
at org.elasticsearch.action.search.AbstractSearchAsyncAction.(AbstractSearchAsyncAction.java:94) ~[graylog.jar:?]
at org.elasticsearch.action.search.SearchQueryThenFetchAsyncAction.(SearchQueryThenFetchAsyncAction.java:53) ~[graylog.jar:?]
at org.elasticsearch.action.search.TransportSearchAction.doExecute(TransportSearchAction.java:93) ~[graylog.jar:?]
at org.elasticsearch.action.search.TransportSearchAction.doExecute(TransportSearchAction.java:47) ~[graylog.jar:?]
at org.elasticsearch.action.support.TransportAction.doExecute(TransportAction.java:149) ~[graylog.jar:?]
at org.elasticsearch.action.support.TransportAction.execute(TransportAction.java:137) ~[graylog.jar:?]
at org.elasticsearch.action.support.TransportAction.execute(TransportAction.java:85) ~[graylog.jar:?]
at org.elasticsearch.client.node.NodeClient.doExecute(NodeClient.java:58) ~[graylog.jar:?]
at org.elasticsearch.client.support.AbstractClient.execute(AbstractClient.java:359) ~[graylog.jar:?]
at org.elasticsearch.client.FilterClient.doExecute(FilterClient.java:52) ~[graylog.jar:?]
at org.graylog2.indexer.elasticsearch.GlobalTimeoutClient.doExecute(GlobalTimeoutClient.java:58) ~[graylog.jar:?]
at org.elasticsearch.client.support.AbstractClient.execute(AbstractClient.java:359) ~[graylog.jar:?]
at org.elasticsearch.client.support.AbstractClient.execute(AbstractClient.java:348) ~[graylog.jar:?]
at org.elasticsearch.client.support.AbstractClient.search(AbstractClient.java:577) ~[graylog.jar:?]
at org.graylog2.indexer.elasticsearch.GlobalTimeoutClient.search(GlobalTimeoutClient.java:64) ~[graylog.jar:?]
at org.graylog2.indexer.searches.Searches.search(Searches.java:264) ~[graylog.jar:?]
at org.graylog2.rest.resources.search.RelativeSearchResource.searchRelative(RelativeSearchResource.java:113) ~[graylog.jar:?]
at sun.reflect.GeneratedMethodAccessor78.invoke(Unknown Source) ~[?:?]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_131]
at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_131]
at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory$1.invoke(ResourceMethodInvocationHandlerFactory.java:81) ~[graylog.jar:?]
at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:144) ~[graylog.jar:?]
at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:161) ~[graylog.jar:?]
at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$TypeOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:205) ~[graylog.jar:?]
at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:99) ~[graylog.jar:?]
at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:389) ~[graylog.jar:?]
at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:347) ~[graylog.jar:?]
at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:102) ~[graylog.jar:?]
at org.glassfish.jersey.server.ServerRuntime$2.run(ServerRuntime.java:326) [graylog.jar:?]
at org.glassfish.jersey.internal.Errors$1.call(Errors.java:271) [graylog.jar:?]
at org.glassfish.jersey.internal.Errors$1.call(Errors.java:267) [graylog.jar:?]
at org.glassfish.jersey.internal.Errors.process(Errors.java:315) [graylog.jar:?]
at org.glassfish.jersey.internal.Errors.process(Errors.java:297) [graylog.jar:?]
at org.glassfish.jersey.internal.Errors.process(Errors.java:267) [graylog.jar:?]
at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:317) [graylog.jar:?]
at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:305) [graylog.jar:?]
at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:1154) [graylog.jar:?]
at org.glassfish.jersey.grizzly2.httpserver.GrizzlyHttpContainer.service(GrizzlyHttpContainer.java:384) [graylog.jar:?]
at org.glassfish.grizzly.http.server.HttpHandler$1.run(HttpHandler.java:224) [graylog.jar:?]
at com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:176) [graylog.jar:?]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [?:1.8.0_131]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [?:1.8.0_131]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_131]

[jan (Jan Doberstein)]

Do not HighJack threads and follow the same rules I had posted into your question thread:

I AM unable to get the SEARCH in graylog

what did you try to ask and why didn’t you format your post in a way that someone who is willing to help you can actually read ? Configuration and Logs inside of a Code block ``` Text ``` and asking a question would be more nice than just spit a subject and loglines here.

[DRDL]

Dear JAN,

Thank you for the help

My configuration is
Elasticsearch 2.4.4
Graylog 2.2.3
Mongo db version v2.6.10

I am running graylog server in seperate system and elasticsearch in seperate
system’
kindly please find the logs and please respond sooner.

thanks,
Suman Gupta G

[DRDL]

Dear Jan,

Also please find my graylog file and elasticsearch file

Thanks,
Suman Gupta G