Timestamp in Elastisearch query

[Erwin]

Hello fellow Graylog users,

I’m trying to setup streams at my graylog node. I’m able to successfully create a stream and enable rules. After enabling those rules i do not recieve any syslog messages on those streams.

I can confirm that my inputs do recieve messages, and the standard ‘All messages’ stream does also. one thing i noticed is that the elasticearch query uses a different timestamp.

Time configuration

User admin:
** 2017-12-14 12:38:32 +01:00**
Your web browser:
** 2017-12-14 12:38:32 +01:00**
Graylog server:
** 2017-12-14 12:38:32 +01:00**

And below the elasticsearch query using the ‘last 5 min’:

“from”: 0,
“size”: 150,
“query”: {
“bool”: {
“must”: {
“match_all”: {}
},
“filter”: {
“bool”: {
“must”: [
{
“range”: {
“timestamp”: {
“from”: “2017-12-14 11:33:57.663”,
“to”: “2017-12-14 11:38:57.663”,
“include_lower”: true,
“include_upper”: true
}
}
},
{
“query_string”: {
“query”: “streams:5a324f3f3afd3c03b485d681”
}
}
]
}
}
}
},
“sort”: [
{
“timestamp”: {
“order”: “desc”
}
}
]
}

As you can see there is a difference of a hour. and i’m wondering how i do solve this, and if this may be the issue.

Graylog v2.3.2+3df951e
elasticsearch.noarch 5.6.5-1
mongodb-org.x86_64 3.6.0-1.el7

[jochen]

@Erwin

As you can see there is a difference of a hour.

The Messages are stored with a timestamp in UTC in Elasticsearch.

Your Graylog cluster is configured for GMT+1, so the translation is correct.

[Erwin]

Hello Jochen,

Ok, so do you have any troubleshooting steps to find out why my stream is not recieving any messages?

[jochen]

Are you sure that your messages have the correct timestamp?
Do you have examples?

[Erwin]

Yes i’m sure they are correct. See example below.

message:

<141>88184: Dec 20 14:49:57: %SEC_LOGIN-5-LOGIN_SUCCESS: Login Success [user: erwin] [Source: x.x.x.x] [localport: 22] at 14:49:57 CEDT Wed Dec 20 2017

source : x.x.x.x
timestamp: 2017-12-20T14:49:58.510Z

The IP addresses are replaced with ‘x’.

[Erwin]

Also, i noticed that my extractors are not working either.

If i’m configuring the extractor and try my regular expression it works fine.
After saving the extractor it just does nothing at all.

I’m having exact the same setup configured on another server and there are no problems at all.

[jcspino]

Erwin,

Did you confirm that the “Message Processors Configuration” matches between the 2 servers? If you go to system>configuration there is a control for the order in which messages are processed. I banged my head against the wall for a few hours one day before realizing that my message filter was processing after my piplines.

[Erwin]

Hello jcspino,

Thank you for your reply.

My message configuration was indeed in the wrong order. After editing the order everything seems to work fine.

Only issue left is that not every IP address is being resolved to a hostname, but i’m probally able to resolve this myself.