Nginx access log timestamp problem

Graylog Central (peer support)
1

Hey Gys. I have two extractor among other but I cant see the prblem.
I have accese log hwere Im parsing

78.8.210.114 - - [23/Oct/2017:10:34:04 +0200] GET /temp/sma/alcon/inne-b-miesieczne_08.jpg HTTP/1.1 "200" 2289 "http://xxx.pl/soczewki-kontaktowe-6-szt-" "Mozilla/5.0 (Windows NT 10.0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.79 Safari/537.36 Edge/14.14393" ***{0.000}*** "78.8.210.114"

But in /var/log/graylog-server/server.log I can see:

java.lang.IllegalArgumentException: Invalid format: "23/Oct/2017:10:38:28 +0200" is malformed at "Oct/2017:10:38:28 +0200"
	at org.joda.time.format.DateTimeFormatter.parseDateTime(DateTimeFormatter.java:945) ~[graylog.jar:?]
	at org.joda.time.DateTime.parse(DateTime.java:160) ~[graylog.jar:?]
	at org.graylog2.inputs.converters.DateConverter.convert(DateConverter.java:79) ~[graylog.jar:?]
	at org.graylog2.plugin.inputs.Extractor.runConverters(Extractor.java:260) [graylog.jar:?]
	at org.graylog2.plugin.inputs.Extractor.runExtractor(Extractor.java:247) [graylog.jar:?]
	at org.graylog2.filters.ExtractorFilter.filter(ExtractorFilter.java:77) [graylog.jar:?]
	at org.graylog2.messageprocessors.MessageFilterChainProcessor.process(MessageFilterChainProcessor.java:100) [graylog.jar:?]
	at org.graylog2.shared.buffers.processors.ProcessBufferProcessor.handleMessage(ProcessBufferProcessor.java:114) [graylog.jar:?]
	at org.graylog2.shared.buffers.processors.ProcessBufferProcessor.dispatchMessage(ProcessBufferProcessor.java:100) [graylog.jar:?]
	at org.graylog2.shared.buffers.processors.ProcessBufferProcessor.onEvent(ProcessBufferProcessor.java:77) [graylog.jar:?]
	at org.graylog2.shared.buffers.processors.ProcessBufferProcessor.onEvent(ProcessBufferProcessor.java:42) [graylog.jar:?]
	at com.lmax.disruptor.WorkProcessor.run(WorkProcessor.java:143) [graylog.jar:?]
	at com.codahale.metrics.InstrumentedThreadFactory$InstrumentedRunnable.run(InstrumentedThreadFactory.java:66) [graylog.jar:?]
	at java.lang.Thread.run(Thread.java:745) [?:1.8.0_111]

my extractors:

{
      "title": "prod.access_log.with_response_time",
      "extractor_type": "grok",
      "converters": [],
      "order": 3,
      "cursor_strategy": "copy",
      "source_field": "message",
      "target_field": "",
      "extractor_config": {
        "grok_pattern": "%{IPORHOST:http_host} - (?:%{WORD:auth}|-) \\[%{HTTPDATE:timestamp_string}\\] %{WORD:verb} %{NOTSPACE:request} HTTP/%{NUMBER:httpversion} \"%{NUMBER:response}\" (?:%{NUMBER:bytes}|-) \"(?:%{URI:referrer}|-)\" %{QS:agent} \\*\\*\\*\\{%{NUMBER:repsonse_time}\\}\\*\\*\\* \"(?<ips>%{IP}(, %{IP})*|-)\""
      },
      "condition_type": "string",
      "condition_value": "***{"
    },
 {
      "title": "[access_log] timestamp_string to timestamp",
      "extractor_type": "copy_input",
      "converters": [
        {
          "type": "date",
          "config": {
            "date_format": "dd/MMM/yyyy:HH:mm:ss Z",
            "time_zone": "Europe/Warsaw",
            "locale": "pl-PL"
          }
        }
      ],
      "order": 4,
      "cursor_strategy": "copy",
      "source_field": "timestamp_string",
      "target_field": "timestamp_string_exttra",
      "extractor_config": {},
      "condition_type": "none",
      "condition_value": ""
    }
2

did you checked your GROK in the Grok Debugger?

http://grokdebug.herokuapp.com

3

Ok I found OUT that when adding converter
I DO NOT set locale AND time_zone.
Then Its Working properly:

{
      "title": "[access_log] timestamp_string to timestamp",
      "extractor_type": "copy_input",
      "converters": [
        {
          "type": "date",
          "config": {
            "date_format": "dd/MMM/YYYY:HH:mm:ss Z",
            "time_zone": "",
            "locale": ""
          }
        }
      ],
      "order": 4,
      "cursor_strategy": "copy",
      "source_field": "timestamp_string",
      "target_field": "timestamp",
      "extractor_config": {},
      "condition_type": "none",
      "condition_value": ""
    }
4

FWIW, when you’re setting the locale to Polish (pl-PL), the converter expects the month name to be Polish, e. g. “październik” instead of “October” or “pazdz” instead of “Oct” respectively.

5

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.