Bug: Invalid format: "05/09/17" is malformed at "/09/17"

olivpass · 2017-05-09 09:29:01 UTC

i get the following error if send Cisco Prime logs to graylog:

2017-05-09 11:09:21,213 ERROR: org.graylog2.shared.buffers.processors.DecodingProcessor - Error processing message RawMessage{id=2ff57150-3497-11e7-94c9-0242ac120007, journalOffset=420970440, codec=syslog, payloadSize=170, timestamp=2017-05-09T09:09:20.741Z, remoteAddress=/10.255.0.3:33025}
java.lang.IllegalArgumentException: Invalid format: "05/09/17" is malformed at "/09/17"
        at org.joda.time.format.DateTimeFormatter.parseDateTime(DateTimeFormatter.java:945) ~[graylog.jar:?]
        at org.joda.time.DateTime.parse(DateTime.java:160) ~[graylog.jar:?]
        at org.joda.time.DateTime.parse(DateTime.java:149) ~[graylog.jar:?]
        at org.graylog2.syslog4j.server.impl.event.SyslogServerEvent.parseDate(SyslogServerEvent.java:108) ~[graylog.jar:?]
        at org.graylog2.syslog4j.server.impl.event.SyslogServerEvent.parsePriority(SyslogServerEvent.java:136) ~[graylog.jar:?]
        at org.graylog2.syslog4j.server.impl.event.SyslogServerEvent.parse(SyslogServerEvent.java:152) ~[graylog.jar:?]
        at org.graylog2.syslog4j.server.impl.event.SyslogServerEvent.<init>(SyslogServerEvent.java:50) ~[graylog.jar:?]
        at org.graylog2.inputs.codecs.SyslogCodec.parse(SyslogCodec.java:132) ~[graylog.jar:?]
        at org.graylog2.inputs.codecs.SyslogCodec.decode(SyslogCodec.java:96) ~[graylog.jar:?]
        at org.graylog2.shared.buffers.processors.DecodingProcessor.processMessage(DecodingProcessor.java:146) ~[graylog.jar:?]
        at org.graylog2.shared.buffers.processors.DecodingProcessor.onEvent(DecodingProcessor.java:87) [graylog.jar:?]
        at org.graylog2.shared.buffers.processors.ProcessBufferProcessor.onEvent(ProcessBufferProcessor.java:79) [graylog.jar:?]
        at org.graylog2.shared.buffers.processors.ProcessBufferProcessor.onEvent(ProcessBufferProcessor.java:45) [graylog.jar:?]
        at com.lmax.disruptor.WorkProcessor.run(WorkProcessor.java:143) [graylog.jar:?]
        at com.codahale.metrics.InstrumentedThreadFactory$InstrumentedRunnable.run(InstrumentedThreadFactory.java:66) [graylog.jar:?]
        at java.lang.Thread.run(Thread.java:745) [?:1.8.0_72-internal]

here is a tcpdump of the cisco prime logs:

09:00:13.645127 IP 172.16.137.30.33025 > 172.16.2.94.514: SYSLOG local7.info, length: 102
        0x0000:  0050 56b6 04a0 8c60 4f48 4581 0800 4500  .PV....`OHE...E.
        0x0010:  0082 0000 4000 3f11 57ce ac10 891e ac10  ....@.?.W.......
        0x0020:  025e 8101 0202 006e 0e9d 3c31 3930 3e30  .^.....n..<190>0
        0x0030:  352f 3039 2f31 3720 3131 3a30 303a 3133  5/09/17.11:00:13
        0x0040:  2e36 3434 2049 4e46 4f20 205b 6165 734d  .644.INFO..[aesM
        0x0050:  6f6e 6974 6f72 5d20 5b73 6571 7461 736b  onitor].[seqtask
        0x0060:  6578 6563 7574 6f72 2d36 3034 3338 5d20  executor-60438].
        0x0070:  5374 6172 7469 6e67 2072 656d 6169 6e69  Starting.remaini
        0x0080:  6e67 2064 6576 6963 6573 3a20 3238 200a  ng.devices:.28..
09:00:14.202847 IP 172.16.137.30.33025 > 172.16.2.94.514: SYSLOG local7.info, length: 184
        0x0000:  0050 56b6 04a0 8c60 4f48 4581 0800 4500  .PV....`OHE...E.
        0x0010:  00d4 0000 4000 3f11 577c ac10 891e ac10  ....@.?.W|......
        0x0020:  025e 8101 0202 00c0 218b 3c31 3930 3e30  .^......!.<190>0
        0x0030:  352f 3039 2f31 3720 3131 3a30 303a 3134  5/09/17.11:00:14
        0x0040:  2e32 3032 2049 4e46 4f20 205b 6a6f 626d  .202.INFO..[jobm
        0x0050:  616e 6167 6572 5d20 5b61 6c6c 5472 6967  anager].[allTrig
        0x0060:  6765 7265 644a 6f62 7345 7865 6375 746f  geredJobsExecuto
        0x0070:  722d 3234 5d20 494e 464f 3a20 5b53 6368  r-24].INFO:.[Sch
        0x0080:  6564 756c 6564 2074 6865 206a 6f62 3d41  eduled.the.job=A
        0x0090:  7574 6f6e 6f6d 6f75 7320 4150 204f 7065  utonomous.AP.Ope
        0x00a0:  7261 7469 6f6e 616c 2053 7461 7475 7320  rational.Status.
        0x00b0:  7769 7468 206e 6578 7420 7469 6d65 2061  with.next.time.a
        0x00c0:  7320 5475 6520 4d61 7920 3039 2031 313a  s.Tue.May.09.11:
        0x00d0:  3035 3a31 3420 4345 5354 2032 3031 375d  05:14.CEST.2017]
        0x00e0:  2e0a                                     ..

Version: 2.2.3

thanks

jochen · 2017-05-09 12:25:01 UTC

Try using a Raw/Plaintext input and some extractors for this type of device.

desertjedi85 · 2017-05-09 16:19:24 UTC

I second the Plaintext input. We did this the other day because of this issue with Cisco logs and it cleared it right up.

olivpass · 2017-05-12 09:25:00 UTC

Thank you, this helps to get the logs in. But a check shows the next problem. Have a look:

1	0.000000	172.16.137.30	172.16.2.94	UDP	643	Source port: 20960  Destination port: 515
0000   00 50 56 b6 04 a0 8c 60 4f 48 45 81 08 00 45 00  .PV....`OHE...E.
0010   02 75 00 00 40 00 3f 11 55 db ac 10 89 1e ac 10  .u..@.?.U.......
0020   02 5e 51 e0 02 03 02 61 52 f1 3c 31 39 30 3e 30  .^Q....aR.<190>0
0030   35 2f 30 39 2f 31 37 20 31 36 3a 34 38 3a 31 38  5/09/17 16:48:18
0040   2e 38 37 37 20 49 4e 46 4f 20 20 5b 61 65 73 46  .877 INFO  [aesF
0050   61 75 6c 74 73 5d 20 5b 41 75 74 68 6d 67 72 53  aults] [AuthmgrS
0060   79 73 6c 6f 67 48 61 6e 64 6c 65 72 5d 20 46 61  yslogHandler] Fa
0070   69 6c 75 72 65 20 74 6f 20 64 65 74 65 72 6d 69  ilure to determi
0080   6e 65 20 69 66 49 6e 64 65 78 2e 20 20 46 69 65  ne ifIndex.  Fie
0090   6c 64 20 43 6f 6c 6c 65 63 74 69 6f 6e 3a 20 50  ld Collection: P
00a0   61 63 6b 61 67 65 3a 63 6f 6d 2e 63 69 73 63 6f  ackage:com.cisco
00b0   2e 78 6d 70 2e 64 65 63 61 70 2e 62 61 73 65 0a  .xmp.decap.base.
00c0   4e 61 6d 65 3a 73 73 6e 67 53 74 61 74 65 3b 4b  Name:ssngState;K
00d0   69 6e 64 3a 6f 63 74 65 74 53 74 72 69 6e 67 3b  ind:octetString;
00e0   56 61 6c 75 65 3a 75 70 0a 4e 61 6d 65 3a 73 73  Value:up.Name:ss
00f0   6e 67 4c 69 6e 6b 49 44 3b 4b 69 6e 64 3a 6f 63  ngLinkID;Kind:oc
0100   74 65 74 53 74 72 69 6e 67 3b 56 61 6c 75 65 3a  tetString;Value:
0110   47 69 67 61 62 69 74 45 74 68 65 72 6e 65 74 31  GigabitEthernet1
0120   2f 30 2f 34 30 0a 4e 61 6d 65 3a 50 72 6f 63 65  /0/40.Name:Proce
0130   73 73 49 64 3b 4b 69 6e 64 3a 6f 63 74 65 74 53  ssId;Kind:octetS
0140   74 72 69 6e 67 3b 56 61 6c 75 65 3a 0a 4e 61 6d  tring;Value:.Nam
0150   65 3a 6e 6f 74 69 66 69 63 61 74 69 6f 6e 44 65  e:notificationDe
0160   6c 69 76 65 72 79 4d 65 63 68 61 6e 69 73 6d 3b  liveryMechanism;
0170   4b 69 6e 64 3a 69 6e 74 65 67 65 72 3b 56 61 6c  Kind:integer;Val
0180   75 65 3a 33 0a 4e 61 6d 65 3a 63 69 73 63 6f 46  ue:3.Name:ciscoF
0190   61 63 69 6c 69 74 79 3b 4b 69 6e 64 3a 6f 63 74  acility;Kind:oct
01a0   65 74 53 74 72 69 6e 67 3b 56 61 6c 75 65 3a 4c  etString;Value:L
01b0   49 4e 4b 0a 4e 61 6d 65 3a 63 69 73 63 6f 4d 6e  INK.Name:ciscoMn
01c0   65 6d 6f 6e 69 63 3b 4b 69 6e 64 3a 6f 63 74 65  emonic;Kind:octe
01d0   74 53 74 72 69 6e 67 3b 56 61 6c 75 65 3a 55 50  tString;Value:UP
01e0   44 4f 57 4e 0a 4e 61 6d 65 3a 70 72 6f 63 65 73  DOWN.Name:proces
01f0   73 6f 72 41 74 74 72 43 6f 75 6e 74 3b 4b 69 6e  sorAttrCount;Kin
0200   64 3a 69 6e 74 65 67 65 72 3b 56 61 6c 75 65 3a  d:integer;Value:
0210   32 35 0a 4e 61 6d 65 3a 73 79 73 6c 6f 67 54 79  25.Name:syslogTy
0220   70 65 3b 4b 69 6e 64 3a 69 6e 74 65 67 65 72 3b  pe;Kind:integer;
0230   56 61 6c 75 65 3a 33 31 39 0a 4e 61 6d 65 3a 73  Value:319.Name:s
0240   73 6e 67 49 6e 74 65 72 66 61 63 65 4e 61 6d 65  sngInterfaceName
0250   3b 4b 69 6e 64 3a 6f 63 74 65 74 53 74 72 69 6e  ;Kind:octetStrin
0260   67 3b 56 61 6c 75 65 3a 47 69 67 61 62 69 74 45  g;Value:GigabitE
0270   74 68 65 72 6e 65 74 31 2f 30 2f 34 30 0a 4e 61  thernet1/0/40.Na
0280   2e 2e 2e                                         ...

2	0.000050	172.16.137.30	172.16.2.94	UDP	643	Source port: 20960  Destination port: 515
0000   00 50 56 b6 04 a0 8c 60 4f 48 45 81 08 00 45 00  .PV....`OHE...E.
0010   02 75 00 00 40 00 3f 11 55 db ac 10 89 1e ac 10  .u..@.?.U.......
0020   02 5e 51 e0 02 03 02 61 5c 76 3c 31 39 30 3e 2e  .^Q....a\v<190>.
0030   2e 2e 6d 65 3a 70 72 6f 78 79 49 50 3b 4b 69 6e  ..me:proxyIP;Kin
0040   64 3a 69 70 41 64 64 72 65 73 73 3b 56 61 6c 75  d:ipAddress;Valu
0050   65 3a 31 37 32 2e 31 36 2e 31 33 37 2e 32 32 38  e:172.16.137.228
0060   0a 4e 61 6d 65 3a 73 65 76 65 72 69 74 79 3b 4b  .Name:severity;K
0070   69 6e 64 3a 69 6e 74 65 67 65 72 3b 56 61 6c 75  ind:integer;Valu
0080   65 3a 33 0a 4e 61 6d 65 3a 63 6c 61 73 73 49 64  e:3.Name:classId
0090   3b 4b 69 6e 64 3a 69 6e 74 65 67 65 72 3b 56 61  ;Kind:integer;Va
00a0   6c 75 65 3a 32 0a 4e 61 6d 65 3a 73 79 73 6c 6f  lue:2.Name:syslo
00b0   67 46 6f 72 6d 61 74 54 79 70 65 3b 4b 69 6e 64  gFormatType;Kind
00c0   3a 69 6e 74 65 67 65 72 3b 56 61 6c 75 65 3a 30  :integer;Value:0
00d0   0a 4e 61 6d 65 3a 72 61 77 52 63 76 53 65 63 3b  .Name:rawRcvSec;
00e0   4b 69 6e 64 3a 69 6e 74 65 67 65 72 3b 56 61 6c  Kind:integer;Val
00f0   75 65 3a 31 34 39 34 33 34 31 32 39 38 0a 4e 61  ue:1494341298.Na
0100   6d 65 3a 72 61 77 52 63 76 55 73 65 63 3b 4b 69  me:rawRcvUsec;Ki
0110   6e 64 3a 69 6e 74 65 67 65 72 3b 56 61 6c 75 65  nd:integer;Value
0120   3a 37 36 36 38 32 39 0a 4e 61 6d 65 3a 54 69 6d  :766829.Name:Tim
0130   65 73 74 61 6d 70 3b 4b 69 6e 64 3a 6f 63 74 65  estamp;Kind:octe
0140   74 53 74 72 69 6e 67 3b 56 61 6c 75 65 3a 4d 61  tString;Value:Ma
0150   79 20 20 39 20 31 34 3a 34 38 3a 31 37 2e 37 36  y  9 14:48:17.76
0160   32 0a 4e 61 6d 65 3a 63 61 74 65 67 6f 72 79 3b  2.Name:category;
0170   4b 69 6e 64 3a 6f 63 74 65 74 53 74 72 69 6e 67  Kind:octetString
0180   3b 56 61 6c 75 65 3a 4c 49 4e 4b 0a 4e 61 6d 65  ;Value:LINK.Name
0190   3a 4d 65 73 73 61 67 65 54 79 70 65 3b 4b 69 6e  :MessageType;Kin
01a0   64 3a 6f 63 74 65 74 53 74 72 69 6e 67 3b 56 61  d:octetString;Va
01b0   6c 75 65 3a 4c 49 4e 4b 2d 33 2d 55 50 44 4f 57  lue:LINK-3-UPDOW
01c0   4e 0a 4e 61 6d 65 3a 67 72 6f 75 70 3b 4b 69 6e  N.Name:group;Kin
01d0   64 3a 6f 63 74 65 74 53 74 72 69 6e 67 3b 56 61  d:octetString;Va
01e0   6c 75 65 3a 0a 4e 61 6d 65 3a 4d 65 73 73 61 67  lue:.Name:Messag
01f0   65 54 65 78 74 3b 4b 69 6e 64 3a 6f 63 74 65 74  eText;Kind:octet
0200   53 74 72 69 6e 67 3b 56 61 6c 75 65 3a 49 6e 74  String;Value:Int
0210   65 72 66 61 63 65 20 47 69 67 61 62 69 74 45 74  erface GigabitEt
0220   68 65 72 6e 65 74 31 2f 30 2f 34 30 2c 20 63 68  hernet1/0/40, ch
0230   61 6e 67 65 64 20 73 74 61 74 65 20 74 6f 20 75  anged state to u
0240   70 0a 4e 61 6d 65 3a 50 72 6f 63 65 73 73 4e 61  p.Name:ProcessNa
0250   6d 65 3b 4b 69 6e 64 3a 6f 63 74 65 74 53 74 72  me;Kind:octetStr
0260   69 6e 67 3b 56 61 6c 75 65 3a 0a 4e 61 6d 65 3a  ing;Value:.Name:
0270   4e 6f 64 65 49 64 3b 4b 69 6e 64 3a 6f 63 74 65  NodeId;Kind:octe
0280   2e 2e 2e                                         ...

<190>05/09/17 16:48:18.877 INFO  [aesFaults] [AuthmgrSyslogHandler] Failure to determine ifIndex.  Field Collection: Package:com.cisco.xmp.decap.base
Name:ssngState;Kind:octetString;Value:up
Name:ssngLinkID;Kind:octetString;Value:GigabitEthernet1/0/40
Name:ProcessId;Kind:octetString;Value:
Name:notificationDeliveryMechanism;Kind:integer;Value:3
Name:ciscoFacility;Kind:octetString;Value:LINK
Name:ciscoMnemonic;Kind:octetString;Value:UPDOWN
Name:processorAttrCount;Kind:integer;Value:25
Name:syslogType;Kind:integer;Value:319
Name:ssngInterfaceName;Kind:octetString;Value:GigabitEthernet1/0/40
Na...<190>...me:proxyIP;Kind:ipAddress;Value:172.16.137.228
Name:severity;Kind:integer;Value:3
Name:classId;Kind:integer;Value:2
Name:syslogFormatType;Kind:integer;Value:0
Name:rawRcvSec;Kind:integer;Value:1494341298
Name:rawRcvUsec;Kind:integer;Value:766829
Name:Timestamp;Kind:octetString;Value:May  9 14:48:17.762
Name:category;Kind:octetString;Value:LINK
Name:MessageType;Kind:octetString;Value:LINK-3-UPDOWN
Name:group;Kind:octetString;Value:
Name:MessageText;Kind:octetString;Value:Interface GigabitEthernet1/0/40, changed state to up
Name:ProcessName;Kind:octetString;Value:
Name:NodeId;Kind:octe...

It seems that Cisco Prime sends UDP with maximal length of 644 bytes. In my example the single message are spitted into 8 UDP Packets. The first seven UDP have length of 643 (ending with “2e 2e 2e”) and the last is 644 bytes long and ends with “0a 20 0a”. Do you know something like this? Is a solution for this?

thanks

jochen · 2017-05-12 09:27:02 UTC

Use a TCP-based protocol.

system · 2017-05-26 09:29:54 UTC

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.