Hi –
New Graylog user here. I’ve run into a problem with sending a resin (java JSP server similar to tomcat) log file into graylog. The log file has multiple “types” of multi-line log messages, which makes using a single filebeat rule difficult (even if I use multiple OR statements in the regexp). I’ve included a sample here showing some single line, and multi line entries.
[00:54:04.866] {http--8000-15$268023904} Serious error occurrred: java.lang.NullPointerException
[00:54:04.866] {http--8000-15$268023904} java.lang.NullPointerException
[00:54:04.866] {http--8000-15$268023904} Error: java.lang.NullPointerException
[00:54:04.866] {http--8000-15$268023904}
[00:55:47.533] {Timer-6} 8/11/17 12:55 AM | SessionCache.Perge - 0ms (75/171) n=30
[00:55:56.359] {DefaultQuartzScheduler_QuartzSchedulerThread} 00:55:56.359 [DefaultQuartzScheduler_QuartzSchedulerThread] DEBUG org.quartz.core.QuartzSchedulerThread - batch acquisition of 0 triggers
[01:11:33.155] {http--8000-10$1894935270} boards.exceptions.RedirectException
[01:11:33.155] {http--8000-10$1894935270} at boards.request.Request.redirect(Request.java:703)
[01:11:33.156] {http--8000-10$1894935270} at com.caucho.util.ThreadPool$Item.runTasks(ThreadPool.java:743)
[01:11:33.156] {http--8000-10$1894935270} at com.caucho.util.ThreadPool$Item.run(ThreadPool.java:662)
[01:11:33.156] {http--8000-10$1894935270} at java.lang.Thread.run(Thread.java:619)
[01:28:02.403] {DefaultQuartzScheduler_QuartzSchedulerThread} 01:28:02.403 [DefaultQuartzScheduler_QuartzSchedulerThread] DEBUG org.quartz.core.QuartzSchedulerThread - batch a
cquisition of 0 triggers
[01:28:07.357] {http--8000-4$1868584300} Error: com.caucho.java.JavaCompileException: /boards/test/realcategorystats.jsp:42: cannot find symbol
[01:28:07.357] {http--8000-4$1868584300} symbol : method getRealStatsURL(java.lang.String)
[01:28:07.357] {http--8000-4$1868584300} location: class boards.util.URL
[01:28:07.357] {http--8000-4$1868584300} out.print(( URL.getRealStatsURL(mr.getParameter(Schema.TEST_ID))));
[01:28:07.357] {http--8000-4$1868584300} ^
[01:28:07.357] {http--8000-4$1868584300} 1 error
[01:28:07.357] {http--8000-4$1868584300} at com.caucho.java.AbstractJavaCompiler.run(AbstractJavaCompiler.java:102)
[01:28:07.357] {http--8000-4$1868584300} at java.lang.Thread.run(Thread.java:619)
[01:28:07.357] {http--8000-4$1868584300}
[01:28:28.923] {DefaultQuartzScheduler_QuartzSchedulerThread} 01:28:28.923 [DefaultQuartzScheduler_QuartzSchedulerThread] DEBUG org.quartz.core.QuartzSchedulerThread - batch acquisition of 0 triggers
[01:38:02.776] {http--8000-20$2105617913} Error: java.lang.NumberFormatException: For input string: "5 and 1=1"
[01:38:02.776] {http--8000-20$2105617913} at java.lang.NumberFormatException.forInputString(NumberFormatException.java:48)
[01:38:02.776] {http--8000-20$2105617913} at com.caucho.server.http.HttpRequest.handleRequest(HttpRequest.java:273)
[01:38:02.776] {http--8000-20$2105617913} at java.lang.Thread.run(Thread.java:619)
[01:38:02.776] {http--8000-20$2105617913}
[01:38:13.803] {http--8000-18$1403411429} Error: java.lang.NumberFormatException: For input string: "5 or (1,2)=(select*from(select name_const(CHAR(111,108,111,108,111,115,104,101,114),1),name_const(CHAR(111,108,111,108,111,115,104,101,114),1))a) -- and 1=1"
[02:43:46.351] {http--8000-19$302001047} No random. Size: 5
Because there are several types of multi-line log entries, I’m trying to figure out the best way to get these into graylog as a single message. All of the multiline messages do share a common string, but the string changes with each message. (4$1868584300 and 15$268023904 from above, for example). Is it possible to group together any messages that share a common string that changes per message grouping?
Alternatively, I’m open to any other ideas that folks may have, since I’m not very well versed in this world. I’m open to doing the manipulation client or server side. Thanks in advance.