Hi everybody,
I am currently facing the issue that I am not able to configure the multiline parser within filebeat to parse the logs which are generated by the java service wrapper. We are using log4j2 inside hybris to log everything of our application logs in json format, but the java service wrapper will write the starting logs into console.log until the log4j2 logger is started.
I have configured the parser after reading this manual: Manage multiline messages | Filebeat Reference [6.4] | Elastic
The regex should match all lines from the exception (starting with “| at”, “| Caused by”, “| …” and “|” and combine it with the previous message.
I am using the following configuration to parse the logs:
Tail files: true
Enable Multiline: true
Start pattern of a multiline message: ^.(|\s{6,10}at.||\sCaused\sby.||\s{6,10}....||\s?)$
Multiline pattern is negated: false
How are matching lines combined into one event: after
The exceptions in the logs are looking like this:
INFO | jvm 1 | main | 2018/10/16 09:54:23.791 | SEVERE: Exception sending context initialized event to listener instance of class [com.hybris.backoffice.BackofficeContextLoaderListener]
INFO | jvm 1 | main | 2018/10/16 09:54:23.791 | org.springframework.beans.factory.BeanCreationException: Error creating bean with name ‘org.springframework.security.filterChains’: Cannot resolve reference to bean ‘org.springframework.security.web.DefaultSecurityFilterChain#4’ while setting bean property ‘sourceList’ with key [4]; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name ‘org.springframework.security.web.DefaultSecurityFilterChain#4’: Cannot resolve reference to bean ‘org.springframework.security.web.authentication.UsernamePasswordAuthenticationFilter#0’ while setting constructor argument with key [4]; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name ‘org.springframework.security.web.authentication.UsernamePasswordAuthenticationFilter#0’: Cannot resolve reference to bean ‘backofficeAuthenticationSuccessHandler’ while setting bean property ‘authenticationSuccessHandler’; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name ‘resetConfigurationAuthenticationSuccessHandler’: Invocation of init method failed; nested exception is de.hybris.platform.jalo.JaloSystemException: Got JaloSecurityException in getCurrentSession: de.hybris.platform.jalo.security.JaloSecurityException: system not initialized[HY-0][HY–1]
INFO | jvm 1 | main | 2018/10/16 09:54:23.792 | at org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveReference(BeanDefinitionValueResolver.java:359)
INFO | jvm 1 | main | 2018/10/16 09:54:23.792 | at org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveValueIfNecessary(BeanDefinitionValueResolver.java:108)
INFO | jvm 1 | main | 2018/10/16 09:54:23.792 | at org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveManagedList(BeanDefinitionValueResolver.java:382)
INFO | jvm 1 | main | 2018/10/16 09:54:23.792 | at org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveValueIfNecessary(BeanDefinitionValueResolver.java:157)
INFO | jvm 1 | main | 2018/10/16 09:54:23.793 | at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.applyPropertyValues(AbstractAutowireCapableBeanFactory.java:1537)
INFO | jvm 1 | main | 2018/10/16 09:54:23.793 | at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.populateBean(AbstractAutowireCapableBeanFactory.java:1284)
INFO | jvm 1 | main | 2018/10/16 09:54:23.793 | at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:553)
INFO | jvm 1 | main | 2018/10/16 09:54:23.793 | at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:483)
INFO | jvm 1 | main | 2018/10/16 09:54:23.794 | at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:312)
INFO | jvm 1 | main | 2018/10/16 09:54:23.794 | at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:230)
INFO | jvm 1 | main | 2018/10/16 09:54:23.794 | at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:308)
INFO | jvm 1 | main | 2018/10/16 09:54:23.794 | at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:197)
INFO | jvm 1 | main | 2018/10/16 09:54:23.795 | at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:742)
INFO | jvm 1 | main | 2018/10/16 09:54:23.795 | at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:867)
INFO | jvm 1 | main | 2018/10/16 09:54:23.795 | at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:543)
INFO | jvm 1 | main | 2018/10/16 09:54:23.795 | at org.springframework.web.context.ContextLoader.configureAndRefreshWebApplicationContext(ContextLoader.java:443)
INFO | jvm 1 | main | 2018/10/16 09:54:23.795 | at org.springframework.web.context.ContextLoader.initWebApplicationContext(ContextLoader.java:325)
INFO | jvm 1 | main | 2018/10/16 09:54:23.796 | at de.hybris.platform.spring.HybrisContextLoaderListener.doInitWebApplicationContext(HybrisContextLoaderListener.java:225)
INFO | jvm 1 | main | 2018/10/16 09:54:23.796 | at de.hybris.platform.spring.HybrisContextLoaderListener.initWebApplicationContext(HybrisContextLoaderListener.java:199)
INFO | jvm 1 | main | 2018/10/16 09:54:23.796 | at org.springframework.web.context.ContextLoaderListener.contextInitialized(ContextLoaderListener.java:107)
INFO | jvm 1 | main | 2018/10/16 09:54:23.796 | at de.hybris.platform.spring.HybrisContextLoaderListener.contextInitialized(HybrisContextLoaderListener.java:95)
INFO | jvm 1 | main | 2018/10/16 09:54:23.797 | at org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:4745)
INFO | jvm 1 | main | 2018/10/16 09:54:23.797 | at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5207)
INFO | jvm 1 | main | 2018/10/16 09:54:23.797 | at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:150)
INFO | jvm 1 | main | 2018/10/16 09:54:23.797 | at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1419)
INFO | jvm 1 | main | 2018/10/16 09:54:23.798 | at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1409)
INFO | jvm 1 | main | 2018/10/16 09:54:23.798 | at java.util.concurrent.FutureTask.run(FutureTask.java:266)
INFO | jvm 1 | main | 2018/10/16 09:54:23.798 | at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
INFO | jvm 1 | main | 2018/10/16 09:54:23.798 | at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
INFO | jvm 1 | main | 2018/10/16 09:54:23.799 | at java.lang.Thread.run(Thread.java:748)
INFO | jvm 1 | main | 2018/10/16 09:54:23.799 | Caused by: org.springframework.beans.factory.BeanCreationException: Error creating bean with name ‘org.springframework.security.web.DefaultSecurityFilterChain#4’: Cannot resolve reference to bean ‘org.springframework.security.web.authentication.UsernamePasswordAuthenticationFilter#0’ while setting constructor argument with key [4]; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name ‘org.springframework.security.web.authentication.UsernamePasswordAuthenticationFilter#0’: Cannot resolve reference to bean ‘backofficeAuthenticationSuccessHandler’ while setting bean property ‘authenticationSuccessHandler’; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name ‘resetConfigurationAuthenticationSuccessHandler’: Invocation of init method failed; nested exception is de.hybris.platform.jalo.JaloSystemException: Got JaloSecurityException in getCurrentSession: de.hybris.platform.jalo.security.JaloSecurityException: system not initialized[HY-0][HY–1]
The problem is that the exception will not be parsed correctly into one message entry in graylog.
Looking forward to get some feedback.
Thanks and greetings,
ka51m1r