Office 365 input keeps stopping

Hi @gsmith,

Yes, I’m using the “Office 365 Log events” input that is built into Graylog. Here is my current configuration :

Hre’s the sudo journalctl -xeu graylog-server command output :

juil. 18 08:37:10 machine_hostname graylog-server[22172]: WARNING: Illegal reflective access by retrofit2.Platform (file:/usr/share/graylog-serjuil. 18 08:37:10 machine_hostname graylog-server[22172]: WARNING: Please consider reporting this to the maintainers of retrofit2.Platform
juil. 18 08:37:10 machine_hostname graylog-server[22172]: WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective ajuil. 18 08:37:10 machine_hostname graylog-server[22172]: WARNING: All illegal access operations will be denied in a future release
juil. 18 08:37:21 machine_hostname systemd[1]: Stopping Graylog server...
-- Subject: L'unité (unit) graylog-server.service a commencé à s'arrêter
-- Defined-By: systemd
-- Support: https://www.debian.org/support
--
-- L'unité (unit) graylog-server.service a commencé à s'arrêter.
juil. 18 08:37:21 machine_hostname systemd[1]: graylog-server.service: Main process exited, code=killed, status=15/TERM
-- Subject: Unit process exited
-- Defined-By: systemd
-- Support: https://www.debian.org/support
--
-- An ExecStart= process belonging to unit graylog-server.service has exited.
--
-- The process' exit code is 'killed' and its exit status is 15.
juil. 18 08:37:28 machine_hostname systemd[1]: graylog-server.service: Succeeded.
-- Subject: Unit succeeded
-- Defined-By: systemd
-- Support: https://www.debian.org/support
--
-- The unit graylog-server.service has successfully entered the 'dead' state.
juil. 18 08:37:28 machine_hostname systemd[1]: Stopped Graylog server.
-- Subject: L'unité (unit) graylog-server.service a terminé son arrêt
-- Defined-By: systemd
-- Support: https://www.debian.org/support
--
-- L'unité (unit) graylog-server.service a terminé son arrêt.
juil. 18 08:37:28 machine_hostname systemd[1]: Started Graylog server.
-- Subject: L'unité (unit) graylog-server.service a terminé son démarrage
-- Defined-By: systemd
-- Support: https://www.debian.org/support
--
-- L'unité (unit) graylog-server.service a terminé son démarrage, avec le résultat done.
juil. 18 08:37:28 machine_hostname graylog-server[22441]: OpenJDK 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in versionjuil. 18 08:37:28 machine_hostname graylog-server[22441]: WARNING: sun.reflect.Reflection.getCallerClass is not supported. This will impact perjuil. 18 08:37:30 machine_hostname graylog-server[22441]: WARNING: An illegal reflective access operation has occurred
juil. 18 08:37:30 machine_hostname graylog-server[22441]: WARNING: Illegal reflective access by retrofit2.Platform (file:/usr/share/graylog-serjuil. 18 08:37:30 machine_hostname graylog-server[22441]: WARNING: Please consider reporting this to the maintainers of retrofit2.Platform
juil. 18 08:37:30 machine_hostname graylog-server[22441]: WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective ajuil. 18 08:37:30 machine_hostname graylog-server[22441]: WARNING: All illegal access operations will be denied in a future release
juil. 18 08:37:51 machine_hostname graylog-server[22441]: SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
juil. 18 08:37:51 machine_hostname graylog-server[22441]: SLF4J: Defaulting to no-operation (NOP) logger implementation
juil. 18 08:37:51 machine_hostname graylog-server[22441]: SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
lines 491-536/536 (END)
juil. 18 08:37:10 machine_hostname graylog-server[22172]: WARNING: Illegal reflective access by retrofit2.Platform (file:/usr/sjuil. 18 08:37:10 machine_hostname graylog-server[22172]: WARNING: Please consider reporting this to the maintainers of retrofijuil. 18 08:37:10 machine_hostname graylog-server[22172]: WARNING: Use --illegal-access=warn to enable warnings of further illejuil. 18 08:37:10 machine_hostname graylog-server[22172]: WARNING: All illegal access operations will be denied in a future reljuil. 18 08:37:21 machine_hostname systemd[1]: Stopping Graylog server...
-- Subject: L'unité (unit) graylog-server.service a commencé à s'arrêter
-- Defined-By: systemd
-- Support: https://www.debian.org/support
--
-- L'unité (unit) graylog-server.service a commencé à s'arrêter.
juil. 18 08:37:21 machine_hostname systemd[1]: graylog-server.service: Main process exited, code=killed, status=15/TERM
-- Subject: Unit process exited
-- Defined-By: systemd
-- Support: https://www.debian.org/support
--
-- An ExecStart= process belonging to unit graylog-server.service has exited.
--
-- The process' exit code is 'killed' and its exit status is 15.
juil. 18 08:37:28 machine_hostname systemd[1]: graylog-server.service: Succeeded.
-- Subject: Unit succeeded
-- Defined-By: systemd
-- Support: https://www.debian.org/support
--
-- The unit graylog-server.service has successfully entered the 'dead' state.
juil. 18 08:37:28 machine_hostname systemd[1]: Stopped Graylog server.
-- Subject: L'unité (unit) graylog-server.service a terminé son arrêt
-- Defined-By: systemd
-- Support: https://www.debian.org/support
--
lines 491-519/536 95%
juil. 18 08:37:10 machine_hostname graylog-server[22172]: WARNING: Illegal reflective access by retrofit2.Platform (file:/usr/share/graylog-server/graylog.jar) to constructor java.lang.invoke.MethodHandles$Lookup(j
juil. 18 08:37:10 machine_hostname graylog-server[22172]: WARNING: Please consider reporting this to the maintainers of retrofit2.Platform
juil. 18 08:37:10 machine_hostname graylog-server[22172]: WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
juil. 18 08:37:10 machine_hostname graylog-server[22172]: WARNING: All illegal access operations will be denied in a future release
juil. 18 08:37:21 machine_hostname systemd[1]: Stopping Graylog server...
-- Subject: L'unité (unit) graylog-server.service a commencé à s'arrêter
-- Defined-By: systemd
-- Support: https://www.debian.org/support
--
-- L'unité (unit) graylog-server.service a commencé à s'arrêter.
juil. 18 08:37:21 machine_hostname systemd[1]: graylog-server.service: Main process exited, code=killed, status=15/TERM
-- Subject: Unit process exited
-- Defined-By: systemd
-- Support: https://www.debian.org/support
--
-- An ExecStart= process belonging to unit graylog-server.service has exited.
--
-- The process' exit code is 'killed' and its exit status is 15.
juil. 18 08:37:28 machine_hostname systemd[1]: graylog-server.service: Succeeded.
-- Subject: Unit succeeded
-- Defined-By: systemd
-- Support: https://www.debian.org/support
--
-- The unit graylog-server.service has successfully entered the 'dead' state.
juil. 18 08:37:28 machine_hostname systemd[1]: Stopped Graylog server.
-- Subject: L'unité (unit) graylog-server.service a terminé son arrêt
-- Defined-By: systemd
-- Support: https://www.debian.org/support
--
-- L'unité (unit) graylog-server.service a terminé son arrêt.
juil. 18 08:37:28 machine_hostname systemd[1]: Started Graylog server.
-- Subject: L'unité (unit) graylog-server.service a terminé son démarrage
-- Defined-By: systemd
-- Support: https://www.debian.org/support
--
-- L'unité (unit) graylog-server.service a terminé son démarrage, avec le résultat done.
juil. 18 08:37:28 machine_hostname graylog-server[22441]: OpenJDK 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
juil. 18 08:37:28 machine_hostname graylog-server[22441]: WARNING: sun.reflect.Reflection.getCallerClass is not supported. This will impact performance.
juil. 18 08:37:30 machine_hostname graylog-server[22441]: WARNING: An illegal reflective access operation has occurred
juil. 18 08:37:30 machine_hostname graylog-server[22441]: WARNING: Illegal reflective access by retrofit2.Platform (file:/usr/share/graylog-server/graylog.jar) to constructor java.lang.invoke.MethodHandles$Lookup(j
juil. 18 08:37:30 machine_hostname graylog-server[22441]: WARNING: Please consider reporting this to the maintainers of retrofit2.Platform
juil. 18 08:37:30 machine_hostname graylog-server[22441]: WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
juil. 18 08:37:30 machine_hostname graylog-server[22441]: WARNING: All illegal access operations will be denied in a future release
juil. 18 08:37:51 machine_hostname graylog-server[22441]: SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
juil. 18 08:37:51 machine_hostname graylog-server[22441]: SLF4J: Defaulting to no-operation (NOP) logger implementation
juil. 18 08:37:51 machine_hostname graylog-server[22441]: SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.

Here is a part that comes around several times il the 30 last days…

2022-07-11T07:19:23.370+02:00 INFO  [EventProcessorExecutionJob] Event processor <62b07ae129fd030c6f0b5181> is catching up on old data. Combining 60 search windows with catchUpWindowSize=3600000ms: from=2022-07-11T04:02:31.684Z to=2022-07-11T05:02:31.684Z
2022-07-11T07:19:23.506+02:00 INFO  [EventProcessorExecutionJob] Event processor <62b1863629fd030c6f0dc0c3> is catching up on old data. Combining 60 search windows with catchUpWindowSize=3600000ms: from=2022-07-11T04:01:56.491Z to=2022-07-11T05:01:56.491Z
2022-07-11T08:01:38.365+02:00 INFO  [O365PollerTask] Error fetching manifest for Content Type [AZURE_ACTIVE_DIRECTORY]: [java.lang.RuntimeException: Unable to communicate with O365 servers: Retrying failed to complete successfully after 40 attempts.]
2022-07-11T08:01:38.365+02:00 INFO  [InputStateListener] Input [Office 365 Log Events/626639d4f50e1e1453fdf9de] is now STOPPING
2022-07-11T08:01:38.373+02:00 INFO  [InputStateListener] Input [Office 365 Log Events/626639d4f50e1e1453fdf9de] is now STOPPED
2022-07-11T08:01:38.374+02:00 INFO  [InputStateListener] Input [Office 365 Log Events/626639d4f50e1e1453fdf9de] is now TERMINATED
2022-07-11T08:01:41.299+02:00 INFO  [InputStateListener] Input [Office 365 Log Events/626639d4f50e1e1453fdf9de] is now STARTING
2022-07-11T08:01:41.299+02:00 INFO  [InputStateListener] Input [Office 365 Log Events/626639d4f50e1e1453fdf9de] is now RUNNING
2022-07-11T08:01:42.977+02:00 INFO  [O365PollerTask] Error fetching manifest for Content Type [SHAREPOINT]: [java.lang.RuntimeException: Unable to communicate with O365 servers: Retrying failed to complete successfully after 1 attempts.]
2022-07-11T08:01:43.141+02:00 INFO  [O365PollerTask] Error fetching manifest for Content Type [EXCHANGE]: [java.lang.RuntimeException: Unable to communicate with O365 servers: Retrying failed to complete successfully after 1 attempts.]
2022-07-11T08:01:44.325+02:00 INFO  [O365PollerTask] Error fetching manifest for Content Type [GENERAL]: [java.lang.RuntimeException: Unable to communicate with O365 servers: Retrying failed to complete successfully after 1 attempts.]
2022-07-11T08:01:44.663+02:00 INFO  [O365PollerTask] Error fetching manifest for Content Type [DLP_ALL]: [java.lang.RuntimeException: Unable to communicate with O365 servers: Retrying failed to complete successfully after 1 attempts.]
2022-07-11T08:01:44.663+02:00 ERROR [Office365Input] The input has encountered errors while fetching data from Microsoft's O365 servers :: All attempts to fetch logs from 0365 failed
java.lang.RuntimeException: All attempts to fetch logs from 0365 failed
	at org.graylog.enterprise.integrations.office365.O365PollerTask.doRun(O365PollerTask.java:131) ~[graylog-plugin-enterprise-integrations-4.3.2.jar:?]
	at org.graylog.enterprise.integrations.office365.O365PollerTask.run(O365PollerTask.java:51) [graylog-plugin-enterprise-integrations-4.3.2.jar:?]
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) [?:?]
	at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:305) [?:?]
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:305) [?:?]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [?:?]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [?:?]
	at java.lang.Thread.run(Thread.java:829) [?:?]
2022-07-11T10:01:21.489+02:00 INFO  [ReportPeriodical] Starting report generation for: Graylog
2022-07-11T10:01:21.492+02:00 INFO  [SystemJobManager] Submitted SystemJob <a6db0a30-00ef-11ed-a881-000c2946405a> [org.graylog.plugins.report.scheduler.ReportRenderSystemJob]
2022-07-11T10:01:21.493+02:00 INFO  [ReportPeriodical] Setting report scheduler configuration: 0 0 10 * * ? *
2022-07-11T10:01:21.493+02:00 INFO  [ReportPeriodical] Next report generation will be at: 2022-07-12T10:00+02:00[Europe/Paris]
2022-07-11T10:01:28.342+02:00 WARN  [RemoteBrowserService] Chrome sandboxing is currently disabled. Please validate your security settings!
2022-07-11T10:01:29.287+02:00 INFO  [SystemJobManager] SystemJob <a6db0a30-00ef-11ed-a881-000c2946405a> [org.graylog.plugins.report.scheduler.ReportRenderSystemJob] finished in 7794ms.
2022-07-11T10:52:09.084+02:00 INFO  [connection] Opened connection [connectionId{localValue:26, serverValue:13}] to localhost:27017

I can’t send you the entire file here, because it’s like 5Mb but I can email you if you want it.

Many thanks for your help, because I can’t figure out what the hell is going on…

Hello,

Thank you , I was able to see what’s going on. Couple things I noticed and to be honest some of this is out of my scope for troubleshooting. I have picked some issue that happened and perhaps it may help further our troubleshooting.

Not sure, but I did find this link about that error.

"SLF4J: Failed to load class “org.slf4j.impl.StaticLoggerBinder”.

This error from my understanding shows that it tried to connect 40 time and failed. Its either a configuration issue but not sure if it on Office side or Graylog side. Or a network issue ( i.e. ports, Selinux, etc…)

[O365PollerTask] Error fetching manifest for Content Type [AZURE_ACTIVE_DIRECTORY]: [java.lang.RuntimeException: Unable to communicate with O365 servers: Retrying failed to complete successfully after 40 attempts.]

After failed connection attempts it just failed to get logs from Microsoft’s O365 servers.

ERROR [Office365Input] The input has encountered errors while fetching data from Microsoft’s O365 servers :: All attempts to fetch logs from 0365 failed

Last, I seen this Warning, Is this Docker Environment? If so are you running Docker-Compose? If not is this Enterprise Version?

WARN [RemoteBrowserService] Chrome sandboxing is currently disabled. Please validate your security settings!

EDIT:
After looking over the Error again, I did a quick search.

All attempts to fetch logs from 0365 failed
at org.graylog.enterprise.integrations.office365.O365PollerTask.doRun(O365PollerTask.java:131) ~[graylog-plugin-enterprise-integrations-4.3.2.jar:?]

By chance do you have Enterprise/Operation license installed? If so have you contacted Customer Support?

Hi @gsmith,
Thanks for your fast reply.

I will try to understand what all these logs means. If not, maybe a software reinstallation will be the solution but I hope to not have to get there.

No, it’s a virtual machine running on esxi cluster.

Well, I have an Enterprise license installed, but it’s the free one with a 2gb/day limit. I don’t really know if I can reach out to the customer support with that… But I can try ! :slight_smile:

Hello,

To be honest, I don’t think you need a fresh installation. Were missing something. What I do know about INPUTs stopping, like TLS/TCP INPUT for example. If the certificates are out dated or incorrect the input might stop, Or if there is a network port confliction, meaning you have an other port opened that the INPUT is also using it will stop.

I understand, You can also post on GitHub the dev’s might be able to shed some light on this issue.

Or you can post in Graylog Discord channel. perhaps one of these places might be able to help. To be honest I running out of ideas.

1 Like

Hi @gsmith,

Well, thanks for the help you provided and the time you spent on this issue !

I’ll try these ideas, GitHub and Discord.

1 Like

I wonder if it’s an issue in the OpenJDK version you have? There are depreciation warnings and I think I saw you are on version 9 where OpenJDK 17 is supported (Listed here)

Good catch :+1: defiantly over looked that.

Hi @tmacgbay,

Thanks for this hint ! :star_struck:

I’ve installed Java 17 from the official website, and uninstalled the openjdk 11 I was previously running.

I keep you updated on the behaviours of Graylog in the next few days ! :wink:

1 Like

It works great again since yesterday ! :yum:

If all is still ok on wednesday, I’ll mark the @tmacgbay post as a solution.

2 Likes

Well, sorry guys but it didn’t work out for me. I still have this issuen, even after the Java 17 environnent was installed. :confused:

Have you tries changing your polling interval?