Upgrading Kafka from 0.9.0.1 to 0.10.0.1

[pbrocker]

I was wondering what the timeline was for upgrading kafka from 9 to 10. If that was in the plans for the next graylog minor release (2.3.2) or major release (2.4). There are some bugs (connection issues) in kafka 9 that are causing production issues in graylog.

[jochen]

There are currently no plans to upgrade Kafka.

If you have problems with the currently used version of Kafka in Graylog, please open bug reports for each problem at https://github.com/Graylog2/graylog2-server/issues

[pbrocker]

@jochen… the reason why I am wanting to upgrade kafka is because it handles traffic better when it is has high volume.

[2017-11-30 10:38:40,081] ERROR Processor got uncaught exception. (kafka.network.Processor)
java.lang.ArrayIndexOutOfBoundsException: 18
        at org.apache.kafka.common.protocol.ApiKeys.forId(ApiKeys.java:68)
        at org.apache.kafka.common.requests.AbstractRequest.getRequest(AbstractRequest.java:39)
        at kafka.network.RequestChannel$Request.<init>(RequestChannel.scala:79)
        at kafka.network.Processor$$anonfun$run$11.apply(SocketServer.scala:426)
        at kafka.network.Processor$$anonfun$run$11.apply(SocketServer.scala:421)
        at scala.collection.Iterator$class.foreach(Iterator.scala:742)
        at scala.collection.AbstractIterator.foreach(Iterator.scala:1194)
        at scala.collection.IterableLike$class.foreach(IterableLike.scala:72)
        at scala.collection.AbstractIterable.foreach(Iterable.scala:54)
        at kafka.network.Processor.run(SocketServer.scala:421)
        at java.lang.Thread.run(Thread.java:748)

As well 10 has many fixes for zookeeper timeouts as well.

[jan (Jan Doberstein)]

we will happily review a pr that includes the update.