Preview: Apache Kafka Log4j2 Support (KIP-653 & KIP-719)

Lee Dongjin · February 26, 2021

Note: Updated in Dec 29th, 2021 for the log4j vulnerability crisis of 2021, like CVE-2021-44228, CVE-2021-45046, CVE-2021-4104, CVE-2021-45105 and CVE-2021-44832. Please use the previews based on 2.8.1 and 3.0.0.

In May 2012, the log4j dev team released log4j 1.2.17 and stopped their support to 1.x releases. However, as of 2.7.0, Apache Kafka is still using deprecated log4j 1.2.17 and is exposed to its security vulnerability, like CVE-2019-17571.

KIP-653: Upgrade log4j to log4j2 and KIP-719: Deprecate logj4-appender are aiming to cope with this problem by upgrading log4j to log4j2. The development of KIP-653 and KIP-719 is working in progress here and here, but for the users who need this feature urgently or hope to test its log4j2 configuration functionality, here is the preview.

DISCLAIMER: For testing purposes only - I can’t guarantee anything.

How to use the Preview

You can download the preview version based on the official Kafka release here:

Each preview is identical to the corresponding official release, except it is cherry-picked with the log4j2 patch. If you are running your own custom build, apply the appropriate patch with git apply at the project root and build.

If you are running Kafka with Docker image (and probably in kubernetes cluster), do the folllowing:

Using wurstmeister/kafka

Change the Docker image to one of the following:

dongjinleekr/kafka is a compatible variant of wurstmeister/kafka, with Oracle Linux + GraalVM Community edition.

Using confluentc/cp-kafka or Confluent operator

dongjinleekr/kafka may work well with the Confluent operator, but I can’t sure. If anyone tried, please mention me to my twitter account, @dongjinleekr.

Using Strimzi operator

Change the download URL and md5 checksum to the following. (see: strimzi/strimzi-kafka-operator#4468)

- version: 2.7.0
  ...
  url: https://github.com/dongjinleekr/kafka/releases/download/2.7.0%2Blog4j2/kafka_2.13-2.7.0+log4j2-1.tgz
  checksum: 99c9092fe9aa430880cf660eacbabdb4
  ...

Using Banzaicloud operator

Change clusterImage to dongjinleekr/kafka:2.13-2.7.0-log4j2-0. (see: banzaicloud/kafka-operator#565)

spec:
  headlessServiceEnabled: true
  zkAddresses:
    - "zookeeper-client.zookeeper:2181"
  propagateLabels: false
  oneBrokerPerNode: false
  clusterImage: "dongjinleekr/kafka:2.13-2.7.0-log4j2-0"

How to Use this feature?

Since this feature is transparently-replaceable, You don’t have to change your log4j configuration. Unless you are using the log4j feature removed in log4j2, it will not cause any problem.

However, if you want to activate log4j2, make the process to pick up the log4j2 configuration by setting KAFKA_LOG4J_OPTS environment variable to "-Dlog4j.configurationFile=file:{kafka.home}/bin/../config/log4j2.properties". You can confirm that it now runs with log4j2 from the initialization log like the following:

Conclusion

If you experience any trouble, please leave a mention to the PR here. Thanks in advance.

Twitter, Facebook