kafka(2.3.1)-zookeeper(3.4.12)集群开启kerberos认证部署详解

kafka(2.3.1)-zookeeper(3.4.12)集群开启kerberos认证部署详解

Aug 5, 2026 // 大蚊子
kafka
kafka


前置条件：服务器及安装情况 hostnameprincipal 192.168.2.123db-123kerberos zookeeper、kafkakafka/db-124@DB-123.COM zookeeper、kafkakafka/db-125@DB-123.COM zookeeper、kafkakafka/db-126@DB-123.COM java version 1.8.0_241 zookeeper 版本:zookeeper-3.4.12 安装路径 /usr/local/kafka_zk/zookeeper-3.4.12 kafka 版本：kafka_2.12-2.3.1 安装路径 /usr/local/kafka_zk/kafka_2.12-2.3.1 配置文件路径： /usr/local/kafka_zk/config, 包括： kafka124.keytab kafka与zookeeper所需的principal（两个principal放在了一起） kafka.jaas kafka的jaas文件 ker.pro kafka的consumer与producer所需的认证文件 zookeeper.jaas zookeeper的jaas文件 一、安装kerberos服务 请参考Kerberos认证原理与环境部署 二、创建principal导出keytable文件以及其他相关的配置 三台服务器分别按照前置条件修改hostname与hosts 分别添加san’taisantai服务器的principal并生成ketable 将keytable文件放到三台服务器中对应的服务器的指定的位置/usr/local/kafka_zk/config下 三、部署zookeeper并开启kerberos 分别在三台服务器做以下配置，注意相关地址的修改，以下以db-124为例 3.1、修改zoo.cfg 添加以下安全相关的配置追加下文 authProvider.1=org.apache.zookeeper.server.auth.SASLAuthenticationProvider requireClientAuthScheme=sasl jaasLoginRenew=3600000

#完整配置 tickTime=2000 initLimit=10 syncLimit=5 dataDir=/usr/local/kafka_zk/zookeeper-3.4.12/data clientPort=2181 server.124=db-124:2888:3888 server.125=db-125:2888:3888 server.126=db-126:2888:3888 authProvider.1=org.apache.zookeeper.server.auth.SASLAuthenticationProvider requireClientAuthScheme=sasl jaasLoginRenew=3600000 3.2、在conf目录创建java.env文件，添加以下内容 export JVMFLAGS="- Djava.security.auth.login.config=/usr/local/kafka_zk/config/zookeeper.jaas" 3.3、在/usr/local/kafka_zk/config目录创建zookeeper.jaas文件 Server{ com.sun.security.auth.module.Krb5LoginModule required useKeyTab=true storeKey=true keyTab="/usr/local/kafka_zk/config/kafka124.keytab" principal=“zookeeper/db-124@DB-123.COM ” userTicketCache=false; }; Client{ com.sun.security.auth.module.Krb5LoginModule required useKeyTab=true

storeKey=true keyTab="/usr/local/kafka_zk/config/kafka124.keytab" principal=“zookeeper/db-124@DB-123.COM ” userTicketCache=false; }; 3.4、启动zookeeper服务 ./bin/zkServer.sh start 由上图可知，服务端开启kerberos成功。 启动完毕 zookeeper 可以在 zk 节点上进入 zkCli zkCli.sh -server db-124:2181 四、部署kafka 分别在三台服务器做以下配置，注意相关地址的修改，以下以db-124为例 4.1、配置kafka.jaas文件 KafkaServer { com.sun.security.auth.module.Krb5LoginModule required useKeyTab=true

storeKey=true keyTab="/usr/local/kafka_zk/config/kafka124.keytab" principal=“kafka/db-124@DB-123.COM ”; }; KafkaClient { com.sun.security.auth.module.Krb5LoginModule required useKeyTab=true useKeyTab=true keyTab="/usr/local/kafka_zk/config/kafka124.keytab" principal=“kafka/db-124@DB-123.COM ”; }; Client { com.sun.security.auth.module.Krb5LoginModule required useKeyTab=true storeKey=true keyTab="/usr/local/kafka_zk/config/kafka124.keytab" principal=“kafka/db-124@DB-123.COM ”; }; KakfaServer配置代表了kafka集群中Server彼此间通信所使用的通信认证方式。 Client配置代表了kafka集群中Server与Zookeeper通信时的认证方式。 KafkaClient配置代表了命令行客户端与kafka集群通信时的认证方式。 4.2、修改kafka-run-class.sh文件 在kafka-run-class.sh文件中找到KAKFA_JVM_PERFORMANCE_OPTS,并增加两个JVM参数： -Djava.awt.headless=true -Djava.security.krb5.conf=/etc/krb5.conf - Djava.security.auth.login.config=/usr/local/kafka_zk/config/kafka.jaas 如图所示： 4.3、创建生产者、消费者配置文件 ker.pro

security.protocol=SASL_PLAINTEXT sasl.kerberos.service.name=kafka sasl.mechanism=GSSAPI 4.4、修改Kafka Server配置文件 开启Kerberos服务，需要在配置文件中开启相应的配置项。修改server.properties文件，确保配置项的值如下： authorizer.class.name=kafka.security.auth.SimpleAclAuthorizer security.inter.broker.protocol=SASL_PLAINTEXT sasl.mechanism.inter.broker.protocol=GSSAPI sasl.enabled.mechanisms=GSSAPI sasl.kerberos.service.name=kafka super.users=User:kafka #暂时没发现此配置的作用，配不配都行 allow.everyone.if.no.acl.found=false #完整配置 broker.id=124 listeners=SASL_PLAINTEXT://db-124:9092 advertised.listeners=SASL_PLAINTEXT://db-124:9092 num.network.threads=3 num.io.threads=8 socket.send.buffer.bytes=102400 socket.receive.buffer.bytes=102400 socket.request.max.bytes=104857600 ############################# Log Basics ############################# log.dirs=/usr/local/kafka_zk/kafka_2.12-2.3.1/data/kafka-logs num.partitions=1 num.recovery.threads.per.data.dir=1 ############################# Internal Topic Settings ############################# offsets.topic.replication.factor=1 transaction.state.log.replication.factor=1 transaction.state.log.min.isr=1 ############################# Log Flush Policy ############################# log.retention.hours=168 log.segment.bytes=1073741824 log.retention.check.interval.ms=300000 ############################# Zookeeper ############################# zookeeper.connect=db-124:2181,db-125:2181,db-126:2181

#zookeeper.connect=db-124:2181 zookeeper.connection.timeout.ms=6000 ############################# Group Coordinator Settings ############################# group.initial.rebalance.delay.ms=0 authorizer.class.name=kafka.security.auth.SimpleAclAuthorizer security.inter.broker.protocol=SASL_PLAINTEXT sasl.mechanism.inter.broker.protocol=GSSAPI sasl.enabled.mechanisms=GSSAPI sasl.kerberos.service.name=kafka super.users=User:kafka #暂时没发现此配置的作用，配不配都行 allow.everyone.if.no.acl.found=false #此版本不能配置 #advertised.host.name=db-124 #zookeeper.set.acl=true #zookeeper.sasl.clientconfig=kafka 4.5、启动Kafka Server 启动kafka之前确保Zookeeper集群已启动。启动指令： ./bin/kafka-server-start.sh -daemon start config/server.properties 观察启动日志可以发现，权限认证已成功通过并登陆 后记：相关命令 kadmin.local

getprincs ./bin/kafka-server-start.sh -daemon config/server.properties ./bin/kafka-server-start.sh config/server.properties ./bin/kafka-topics.sh –create –zookeeper db-124:2181,db-125:2181,db-126:2181 – replication-factor 3 –partitions 3 –topic test ./bin/kafka-console-consumer.sh –topic test1 –from-beginning –bootstrap-server db- 124:9092,db-125:9092,db-126:9092 –consumer.config ../config/ker.pro ./bin/kafka-console-producer.sh –broker-list db-124:9092,db-125:9092,db-126:9092 – topic test-zk –producer.config ../config/ker.pro ./bin/kafka-topics.sh –zookeeper db-124:2181,db-125:2181,db-126:2181 –list ./bin/kafka-topics.sh –zookeeper db-124:2181,db-125:2181,db-126:2181 –list ./bin/kafka-acls.sh –authorizer-properties zookeeper.connect=db-124:2181,db-125:2181,db- 126:2181 –list ./bin/kafka-acls.sh –authorizer-properties zookeeper.connect=db-124:2181,db-125:2181,db- 126:2181 –add –allow-principal User:zhangsan –operation Read –operation Write – topic test ./bin/kafka-acls.sh –authorizer-properties zookeeper.connect=db-124:2181,db-125:2181,db- 126:2181 –add –allow-principal User:zhangsan –group test ./bin/zkServer.sh start ./bin/zkServer.sh stop ./bin/zkServer.sh status rm -rf data/version-2/* rm -rf data/kafka-logs/* ./bin/zkCli.sh -server db-125:2181

page3

page4

page6