元数据信息包括消息的位移、创建时间戳、压缩类型、字节数等。
首先,创建测试topic,之后生产若干条消息:
bin/kafka-topics.sh --zookeeper localhost:2181 --create --partitions 1 --replication-factor 1 --topic test764-1
bin/kafka-console-producer.sh --broker-list localhost:9092 --topic test764-1
>message1 for t1
>message2 for t1
>message3 for t1
>message4 for t1
>^C
查看 test764-1 的日志存储目录(日志目录位置定义在 config/server.properties 中):
$ ll /tmp/kafka-logs/test764-1-0/
total 16
-rw-r--r-- 1 dys wheel 10485760 Nov 6 19:33 00000000000000000000.index
-rw-r--r-- 1 dys wheel 332 Nov 6 19:34 00000000000000000000.log
-rw-r--r-- 1 dys wheel 10485756 Nov 6 19:33 00000000000000000000.timeindex
-rw-r--r-- 1 dys wheel 8 Nov 6 19:34 leader-epoch-checkpoint
00000000000000000000.log 就是test764-1消息的存储文件,查看元数据:
bin/kafka-run-class.sh kafka.tools.DumpLogSegments --files /tmp/kafka-logs/test764-1-0/00000000000000000000.log --print-data-log
# 输出信息
Dumping /tmp/kafka-logs/test764-1-0/00000000000000000000.log
Starting offset: 0
offset: 0 position: 0 CreateTime: 1541504051699 isvalid: true keysize: -1 valuesize: 15 magic: 2 compresscodec: NONE producerId: -1 producerEpoch: -1 sequence: -1 isTransactional: false headerKeys: [] payload: message1 for t1
offset: 1 position: 83 CreateTime: 1541504061031 isvalid: true keysize: -1 valuesize: 15 magic: 2 compresscodec: NONE producerId: -1 producerEpoch: -1 sequence: -1 isTransactional: false headerKeys: [] payload: message2 for t1
offset: 2 position: 166 CreateTime: 1541504066575 isvalid: true keysize: -1 valuesize: 15 magic: 2 compresscodec: NONE producerId: -1 producerEpoch: -1 sequence: -1 isTransactional: false headerKeys: [] payload: message3 for t1
offset: 3 position: 249 CreateTime: 1541504072007 isvalid: true keysize: -1 valuesize: 15 magic: 2 compresscodec: NONE producerId: -1 producerEpoch: -1 sequence: -1 isTransactional: false headerKeys: [] payload: message4 for t1
--files参数指定了该topic日志段文件的完整路径。从输出可知,当前共有4条消息(console producer生产的4条),每行表示一条消息,详细地列出了该消息的位移信息、物理文件位置、消息长度、压缩类型、CRC码等元数据信息。