kafka中的一個topic(Topic-A)每次啟動消費者都重置偏移量到一個固定的位置,而且這個出問題的Topic-A不保存消費者組資訊,用下面命令
./kafka-consumer-groups.sh --bootstrap-server localhost:9092 --list
./kafka-consumer-groups.sh --bootstrap-server localhost:9092 --describe --group group-5
當我程式開啟時查到group資訊為:
Consumer group 'group-5' has no active members.
關閉之后查看消費者組list和查看指定group的資訊都查不到,回傳
Error: Consumer group 'group-5' does not exist.
下面是啟動程式時Topic-A輸出的資訊:
o.a.k.c.c.i.ConsumerCoordinator - [Consumer clientId=consumer-1, groupId=group-5] Revoking previously assigned partitions []
o.a.k.c.c.i.AbstractCoordinator - [Consumer clientId=consumer-1, groupId=group-5] (Re-)joining group
o.a.k.c.c.i.AbstractCoordinator - [Consumer clientId=consumer-1, groupId=group-5] Successfully joined group with generation 1
o.a.k.c.c.i.ConsumerCoordinator - [Consumer clientId=consumer-1, groupId=group-5] Setting newly assigned partitions [A1-0]
o.a.k.c.consumer.internals.Fetcher - [Consumer clientId=consumer-1, groupId=group-5] Resetting offset for partition A1-0 to offset 25143.
最后一行資訊:
參考 o.a.k.c.consumer.internals.Fetcher - [Consumer clientId=consumer-1, groupId=group-5] Resetting offset for partition A1-0 to offset 25143.
每次啟動Topic-A都會出現這一行資訊,把offset重置到25143。
如果就算我運行一段時間,消費到25200,然后關閉程式,下次啟動又自動給我重置到25143。
多次啟動都是用的group-5.
啟動程式后用groups.list命令可以看到group-5,用describe 命令查group-5顯示Consumer group 'group-5' has no active members.
關閉程式后用groups.list看不到group-5,,用describe 回傳Error: Consumer group 'group-5' does not exist.
下面是另一個正常的Topic啟動輸出的資訊,沒有上面那條Resetting offset for partition xxx to offset xxx的資訊,每次啟動都可以從上次停止位置繼續消費。
org.apache.kafka.clients.Metadata - Cluster ID: iQU30Fo1TViA2rkH9cxVYQ
o.a.k.c.c.i.AbstractCoordinator - [Consumer clientId=consumer-3, groupId=wallet-1] Discovered group coordinator localhost:9092 (id: 2147483646 rack: null)
o.a.k.c.c.i.ConsumerCoordinator - [Consumer clientId=consumer-3, groupId=wallet-1] Revoking previously assigned partitions []
o.a.k.c.c.i.AbstractCoordinator - [Consumer clientId=consumer-3, groupId=wallet-1] (Re-)joining group
o.a.k.c.c.i.AbstractCoordinator - [Consumer clientId=consumer-3, groupId=wallet-1] Successfully joined group with generation 13
o.a.k.c.c.i.ConsumerCoordinator - [Consumer clientId=consumer-3, groupId=wallet-1] Setting newly assigned partitions [blockaddresscomplete3-0]
請問我該怎么設定topic-A,讓他在每次啟動時繼續從上次停止位置消費訊息,而不是每次啟動都重置到固定的offset。
uj5u.com熱心網友回復:
參考 樓主 deantzzz 的回復: kafka中的一個topic(Topic-A)每次啟動消費者都重置偏移量到一個固定的位置,而且這個出問題的Topic-A不保存消費者組資訊,用下面命令
./kafka-consumer-groups.sh --bootstrap-server localhost:9092 --list
./kafka-consumer-groups.sh --bootstrap-server localhost:9092 --describe --group group-5
當我程式開啟時查到group資訊為:
Consumer group 'group-5' has no active members.
關閉之后查看消費者組list和查看指定group的資訊都查不到,回傳
Error: Consumer group 'group-5' does not exist.
下面是啟動程式時Topic-A輸出的資訊:
o.a.k.c.c.i.ConsumerCoordinator - [Consumer clientId=consumer-1, groupId=group-5] Revoking previously assigned partitions []
o.a.k.c.c.i.AbstractCoordinator - [Consumer clientId=consumer-1, groupId=group-5] (Re-)joining group
o.a.k.c.c.i.AbstractCoordinator - [Consumer clientId=consumer-1, groupId=group-5] Successfully joined group with generation 1
o.a.k.c.c.i.ConsumerCoordinator - [Consumer clientId=consumer-1, groupId=group-5] Setting newly assigned partitions [A1-0]
o.a.k.c.consumer.internals.Fetcher - [Consumer clientId=consumer-1, groupId=group-5] Resetting offset for partition A1-0 to offset 25143.
最后一行資訊:
參考 o.a.k.c.consumer.internals.Fetcher - [Consumer clientId=consumer-1, groupId=group-5] Resetting offset for partition A1-0 to offset 25143.
每次啟動Topic-A都會出現這一行資訊,把offset重置到25143。
如果就算我運行一段時間,消費到25200,然后關閉程式,下次啟動又自動給我重置到25143。
多次啟動都是用的group-5.
啟動程式后用groups.list命令可以看到group-5,用describe 命令查group-5顯示Consumer group 'group-5' has no active members.
關閉程式后用groups.list看不到group-5,,用describe 回傳Error: Consumer group 'group-5' does not exist.
下面是另一個正常的Topic啟動輸出的資訊,沒有上面那條Resetting offset for partition xxx to offset xxx的資訊,每次啟動都可以從上次停止位置繼續消費。
org.apache.kafka.clients.Metadata - Cluster ID: iQU30Fo1TViA2rkH9cxVYQ
o.a.k.c.c.i.AbstractCoordinator - [Consumer clientId=consumer-3, groupId=wallet-1] Discovered group coordinator localhost:9092 (id: 2147483646 rack: null)
o.a.k.c.c.i.ConsumerCoordinator - [Consumer clientId=consumer-3, groupId=wallet-1] Revoking previously assigned partitions []
o.a.k.c.c.i.AbstractCoordinator - [Consumer clientId=consumer-3, groupId=wallet-1] (Re-)joining group
o.a.k.c.c.i.AbstractCoordinator - [Consumer clientId=consumer-3, groupId=wallet-1] Successfully joined group with generation 13
o.a.k.c.c.i.ConsumerCoordinator - [Consumer clientId=consumer-3, groupId=wallet-1] Setting newly assigned partitions [blockaddresscomplete3-0]
請問我該怎么設定topic-A,讓他在每次啟動時繼續從上次停止位置消費訊息,而不是每次啟動都重置到固定的offset。 解決了嗎
轉載請註明出處,本文鏈接:https://www.uj5u.com/qiye/7190.html
標籤:中間件
上一篇:心跳線原理總結:
下一篇:layui table中select顯示優化