Kafka Share Groups Change Acknowledgement, Not Business Atomicity

Consumer A acquires record 10 and calls an external provider. The effect finishes. Before A commits the acknowledgement, the acquisition lock expires. Consumer B receives record 10 and calls the provider again.

There is no traditional committed-offset hole to fix. A share group uses per-record acquisition and acknowledgement state, not exclusive partition ownership plus one committed next-offset frontier.

Kafka 4.3.1 makes share groups production-ready at the Apache release level. That does not make every managed service, client, sink, or business effect share the same guarantee. The 4.3 upgrade notes and KafkaShareConsumer 4.3.1 source are the baseline here.

More consumers than partitions changes the ownership model

Traditional consumer groups assign a partition to one member at a time. Share groups can let several members acquire different records from one partition. That allows record-level cooperative work and gives up the traditional partition-owner ordering assumption.

A share group can fit queue-like work where record-level distribution matters more than partition-owner order. It does not turn a Kafka topic into a workflow system with universal scheduling, cancellation, priority, compensation, or provider transactions.

Acquisition locks are temporary authority

When poll() returns a record, Kafka acquires it for one share consumer under a time-limited acquisition lock. The current 4.3.1 Java API describes four acknowledgement types in AcknowledgeType:

The current group setting is share.record.lock.duration.ms. Some KafkaShareConsumer prose still shows the older group.share.record.lock.duration.ms spelling. The generated/current GroupConfig definition is the source used by this guide.

Lock expiry makes an unacknowledged record available again. It cannot cancel a request already accepted by a database or HTTP provider. Use an operation key, conditional mutation, or downstream fence when the effect must reject a second owner.

Local acknowledgement and committed acknowledgement are separate

In explicit acknowledgement mode, the application calls:

consumer.acknowledge(record, AcknowledgeType.ACCEPT);

That call updates local client information. The KafkaShareConsumer contract states that commitSync(), commitAsync(), or a later valid poll() commits pending acknowledgement state to Kafka. A returned local method call is not by itself a durable server acknowledgement.

Explicit mode requires an acknowledgement type for every record returned by the prior poll() before the next poll(). Otherwise, Java throws IllegalStateException; the unacknowledged records can still be acknowledged. This constraint makes a traditional “poll a batch, hand every record to an unbounded executor, poll again” recipe invalid without an explicit acknowledgement/renewal design.

In implicit acknowledgement mode, the next poll() or an explicit commit can acknowledge the previous delivered batch as successfully processed. That is a strong coupling between poll lifecycle and application completion. If the application hands work to background handlers and polls again too early, it can acknowledge records that have not completed under its business contract.

For both modes, a lost commit response creates an observer problem. Server state and client knowledge can differ. Do not translate that uncertainty into a traditional committed offset or assume a second effect-capable attempt is free.

Renewal keeps the poll loop in the design

Kafka 4.3.1 includes AcknowledgeType.RENEW. The KIP-1222 design and current Java API require explicit mode and continued polling. The application acknowledges the record with RENEW on each renewal iteration; when renewal succeeds, the record returns through poll() again.

The current group setting share.renew.acknowledge.enable can permit or deny renewal. A renewal response can fail or disappear. Even a successful renewal extends one bounded lock; it does not guarantee that the member will hold the record forever or that the external effect remains cancellable.

Keep three times separate:

1. acquisition expiry;
2. poll liveness deadline;
3. business-operation deadline.

Extending the first does not reset the third.

Share isolation is group-wide in the current definition

Traditional KafkaConsumer uses the client setting isolation.level. KafkaShareConsumer explicitly rejects that traditional setting in ShareConsumerConfig.

The current group setting is share.isolation.level, with read_uncommitted as the documented default and read_committed supported. Again, older class prose can show group.share.isolation.level; this guide keeps the current generated definition for configuration work and the older literal only so readers can recognize the source discrepancy.

Under read_committed, eligible share records are bounded by the partition's last stable offset. An open transaction can block later records. Under read_uncommitted, delivery is bounded by the high watermark and can include aborted transactional data. The same exclusive LSO/HW rules from the transaction protocol article still matter; the acquisition state above them is different.

Delivery count and rejection do not create a DLQ

The group setting share.delivery.count.limit bounds the current share delivery-count policy. Reaching a limit or rejecting a record changes its share-group state. It does not:

• publish a record to a dead-letter topic;
• preserve the original payload after every retention event;
• record a successful business outcome;
• remove the record from every other group;
• authorize later redrive.

If you need a DLQ, define publication/provenance/progress consistency and redrive identity separately. The Kafka recovery planner models those boundaries for traditional transactional offsets. Do not apply sendOffsetsToTransaction() to share acknowledgements or claim an atomic share-ack-plus-output transaction without a separately verified API.

The six advanced cases have their own reducer and evidence gate

The Kafka Reliability Field Guide includes these source-reviewed scenarios:

1. SG01: two consumers acquire different records from one partition;
2. SG02: effect completes, lock expires, another consumer repeats it;
3. SG03: local accept, committed server acknowledgement, and lost response;
4. SG04: release, reject/archive, and delivery-limit exhaustion;
5. SG05: renewal permitted, denied, or response lost;
6. SG06: group-wide committed isolation and an open transaction.

They use lib/kafka-reliability/share-acquisition.ts, not the traditional completed-offset frontier.

Interactive release has a stronger gate: fixture F08 must compile and run against pinned Apache Kafka 4.3.1 and the Java 4.3.1 share client, covering explicit/implicit acknowledgement, renewal, rejection, delivery count, same-partition distribution, and group isolation. The isolated 2026-09-08 evidence summary records F08 as passed, so the consumer lab exposes the SG selectors. The same manifest records InvalidRecordStateException for RENEW when share.renew.acknowledge.enable=false. It also captured successful ShareAcknowledge responses for ACCEPT and RENEW, dropped each correlated reply, and recorded the client-side UnknownServerException. The accepted record did not return to another member after the first consumer closed. That exception class is one observed Java 4.3.1 path, not a universal mapping for every retry or broker schedule. Managed-service support, correlated failure, acknowledgement-plus-output transactions, and external-effect atomicity remain unestablished.

The gate matters because KIPs record design history while method behavior, current config names, and supported wire paths can change. The accepted KIP-932 does not replace the 4.3.1 API, source, and fixture.

What could go wrong

• Traditional offset arithmetic enters the share wrapper. One committed next offset cannot represent independent acquired records.
• acknowledge() is treated as server commit. The local client update is mistaken for durable acknowledgement.
• An explicit-mode consumer polls before every record has an acknowledgement type. The Java API rejects the poll while background handlers still run.
• Implicit mode polls while async handlers remain unfinished. The previous batch can be acknowledged too early for the application contract.
• Renewal is treated as a lease forever. Group policy, poll liveness, and response uncertainty still bound authority.
• REJECT is labelled “sent to DLQ.” No quarantine publication happened.
• The old group.share.isolation.level prose wins over the current definition. Configuration and source evidence drift apart.
• Share delivery is called exactly-once processing. Lock expiry and acknowledgement loss can repeat attempts; external effects need another identity contract.

Share groups move the Kafka ownership boundary from partitions toward records. They do not move your provider inside Kafka.

Sources and references

• Apache Kafka 4.3 upgrade notes
• KafkaShareConsumer 4.3.1 source
• KafkaShareConsumer 4.3 Java API
• AcknowledgeType 4.3.1
• ShareConsumerConfig 4.3.1
• GroupConfig 4.3.1 share settings
• KIP-932: Queues for Kafka
• KIP-1222: acquisition lock renewal
• KIP-1240: additional share-group configurations