Kafka reliability / Recovery planner

Kafka Recovery, Retry & DLQ Planner

What evidence and consistency boundary must exist before you retry, quarantine, or replay these records?

Baseline: apache-java-4.3.1 · source 26b251a451ce

This deterministic teaching model runs in your browser and never connects to Kafka. It accepts only bounded synthetic controls. Curated scenario and variant IDs may appear in a shareable URL; custom local values, payloads, brokers, credentials, and production identifiers never do. Exports include only what you request.

Deterministic teaching model · apache-java-4.3.1

R03: One Kafka transaction can join DLQ and source progress

Which participants share the decision?

Scenario

Curated variant

Change bounded local inputs

Changing these controls creates a custom local example. The share link keeps only the original curated preset; exports can include your bounded local values.

Recovery flow

The planner never executes the selected operation.

DLQ/progress order

Only one same-Kafka transaction can join those Kafka participants.

Required ordering

Retry topics do not preserve partition processing order by default.

Source retention (days)

Independent from group checkpoints and sink protection.

Sink protection expiry day

A source record can outlive this boundary.

Replay day

At/after expiry is tested explicitly.

Client total attempts

Includes the first attempt.

Application total attempts

Conditional nested upper bound only.

Framework total attempts

Do not assume all layers multiply independently in production.

Absolute deadline (ms)

0–600,000 virtual milliseconds.

Current outcome · step 0 of 5

The failed source record remains at offset 10; no quarantine publication, progress change, or recovery authority exists yet.

curated preset

Disposition consistency

Source stored next offset: 10
Quarantine publications / visible: 0 / no
Transaction decision / response: none / none
Input restored: no

Retry and ordering

Conditional attempt upper bound: 8
Attempts inside deadline: 0; 0 ms of 30000 ms.
Ordering contract: partition-order
Later record completed first: no

Evidence and authority

effect: unknown
identity: unknown
authority: unknown
dependency: unknown
schema: unknown
sourceRange: unknown
Protection current: yes
Suggested disposition: needs reconciliation

Runbook

Canary: not recorded
Stop / verification items: 0 / 0
Manifest: not requested
Cross-cluster profile: same-cluster

Invariant results

Every normally skipped failed record has a durable disposition
holds in model
Source progress has not crossed the failed record without visible quarantine.
Transactional DLQ and source progress share one decision
holds in model
A committed recovery transaction exposes both Kafka participants together in the bounded model.
Retry flow preserves the declared ordering contract
holds in model
No selected action contradicts the declared order.
Reported evidence establishes automatic replay permission
violated in model
Missing, negative, or expired evidence blocks automatic effect-capable replay.

Ordered trace

The state table shows model truth. Each observer result says what the current caller can establish at that step.

Ordered deterministic actions through step 0. Print and exports contain the complete bounded trace.
Step	Actor	Action	Observer result
10 ms	operator	Begin ordinary Kafka transaction	An ordinary same-cluster Kafka transaction begins. No source progress or quarantine record is committed yet.
21 ms	operator	Append synthetic DLQ record	The synthetic quarantine record is acknowledged under the selected Kafka publication contract.
32 ms	operator	Attach source next offset 11	Source next offset 11 is attached to the pending Kafka transaction.
43 ms	process	Crash before commit request	The crash occurs before any commit request. The bounded ordinary transaction resolves as abort; source progress remains unchanged.
54 ms	operator	Restore input position	Input position is restored to stored next offset 10.

Evidence and limits

Claims

Primary sources

Assumptions and known limits

All ranges, identities, and evidence answers are synthetic and bounded.
The planner never contacts Kafka or emits executable reset, delete, or redrive commands.

Reported evidence is not verified evidence or replay authorization.

Default synthetic example loaded.

The planner can block a replay; it cannot authorize one

Commit-first and publish-first traces expose missing or duplicate quarantine. A same-cluster Kafka transaction can join a DLQ record and source offsets, but redrive effects still need identity, retention, schema, authority, dependency, canary, stop, and verification evidence. All answers remain reported for a synthetic example rather than verified against a real system.

Assumptions and limits

Feature levels, protocol versions, client settings, and binary versions remain separate facts.
Replication examples use stable committed ISR membership between declared transitions.
The ordinary transaction profile disables transaction.two.phase.commit.enable.
Browser traces are deterministic teaching models, not broker observations or production recommendations.