Kafka: Offsets, Commits, and Why Your Consumer Got Stuck

You have an order service, a payment service, an inventory service, and an email service. When a user places an order, all four services need to know about it.

You could have the order service call each one directly. Four HTTP requests every time someone clicks "Buy Now." But what happens when the payment service is down? Or slow? The order service blocks, times out, and the user sees an error. You add retry logic. Now you have duplicate payments. You add a database to track which services you've called. Now you have distributed state management. This is getting complicated.

What if instead of calling services directly, the order service just wrote "order created" to a log? The other services could read from that log at their own pace. If payment is down, the order still succeeds and payment catches up later. If email is slow, it doesn't block anyone else.

This is the idea behind Kafka. A durable, distributed log that decouples producers from consumers.

Kafka architecture: brokers, topics, and partitions

Before we dive into offsets and commits, let's understand the physical architecture.

A broker is a server that stores and serves Kafka data. A Kafka cluster typically has multiple brokers for fault tolerance and scalability.

A topic is a category or feed name, like "orders" or "payments." Topics are split into partitions for parallelism. Each partition is an independent, ordered log stored on one or more brokers.

In the visualization above, topic "orders" has 6 partitions distributed across 3 brokers. Partition 0 lives on Broker 1, Partition 1 on Broker 2, and so on. Each partition can be replicated to multiple brokers (we'll see replication later).

What's in a Kafka message?

Every Kafka message has several fields. Adjust the inputs to see how messages are structured.

A message contains:

Key (optional): Used for partitioning. Messages with the same key always go to the same partition. If null, Kafka distributes messages round-robin.
Value: The actual payload (your data).
Partition: Calculated from the key using hash(key) % num_partitions. Same key always maps to the same partition, preserving ordering per key.
Offset: A sequential number assigned by the broker when the message is written. This is the message's position in the partition.

The partition is calculated by the producer based on the key using a hash function. The offset is assigned by the broker when it writes the message to the partition log. These two numbers define exactly where a message lives: partition 2, offset 1234.

The simplest version: an append-only log

Kafka is an append-only log. Producers write messages to the end. Consumers read messages starting from wherever they want.

Walk through the steps. A producer writes messages to the log (a Kafka topic). Each message gets a sequential number called an offset. Message 0, message 1, message 2. A consumer reads messages starting from a specific offset.

The producer doesn't care about offsets. It just appends messages to the end. The consumer tracks its own position: "I read up to offset 3, so next time I'll start at 3."

That position is everything. Too far back and you reprocess old data. Too far forward and you skip messages. Crash at the wrong time and you're stuck in a loop.

Multiple readers need different positions

Back to our example. The payment service needs to process every order exactly once. The analytics service wants to reprocess old orders to recalculate metrics. Both read from the same "orders" topic, but they need different positions in the log.

If they shared a single offset pointer, they'd interfere with each other. When analytics seeks back to reprocess old data, it would break payment processing.

The solution: consumer groups. Each group maintains its own independent set of offsets.

Poll messages from both consumer groups. Notice they're independent. The analytics group can reprocess old data while the billing group continues from where it left off.

This is how Kafka enables multiple applications to consume the same stream. Each consumer group maintains separate offsets. Same topic, but different positions for different purposes.

Offsets need to survive crashes

So far we've said "the consumer tracks its position." But where exactly is that position stored?

If it's in memory, the position is lost when the consumer crashes. The consumer restarts and has no idea where it left off. Should it start from the beginning (reprocessing everything) or from the end (skipping unprocessed messages)?

Neither is acceptable. We need to save the offset somewhere durable.

Offsets are written to Kafka itself, in a special internal topic called __consumer_offsets.

This is a real Kafka topic, partitioned and replicated like any other. When your consumer commits offset 1234 for partition 0 of topic "orders", Kafka writes that as a message to __consumer_offsets.

Committing an offset is a write operation. It can fail. It can be slow. It requires a round-trip to the broker, a machine that stores and serves Kafka data.

But who manages the __consumer_offsets topic?

Kafka does, automatically. The topic is compacted, meaning Kafka keeps only the latest offset for each (consumer group, topic, partition) tuple and discards older entries. Old offsets are garbage collected.

When your consumer starts up, it reads from __consumer_offsets to find where it left off. If there's no offset stored (new consumer group), it uses the auto.offset.reset config (either earliest to start from the beginning or latest to start from the end).

When should offsets be saved?

We know offsets need to be saved to __consumer_offsets. But when? After every message? Every 100 messages? When the consumer shuts down?

By default, Kafka uses auto-commit. Every 5 seconds (configurable with auto.commit.interval.ms), your consumer automatically commits the offsets of messages you've polled.

This seems convenient. It's also where most problems start.

Step through carefully. Watch what happens at 5.5 seconds.

The consumer polled messages 100-104. It's still processing them. The 5-second interval fires and commits offset 105 (next message to read). Then the consumer crashes.

When it restarts, it seeks to offset 105. Messages 100-104 were already committed but never fully processed.

If your processing is idempotent, fine. If you're writing to a database or sending emails, you just skipped work.

Can't auto-commit wait until processing finishes?

No. Auto-commit has no idea when "processing" is done. It just commits what you've polled at fixed intervals. The poll happens in your code. The commit happens in the background.

This is the timing gap. It's invisible until you crash.

Manual commit: taking control

Turn off auto-commit. Commit manually after processing each batch.

Sync commit (blocks for safety)

Sync commit (commitSync()) blocks until Kafka acknowledges the write. The offset is guaranteed to be saved before you continue processing, but throughput suffers because you're waiting on the network for every batch.

Async commit (non-blocking for speed)

Async commit (commitAsync()) returns immediately, giving you higher throughput since you're not waiting. But the commit might fail and you won't know unless you check the callback.

Most production systems use async commits during normal operation and sync commit before shutdown. This balances throughput with safety.

try {
  while (true) {
    ConsumerRecords<String, String> records = consumer.poll(Duration.ofMillis(100));
 
    for (ConsumerRecord<String, String> record : records) {
      process(record);
    }
 
    consumer.commitAsync(); // Non-blocking
  }
} finally {
  consumer.commitSync(); // Blocking, ensure final commit succeeds
  consumer.close();
}

The pattern: async during the loop (fast), sync on shutdown (safe).

Scaling beyond one log

We've been talking about "the log" and "the topic" as if they're one thing. For low-throughput systems, they are. But what happens when you need to handle 100,000 orders per second?

A single log on a single machine has limits. Disk bandwidth. CPU for encoding/decoding. Network throughput. You hit the ceiling.

The solution: split the topic into multiple partitions. Each partition is an independent log. Partition 0 stores some messages, partition 1 stores others, partition 2 stores others. Together they form the topic, but each partition can live on a different machine.

This unlocks parallelism. Multiple brokers can handle writes. Multiple consumers can read in parallel.

But now we have a new question: how do producers decide which partition to write to?

How producers choose where to write

How does a producer decide which partition to write to?

Key-based partitioning (same key → same partition)

If you provide a key, Kafka uses key-based partitioning. The message producer.send(new ProducerRecord<>("orders", "user-123", "login")) hashes the key (user-123) and uses hash(key) % num_partitions to pick a partition. Messages with the same key always go to the same partition, preserving ordering per key. Use this when you want all events for user-123 in order, but you don't care about ordering across different users.

Round-robin partitioning (even distribution)

If the key is null, Kafka uses round-robin partitioning. The message producer.send(new ProducerRecord<>("orders", null, "login")) distributes messages evenly across partitions. Even load distribution, no ordering. Use this when you don't care about ordering and just want maximum throughput.

You can also write a custom partitioner by implementing the Partitioner interface and configuring it with partitioner.class.

Dividing partitions across consumers

Now we have multiple partitions. We also have multiple consumers in the same consumer group (to parallelize the work). How are partitions assigned to consumers?

If you have 6 partitions and 3 consumers in the "billing" group, who reads what?

Slide the number of consumers and partitions. Change the assignment strategy. Watch how partitions move.

Kafka has three built-in strategies:

Range (the default) divides partitions into contiguous ranges. If you have 6 partitions and 3 consumers, Consumer 1 gets P0 and P1, Consumer 2 gets P2 and P3, and Consumer 3 gets P4 and P5. Simple, but if you have multiple topics, Consumer 1 might get P0-P1 of every topic, leading to uneven load.

Round-robin distributes partitions one by one. Consumer 1 gets P0 and P3, Consumer 2 gets P1 and P4, Consumer 3 gets P2 and P5. This provides better load balance across topics.

Sticky works like round-robin but minimizes partition movement during rebalancing. If Consumer 3 leaves, sticky tries to keep P0-P4 where they are and only reassigns P5, reducing the overhead of shuffling partitions around.

Every time a consumer joins or leaves the group, Kafka rebalances. During rebalancing, no messages are processed.

When the group reshuffles

A consumer crashes. A new consumer joins. Kafka detects it and triggers a rebalance.

Watch what happens. All consumers stop processing. They send a JoinGroup request to the group coordinator, a broker that manages consumer group membership. The coordinator picks a new partition assignment. The consumers receive their new assignments and resume.

This is called "stop-the-world" rebalancing. For those few seconds (or minutes, in large clusters), no messages are consumed from any partition in the group.

Rebalancing triggers:

Consumer joins the group (scale up)
Consumer leaves the group (crash or graceful shutdown)
Consumer misses its heartbeat (network partition or slow processing)
Partition count changes (rare, but triggers rebalance)

Newer Kafka versions (2.4+) support cooperative (incremental) rebalancing. Instead of stopping all consumers, only the partitions being reassigned pause. The rest keep processing.

But the default is still stop-the-world. If you're on an older cluster or haven't enabled cooperative rebalancing, expect downtime during rebalances.

The rebalancing offset trap

Rebalancing has a hidden danger with uncommitted offsets.

Step through carefully. Consumer 1 is processing messages 101-105. It hasn't committed yet. A rebalance triggers (Consumer 2 leaves). Consumer 1 stops. The uncommitted work (101-105) is lost.

After rebalancing, Consumer 1 seeks to the last committed offset: 100. It re-reads messages 101-105. Duplicates.

How to avoid this:

Use a rebalance listener

Commit offsets before rebalancing:

consumer.subscribe(topics, new ConsumerRebalanceListener() {
  @Override
  public void onPartitionsRevoked(Collection<TopicPartition> partitions) {
    // Called before rebalance. Commit offsets now.
    consumer.commitSync();
  }
 
  @Override
  public void onPartitionsAssigned(Collection<TopicPartition> partitions) {
    // Called after rebalance. Resume processing.
  }
});

The onPartitionsRevoked callback fires before the rebalance. Commit your offsets here to avoid losing work.

Alternative: Accept at-least-once semantics and make your processing idempotent. If you reprocess messages 101-105, it shouldn't matter.

When bad messages break your consumer

Now we understand commits and rebalancing. Let's look at a scenario that combines both to create production nightmares.

Step through the entire sequence. This is your production incident in slow motion.

The consumer reads offset 1234, and the parse fails with an exception. The consumer crashes. Kubernetes restarts it, and the consumer seeks to the last committed offset: 1234. It reads the same message and crashes again. This creates an infinite loop while your alerts fire and the queue backs up.

Three ways to recover:

1. Advance the offset

Manually seek past the poison message:

kafka-consumer-groups --bootstrap-server localhost:9092 \
  --group my-consumer-group \
  --topic orders \
  --reset-offsets --to-offset 1235 \
  --execute

This tells Kafka "set the offset for this consumer group to 1235." The consumer will resume from there, skipping the bad message at 1234.

When to use: The poison message is unrecoverable and can be safely skipped.

2. Dead letter queue

Catch the exception, send the bad message to a separate "dead letter" topic, commit the offset, and move on:

try {
  process(record);
} catch (Exception e) {
  producer.send(new ProducerRecord<>("orders-dlq", record.key(), record.value()));
  // Commit offset to skip this message
}

Later, you can analyze the DLQ, fix the issue, and optionally replay those messages.

When to use: You want to investigate the bad messages later without blocking the consumer.

3. Fix the message schema

If the schema is wrong, fix it at the source. Update the producer to not send null customerId. Then reprocess from the beginning or from a checkpoint.

When to use: The schema issue affects many messages and needs to be fixed upstream.

Most production systems use a combination: DLQ for immediate recovery, schema fixes for long-term prevention.

When retries create duplicates

We've focused on the consumer side. But producers have their own reliability challenges.

A network glitch occurs. The broker receives your message and writes it to the log, but the acknowledgment is lost. Your producer times out and retries. Now you have a duplicate.

Step through the retry scenario. Watch how the broker detects the duplicate.

How it works:

Enable idempotence:

props.put("enable.idempotence", "true");

Kafka assigns your producer a unique Producer ID (PID). Each message gets a sequence number (per partition). The broker tracks the highest sequence number it's seen for each PID.

When the producer retries with the same (PID, sequence number), the broker recognizes it's a duplicate and discards it. The producer gets an ACK without writing the message twice.

Guarantees:

Exactly-once delivery to the log
In-order delivery per partition

Trade-off: Slightly higher latency (sequence numbers add overhead). In practice, negligible.

When to use: Always. Idempotence is free and eliminates a major class of bugs. Modern Kafka enables it by default.

What happens when a broker crashes

Idempotence protects against retry duplicates on a single broker. But what if the broker itself fails?

Each partition is replicated across multiple brokers. One broker is the leader, the others are followers.

Step through the leader crash. Watch the follower promotion.

Replication factor 3 means one leader + two followers. All writes go to the leader. The leader replicates to followers asynchronously.

What happens on leader failure:

ZooKeeper (Kafka's coordination service, or KRaft in newer versions which embeds coordination directly into Kafka) detects the leader is down
The controller, a special broker responsible for administrative tasks, selects a new leader from the In-Sync Replicas (ISR)
Producers and consumers transparently fail over to the new leader

In-Sync Replicas (ISR): Followers that are caught up with the leader. If a follower falls too far behind (replica.lag.time.max.ms), it's removed from the ISR.

Only ISR members can become leader. This prevents data loss.

When replicas fall behind

Not all replicas are equal. Some are caught up. Some lag behind.

Click "Produce Messages" and watch Follower 2 fall behind. Once its lag exceeds the threshold, it's removed from the ISR.

ISR membership criteria:

A follower is in-sync if it has fetched messages from the leader within replica.lag.time.max.ms (default 10 seconds).

If a follower's disk is slow, or the network is congested, it might fall behind. Kafka removes it from the ISR.

Producer acknowledgment timing

The acks setting controls when the producer considers a write successful. Step through each configuration to see the difference.

Watch when the producer receives the ACK in each case:

acks=0: Producer doesn't wait for any acknowledgment. Fire and forget. Fastest, but no durability guarantee. If the broker crashes before writing, the message is lost.
acks=1: Producer waits for the leader to write. The leader writes to its local log and immediately sends an ACK. If the leader crashes before replication, the message is lost.
acks=all: Producer waits for all ISR members to write. Strongest guarantee. If the leader crashes, the message is on at least one follower. Slowest, but most durable.

With acks=all, the producer waits until all in-sync replicas have written the message. If only the leader is in-sync (followers are too slow), the producer only waits for the leader.

Minimum ISR: You can configure min.insync.replicas=2 to require at least 2 replicas in the ISR. If ISR shrinks below 2, writes fail with NotEnoughReplicasException.

This protects against data loss: you refuse to write unless you have at least 2 copies.

Trade-off: Availability vs durability. Requiring 2 in-sync replicas means you can tolerate 1 replica failure. But if 2 replicas fail, you can't write.

The hardest guarantee: exactly-once

We've built protection against duplicates at the producer (idempotence) and against data loss at the broker (replication). But crashes can still cause duplicates on the consumer side. A consumer processes a message, crashes before committing the offset, and reprocesses on restart.

At-least-once is easy: commit after processing. You might reprocess on crash, but nothing is lost.

Exactly-once is hard. It requires coordination across the entire pipeline.

Step through the transaction flow. See how messages become visible atomically.

Three pieces:

1. Idempotent producer

As we saw earlier, idempotent producers prevent duplicate writes.

props.put("enable.idempotence", "true");

2. Transactional producer

Write multiple messages atomically:

props.put("transactional.id", "my-transactional-id");
 
producer.initTransactions();
producer.beginTransaction();
producer.send(new ProducerRecord<>("orders", "order-100"));
producer.send(new ProducerRecord<>("inventory", "reserve-100"));
producer.commitTransaction(); // Both messages visible together

If you crash before commitTransaction(), both messages are aborted. Kafka writes transaction markers to the log.

3. Read committed consumer

Configure the consumer to only see committed messages:

props.put("isolation.level", "read_committed");

This filters out aborted messages. Your consumer never sees them.

Put it together:

// Producer writes orders + inventory updates transactionally
producer.beginTransaction();
producer.send(new ProducerRecord<>("orders", order));
producer.send(new ProducerRecord<>("inventory", inventoryUpdate));
 
// Also commit consumer offset as part of the same transaction
producer.sendOffsetsToTransaction(offsets, consumerGroupMetadata);
producer.commitTransaction();

Now the entire read → process → write pipeline is atomic. If the transaction commits, both the output messages and the consumer offset are committed. If it aborts, neither is.

This is exactly-once end-to-end.

Trade-offs:

Significant performance overhead (coordinator manages transactions)
More complex code (transaction lifecycle)
Only works if your entire pipeline is Kafka (can't transactionally write to external databases)

For external systems (databases, APIs), you need distributed transactions like two-phase commit (a protocol where a coordinator ensures all participants commit or abort together) or Sagas (a pattern where each step has a compensating action to undo it), or you need to make your operations idempotent.

Picking the right commit strategy

We've seen auto-commit, manual sync, manual async, and transactional commits. Which should you use?

Strategy	Duplicates	Lost Messages	Throughput	Complexity	Best For
Auto-commit	Possible	Possible	High	Low	Logging, analytics, can tolerate loss
Manual sync	No	No	Medium	Medium	Financial transactions, critical data
Manual async	Possible	No	High	Medium	High-throughput, willing to monitor
Transactional	No	No	Low	High	Exactly-once required (billing, inventory)

Auto-commit

Config:

enable.auto.commit=true
auto.commit.interval.ms=5000

Pros:

Simple to use
High throughput
No manual offset management

Cons:

Duplicates on crash (uncommitted work)
Lost messages if crash before processing
No control over commit timing

Manual sync commit

Config:

enable.auto.commit=false
consumer.commitSync()

Pros:

Guarantees offset is saved
No duplicates (commit after processing)
No lost messages

Cons:

Blocks until commit succeeds
Lower throughput
Increases latency

Manual async commit

Config:

enable.auto.commit=false
consumer.commitAsync()

Pros:

Non-blocking (high throughput)
Control over commit timing
Better than auto-commit

Cons:

Commit might fail silently
Need callback to handle failures
Duplicates possible on failure

Transactional

Config:

transactional.id=producer-1
isolation.level=read_committed
Coordinated producer + consumer

Pros:

True exactly-once semantics
Atomic writes across partitions
Offset commits within transaction

Cons:

Significant performance overhead
Complex to implement correctly
Requires transaction coordinator

Quick decision tree:

Need exactly-once? → Transactional
Financial or critical data? → Manual sync
High throughput + can monitor? → Manual async
Logging/analytics (ok with some loss)? → Auto-commit

Most systems start with auto-commit and move to manual commits when they hit their first production incident. Skip the incident. Choose manual commits up front.

Patterns for production use

The decision tree tells you which strategy to use. Here's how to implement each one correctly.

Pattern 1: Batch Processing with Sync Commit

int batchSize = 0;
for (ConsumerRecord<String, String> record : records) {
  process(record);
  batchSize++;
 
  if (batchSize >= 100) {
    consumer.commitSync();
    batchSize = 0;
  }
}
consumer.commitSync(); // Commit remainder

Commit every 100 messages. Balances throughput (not committing every message) with safety (not auto-committing blindly).

Pattern 2: Async Commits with Error Handling

consumer.commitAsync((offsets, exception) -> {
  if (exception != null) {
    log.error("Commit failed for offsets {}", offsets, exception);
    // Optional: retry, alert, etc.
  }
});

Async commits can fail silently. Use a callback to detect failures and handle them (log, retry, alert).

Pattern 3: Manual Offset Management

TopicPartition partition = new TopicPartition("orders", 0);
OffsetAndMetadata offset = new OffsetAndMetadata(1234);
Map<TopicPartition, OffsetAndMetadata> offsets = Map.of(partition, offset);
 
consumer.commitSync(offsets);

Instead of committing the current polled offsets, you can commit arbitrary offsets. Useful for custom checkpointing or recovery logic.

Pattern 4: Rebalance-Safe Processing

ConsumerRebalanceListener listener = new ConsumerRebalanceListener() {
  @Override
  public void onPartitionsRevoked(Collection<TopicPartition> partitions) {
    // Flush any in-memory state
    // Commit offsets
    consumer.commitSync();
  }
 
  @Override
  public void onPartitionsAssigned(Collection<TopicPartition> partitions) {
    // Initialize state for new partitions
  }
};
 
consumer.subscribe(List.of("orders"), listener);

Always commit before partitions are revoked. This prevents losing uncommitted work during rebalancing.

Knowing when things break

You've built the system correctly. Now you need to know when it stops working. You can't fix what you can't see.

Key metrics to track:

Consumer lag

Adjust the producer and consumer rates, then hit Run. Watch what happens when the producer writes faster than the consumer can process.

Lag is the difference between the current offset (where messages are being written) and the committed offset (where the consumer has processed up to). If lag keeps growing, your consumer can't keep up.

You can check lag with:

kafka-consumer-groups --bootstrap-server localhost:9092 \
  --describe --group my-consumer-group

Shows current offset vs log-end-offset per partition.

Alert on: Lag > threshold for more than N minutes.

Commit rate

Track consumer.commitSync() and consumer.commitAsync() calls. If commits are failing, you'll see exceptions.

Alert on: Commit failure rate > 1%.

Rebalance frequency

Frequent rebalancing means unstable consumer groups. Could be crashes, slow processing, or network issues.

Alert on: More than 1 rebalance per hour.

Consumer offset reset

If a consumer seeks to auto.offset.reset (earliest or latest), it means offsets were lost or expired.

Alert on: Any offset reset in production (shouldn't happen if commits are working).

Tools:

Kafka's built-in metrics (exposed via JMX, Java's standard monitoring interface)
Prometheus (metrics collector) + Grafana (visualization dashboards)
Confluent Control Center (commercial monitoring UI)
Burrow (LinkedIn's open-source lag monitoring tool)

Set up dashboards before the incident. Not during.

When things go wrong

Alerts are firing. Here's how to diagnose the four most common failures.

Issue 1: Consumer Not Consuming

Symptoms: Lag increasing, no errors in logs.

Likely causes:

Rebalancing loop (consumer keeps timing out)
Processing is too slow (max.poll.interval.ms exceeded)
Consumer is paused or suspended

How to debug:

Check consumer group state: kafka-consumer-groups --describe
Look for frequent rebalances in logs
Check processing time per message
Verify max.poll.interval.ms is high enough for your workload

Issue 2: Duplicate Messages

Symptoms: Same message processed multiple times.

Likely causes:

Crash between processing and commit
Rebalancing before commit
Auto-commit timing gap

How to debug:

Check commit strategy (auto vs manual)
Add rebalance listener to commit before revocation
Make processing idempotent (deduplicate on message ID)

Issue 3: Lost Messages

Symptoms: Offsets committed, but messages weren't processed.

Likely causes:

Auto-commit committed before processing finished
Crash after commit, before processing

How to debug:

Switch to manual commits
Commit only after processing succeeds
Add transaction boundaries if needed

Issue 4: Poison Message Loop

Symptoms: Consumer stuck on one offset, restarting repeatedly.

Solution: See the PoisonMessageDemo above. Skip the bad message or send to DLQ.

Best practices

Disable auto-commit in production. Use manual commits. Control when offsets are saved.
Commit frequently, but not too frequently. Every 100-1000 messages is a good starting point. Tune based on throughput.
Use rebalance listeners. Commit offsets before partitions are revoked to avoid losing work.
Make processing idempotent. Duplicates are inevitable (rebalancing, crashes). Design for at-least-once.
Monitor consumer lag. Set up alerts before lag grows unbounded.
Enable idempotent producers. Free and prevents a class of duplicates.
Use Dead Letter Queues. Don't let poison messages kill your consumer. Skip them and investigate later.
Set max.poll.interval.ms generously. If your processing takes 30 seconds per batch, set this to 60+ seconds. Otherwise, Kafka thinks your consumer is dead.
Test rebalancing in staging. Kill consumers, add consumers, see how your application behaves.
Use transactions only if you need exactly-once. The overhead is real. Most systems can tolerate at-least-once with idempotent processing.

Summary

Kafka is an append-only log. Consumers track offsets. Commits save those offsets. Rebalancing shuffles partition assignments. Crashes cause duplicates or data loss depending on commit timing.

You now understand:

Offsets are position pointers stored in __consumer_offsets
Auto-commit has a timing gap that causes duplicates or lost messages
Manual commits (sync or async) give you control over when offsets are saved
Poison messages cause crash loops; skip them or send to a DLQ
Partition assignment strategies distribute partitions across consumers (range, round-robin, sticky)
Rebalancing pauses consumption and can lose uncommitted work
Producer partitioning uses key hashing to preserve ordering per key
Idempotent producers eliminate duplicate writes
Replication and ISR provide fault tolerance; acks=all waits for all in-sync replicas
Exactly-once semantics requires transactional producers + read_committed consumers
Commit strategies trade off throughput, safety, and complexity

The next time your consumer gets stuck, you won't just run a command. You'll know exactly why it happened and how to prevent it.