IMAP UID FETCH after reconnect: resume from a UID cursor without turning sequence numbers into durable state

Separate the transport fact from reputation guesses

IMAP message sequence numbers are positions that can change as the mailbox changes. UIDs are designed for stable reference within a mailbox while UIDVALIDITY remains unchanged.

After reconnecting to an IMAP mailbox, sequence numbers are not a durable cursor. They can be renumbered as messages are expunged or as the mailbox view changes. UIDs are designed to persist within a mailbox while UIDVALIDITY remains unchanged, so a worker can save the last processed UID and request a range above that value after reconnect. The important qualifier is the UIDVALIDITY value: if it changes, the old UID namespace can no longer be trusted. A robust reconnect routine therefore restores the mailbox, compares UIDVALIDITY, then resumes UID-based discovery or performs a controlled resynchronization if the namespace changed.

How a green checker can still hide the problem

A sequence number and a UID can both look like simple integers, so logging only the number hides which namespace the code used. The bug often appears only after deletions or expunges shift positions.

Do not assume that “last UID + 1” must exist or that every UID up to UIDNEXT is assigned. UIDs can have gaps, and UIDNEXT is a prediction for a future assignment rather than a message count. Requesting a range such as `saved_uid+1:*` asks the server for whatever matching UIDs actually exist; your application should then deduplicate each returned message against durable state. Another mistake is saving only the highest sequence number observed before disconnect. That cursor can point at a completely different message after expunge, creating both misses and duplicate processing.

Recreate the failure with one controlled path

Reconnect to a mailbox after deleting an earlier message from another client. Compare ordinary FETCH 185 with UID FETCH on the stored UID range and verify your worker follows the original messages rather than the new positions.

Test reconnect behavior by delivering several messages, recording UIDs, expunging one older message from another client, and then reconnecting the worker. Confirm the sequence positions change while the remaining UIDs keep their identity under the same UIDVALIDITY. Next, simulate a stored cursor and fetch the UID range above it; verify only genuinely unprocessed messages create events. Finally, test the UIDVALIDITY-change branch using a disposable mailbox or a mocked server so the worker refuses to reuse stale UID state. A reconnect path is not complete until both normal and namespace-reset behavior are covered.

Make the smallest reversible change

Persist mailbox name, UIDVALIDITY and last processed UID. After SELECT, invalidate the cursor if UIDVALIDITY changed; otherwise request a UID range above the stored cursor and process responses idempotently.

Persist mailbox identity, UIDVALIDITY, highest safely processed UID, and per-message idempotency keys in the same synchronization record. Advance the cursor only after downstream processing is committed, or keep enough state to replay safely after a crash. If IDLE is used, treat it as a wake-up optimization; reconciliation after reconnect should still use the durable UID rules. This design tolerates missed notifications and transient connections without turning them into missed replies. The operator log should show where the cursor started, which UIDs were returned, and when the durable checkpoint advanced.

A practical mailbox or domain example

If the last durable UID is 9021, a worker can search or fetch UIDs greater than that even when the mailbox now contains only 120 sequence positions. The two number spaces should never be interchanged.

Cursor recovery should be crash-safe as well as reconnect-safe. If the worker fetches UIDs 501 through 510, processes 501–505, then crashes before persisting 510, restarting from 500 will replay some messages. That is acceptable only if each downstream action is idempotent. Advancing directly to 510 before processing, on the other hand, risks losing 506–510 forever after a crash. A practical design checkpoints after durable processing or stores per-message processed keys so ranges can be replayed safely. Measure the distance between UIDNEXT and your processed cursor as operational lag, but remember gaps mean it is not a literal unread-message count. This state model lets a tiny service survive network drops and deploy restarts without using destructive assumptions about sequence numbers or notification delivery.

The minimum useful change record

Record selected mailbox, UIDVALIDITY, UIDNEXT hint, last processed UID, highest observed UID, reconnect time, and any cursor reset. This makes missed-reply investigations reproducible.

Conditions that should remain unknown or retryable

UIDNEXT is only a prediction/hint and has gaps; do not assume every integer below it exists or that the next arriving message will receive exactly that value.

Keep reconnect testing in deployment checks, not only unit tests. Restart the worker while messages are arriving, verify the saved UID checkpoint, and confirm the same inbound message never creates two reply events. A real restart exercises persistence, connection setup, SELECT state, UIDVALIDITY comparison, and downstream idempotency together—the exact chain that tends to fail when a service is upgraded under live mailbox traffic.

Field checklist

• Capture the raw evidence for IMAP UID FETCH range after reconnect incremental sync before editing DNS, queue state, or mailbox metadata.
• Confirm the cited mechanism using imap-rfc, imap-sync-rfc rather than a generic deliverability score.
• Keep a known-good control case so the failing layer can be compared without changing several variables at once.
• Apply the narrow repair described for IMAP UID FETCH after reconnect: resume from a UID cursor without turning sequence numbers into durable state and preserve a rollback or retry path.
• Repeat the original failing test after the change; do not substitute a different checker as proof of recovery.
• Store timestamp, affected identity, raw result, action taken, and the post-change result in the operator log.