Unread and unprocessed are different states
The IMAP \Seen flag records a user-visible mailbox state. It does not mean “this message has been handled by the outbound application.” A person can read the message, a mobile client can preview it, a filter can mark it read, or another integration can change flags. If automation searches only UNSEEN, any of those actions can remove the message from the worker’s candidate set without creating an application event. The system needs its own durable notion of processed replies.
Use UID progression for mailbox traversal
Persist the last safely processed UID together with UIDVALIDITY for the mailbox. On each poll, search or fetch UIDs beyond that cursor and process them regardless of Seen state. After successful classification and durable side effects, advance the cursor. This approach matches the role UIDs play in IMAP synchronization and remains independent from human interaction. If UIDVALIDITY changes, stop the incremental path and resynchronize rather than trusting the old integer range.
Fetch without changing Seen when possible
If preserving unread state matters to the human operator, retrieve content with an IMAP method that does not set \Seen, such as BODY.PEEK semantics. This is useful courtesy, but it is not the core reliability control. Even a perfect non-marking fetch cannot prevent another client from changing flags. Reliability comes from tracking what the automation processed in its own database, while BODY.PEEK reduces unnecessary side effects in the shared inbox.
Treat flags as metadata, not identity
Flags such as \Seen, \Answered, \Flagged, and provider keywords can enrich workflow, but they are mutable. The message identity for mailbox traversal is tied to UID and UIDVALIDITY; the logical conversation identity can use Message-ID and reply headers. A parser can record the flags it observed at processing time without basing cancellation on them. This layered model survives far more real-world mailbox behavior than a single “unread = new reply” rule.
Close the race with idempotent processing
Two workers may fetch the same new UID around the same time, or a reconnect catch-up may rediscover a reply. The database should reject duplicate side effects using a deterministic processing key. After one worker records the reply and cancels future steps, the other can classify the same message but must not create another CRM event or cancellation. This is especially important when moving away from UNSEEN, because the candidate set intentionally includes messages that a previous session may already have seen.
Use UNSEEN only for optional user-experience features
There are legitimate uses for UNSEEN: showing the salesperson how many unread replies remain, selecting messages for a manual review queue, or reducing a one-off mailbox scan when no correctness depends on it. Keep those features separate from the core reply detector. If the UNSEEN query returns zero while the UID cursor says three new messages arrived, process the three. Application correctness should win over the visual mailbox flag.
A simple failure reproduction test
Send a test reply into INBOX. Before the worker polls, open the message in webmail so the server sets \Seen. A correct detector using UID progression should still process the reply and cancel the sequence. Then repeat without opening it and confirm the same result. Add a third test where two workers poll simultaneously and verify the cancellation occurs once. These tests reproduce common human and concurrency behavior instead of relying on idealized mailbox conditions.
Monitor cursor lag rather than unread count
For health monitoring, measure the difference between the newest known UID and the last processed UID, plus the age of the oldest unprocessed candidate. Unread count can be zero while the detector is hours behind if someone reads messages manually. Cursor lag tells you whether automation has actually caught up. Alert on stale processing time, connection failures, and UIDVALIDITY changes; expose unread count only as a user-interface statistic.
Reproduce the missed-reply bug with two clients
A simple test demonstrates why `SEARCH UNSEEN` is a weak processing cursor. Deliver a reply, open it in a phone or web client so the server sets `\Seen`, and only then run the automation. A detector that searches only UNSEEN will never see the reply even though it is new to the automation. Reverse the order and another failure appears: if the automation fetches content in a way that sets `\Seen`, it can change the user’s mailbox state merely by processing the message. These are user-interface semantics leaking into backend delivery logic.
Use UID progression for discovery instead. Persist the mailbox’s UIDVALIDITY and highest safely processed UID, fetch new messages by UID range, and request body content with a non-marking form such as BODY.PEEK when appropriate. The `\Seen` flag can still be recorded as metadata or used for a dashboard filter, but it should not decide whether a reply has ever been processed. Make the downstream correlation idempotent so a safety rescan does not send duplicate notifications or cancel the same follow-up twice. Monitor cursor lag—the distance between the last processed position and current mailbox state—because it measures whether the automation is keeping up, whereas unread count measures how a person happens to use the mailbox.
Field checklist
- Persist UIDVALIDITY and the last processed UID for each mailbox.
- Process new UIDs regardless of their Seen flag.
- Use BODY.PEEK when you want to preserve human unread state.
- Make reply side effects idempotent across workers and reconnects.
- Reserve UNSEEN for optional UX or review filters.
- Monitor processing cursor lag instead of unread count.
Primary sources
Standards and provider policies can change. These links are the reference points used for this field note.
- RFC 9051 — IMAP4rev2IETF / RFC Editor — Mailbox flags, UIDs, BODY.PEEK and IMAP4rev2 behavior.
- RFC 4549 — Synchronization Operations for Disconnected IMAP4 ClientsIETF / RFC Editor — Operational guidance for UIDVALIDITY checks, cache invalidation and mailbox synchronization.
- RFC 5322 — Internet Message FormatIETF / RFC Editor — Message header structure, Message-ID, In-Reply-To and References threading fields.