feat(sync): idempotent email import so retries do not duplicate IMAP (#85)#86
Merged
Merged
Conversation
🔍 CI ResultsOverall: ❌ 12/12 lanes failed TL;DR: ❌ Quality issues: PHPUnit: 54 tests failed; PHPStan: 328 unique errors in 8 lanes; PHP-CS-Fixer: 24 files. Summary by PHP Version
Quality Metrics
❌ Failed Lanesphp8.0-dev
php8.0-stable
php8.1-dev
php8.1-stable
php8.2-dev
php8.2-stable
php8.3-dev
php8.3-stable
php8.4-dev
php8.4-stable
php8.5-dev
php8.5-stable
CI powered by horde-components • View full results |
4 tasks
ralflang
force-pushed
the
feat/sync-idempotent-import
branch
from
July 18, 2026 08:21
747a9b8 to
b12f517
Compare
Record applied Add/Modify/Remove (and remove-as-move) under the current SyncKey so a lost Sync response can be retried without appending another generation of Draft messages. Dual-write email Add clientids into the sync map; map Draft Modify old→new UIDs and suppress mirror Add/Delete export via mailmap. Closes #85.
TDannhauer
force-pushed
the
feat/sync-idempotent-import
branch
from
July 19, 2026 19:51
b12f517 to
b7f383e
Compare
Guard importMessageDeletion mailmap checks to CLASS_EMAIL so Notes (and other PIM) UUID server ids are not queried against the integer mailmap.message_uid column. Reject non-integer UIDs in isMailMapChangeApplied and roll back an open DB transaction on query failure to avoid PostgreSQL 22P02/25P02 cascades during Sync.
Contributor
Author
|
@ralflang bitte reviewen und mergen |
IMAP append+delete gives an edited Draft a new UID, but MS-ASCMD Sync Responses must reference the ServerId the client sent. Returning the new UID made Gmail reject the response and re-send the whole Drafts batch under the old SyncKey. _draftModifyStat() now returns the client ServerId as 'id' (the new UID stays in the applied map and mailmap and still feeds conversationindex). Also fall back to a synthetic conversationid for drafts without a subject, since empty conversation data suppresses the Modify reply entirely.
Contributor
Author
|
Follow-up from the latest reporter log bundle: idempotent retry now works (78 duplicate Modifies, 0.0s import), but Gmail still refused the new SyncKey because |
3 tasks
ralflang
added a commit
that referenced
this pull request
Jul 24, 2026
Release version 3.2.0 Merge pull request #91 from horde/feat/sync-performance docs(activesync): move #88 entry to recently completed in todo.md feat(activesync): cut steady-state polling and export cost (#88) feat(sync): evict ghost items via deferred synthetic deletions refactor(activesync): centralize client quirks in Device::hasQuirk() feat(activesync): persist folder UID map across FolderSync resets (#87) feat(sync): idempotent email import so retries do not duplicate IMAP (#85) (#86)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Add/Modify/Remove(including remove-as-move) idempotent under the same SyncKey so a lost or rejected Sync response cannot duplicate IMAP messagesModify(UID replace via append+delete) records old→new mapping and returns the same successful reply on retryMotivation
Tracked in #85 (observed with Gmail Android
FullDraftsUpSync). Streaming / transport work (#84) reduces client read timeouts but does not fix the durability mismatch: client commands are committed to IMAP before the client acks the response. On retry under SyncKey N, a non-idempotent Draft Modify appends another generation of messages.Complementary to #84 — not stacked on it; this PR targets
FRAMEWORK_6_0directly.Changes
State_Base/State_Sql/State_Mongo:getAppliedPIMChange,recordAppliedPIMChange,recordPIMAddition,isMailMapChangeApplied, mail-move mapping; tightenisDuplicatePIMAddition; MongoCHANGE_TYPE_DRAFT+isDuplicatePIMChangeConnector_Importer: short-circuit retries; Draft Modify dual-writes map then mailmap (sync_draftnew +sync_deletedold); rebuild stable conversation fields for EAS 16 SyncRepliesImporterIdempotentImportTestRelated Core PR: refuse Draft Modify append-as-new when the old IMAP UID is already missing (defense in depth).
Test plan
vendor/bin/phpunit … ImporterIdempotentImportTest.php(6 tests)FullDraftsUpSyncagainst a small Drafts folder does not grow across failed/partial responses