[2.x] Add incremental mailbox synchronization

[stevebauman]

This PR adds the IMAP primitives needed to keep a local mailbox in sync: fetching changes since a saved checkpoint, discovering deleted messages, and updating flags without overwriting changes made by another client.

Incremental synchronization

• Add CONDSTORE and QRESYNC selection options, with QRESYNC enabled before selection.
• Return a typed Selection\Result containing message counts, UIDVALIDITY, UIDNEXT, HIGHESTMODSEQ, permanent flags, changes returned during selection, and the raw responses.
• Add MessageData::modSequence() and changesSince() for fetching changed messages directly by UID, including optional VANISHED responses.
• Support CHANGEDSINCE and VANISHED directly through fetch() with typed modifiers. Ordinary and modified fetches share the same FetchResult, containing parsed message data, vanished UIDs, and raw responses.
• Support conditional flag updates directly through store() with an UnchangedSince modifier. Ordinary and conditional updates return StoreResult, preserving returned message data and exposing conflicting identifiers through MODIFIED.
• Support QRESYNC sequence-match data with or without a known-UID filter, including validation of UIDVALIDITY, modification-sequence checkpoints, UID sets, and sequence pairs.
• Filter FETCH and STORE results to the requested message set while retaining unsolicited updates in the raw responses. Message-number results retain their source sequence number.
• Preserve selected-folder state across queries, invalidate it correctly around SELECT and EXAMINE failures, and clear connection-specific state when disconnecting, reconnecting, or cloning.
• Return an empty FetchResult immediately when changesSince() receives an empty UID array.
• Model vanished-message history in the testing fakes so synchronization behavior matches a real mailbox.

Applications remain responsible for storing checkpoints, handling UIDVALIDITY changes, and falling back when the server does not support these extensions.

Extensible fetched data

FetchedMessageData now retains the complete returned attribute map instead of copying a fixed list into constructor properties. It provides has(), get(), and immutable merge() methods alongside the existing typed accessors.

This preserves arbitrary body sections, partial offsets, extension attributes, nested lists, and explicit NIL values. It also distinguishes an attribute that was not fetched from one that was returned empty.

Message retains the complete data container through serialization. Lazy fetches merge into the existing data, STORE responses update local message state, and previously fetched complete body parts can be reused when peeking.

Authentication and capabilities

• Add a SASL AuthenticatorInterface and an Authentication coordinator that owns the challenge-response exchange.
• Add XOAUTH2 through the mailbox's xoauth2 configuration. Initial responses remain opt-in for servers supporting SASL-IR, and outgoing credentials are redacted.
• Replace capability arrays with immutable Capabilities and Capability objects that expose supported and enabled state without allowing callers to mutate mailbox state.
• Keep capability enablement on the mailbox, enforce ENABLE before selecting or examining a folder, and mirror the behavior in FakeMailbox.
• Add Mailbox::reconnect(?string $password = null) so an expired password or token can be replaced without swapping the mailbox account, folders, or server configuration.

Command and parameter handling

• Keep connection methods aligned one-to-one with IMAP commands and remove the connection-level uid(), bodyText(), bodyHeader(), bodyStructure(), bodyPart(), flags(), and size() shortcuts in favor of fetch().
• Validate command framing, sequence sets, SASL mechanisms, capabilities, charsets, LIST/STATUS options, STORE flags, QRESYNC data, and synchronization checkpoints before writing to the stream.
• Preserve ID field names and NIL values, framing multiline strings as literals.
• Always send APPEND messages as literals, including messages without line breaks.
• Expand ALL, FAST, and FULL fetch macros, and recognize PEEK and partial-body response attributes when fetching by message number.
• Support LIST selection options, multiple patterns, and additional return responses such as STATUS.
• Make FETCH and STORE result filtering linear for large or sparse message sets.

V2 API changes

• SELECT and EXAMINE return Selection\Result; raw responses remain available through responses(). SELECT, EXAMINE, and STATUS consistently default to INBOX.
• FETCH, STORE, COPY, and MOVE take the message set first. A set can be an ID, an array of IDs, or a string such as 1:3,7:*; the separate from/to arguments are removed. UID EXPUNGE also accepts a string set.
• STORE is explicitly flags-only; the generic item argument is removed.
• SEARCH and SORT accept criteria and an explicit charset option. SEARCH omits CHARSET by default; SORT defaults to UTF-8.
• LIST accepts pattern, selection, and return options, preserving all untagged responses. The folder repository filters folder entries itself.
• STATUS calls its requested attributes items and omits RECENT from the default request. IMAP4rev1 callers can still request RECENT explicitly.
• AUTHENTICATE accepts a SASL mechanism and optional initial response. The Authentication coordinator accepts an AuthenticatorInterface and manages the exchange.
• fetch() returns FetchResult. The separate fetchChanges() method is removed; changesSince() is a query convenience built on fetch().
• store() returns StoreResult and accepts typed modifiers instead of a separate storeConditionally() method. Adding flags remains the default; mode: null replaces flags and mode: '-' removes them. StoreResult::modified() returns identifiers matching the command's addressing mode.
• Rename ImapFetchIdentifier to ImapIdentifier and support it consistently on FETCH, STORE, SEARCH, SORT, COPY, and MOVE. UIDs remain the default.
• Rename connection quota() and quotaRoot() to getQuota() and getQuotaRoot(), preserving both QUOTAROOT and QUOTA responses.
• LOGOUT waits for completion and closes the local connection, including on failure.
• Folder and mailbox selection accept typed selection options.
• FetchedMessageData accepts an attribute array.
• Message is constructed with new Message($folder, $data), and data() exposes the fetched attributes.
• Message array and JSON output use the complete IMAP attribute map instead of the previous fixed lowercase fields.
• Require PHP 8.2 on a 64-bit build so RFC 7162's unsigned 63-bit modification-sequence range can be represented safely.

For example:

use DirectoryTree\ImapEngine\Selection\QuickResync;

$selection = $folder->select(
    options: new QuickResync(
        $uidValidity,
        $highestModSequence,
        knownUids: $knownUids,
        sequenceMatch: [$knownMessageNumbers, $correspondingUids],
    ),
);

$changes = $selection->changes();

foreach ($changes->messages() as $data) {
    $uid = $data->uid();
    $flags = $data->flags();
    $modSequence = $data->modSequence();
}

$vanishedUids = $changes->vanishedUids();

Before applying these changes, the application should compare the returned UIDVALIDITY with its saved value. It should only save the new checkpoint after successfully applying the synchronization results.

To fetch changes directly on the connection:

use DirectoryTree\ImapEngine\Fetch\ChangedSince;

$connection->enable('QRESYNC');
$connection->select('INBOX');

$result = $connection->fetch(
    $knownUids,
    ['FLAGS', 'MODSEQ'],
    modifiers: new ChangedSince($highestModSequence, vanished: true),
);

$messages = $result->messages();
$vanishedUids = $result->vanishedUids();
$responses = $result->responses();

CHANGEDSINCE requires CONDSTORE support, which is also provided by QRESYNC. Requesting VANISHED additionally requires UID FETCH and QRESYNC to be enabled. New modifiers can implement Fetch\ModifierInterface without adding another connection method.

Conditional flag updates follow the same pattern:

use DirectoryTree\ImapEngine\Store\UnchangedSince;

$result = $connection->store(
    $knownUids,
    ['\Seen'],
    modifiers: new UnchangedSince($highestModSequence),
);

$conflictingUids = $result->modified();

The set-first connection API follows the same ordering across message operations:

$connection->fetch('1:*', ['UID', 'FLAGS']);
$connection->store([7, 8], ['\Seen']);
$connection->copy([7, 8], 'Archive');
$connection->move([7, 8], 'Trash');

Higher-level message and query APIs keep their existing argument order. Extension options still require the corresponding server capabilities.

[Copilot]

🟡 Changes recommended

FETCH filtering and mailbox selection tracking can return incorrect data, while empty synchronization sets generate invalid commands.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Pull request overview

Adds incremental IMAP synchronization and modernizes the v2 connection API.

Changes:

• Adds CONDSTORE/QRESYNC synchronization and conditional STORE support.
• Preserves complete fetched attributes and introduces typed results.
• Updates authentication, command parameters, selection state, and tests.

File summaries

File	Description
tests/Unit/Support/StrTest.php	Tests sequence parsing and literals.
tests/Unit/MessageTest.php	Updates message construction tests.
tests/Unit/MessageQueryTest.php	Tests lookup and APPEND behavior.
tests/Unit/MessageDataTest.php	Tests MODSEQ fetch items.
tests/Unit/IncrementalSyncTest.php	Covers incremental synchronization.
tests/Unit/FolderTest.php	Tests folder examination.
tests/Unit/FetchedMessageDataTest.php	Covers attribute preservation and caching.
tests/Unit/Connection/ImapConnectionTest.php	Updates connection API tests.
tests/Unit/Connection/ImapConnectionParametersTest.php	Tests command parameter handling.
tests/Unit/Connection/ImapConnectionOperationsTest.php	Tests message operations and results.
tests/Unit/Connection/ImapConnectionAuthenticationTest.php	Tests SASL authentication.
tests/Integration/FoldersTest.php	Updates STATUS expectations.
src/Vanished.php	Models VANISHED responses.
src/Testing/FakeMessageQuery.php	Adds fake incremental fetching.
src/Testing/FakeMessage.php	Adds fake modification sequences.
src/Testing/FakeMailbox.php	Supports selection results and ENABLE.
src/Testing/FakeFolder.php	Updates fake folder selection.
src/Support/Str.php	Adds literal-list and sequence parsing.
src/StoreResult.php	Models STORE results.
src/StoreModifier.php	Defines STORE modifiers.
src/Store/UnchangedSince.php	Adds UNCHANGEDSINCE.
src/SelectionResult.php	Models SELECT/EXAMINE metadata.
src/SelectionOption.php	Defines selection options.
src/Selection/QuickResync.php	Adds QRESYNC parameters.
src/Selection/CondStore.php	Adds CONDSTORE selection.
src/MessageQueryInterface.php	Extends query synchronization API.
src/MessageQuery.php	Implements changed-message fetching.
src/MessageInterface.php	Exposes modification sequences.
src/MessageData/Attribute.php	Adds MODSEQ attribute.
src/MessageData.php	Adds MODSEQ factory.
src/Message.php	Stores and merges fetched attributes.
src/MailboxInterface.php	Extends mailbox selection API.
src/Mailbox.php	Tracks selection and enabled capabilities.
src/FolderRepository.php	Filters extended LIST responses.
src/FolderInterface.php	Returns typed selection results.
src/Folder.php	Updates selection and quota handling.
src/FileMessage.php	Implements modification sequence accessor.
src/FetchResult.php	Models FETCH results.
src/FetchModifier.php	Defines FETCH modifiers.
src/FetchedMessageData.php	Preserves arbitrary fetched attributes.
src/Fetch/ChangedSince.php	Adds CHANGEDSINCE/VANISHED.
src/Enums/ImapIdentifier.php	Generalizes message identifiers.
src/Connection/ImapTokenizer.php	Parses binary literals.
src/Connection/ImapConnection.php	Implements revised IMAP operations.
src/Connection/ConnectionInterface.php	Defines the v2 connection API.
src/Authenticator.php	Defines SASL authenticators.
src/Authentication/XOAuth2.php	Implements XOAUTH2 authentication.

Review details

• Files reviewed: 47/47 changed files
• Comments generated: 4
• Review effort level: Balanced

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[Copilot]

🟡 Changes recommended

Synchronization state, unsolicited response filtering, capability caching, and 64-bit checkpoint handling remain incorrect.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Review details

Suppressed comments (4)

Previously missed (3) — in code that hasn't changed since the last review.

src/Mailbox.php:83

• The clone still inherits the cached server capabilities even though it opens a new connection. Capability sets can differ after reconnecting (for example across failover nodes or authentication state), so the clone may skip required checks or attempt unsupported QRESYNC/CONDSTORE operations. Clear $capabilities with the other connection-specific fields.

This issue also appears on line 194 of the same file.
src/SelectionResult.php:62

• IMAP modification sequences are unsigned 64-bit values, but casting HIGHESTMODSEQ to PHP's signed int corrupts valid checkpoints above 9223372036854775807 (for example, 18446744073709551615 saturates to PHP_INT_MAX). Such checkpoints cannot round-trip into CHANGEDSINCE/UNCHANGEDSINCE. Preserve them as decimal strings consistently across selection results, fetched data, query methods, and modifiers.
  src/StoreResult.php:31
• Every untagged FETCH response is exposed as a successfully changed message, including unsolicited updates for messages outside the STORE set. This makes messages() contradict its contract and can cause callers to apply unrelated state. Pass the command's set/addressing mode into result parsing and filter these entries, leaving unsolicited responses available through responses().

src/Mailbox.php:197

• Disconnecting clears the enabled and selection caches but leaves $capabilities cached for the next connection. A subsequent connection can therefore make capability decisions from the previous session, despite capabilities being connection/server-state dependent. Reset the capability cache here as well.

            $this->connection = null;
            $this->selected = null;
            $this->selection = null;
            $this->enabled = [];

• Files reviewed: 49/49 changed files
• Comments generated: 2
• Review effort level: Balanced

[stevebauman]

Addressed the latest review in 0844b7b, including the pending password-aware reconnect and explicit login/xoauth2 configuration changes from our discussion.

The additional findings in the review summary were checked as well:

• Capability caches now reset on both disconnect and clone.
• STORE uses the same requested-set filtering as FETCH, while retaining unrelated updates in responses().
• The unsigned 64-bit checkpoint recommendation is based on the superseded specification. RFC 7162 section 3.1 explicitly changed modification sequences to unsigned 63-bit values. The maximum, 9223372036854775807, round-trips through the existing integer API on 64-bit PHP, so I kept that API unchanged: https://datatracker.ietf.org/doc/html/rfc7162#section-3.1

QRESYNC is now checked as an enabled session capability, not merely an advertised one. ENABLE must happen before folder selection; issuing it inside an already-selected query would violate the protocol.

Verified: 625 tests pass, including the live-server suite, and formatting is clean.

The separate Laravel adapter remains unchanged; its plain default will need to become login when adopting this v2 configuration.

[Copilot]

🟡 Changes recommended

MODSEQ precision, QRESYNC validation, and failed-selection state handling can currently produce incorrect synchronization behavior.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Review details

• Files reviewed: 70/70 changed files
• Comments generated: 6
• Review effort level: Balanced

[Copilot]

Copilot review overview

🟡 Changes recommended

STORE state propagation, parameter validation, and command-injection risks remain unresolved.

Get a fresh assessment by requesting another Copilot review.

Review effort: Balanced
Findings: 1

Open (1)

• Validate STATUS items to prevent command injection · New

Resolved since last review (1)

• EXAMINE incorrectly permits ENABLE

Previously missed (2)

In code that hasn't changed since last review

Merge STORE results before applying local flag changes

src/​Message.php:184

The StoreResult is discarded, so any MODSEQ (or other attributes) returned for this STORE never reaches the message. After a flag change, modSequence() can therefore remain at the old checkpoint and cause a subsequent conditional update to conflict. Merge the returned fetched data before applying the local flag change.

Validate sequenceMatch structure before emitting QRESYNC

src/​Selection/​QuickResync.php:44

$sequenceMatch is only constrained by PHPDoc, so values such as [], a one-element array, or sets with different cardinalities are serialized into malformed QRESYNC parameters. Validate that this is exactly a pair of non-empty sets with equal numbers of sequence numbers and UIDs (and that the message-number set is ascending) before emitting it.

[Copilot]

Copilot review overview

🟡 Changes recommended

Valid QRESYNC sequence-match data is rejected when known UIDs are omitted.

Review effort: Balanced
Findings: 1

Open (1)

• Rejects valid QRESYNC sequence-match-only selections · New

Resolved since last review (1)

• Validate STATUS items to prevent command injection

[Copilot]

Copilot review overview

🔵 Needs a closer look

New synchronization modifiers can serialize invalid negative or out-of-range checkpoint values.

Review effort: Balanced
Findings: None

Resolved since last review (1)

• Rejects valid QRESYNC sequence-match-only selections

Previously missed (3)

In code that hasn't changed since last review

Reject negative checkpoints before serializing CHANGEDSINCE

src/​Fetch/​ChangedSince.php:24

Negative checkpoints are accepted and serialized as CHANGEDSINCE -1, which is not a valid IMAP modification-sequence value and guarantees a server-side command failure. Reject negative values when constructing the modifier so invalid checkpoints fail before any command is sent.

Validate QRESYNC UIDVALIDITY and modification sequence ranges

src/​Selection/​QuickResync.php:22

The QRESYNC checkpoint fields are inserted directly into the command, so values such as UIDVALIDITY 0/4294967296 or a negative modification sequence produce an invalid QRESYNC parameter. Validate UIDVALIDITY against the nonzero 32-bit range and reject negative checkpoints before serialization.

Validate modification sequences before serializing UNCHANGEDSINCE

src/​Store/​UnchangedSince.php:17

This modifier currently emits negative values verbatim (for example, UNCHANGEDSINCE -1), although modification sequences cannot be negative. Validate the checkpoint locally to avoid sending a command the server must reject.

[Copilot]

Copilot review overview

🟡 Changes recommended

The fake synchronization path accepts negative checkpoints that the production implementation rejects.

Review effort: Balanced
Findings: 1

Open (1)

• FakeMailbox accepts negative checkpoints unlike production · New

[Copilot]

Copilot review overview

🔵 Needs a closer look

Message-set filtering becomes quadratic for inputs containing many compact ranges.

Review effort: Balanced
Findings: None

Resolved since last review (1)

• FakeMailbox accepts negative checkpoints unlike production

Previously missed (1)

In code that hasn't changed since last review

Avoid quadratic range membership checks in response filtering

src/​Support/​MessageSetMatcher.php:61

contains() linearly scans every stored range for each FETCH/STORE response. Because Str::set() compresses adjacent IDs, a sparse set made of many short runs (for example, 1:2,4:5,...) creates many ranges and filtering all matching responses becomes O(ranges × responses), quadratic rather than the PR's intended linear filtering. Index or merge sorted intervals and use binary search (or a coordinated sweep) for range membership.

[Copilot]

Copilot review overview

🔵 Needs a closer look

Selection-state correctness and testing-fake parity remain unresolved.

Review effort: Balanced
Findings: None

Previously missed (3)

In code that hasn't changed since last review

Direct connection selection leaves Mailbox selection cache stale

src/​Folder.php:90

Calling the public connection-level select()/examine() API can change the server's selected mailbox without updating Mailbox's cache. Because messages() no longer forces selection, a later query can then run against the wrong mailbox (for example: select INBOX through Folder, examine Archive through connection(), then query INBOX). Either keep the forced selection here or centralize/invalidate selection state for direct connection commands.

Fake disconnect and clone retain connection-specific state

src/​Testing/​FakeMailbox.php:103

Only reconnect() clears the fake's selected/examined and enabled-capability state. disconnect() remains a no-op and cloning preserves all of this connection-specific state, unlike Mailbox and the PR's stated disconnect/reconnect/clone behavior. Tests using a disconnected or cloned fake can therefore pass with capabilities and selections that a real mailbox has already discarded; share this reset logic with disconnect() and __clone().

Fake synchronization ignores configured fetch items

src/​Testing/​FakeMessageQuery.php:90

The fake synchronization path ignores the query's configured fetch items and always returns FLAGS and MODSEQ. For example, only(MessageData::bodyStructure())->changesSince(...) includes body structure in production but omits it in the fake, while only(MessageData::modSequence()) unexpectedly includes flags only in the fake. Build the fake attribute map from $this->fetchItems (using FLAGS only when no items were configured), while retaining UID/MODSEQ protocol fields.