Skip to content

ImapEngine v2.0 - #190

Open
stevebauman wants to merge 90 commits into
masterfrom
v2.0
Open

stevebauman wants to merge 90 commits into
masterfrom
v2.0

Conversation

@stevebauman

Copy link
Copy Markdown
Member

This PR brings the work on the v2.0 branch into master for the next major release of ImapEngine.

I've expanded the library to support applications that need to keep a local mailbox in sync, listen for mailbox changes, and fetch only the message data they need.

What's changed

  • Incremental synchronization with CONDSTORE and QRESYNC, including modification sequences, conditional flag updates, and vanished messages.
  • Composable message fetch items for headers, bodies, envelopes, and synchronization metadata.
  • Separate UID ordering and server-side message sorting APIs, with UTF-8-aware SEARCH and SORT handling.
  • Special-use folder resolution for folders such as Sent, Drafts, and Trash using server-provided attributes.
  • Typed mailbox events through events(), while idle() remains focused on incoming messages and poll() provides periodic message checks.
  • APPEND results containing server-assigned UID information when available, support for message internal dates, and targeted UID EXPUNGE.
  • Partial body fetching and chunked attachment streams so applications can read sections without downloading the entire message into memory.
  • Explicit capability and folder-selection results, updated authentication handling, and corresponding support in the library's fakes.

Breaking changes

This is a major release and is not a drop-in replacement for v1. Notable changes include:

  • PHP 8.2 or later on a 64-bit platform is required.
  • Authentication configuration uses login and xoauth2 instead of plain and oauth.
  • Message fetch configuration and ordering use the new fetch-item and ordering APIs.
  • Mailbox capabilities and several connection operations return structured objects instead of their previous values.
  • ImapFetchIdentifier has been renamed to ImapIdentifier, and connection method signatures have changed.
  • Folder flags are now named attributes.

Before release

I'm opening this PR for a complete review of the v2 changes before tagging a stable release. The remaining release work is to review the public APIs and upgrade documentation, run end-to-end checks against a real IMAP server, and validate the Laravel integration against this branch. Any issues found during that integration work can be addressed here before v2 ships.

stevebauman and others added 30 commits September 1, 2026 09:53
Improve APPEND results and safely expunge specific messages
…tems

Replace per-field message fetch state with composable fetch items
Simplify message ordering and server-side sorting
…ders

Add support for resolving special-use folders

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🔵 Needs a closer look

The major API rewrite still has unresolved protocol-range, memory-exhaustion, and fake-cloning correctness issues.

Review effort: Balanced
Findings: 4 High severity · 1 Medium severity

Open (5)
What changed in this PR

Introduces ImapEngine v2 with synchronization, composable fetching, typed mailbox events, and updated connection APIs.

Changes:

  • Adds CONDSTORE/QRESYNC synchronization, structured results, sorting, and special-use folders.
  • Adds IDLE events, polling improvements, partial body fetching, and streamed attachments.
  • Updates authentication, fakes, integration infrastructure, and PHP requirements.
File Description
.github/​workflows/​run-integration-tests.yml Tests GreenMail and Dovecot.
composer.json Requires 64-bit PHP 8.2+.
docker-compose.yml Pins and configures IMAP servers.
readme.md Documents partial body fetching.
src/​AppendResult.php Represents APPENDUID results.
src/​Attachment.php Streams attachment saves.
src/​Authentication.php Implements SASL exchanges.
src/​Authentication/​XOAuth2.php Adds XOAUTH2 authentication.
src/​AuthenticatorInterface.php Defines authenticator contract.
src/​Capabilities.php Adds capability collection.
src/​Capability.php Adds capability value object.
src/​Collections/​FetchedResponseCollection.php Filters parsed FETCH responses.
src/​Collections/​FolderCollection.php Resolves special-use folders.
src/​Collections/​ResponseCollection.php Filters FETCH and VANISHED responses.
src/​Collections/​VanishedCollection.php Filters vanished UID groups.
src/​Connection/​CommandPart.php Validates command fragments.
src/​Connection/​IdleSession.php Manages IDLE protocol sessions.
src/​Connection/​ImapCommand.php Validates literals and redaction.
src/​Connection/​ImapQueryBuilder.php Adds literal-aware search tokens.
src/​Connection/​ImapTokenizer.php Refactors buffering and binary literals.
src/​Connection/​Loggers/​FakeLogger.php Adds logger test double.
src/​Enums/​ImapIdentifier.php Renames identifier enum.
src/​Enums/​ImapSpecialUse.php Defines special-use attributes.
src/​Enums/​SortDirection.php Defines sort directions.
src/​Fetch/​ChangedSince.php Adds CHANGEDSINCE modifier.
src/​Fetch/​ModifierInterface.php Defines FETCH modifiers.
src/​FetchedMessageData.php Models parsed FETCH data.
src/​FetchedResponse.php Associates FETCH data with responses.
src/​FetchResult.php Returns fetched and vanished data.
src/​FileMessage.php Implements updated message contract.
src/​Folder.php Adds events, selection results, and UID expunge.
src/​FolderData.php Defines LIST return data.
src/​FolderDataItemInterface.php Defines folder-data contract.
src/​FolderInterface.php Expands folder API.
src/​FolderRepository.php Supports extended LIST data.
src/​FolderRepositoryInterface.php Adds folder-data selection.
src/​Idle.php Delivers new messages from events.
src/​Idle/​Events/​EventInterface.php Defines event contract.
src/​Idle/​Events/​FolderSelected.php Represents selection events.
src/​Idle/​Events/​MessageExpunged.php Represents EXPUNGE events.
src/​Idle/​Events/​MessageFetched.php Represents FETCH events.
src/​Idle/​Events/​MessagesExist.php Represents EXISTS events.
src/​Idle/​Events/​MessagesVanished.php Represents VANISHED events.
src/​Idle/​Events/​ResponseEvent.php Provides response event base.
src/​Idle/​Events/​UnknownEvent.php Preserves unknown events.
src/​ImapSort.php Models server-side sorting.
src/​MailboxInterface.php Expands mailbox contract.
src/​MessageData.php Creates composable FETCH items.
src/​MessageData/​Attribute.php Defines fixed FETCH attributes.
src/​MessageData/​Body.php Adds body sections and partials.
src/​MessageData/​FetchItemInterface.php Defines FETCH item contract.
src/​MessageInterface.php Adds MODSEQ and partial bodies.
src/​MessageQueryInterface.php Redesigns querying and ordering.
src/​Poll.php Tracks arrivals across reconnects.
src/​Selection/​CondStore.php Adds CONDSTORE selection option.
src/​Selection/​OptionInterface.php Defines selection options.
src/​Selection/​QuickResync.php Adds QRESYNC selection parameters.
src/​Selection/​RequiresEnablementInterface.php Marks enablement-dependent options.
src/​Selection/​Result.php Models selection metadata.
src/​SortCriterion.php Models SORT criteria.
src/​Store/​ModifierInterface.php Defines STORE modifiers.
src/​Store/​UnchangedSince.php Adds conditional STORE support.
src/​StoreResult.php Models STORE outcomes.
src/​Support/​MessageSetMatcher.php Matches compact message sets.
src/​Support/​Str.php Adds protocol validation and sets.
src/​Testing/​FakeFolder.php Expands fake folder behavior.
src/​Testing/​FakeFolderRepository.php Adds fake LIST data support.
src/​Testing/​FakeMailbox.php Adds fake capabilities and selection.
src/​Testing/​FakeMessage.php Adds fake modification sequences.
src/​Testing/​FakeMessageQuery.php Adds fake synchronization and ordering.
src/​UidOrder.php Models local UID ordering.
src/​Vanished.php Parses VANISHED responses.
src/​Watch.php Produces typed IDLE events.
tests/​Integration/​ExpungeTest.php Tests targeted expunge.
tests/​Integration/​FoldersTest.php Updates folder integration coverage.
tests/​Integration/​IdleTest.php Tests real IDLE behavior.
tests/​Integration/​MailboxTest.php Updates capability assertions.
tests/​Integration/​SynchronizationTest.php Tests synchronization workflows.
tests/​IntegrationTestCase.php Cleans up integration mailboxes.
tests/​Pest.php Registers integration setup.
tests/​Support/​IdleAcknowledgementLogger.php Coordinates real IDLE tests.
tests/​Unit/​AppendResultTest.php Tests APPEND results.
tests/​Unit/​ArchitectureTest.php Enforces interface naming.
tests/​Unit/​AuthenticationTest.php Tests SASL and XOAUTH2.
tests/​Unit/​CapabilitiesTest.php Tests capability objects.
tests/​Unit/​Connection/​CommandPartTest.php Tests command validation.
tests/​Unit/​Connection/​FakeLoggerTest.php Tests fake logging.
tests/​Unit/​Connection/​ImapCommandTest.php Tests literals and redaction.
tests/​Unit/​Connection/​ImapConnectionFilteringTest.php Tests response filtering.
tests/​Unit/​Connection/​ImapQueryBuilderTest.php Tests UTF-8 search compilation.
tests/​Unit/​Fetch/​ChangedSinceTest.php Tests FETCH modifiers.
tests/​Unit/​FetchedResponseTest.php Tests FETCH response parsing.
tests/​Unit/​FolderRepositoryTest.php Tests special-use folders.
tests/​Unit/​IdleTest.php Tests arrival tracking.
tests/​Unit/​MailboxCapabilitiesTest.php Tests capability lifecycle.
tests/​Unit/​MailboxTest.php Tests reconnect and authentication.
tests/​Unit/​MessageCursorTest.php Tests lazy batched fetching.
tests/​Unit/​MessageDataTest.php Tests composable FETCH items.
tests/​Unit/​PartialBodyTest.php Tests partial and streamed bodies.
tests/​Unit/​PollTest.php Tests polling lifecycle.
tests/​Unit/​Selection/​QuickResyncTest.php Tests QRESYNC validation.
tests/​Unit/​Store/​UnchangedSinceTest.php Tests conditional STORE validation.
tests/​Unit/​Support/​IdleAcknowledgementLoggerTest.php Tests IDLE acknowledgement tracking.
tests/​Unit/​Support/​MessageSetMatcherTest.php Tests compact set matching.
tests/​Unit/​Support/​StrTest.php Tests protocol string helpers.
tests/​Unit/​Testing/​FakeFolderRepositoryTest.php Tests fake special-use lookup.
tests/​Unit/​Testing/​FakeFolderTest.php Updates fake folder tests.
tests/​docker/​dovecot.conf Enables test-server cleartext login.
tests/​wait-for-mailbox.php Waits for integration server readiness.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread src/FetchedMessageData.php
Comment thread src/Selection/Result.php
Comment thread src/Support/Str.php Outdated
Comment thread src/Testing/FakeMailbox.php
Comment thread src/MessageData/Body.php

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comment thread src/Connection/CommandArgument.php Outdated
Comment thread src/Folder.php
Comment thread src/Idle.php
Comment thread src/Testing/FakeFolderRepository.php Outdated

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

EXPUNGE response filtering, IDLE timeout restoration, and compact VANISHED range filtering have unresolved correctness or reliability issues.

Review effort: Balanced
Findings: 1 High severity

Open (1)
Resolved since last review (5)
Previously missed (2)

In code that hasn't changed since last review

Medium severity Avoid expanding non-overlapping compact VANISHED ranges

src/​Collections/​VanishedCollection.php:27

Determining whether each filtered group is empty can exhaust a compact server range one UID at a time. For example, filtering VANISHED 1:4294967295 against a non-overlapping exact request iterates billions of values before returning, so a compact response can pin the process despite bounded memory. Intersect compact ranges before expansion and only expose the intersection lazily.

Medium severity Restore the socket timeout after IDLE completes

src/​Connection/​IdleSession.php:167

Each IDLE read overwrites the socket timeout, but the previous configured timeout is never restored when the session finishes successfully. The next normal IMAP command therefore inherits the last renewal/completion timeout and may fail prematurely or block longer than configured. Preserve the connection’s prior read timeout and restore it when IDLE ends.

Comment thread src/Folder.php
Comment on lines +225 to 227
return $this->mailbox->connection()->expunge($uids)->map(
fn (UntaggedResponse $response) => $response->tokenAt(1)->value
)->all();
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants