Harden Testing correctness, parallel ownership, and parity - #495
Merged
Conversation
Track the exact tokens whose setup callbacks are entered and use that bounded local ownership set for teardown. Preserve the protected setup-loop extension point while rebuilding fresh applications for cleanup and retaining setup or runner failures over later teardown errors. Add counterfactual coverage for partial setup, custom token loops, unattempted tokens, runner suppression, application freshness, ordering, resolver restoration, cleanup exhaustion, and both primary-error precedence rules.
Run every registered process and test-case teardown callback in registration order even when an earlier callback fails, then rethrow the first failure. Keep setup callbacks fail-fast and retain the existing parallel-mode boundary and callback parameters. Expand focused coverage for single and competing teardown failures, callback order, first-error precedence, and the unchanged setup behavior.
Give artisan test one explicit ownership boundary spanning profile allocation, temporary PHPUnit configuration, process execution, reporting, and cleanup. Release every owned file and directory on all terminal paths while preserving the operation failure or first cleanup failure. Require complete configuration and profile writes instead of accepting failed or partial publication. Add deterministic coverage for allocation, signal, process, reporting, coverage, publication, and competing cleanup failures, including exact worker-clone file restoration.
Restore numeric-string forbidden-output keys before strict matcher calls and clear all shared expectation state plus the OutputStyle binding after every attempted command. Remove the behaviorally inert expectedTables state so one command cannot contaminate the next. Implement one-shot dd output through BufferedOutput, publish the real command exit code, and prevent destructor replay. Add sequential failure and reuse regressions plus a subprocess fixture proving captured output, exit reporting, stdout delivery, and exactly one execution, and document the public debugging API.
Add the shared rendered-HTML constraint used by TestView and TestComponent while preserving the separate raw ordered assertion path. Retain valid string zero, normalize Unicode whitespace, fall back safely for malformed bytes, reject textless expectations, and emit correctly composed PHPUnit diagnostics. Restore the current fluent View and Component assertion surface and truthful list annotations. Revalidate both Mailable ordered text callers with counterfactual zero-value coverage and document the complete component testing API.
Restore rendered-text, bulk JSON-path, canonicalized-path, missing-path, and missing flashed-input APIs on TestResponse. Preserve logged-exception, redirect, and JSON diagnostic precedence without allowing unsupported context values to replace the original assertion failure. Accept every valid JSON root, validate exact JSON whitespace, retain stored response-exception precedence, and memoize one decoded wrapper per response without disturbing streamed content. Preserve keyed-null session presence semantics while grouping ordinary diagnostics, add comprehensive counterfactual coverage, and update the canonical HTTP testing guide.
Require each discovered registrar to explicitly define a callable public static register method before invoking it. Keep class discovery and registration simple without reflection, instantiation, caching, or a new interface. Cover public instance methods, non-public static methods, abstract declarations, magic-only classes, and inherited public static registrars with contextual failure assertions.
Detect Closure expectations directly instead of constructing and filtering an intermediate collection. Preserve the existing matching contract while removing an unnecessary test-only allocation and an unqualified class-name string. Retain and type the complete fluent assertion coverage so scalar, array, nested, callable, and interaction behavior remains pinned.
Consume class_uses_recursive results directly instead of flipping the trait map before membership checks. This preserves database test behavior while removing a redundant allocation from test setup. Keep the focused database concern coverage fully typed and verify the existing migration and transaction classification paths remain unchanged.
Make touched Testing fixtures preserve whether process-global values existed and restore their exact prior value after every path, including falsey cache flags. Complete the remaining direct test method return types without changing providers or inherited lifecycle methods. Keep assertion, deprecation, cache, view, and parallel console coverage isolated under the existing framework test bases with no production reset hook or fixture manager.
Remove ext-json and ext-hash requirements and suggestions from the root and every affected active split manifest because supported PHP versions guarantee both facilities. Retain ext-filter and every other extension that a supported runtime may actually omit. Update package metadata regressions and completed design records so they assert the real platform boundary instead of preserving stale extension inventory. This changes no runtime behavior or package API.
Declare Testing's direct DOM, mbstring, DI, and Carbon boundaries and publish accurate ParaTest and PHPUnit suggestions. Record the supported ParaTest 7 getExitCode omission and current Laravel provenance without duplicating canonical user documentation. Add executable metadata coverage for root constraint alignment, provider discovery, direct dependencies, suggestion ranges, and the absence of guaranteed-core extension inventory across every active split manifest.
Route the active audit work unit to Testing, add collision-free testing-03 through testing-16 dependency rows, close every revalidated Testing-pending edge, and mark the package complete. Preserve the carried Sentry records and coroutine-08 obligation while restoring precise Database, View, Testbench, and Foundation ownership states. Record the completed Testing implementation, Mail assertion revalidation, metadata coverage boundaries, public documentation, API and performance effects, rejected machinery, and final verification. Remove stale later-Testing prose without rewriting historical implementation records.
Record the final evidence-backed design for parallel process ownership, teardown exhaustion, command resources, assertion parity, JSON roots, registrar validation, metadata, test typing, and fixture restoration. Include the retained core anti-overengineering rules, complete implementation and counterfactual test boundaries, Laravel API and ParaTest compatibility decisions, hot-path assessment, rejected alternatives, cross-package routing, and finalized audit-record requirements. The plan reflects the signed-off implementation without retaining superseded proposals or review history.
…ess-parity-lifecycles # Conflicts: # docs/plans/2026-07-12-0900-framework-coroutine-state-lifecycle-audit.md
Use strict boolean searches when verifying forbidden command output so the implementation no longer depends on loose comparison semantics. Make the test command fixture teardown exhaustive across partial setup, generated profile files, configuration restoration, argv restoration, and parent cleanup while preserving the first failure. Restrict the permission fixture to owner access without changing its failure trigger. Include subprocess stderr in every PendingCommand debug regression assertion so bootstrap failures remain diagnosable in CI.
Clarify that phpstan.types.neon.dist owns only the committed types fixtures and must not be redirected at source or test files. This prevents targeted analysis from overriding the configuration boundary and reporting irrelevant level-max errors for code outside the type-contract suite.
Clarify that any isolated scratch directory should use ParallelTesting::tempDir(), regardless of the base test case. Move every TestCommandTest scratch project onto worker- and process-owned paths. Reuse stable, delete-first paths where leftovers remain writable, retain a unique path for the deliberately read-only fixture, and keep all fixture directories owner-only. This removes group-writable executable test content without adding shared fixture machinery. Document the exact CLI argv snapshot as a nullable string list while preserving absent-versus-present state. The focused TestCommand suite and formatter both pass.
|
Important Review skippedAuto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Pro Plus Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR completes the Testing correctness and parity work. It makes parallel process ownership failure-safe, exhausts teardown without hiding the primary failure, closes test-command resource leaks, restores current Laravel testing APIs, and fixes assertion paths that could skip valid values or produce misleading diagnostics.
It also makes
PendingCommandreusable after failures, adds one-shot command debugging, accepts every valid JSON root while memoizing response decoding, validates test-state registrars at discovery time, and restores exact test fixture state. Guaranteed PHP core facilities are no longer declared as optional Composer extensions.Application request paths are unchanged. Runtime changes are limited to test execution and assertions, with repeated JSON decoding replaced by a per-response cache.
For more details, see:
docs/plans/2026-08-08-2225-testing-correctness-parallel-ownership-and-current-parity.mdParallel ownership and cleanup
Test command resources
artisan testone explicit ownership boundary for temporary configuration, profile, and coverage resources.ParallelTesting::tempDir()paths.Assertions and response parity
0expectations, malformed byte behavior, Unicode whitespace normalization, ordering, negation, and precise PHPUnit diagnostics.nullJSON roots.Console testing
PendingCommand::dd()with buffered output, the real exit code, and one-shot execution so destruction cannot rerun the command.Metadata and maintenance
ext-jsonandext-hashdeclarations from active manifests because supported PHP versions guarantee both facilities.ParallelTesting::tempDir(), regardless of its base test class.Compatibility and performance
Supported Laravel testing APIs are restored or extended without changing their public signatures. Hypervel coroutine execution, streamed responses, HEAD behavior, worker-local Testbench clones, and centralized static cleanup remain intact. The existing ParaTest 7-specific omission of the Laravel
getExitCode()helper remains documented because the supported runner already returns the final exit code directly.Parallel bookkeeping is a local token list bounded by the configured process count. Exhaustive work occurs only during teardown or failure handling. There are no new locks, registries, retries, context slots, production middleware, or application hot-path operations.
Validation
composer fix.0.4changes.