Skip to content

fix(scripts): make PowerShell scripts parse under Windows PowerShell 5.1 - #127

Draft
Evgenii (Vaiz) wants to merge 1 commit into
mainfrom
u/vaiz/2026/09/01/ps1-ascii-and-ps5-compat
Draft

fix(scripts): make PowerShell scripts parse under Windows PowerShell 5.1#127
Evgenii (Vaiz) wants to merge 1 commit into
mainfrom
u/vaiz/2026/09/01/ps1-ascii-and-ps5-compat

Conversation

@Vaiz

Copy link
Copy Markdown
Contributor

🤖 Clawpilot here! Posted automatically by Clawpilot (an AI agent), not by a human. Please verify before acting.

The SDL scan's 14 PSScriptAnalyzer findings are parse failures, not style lints: MissingEndCurlyBrace, UnexpectedToken, MissingTypename, IncompleteHashLiteral, MissingCatchOrFinally, TerminatorExpectedAtEndOfString. They are reported at Note severity so they do not break the build (Results below minimum severity: 15), which is why they have gone unexamined.

They have two unrelated causes. Parsing each file with both engines separates them:

Script Windows PowerShell 5.1 PowerShell 7.6
release-crate.ps1 6 errors 0
test-anvil-container.ps1 4 errors 0
test-anvil-dogfood.ps1 4 errors 0

Cause 1 — encoding, not syntax

The scripts are UTF-8 with no BOM, so Windows PowerShell decodes them as Windows-1252. Emoji bytes then land in the range 0x91-0x94, which in CP1252 are the curly quotes ‘ ’ “ ” — and PowerShell accepts those as string delimiters.

release-crate.ps1 contains six such bytes, the first in the changelog header map at line 72. From there the parser loses quote pairing, and every subsequent error is a cascade. Line 433, which Guardian reports as Unexpected token ')' and Missing type name after '[', is plain ASCII and entirely correct — I dumped its codepoints to confirm.

Proof it is decoding and not syntax, both under 5.1:

ParseInput as UTF-8            -> 0 error(s)
ParseInput as ANSI/1252        -> 6 error(s)
ParseFile (what the tool does) -> 6 error(s)

run-examples.ps1 uses /, whose UTF-8 bytes also include 0x93 and 0x97. It parses today only because the resulting quotes happen to pair up — latent, not safe.

Cause 2 — ?? is PowerShell 7 only

test-anvil-container.ps1 and test-anvil-dogfood.ps1 use the null-coalescing operator. That is genuine syntax the 5.1 tokenizer cannot accept, and no encoding change helps it. This is why fixing the emoji alone would have left 8 of the 14 findings in place.

The fix

  • Replace all non-ASCII with ASCII across scripts/.
  • Replace ?? '' with a [string] cast. [string]$null is the empty string, so behaviour is identical; verified ([string]$null) -ceq ($null ?? '') is True. The scripts remain valid PowerShell 7.

Effects

  • All eight scripts under scripts/ now parse with 0 errors under both Windows PowerShell 5.1 and PowerShell 7.6, and contain 0 non-ASCII bytes.
  • The 14 parse-error findings should disappear from the SDL scan. The one real lint, PSAvoidUsingInvokeExpression at release-crate.ps1:275, is untouched and still reported — deliberately, since it is a separate judgement call.
  • One substantive change: the changelog header map loses its emoji prefixes, so future generated changelog sections read ### Features rather than ### ✨ Features. No committed CHANGELOG.md uses the emoji form today, so nothing becomes inconsistent.
  • No behavioural change otherwise; console messages lose decorative prefixes only.

Alternative considered

A UTF-8 BOM also fixes cause 1, with zero content change, and I verified it does (release-crate.ps1 with a BOM: 6 errors -> 0 under 5.1). I chose ASCII because a BOM can be silently dropped by an editor or a tool that rewrites the file, reintroducing this without warning, and because the emoji are console output on a host whose default console encoding cannot render them anyway. Happy to switch to the BOM if you would rather keep the emoji.

Verification

All parse results above are from the real System.Management.Automation.Language.Parser on both installed engines, before and after. This changes no Rust code and no pipeline config.

Independent of #126, which unblocks the actual build break.

/cc Martin Taillefer (@geeknoid)

The SDL scan reports 14 PSScriptAnalyzer findings across three scripts:
MissingEndCurlyBrace, UnexpectedToken, MissingTypename,
IncompleteHashLiteral, MissingCatchOrFinally,
TerminatorExpectedAtEndOfString. They are parse failures, not style
lints, and they have two unrelated causes. Parsing each file with both
engines separates them:

                              WinPS 5.1   PS 7.6
    release-crate.ps1             6          0
    test-anvil-container.ps1      4          0
    test-anvil-dogfood.ps1        4          0

1. Encoding. The scripts are UTF-8 with no BOM, so Windows PowerShell
   decodes them as Windows-1252. The emoji then land as CP1252 bytes
   0x91-0x94, which are the smart quotes, and PowerShell accepts those
   as string delimiters. release-crate.ps1 contains six such bytes, the
   first at the changelog header map, and the parser loses quote
   pairing from there on. Every reported line is a cascade: line 433 is
   plain ASCII and correct. Re-decoding the same bytes as UTF-8 yields
   zero errors, confirming the syntax was never wrong.

   run-examples.ps1 uses the check and cross marks, whose UTF-8 bytes
   also include 0x93 and 0x97, and parses today only because the
   resulting quotes happen to pair up.

2. `??`. test-anvil-{container,dogfood}.ps1 use the null-coalescing
   operator, which is PowerShell 7 only. This is genuine syntax the 5.1
   tokenizer cannot accept, and no encoding change helps.

Fix both. Replace non-ASCII with ASCII, and replace `?? ''` with a
`[string]` cast: casting $null to string yields the empty string, so
the behaviour is identical on both engines. The scripts remain valid
PowerShell 7.

A UTF-8 BOM also fixes cause 1 with no content change, and was
verified to do so. ASCII is preferred here because it cannot regress
through an editor that drops the BOM, and because the emoji are console
output on a host whose default console encoding cannot render them.

The one substantive change is the changelog header map, which loses its
emoji prefixes. No committed CHANGELOG.md uses them today.

After this change all eight scripts under scripts/ parse with zero
errors under both Windows PowerShell 5.1 and PowerShell 7.6, and
contain no non-ASCII bytes.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI lite review requested due to automatic review settings September 1, 2026 08:23
@codecov-commenter

Codecov Comments Bot (codecov-commenter) commented Sep 1, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 97.7%. Comparing base (17f0aea) to head (6801537).

❌ Your project status has failed because the head coverage (97.7%) is below the target coverage (100.0%). You can increase the head coverage or adjust the target coverage.

Additional details and impacted files
@@          Coverage Diff          @@
##            main    #127   +/-   ##
=====================================
  Coverage   97.7%   97.7%           
=====================================
  Files        286     286           
  Lines      62460   62460           
=====================================
  Hits       61033   61033           
  Misses      1427    1427           
Flag Coverage Δ
linux 97.6% <ø> (?)
linux-arm 97.6% <ø> (?)
windows 97.9% <ø> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR updates the repository’s PowerShell automation scripts to parse cleanly under Windows PowerShell 5.1 by removing non-ASCII characters and replacing PowerShell 7–only syntax, while keeping the scripts compatible with PowerShell 7.

Changes:

  • Replaced the PowerShell 7 null-coalescing operator (??) with a Windows PowerShell 5.1–compatible alternative in Invoke-Native.
  • Replaced non-ASCII characters (emoji, typographic punctuation) with ASCII equivalents across the touched scripts.
  • Simplified console output markers in scripts to avoid non-ASCII output text.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 2 comments.

File Description
scripts/test-anvil-dogfood.ps1 Replaces ?? '' with [string](Get-Content ...) for PS 5.1 parsing compatibility.
scripts/test-anvil-container.ps1 Replaces ?? '' usage, removes non-ASCII punctuation, and adjusts a design-doc reference in comments.
scripts/run-examples.ps1 Replaces non-ASCII pass/fail markers with ASCII PASS: / FAIL: messages.
scripts/release-crate.ps1 Removes emoji from changelog header mapping and status messages to avoid non-ASCII bytes.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread scripts/test-anvil-container.ps1
Comment thread scripts/release-crate.ps1
@Vaiz
Evgenii (Vaiz) enabled auto-merge (squash) September 1, 2026 09:27
Comment thread scripts/release-crate.ps1
@@ -69,17 +69,17 @@ $script:TypeGroupMapping = @{

# Maps the final group key to a user-friendly header in the changelog.
$script:HeaderNameMapping = @{

@sandersaares Sander Saares (sandersaares) Sep 1, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This change is fundamentally wrong. We have no need to support PowerShell 5. If some infrastructure requires PowerShell 5 dependency, that infrastructure needs to be fixed. This limitation of being PowerShell 5 compatible is not acceptable to me without some extremely strong rationale.

@Vaiz
Evgenii (Vaiz) marked this pull request as draft September 1, 2026 13:16
auto-merge was automatically disabled September 1, 2026 13:16

Pull request was converted to draft

@geeknoid

Copy link
Copy Markdown
Member

I wish we'd replace all these powershell scripts with Rust scripts, so we don't need PowerShell in Linux environments.

@Vaiz

Evgenii (Vaiz) commented Sep 2, 2026

Copy link
Copy Markdown
Contributor Author

I wish we'd replace all these powershell scripts with Rust scripts, so we don't need PowerShell in Linux environments.

I support this idea, but at the moment, we have not agreed on using Rust for scripts. The main issue is that they require an installed nigthly toolchain

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants