Skip to content

Add filename normalization for macOS - #16973

Open
gmontalta wants to merge 3 commits into
modxcms:3.xfrom
gmontalta:fix-macos-file-upload-translit
Open

gmontalta wants to merge 3 commits into
modxcms:3.xfrom
gmontalta:fix-macos-file-upload-translit

Conversation

@gmontalta

Copy link
Copy Markdown

Normalize multibyte filenames to NFC for compatibility.

What changed and why

  • Names of files uploaded with Windows are transliterated correctly.
  • Names of files uploaded with macOS are in NFD form and need to be normalized to NFC form first.

How to test

  • Create a file named jäöu.jpg on your computer
  • Upload in media browser using drag&drop
  • The file should be named jaeoeu.jpg if you have german transliteration active.

Related issue(s)/PR(s)

n/a

Compatibility notes

  • Before this change, Windows worked. macOS did not.
  • After this change, both operating systems work as expected.

Breaking change assessment

No

Test coverage

No automated unit test

Contributors

n/a

AI tool use

Claude Code

Normalize multibyte filenames to NFC for compatibility.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR updates the media upload path to normalize macOS-provided filenames from NFD (decomposed Unicode) to NFC (composed Unicode) so that downstream filename handling (including transliteration) behaves consistently across operating systems.

Changes:

  • Normalize uploaded filenames to NFC using PHP Intl’s Normalizer when available.
  • Apply normalization before size checks and transliteration filtering in the upload pipeline.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@Mark-H

Mark-H commented Jul 16, 2026

Copy link
Copy Markdown
Collaborator

Just to share as I was unaware of this myself:

NFC vs NFD background

This is Unicode normalization, and it trips people up because the same visual character can be encoded two different ways:

  • NFC (Normalization Form C — "Composed"): one code point per character. ä is a single code point (U+00E4).
  • NFD (Normalization Form D — "Decomposed"): a base character plus a combining accent mark. ä becomes a (U+0061) + a combining diaeresis (U+0308) — two code points.

They render identically on screen and are "canonically equivalent" per the Unicode spec, but as raw byte sequences they're different strings. String comparison, length checks, substring matching, transliteration tables, hashing — all of that operates on bytes/code points, so NFC "ä" and NFD "ä" are not equal as far as PHP's string functions are concerned.
Why macOS specifically: HFS+ and APFS (Apple's filesystems) store and return filenames in NFD form. Windows (NTFS) and Linux (ext4, etc.) don't do this — they generally preserve whatever bytes you gave them, which in practice tends to be NFC since that's what most text input methods produce. So the exact same filename typed on both platforms arrives at the server as different byte sequences depending on which OS the file came from.
That's exactly what bit this code: the transliteration logic was presumably built/tested against NFC input (Windows-style), so when a decomposed macOS filename hit it, the accent-stripping rules didn't match and produced the wrong output.

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
@rthrash

rthrash commented Aug 14, 2026

Copy link
Copy Markdown
Member

@gmontalta it might make sense to elaborate a bit on the following for ease of testing/reviews:

if you have german transliteration active

(I was going to review this myself just now but couldn't recall the details of how to do this.)

Co-authored-by: Bochkarev Ivan <ivanx86@gmail.com>
@gmontalta

Copy link
Copy Markdown
Author

@gmontalta it might make sense to elaborate a bit on the following for ease of testing/reviews:

if you have german transliteration active

(I was going to review this myself just now but couldn't recall the details of how to do this.)

@rthrash Here are the steps:

  • Install MODX 3.2.3-pl
  • Install package "translit" translit 1.0.0-beta from opengeek
  • Set system setting "friendly_alias_translit" to "german"
  • Open MODX media browser
  • Upload a file "jäöu.jpg" via drag&drop
  • File name in media browser should be "jaeoeu.jpg" (not "jäöu.jpg")

AgelxNash pushed a commit to AgelxNash/revolution that referenced this pull request Sep 6, 2026
AgelxNash pushed a commit to AgelxNash/revolution that referenced this pull request Sep 6, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants