Add GrammarJsonServices: deterministic LCM Grammar JSON export

[johnml1135]

What this adds

The format contract:

• doc/lcm-grammar.md — the LCM Grammar JSON specification: conventions (GUID keys, determinism/ordering rules, omission rules, tagged unions, never-silent skips), per-section structure, additive-only versioning policy, and an appendix of built-in FieldWorks GUIDs (word boundary, the 17 morph types) so future importers can be self-contained.
• doc/lcm-grammar.schema.json — the machine-checkable JSON Schema (draft-07, strict additionalProperties). Enforced by unit tests: empty-project and populated-project exports must validate, so the schema and the exporter cannot drift apart.

The exporter:
SIL.LCModel.DomainServices.GrammarJsonServices.ExportGrammar(LcmCache, TextWriter, ICollection<string> warnings = null) — a deterministic JSON export of the parser-relevant subset of a project (phonology, morphology, lexicon), with the envelope {"format":"lcm-grammar","version":1,...}. It sits next to M3ModelExportServices (the existing parser-model export) and follows the same pattern: one static service class, no new project, no new runtime dependency (Newtonsoft.Json is already in the tree via SIL.LCModel.Utils; NJsonSchema is test-only, MIT).

What it's for

An interchange format for external morphological-parser tooling:

• Grammar verification — "does this change make parsing better?" pipelines that need a stable, diffable projection of the grammar.
• Conformance fixtures — a candidate storage format for an engine-agnostic parser conformance suite (see Engine-agnostic morphological-parser conformance suite (v2: typologically-selected languages) machine#454).
• Field deployment — a small, data-minimal artifact (a real ~56 MB .fwdata project exports to ~2.4 MB pretty-printed, ~250 KB gzipped) that web/wasm parsers can load without LCM.

It is a read-only projection: explicitly not an editing format, not a sync/merge format, and not a replacement for fwdata.

Design points

• Deterministic bytes: owning/reference sequences (OS/RS) keep model order (rule order, slot order, allomorph disjunctive order are semantic); unordered collections (OC/RC, repositories) sort by GUID string; multistring values are {"ws","form"} arrays sorted by tag. Two exports of the same data are byte-identical.
• GUID-keyed: every cross-reference is a FieldWorks GUID (never an Hvo), so references survive across sessions and resolve back to FLEx objects.
• Never-silent skips: data the format cannot represent (unknown morph types, dangling references, malformed parser-parameter XML) is skipped with a message in the optional warnings collection. An affix process with an unrepresentable input part is skipped whole, because its output mappings reference input parts by position.
• The field-by-field mapping follows the semantics of HCLoader/M3ModelExportServices consumption (e.g. natural-class names come from Abbreviation, notOnClitics defaults to true, the word-boundary marker is excluded from boundaryMarkers).

Tests

17 NUnit tests in GrammarJsonServicesTests (memory-only backend): schema validation of empty and populated exports, envelope/sections, byte-determinism, lexical entry round-trip, GUID sort order, phonology, parser-parameter defaults and XML parsing, affix-process export and skip-on-unrepresentable-input, morpheme ad hoc rule adjacency, entry-ref variant/complex-form discrimination, and warn-on-skip behavior.

Review notes

The exporter was additionally reviewed against the format's independent reference implementation (a Rust .fwdata reader that produces the same format) and the findings folded in — notably the entry-ref discriminator edge case (a ref carrying both variant and complex-form types is a variant) and positional-index alignment for affix-process mappings.

Beyond the reference-implementation review, the exporter was adversarially probed at runtime (hostile data driving every skip path — this found and fixed two schema-contract violations where all-items-skipped collections emitted []) and smoke-tested against a real 1,462-entry project opened through LibLCM: the export is schema-clean with zero warnings and semantically identical to the reference implementation's output. That smoke test surfaced and fixed two real-data bugs: phoneme names authored only in vernacular writing systems were silently lost (name fields now fall back analysis-then-vernacular), and output text was NFD (LCM's in-memory form) rather than the NFC of the .fwdata at-rest form — exports are now NFC, preserving the byte-reproducibility contract.

A fourth review round tested the contract itself: a "naive importer" reviewer working from the spec + schema + a real export alone (no source access) drove several contract hardenings — a normative key-order rule, a referential-integrity section stating each reference field's resolution scope (notably: sense.msa resolves document-wide, proven necessary by real Sena 3 data, which the exporter now warns about), documented featureType opacity, and a tightened complexForm schema branch. A second real-project smoke test (Amharic: 417 Ethiopic phonemes, real templatic affix processes) came back fully clean — schema-valid, zero warnings, byte-exact Ethiopic text, and structurally identical to the reference implementation.

A cross-implementation byte-equality gate against real projects (Sena 3, Amharic) is a planned follow-up.

🤖 Generated with Claude Code

---

This change is 

[hahn-kev]

What's the actual schema for this? Also is there a reason you didn't define a class to represent the shape of the data and then just call serialize on it? That feels like it would be much simpler and less error prone than using a json text writer.

[jtmaxwell3]

Jason is out of town for a week. I don't think that this should be approved until he has had a chance to look at it.

[johnml1135]

This is being closed for the time being in favor of using the HC grammar export. This may be revisited in the future, as it does hold promise in the right domain, especially for a richer basis for a conformance suite and a more configurable distribution format.