c: complete the schema table implementation

[gafferongames]

Completes the C table implementation against the current specification and C++ reference. Generated C supports id-table files, bitpacked message batches and announcements, caller-owned retention for variable regions, UnitView, JSON, native regions and builders, canonical cooks in either byte order, and const block views. Maps, lists, arrays, unions, wide scalars, defaults, aliases, shared nodes and blobs use the same public surfaces. Allocation hooks and named refusal reports cover these paths. Consumers should regenerate headers and recompile.

Shared reference repairs cover mixed alignment, collection widening, hidden union-array extents, default values above a shortened array count, message encoding work, and map occurrence/key handling. C++ map callers retain the full count and propagate refusal above the representable cap. Existing wire and cook pins are preserved; added malformed pointer-arm and map-occurrence vectors pin the repaired behavior.

Validation:

• Full Go suite, golangci-lint 2.12.2, generated-source checks, and full local test-c pass.
• The complete C ordinary and retaining rosters pass 154,842 and 85,434 mutations respectively, in both native and ASan/UBSan builds. The longer retaining run passes 183,434 mutations in each mode over all 25 roots.
• Direct C/C++ collection wire and both cook-byte-order comparisons pass 48,789 mutations in each mode. Compiled regressions and negative controls cover the repaired boundaries.
• The five C sabotage controls are distributed across three CI jobs, preserving every target and the existing timeout. The previous combined job exceeded its five-minute limit while rebuilding its third complete driver.

Rowan reviewed specification/retention parity and the reference repairs; Johnny reviewed allocation, lifetime, bounds and the subsequent fixes. Their named findings are addressed, including preserving widening across repeated message keys and explicitly naming repeated map fields in the retention occurrence rules. These are scoped reviews; optional fixture/history cleanup is recorded separately.

Pre-integration C CI: all 65 jobs passed at https://github.com/mas-bandwidth/schema/actions/runs/34187609412.

Landed alongside the completed Go port, preserving both backend implementations. Both prior heads passed CI; combined package validation and the corrected shared integration checks, lint, and negative-control manifest checks passed. Follow-up 5c679a3 removes an identical duplicate regression file missed during integration staging; the tablewire suite passes. Main CI is pending at https://github.com/mas-bandwidth/schema/actions/runs/34188466019.

[rowan-claude]

This is a cold read of head 21e354c against base 2a05db9 by an independent reader, relayed by Rowan. The head has since moved to 6e6b14b, and a second reader takes that delta separately. The reader was given two rulings as fixed: the array count is read against the enclosing reader before L bounds the elements, and the field-versus-element extent asymmetry is being fixed once under #710.

The verdict at 21e354c is merge. None of the four things that would have changed it was found: no accept-or-refuse divergence from C++, no arithmetic reaching past a bounded frame, no allocation on the fixed-class path, and the decode.go oracle change agrees with section 4 of SPEC-TABLES.

The C reader matches cpptable construct for construct. LEB128 canonicality in wire.go at lines 117 to 131 is the same algorithm as cpptable.go at 920 to 946, with the same ten-byte cap, the same rejections, and the cursor restored on refusal. The field-loop order and the reserved-id triple predicate match, including the nested qualifier, at wire_read.go line 88 against codecs.go line 1554. The array header is read against the enclosing body at wire.go 145 to 155, and C's has and room checks carry an explicit offset-not-past-size guard that C++ leaves to the caller, so C is the safer of the two and reaches the same verdict. The element-kind damage order puts malformed before the kind check, matching codecs.go line 1861. On extents, a nested field resets and counts malformed when the subreader does not reach its size, at wire_read.go line 206, while an array element keeps its partial body, matching codecs.go 2033 to 2039, which is the current reference. Union arm refusals match arms.go line 283. The widening dispatch is complete for what C can declare, since ir.RefuseWideTableKinds refuses kinds 18 and 19 in this backend. The refusal vocabulary and the root-slack pre-scan mirror TableBodyEndsEarly. Every offset increment is preceded by a has or room check, every narrowing is of a value already clamped, and the writer's id array is a compile-time-sized stack array with no malloc or alloca on the fixed path.

The oracle change is right. Line 1408 of decode.go removed a mask to the source width that was applied after the declared clamp. Section 4's WIDENED bullet, SPEC-TABLES lines 5781 to 5783, clamps against the declaration and nothing re-narrows afterward, and emitWidenedScalar in cpptable/widen.go at 140 to 176 does the same. The removal is a no-op for every non-widened read and fixes exactly the u8 255 into a uint64 with minimum 1000 case, pinned at all three unsigned widths in widen_test.go.

The evidence reproduced. Build and vet were clean, the ctable and tablewire tests passed, the native wire differential reported 47282 mutants and 0 divergences, the PR's exact number, and the ASan and UBSan differential the same. Conformance under the sanitized drivers passed wire 16 of 16, report 16 of 16, json-read 16 of 16, json-write 16 of 16, json-hostile 74 of 74, cook 6 of 6, forgery 12 of 12 and cook-forgery 113 of 113. The soak made 0 allocator calls over 1024 iterations. The soak, canonical-LEB and conformance negative controls were each red where they should be, and regenerating the C goldens was byte-identical. The full make test-c was not run, because it needs a sibling serialize.c checkout the reader was scoped away from, so the C and C++ perf lock from base is unverified here; everything downstream of that step was run individually.

Six findings, none blocking. First, the write side walks nested-table bodies three times per level: wire.go line 385 runs save_body once as a default probe, then wireFrame at lines 266 to 271 runs it twice more, which is visible at NestedTable.h lines 693 to 703 in the examples-c goldens where root_config_save_body appears three times for one field, and arrays are worse at wire.go line 307 and TablesTable.h 1291 to 1305, with four child walks per element per parent walk. The reference measures arithmetically at cpptable/codecs.go 915 to 916. Nothing goes red at the corpus's depth, but the cost is exponential in schema depth; the smallest repair is a measure_body per type that sums lengths and a wireFrame that calls it. Second, table_writer_id at wire.go line 68 is a linear scan, and every probe copies the whole TableWriter including its 156-entry id array, about 1,280 bytes, twice per frame, where C++ interns into one shared TableIds and undoes elided entries with truncate in constant time at cpptable.go 840 and 849 to 857; the repair is the vocabulary behind a pointer plus a truncate. Third, the SPEC-TABLES edit at lines 166 to 177 drops the per-port row list for #511 to #518 and the ROADMAP pointer, and ROADMAP now has two overlapping wire-form rows, the new C-complete row directly above an older C-incomplete row that C does implement for fixed units; restore the list and reword the older row to say including variable storage. Fourth, the JSON changes are outside the stated scope: json.go adds line and block comments as whitespace, replaces ill-formed sequences with U+FFFD, and switches keyed-slot validity from a nonzero id to a non-null name. All of it is correct against section 16.2 at lines 12179 to 12190, and the slot change is a real fix now that identities can be zero, but two spec-anchored comments were deleted, the RFC 8259 section 8.1 rationale on write_string and the byte-transparency note on scan_string; keep the code, restore the comments, and name the text form's comment gate in the PR body. Fifth, the plain predicate at wire_read.go line 105 gates on kind at most f64 and so emits dead widening branches for bool, i8, u8, f32 and keyed scalars, harmless under -Werror; reuse widenable from cpptable/widen.go 96 to 104. Sixth, TableReport gained three members appended after malformed at ctable.go 335 to 338, source-compatible for designated initializers and breaking for positional ones, which wants a line in the release notes.

[rowan-claude]

This is a cold read of the delta from 21e354c to head 6e6b14b by an independent reader, relayed by Rowan; the author's own commit is 6e6b14b on the merge 475cb68, and everything else in the range came in from main. The verdict for the delta is merge.

Reproduced, with a sibling serialize.c present on this bench: build, vet and the whole Go suite clean, goldens included; the C wire differential at 64505 enumerated plus 100000 random mutants, 164505 in all, with 0 divergences, native and under ASan and UBSan with no sanitizer recovery, which reconciles exactly with the author's 84505 at a smaller random count; both negative controls fire; the C conformance leg under the sanitizers passes every registered surface, wire 53 of 53, report 47 of 47, JSON read and write 53 of 53, hostile JSON 107 of 107, forgery 12 of 12, cook 6 of 6, cook-foreign 6 of 6, cook-forgery 113 of 113, block 2 of 2; make tables-c passes every C step and stops only at a JavaScript control that needs a node binary absent from this machine; the repo's own soak reports zero allocator calls over 6144 iterations, confirmed by a static sweep finding no malloc, calloc, realloc, free, alloca or variable-length array in the generated fixed-class C. No wire pin moved: the set of sixteen-hex ids is byte-identical between the merge base and the head across all 26 changed goldens and the bench output, the form byte is unchanged, and the descriptor gained only the wide and fraction-bits columns and the arm field and size the C++ reference already declares.

The new material against the reference. The 128-bit and fixed-point exact-decimal writer and reader are a line-for-line transliteration of cpptable/json.go at 755 to 850, 1288 to 1331 and 2270 to 2400, with the same saturation, the same forty-digit point bounds, the same rule that an inexact decimal is a kind mismatch leaving storage untouched, the same negative-zero rule and the same clamp order; the 127-fraction-bit bound is enforced upstream by the checker at check.go line 1411, so the shifts by fraction minus 64 never reach 64. The corpus stops at 32 fraction bits, so the reader probed beyond it: fixed 1 by 127, ufixed 64 by 64, fixed 64 by 64 and ufixed 63 by 65 all round-trip bit-exactly including the 128-digit expansion of minus one plus two to the minus 127, 0.3 into 64 fraction bits gives one kind mismatch with storage untouched, and 1e40 into a bounded ufixed saturates then clamps with clamped at two. Wide text rejects an odd byte count, an embedded zero unit and lone or reversed surrogates, backs off one unit rather than splitting a pair, and reads units little-endian never through host memory, with the scan and write helpers matching the reference's including the not-clamped prefix rule. Unions read the arm reference, then the kind, then the length, in the reference's order; the type is cleared only after the span succeeds so a truncated repeated header keeps the prior arm, and an array element clears first; payload-free arms ride kind 32 with zero length and general arms get a fixed-width check per accepted kind. Optional arrays: a foreign element kind counts a kind mismatch and breaks out of the id switch before presence is set, at wire_read.go line 182 against the presence line at 122, verified in emitted code and against the reference. And the deep-frame change is real: wire.go line 273 emits, when the writer has no buffer, one body walk and then the length prefix from the offset difference, and the default check short-circuits the elision probe at the first riding field instead of measuring the whole nested body. Measured on the repo's own C table bench at O3 with NDEBUG, three sittings each, write went from 0.066 to 0.089 million messages per second and round-trip from 0.057 to 0.073, about 35 and 29 percent, so dropping forced inlining on composite codecs is more than paid for; there is no table-bench perf lock, and the packet bench's C row is untouched.

Seven findings, none blocking. First, four normative sentences are now false and the delta touches no spec page: SPEC-TABLES lines 10134 and 10136 say a payload-free arm reached by a table closure, and a table-closure union, are C++ only with the other eight refusing; line 11714 says every ported backend but C++ refuses a union with table arms; SPEC.md line 734 says C++ carries all three and the other eight refuse a default reachable from a table. Name C beside C++ in all four and make eight seven, which also folds in the earlier open row-list finding. Second, the keyed-at macro at ctable.go line 216 now bounds by sizeof of the array over sizeof of its first element, correct for an array and silently wrong for a decayed pointer, and nothing in the fixed-class wire needs it; revert the count argument or make the macro refuse a pointer at compile time. Third, the plain predicate at wire_read.go line 107 is still looser than the reference's plainScalar and the delta widened it, still dead code only; gate on a shared widenable predicate. Fourth, the vocabulary cost is unchanged, the linear id scan at wire.go line 65 and the by-value probe copy of the whole id array at line 73; the delta removed the exponential multiplier on top of this and the bench rose regardless, so a note. Fifth, the inlining change carries no benchmark of its own: every bench, README and PERFORMANCE change in the range came from the merge of main, so the PR's refreshed benchmark output is true of the goldens and not of the perf claim, which the reader measured instead. Sixth, latent: codecs.go line 292 groups wstring with string and bytes for a default memcpy into a sixteen-bit array with a byte-count length, unreachable because the checker refuses a wstring default, but written as if reachable; split it out with an explicit unreachable. Seventh, cosmetic: the wide-kind test is spelled as a range comparison three times where ir.TableKindWide already says it.

Of the prior six findings the exponential body walks are resolved and measured, the JSON changes are resolved by alignment since the text form is now a transliteration of the reference including its comments, and the vocabulary scan, the plain predicate, the spec row list and the report members are unchanged, the spec finding now carrying four more stale sentences. Every stated claim matched what was reproduced; the one imprecision is the benchmark output's provenance. No unchecked arithmetic or read past a bounded frame turned up in reading or under UBSan across the mutants, the hostile suite and 100000 forgery rounds.

[rowan-claude]

This is a cold read of the delta from 6e6b14b through 2842a22, re-verified at the head the branch moved to during the read, 069de05, by an independent reader with the serialize.c sibling pointed at read-only, relayed by Rowan. The verdict is merge after fixes, and the first fix is one that must not merge without: the last commit, roughly twelve hundred lines of new C cook-write code in cookwrite.go, cook.go, block.go and the graph pack path, was read in outline only and needs its own cold read.

The first finding is block-worthy on its own. The enum-keyed accessor lost its upper bound: at ctable.go lines 223 to 231 and 236, table_keyed_slot now refuses only a key at or below zero and returns key minus one, where the base refused a key above the count as well. The repair for the prior decayed-array finding deleted the bound instead of fixing how it was derived. SPEC-TABLES lines 990 to 1005 are explicit that the accessor refuses None and past Max, that the refusal stands in every build, in every port, at both ends, and that the guard is symmetric, since a build that skipped it would read one element before the array at one end and past its end at the other. The C++ reference does both ends in one unsigned compare, visible in the KeyedTable golden at line 2089. The smallest repair is to take the slot count as a parameter and use the reference's single unsigned compare, the macro passing the field's maximum, or to emit a per-field accessor with the bound baked in, which also removes the array-decay problem the change was made to solve. No generated code calls the macro, so only the shipped caller surface regressed, but that surface is what the spec sentence is about.

Second, a blob past the size cap reports the wrong refusal reason at LoadMeasure: graph.go line 544 returns minus one without setting the report's reason, so the extended measure returns the seeded count-over-length reason where the reference sets blob-over-size-cap; the thresholds agree exactly, so C and C++ refuse the same wires and only the label differs, one line now that the head added the full fourteen-value reason enum. The larger version of this finding at 2842a22, the extended measure multiplexing two enums, is fixed by the head. Third, an evidence gap: the whole caller-allocator surface, fifty-six generated symbols in the graph golden alone, all newly claimed in section 11 and in the checker's generated-verb list, is called by nothing under test; the allocator-hook and allocation-failure tests in the tree are C++ only, so the claim of allocation-failure coverage in the collection tests did not reproduce on the C side. Fourth, minor: SPEC-TABLES line 11688 still says the list has its three claimed names while the checker now claims a fourth, the per-field At. Fifth, minor and latent: the arena's span grab advances the cursor before allocating, so a failed allocation loses those slots from the four-gigabyte offset space for the arena's life, harmless because the builder is torn down but unrecoverable by a retry.

The prior findings: the four spec sentences now name C; the decayed accessor was addressed by deleting the derivation, but the fix dropped the refusal, which is the first finding; the hash vocabulary with rollback is resolved and correct, and the reader proved both properties it was asked about: rollback restores first-use order exactly, because at the moment a slot is cleared every later entry is already gone and no earlier entry probes through it, and no probe can leave an interned id behind, checked mechanically across all 27 probe sites in the graph golden and zero unpaired across every C golden, the only unrewound exits being the returns that abort the whole save. The 400-byte blob oracle repair is present and matches the Go task's shape, a framed-record scan retaining body lengths consumed as aligned record storage, byte-identical to C's blob storage, pinned at 400 with a 328 counterfactual. The negative-offset helper is repaired in both readers.

Evidence at the head: build, vet and the whole Go suite clean; make test-c exit zero against the sibling; the wire fuzz at 138751 mutants and 0 divergences native and sanitized, matching the claim exactly; the collections differential at 43016 mutants with identical reports, bytes and sizes across three legs, native and sanitized; all six negative controls held. No wire pin or protocol id changed: the packet goldens moved only in #711's loop conditions, and the short descriptor ids that vanished from the C table goldens were the old nested-form ids, each field now carrying the canonical 64-bit id the C++ golden already had.

Checked clean: the region and node model against section 6, with the layout computed from framing alone, the region path re-checking size against capacity so a measure and load disagreement refuses rather than overruns, the base checked against the unit alignment with a static assertion per node type, and the directory placed after aligned sums; maps with a heapsort over a cached order, the entry terminator checked in the key scan before slot choice in all three implementations, descending refused, a duplicate replacing the last, an over-long key dropped with a clamp, and the index probe bounded; the list cap refusing at LoadMeasure with reason eleven and no report events and propagating refused through every enclosing body on LoadBuilder with no extra counter, with no cap on maps on either side, matching C++; shared identity through an explicit-stack walk with open and close markers, both re-entry and two type ids refusing, edges reversed so declaration order is visit order, pointer arrays and union arms both routed through one numbering; a shared blob in JSON refused exactly as C++ refuses it; and no unchecked arithmetic on counts, lengths or offsets, and no read past a bounded frame.

[gafferongames]

Read dedbe1a (C UnitView and bounds). Independent, not a merge verdict. Scoped to C API lifetime, allocation and bounds.

UnitView is process-static registry data, not a live region view. unit_view() returns a function-local static; nothing to free; a caller using a descriptor still owns the value's storage. Outside-closure ids and JSON keys are zero/NULL.

Keyed: (uint32_t)key - 1u >= count with explicit E_MAX restores both ends on a decayed pointer. Blob cap names SCHEMA_TABLE_REFUSE_BLOB_OVER_SIZE_CAP from body->size before payload. Cook measure/write both orders are in the allocator-failure sweep.

No blocking defect from me on this checkpoint.

Written by Johnny Grok

[gafferongames]

Read bb61510 (nested message payload alignment). Independent, not a merge verdict.

The defect is real: encodeBitField/encodeBitKeyed probed nested bodies in a bitWriter at bit zero and spliced them into the batch, so byte alignment inside nested text/list payloads was wrong at a nonzero bit position. The repair uses the probe only for elision and writes bodies and keyed values at the running batch writer. TestMessageNestedPayloadAlignment passed here, including the pinned 42-byte C++ combined wire.

No golden edits. Not a merge verdict.

[gafferongames]

Read 3fda8a8 after bb61510. Independent, not a merge verdict. Scoped to C lifetimes, allocation, bounds and the three approaches named on the bus.

Announcement: min/max hold source offsets only during validation, then a second linear parse writes resolved facts into the same 96-byte entries. No announcement pointer is stored. Duplicate identity is memcmp of the canonical span, which keeps #722's triples distinct. Capacity refuses before writing past max_entries. On failure, announced is not set.

Graph load: *out = root immediately after place+reset; count in load_messages still increments only on success. Zero-width unbounded list/map counts > INT32_MAX refuse in extent before the element walk (extent_refused → count-over-length). Node count is bounded by remaining ref bits.

Numbering dispose is on the save done path. Sanitizer rerun after the parser adjustment is still owed, as you said.

Not a merge verdict. Retention unknowns remain ahead.