Add w3_validate_<type name> functions#101
Open
MartinSStewart wants to merge 6 commits into
Open
Conversation
…ders When a module defines `w3_validate_<TypeName> : <TypeName> -> Result String ()` for a custom type, the generated `w3_decode_<TypeName>` now calls it after decoding: `Ok ()` decodes successfully, while `Err msg` logs the message via Lamdera.Wire3.debug and fails the decode. Each w3_validate_ function is verified at compile time: the named type must be a custom type defined in the same module (not a type alias, and not missing), the function must have a type annotation, and its signature must be exactly `<TypeName> <tvars> -> Result String ()` using the type's declared variables. Any violation is a compile error. When a module contains validators, the generated wire functions are placed after user definitions so the generated decoders can reference them. Tests: a compiling fixture covering the codegen (plain tvar, number-constrained tvar, and nested usage), plus compile-error fixtures for the undefined-type, type-alias, missing-annotation, wrong-signature, and concrete-tvar cases. https://claude.ai/code/session_01URzKFJwLCrv3r2W28bPiW6
Compile-success fixtures verifying the w3_validate hook behaves with recursive custom types (no codegen loops, compiler crashes, or dependency-graph issues): - Wire_Validate_Recursive: a directly self-referential type (Tree). - Wire_Validate_RecursiveRecord: a custom type referencing a record that references it back (Node -> NodeData -> List Node), i.e. mutually recursive generated decoders plus the validator call. - Wire_Validate_RecursiveExtra: recursion through Maybe (Chain), parameterised recursion through List validated with the type variable (Rose a), and mutual recursion between two separately-validated types (Ping/Pong). https://claude.ai/code/session_01URzKFJwLCrv3r2W28bPiW6
The existing w3_validate tests are all compile-time: they check the generated decoder type-checks and that bad validator definitions are rejected by the compiler. None execute a decoder, so neither the Ok-accept nor the Err-reject path was tested at runtime. This adds an elm-test (run via the project's --compiler=lamdera harness, e.g. `cd test/scenario-alltypes && npx elm-test --compiler=lamdera tests/Wire3ValidateTest.elm`) that encodes Validated values and decodes them back, asserting: - values that pass w3_validate_Validated decode to `Just value` - values that fail it are rejected and decode to `Nothing` Encoding does not run validation, so a validation-failing value can be encoded and then shown to fail on decode. The wire functions are imported from Test.Wire_Validate (cross-module) on purpose. https://claude.ai/code/session_01URzKFJwLCrv3r2W28bPiW6
Compile-error fixtures filling in gaps in the existing wireValidateErrors
suite:
- WrongOkType: Result with Int as the Ok payload (not unit).
- TvarRename: type Holder a validated as Holder b -- strict tvar-name match
rejects alpha-renamed variables.
- TvarSwap: type Pair a b validated as Pair b a -- multi-tvar order matters.
- ArgCount: validator declared as a value (no argument).
Compile-success fixtures (added to wireTestFiles):
- MultiTvar: type Pair a b with a validator using both type variables.
- Phantom: type Phantom a (variable declared but unused in constructors).
Runtime tests (extending tests/Wire3ValidateTest.elm) covering behaviour the
compile tests can't observe:
- Recursive Tree: validation runs at every node, so a deeply-nested invalid
Leaf causes the whole decode to fail.
- Container (a record containing Validated values): validation runs through
aggregate fields, both for the direct field and for list entries.
https://claude.ai/code/session_01URzKFJwLCrv3r2W28bPiW6
…error When a module defines a w3_validate_* function, the generated wire functions are appended after the user's code so the generated decoders can call the validators. That meant any user top-level code in the same module that also referenced a generated w3_encode_*/w3_decode_* function produced a forward reference, which crashed the type solver with an internal Map.! error. Detect that case in addWireGenerations_ (only on the validator path) and emit a normal compile error naming the offending definition and wire function. Adds a total VarTopLevel collector (topLevelRefsInExpr) that handles every Expr_ constructor, since getLvars is specialised to generated code and errors on shapes like multi-branch if. Tests: two fixtures (encoder and decoder reference) asserting the new error.
…e_<T>
Validation should only run on attacker-controlled, backend-inbound data, not on
trusted data (persistence, evergreen migrations). So every custom type and alias
now gets two decoders:
* w3_decode_<T> validating: applies w3_validate_<T> if present and
recurses through the w3_decode_* chain (for the Lamdera
runtime to use on backend-inbound data).
* w3_unsafe_decode_<T> non-validating: behaves like w3_decode_<T> did before
validation existed, recursing through the
w3_unsafe_decode_* chain.
The two chains are threaded through decoder codegen via a DecodeMode parameter.
The validating chain is byte-identical to the previous w3_decode output (only
the prefix and, for unions, the validator hook differ), so existing behaviour is
unchanged. Built-in decoders (decodeList etc.) are mode-agnostic and thread the
mode through their element decoder.
The trusted in-repo consumers are repointed at the unsafe chain to preserve
their pre-validation behaviour: the evergreen migration harness and backend-model
persistence reload.
Also adds w3_unsafe_decode_* stubs + exports and extends the getForeignSig
fallback to the new prefix. The validator-module reference check already covers
w3_unsafe_decode_* (names are derived from the generated defs).
Tests: Wire3ValidateTest now contrasts unsafe-decode-accepts-invalid vs
validating-decode-rejects; the full Test.Wire suite passes with all fixtures
compiling under both chains.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR adds support for
w3_validate_<type name>: <type name> -> Result String ()functions. These are user defined functions thatw3_decode_<type name>functions will use to determine if a custom type can safely be decoded.Here's an example of what w3_validate looks like in real code
Motivation behind this feature
If you have an opaque type that only can be constructed via a function that enforces some guarantee (perhaps a
type Name = Name Stringthat can't be longer than 100 characters) then you might expect that guarantee to hold when you serialize data. The reality however is that w3_decode functions circumvent any opaque type guarantees and just create a instance of the type directly. This means a hacker could manually create a ToBackend request that contains an opaque type that violates some guarantee that type is supposed to have. Since the backend is probably coded with the assumption that opaque type guarantees hold, this could lead to bugs or security vulnerabilities.Drawbacks
This feature has a number of drawbacks.
w3_validate_<type name>implementation such that it returnsErr "some error"for values that can be created normally. If those values get sent to the backend then the decoder will fail and the request will get silently dropped (the "some error" will get logged on the Lamdera server but that's it)w3_unsafe_decode
w3_unsafe_decode_<type name>functions that are generated along side w3_decode. These behave identically to how w3_decode behaved before this PR (or how w3_decode behaves now if the user has not written any w3_validate functions)In conclusion, I'm not particularly happy with this feature, but I don't know of any better way to solve the motivating security issue. I've spoken with a couple people and brainstormed ideas and this seems like the best option.