Skip to content

feat: Human-in-the-loop effect for APL and elicitation plugin#115

Merged
araujof merged 8 commits into
devfrom
feat/hil_apl
Jul 15, 2026
Merged

feat: Human-in-the-loop effect for APL and elicitation plugin#115
araujof merged 8 commits into
devfrom
feat/hil_apl

Conversation

@terylt

@terylt terylt commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

Summary

Adds human-in-the-loop (HIL) elicitation to APL: a policy can pause an operation to ask a human — manager approval, a confirm, a step-up re-auth, an attestation — and resume once the human responds, without blocking the request path. The human's decision is bound to the actual request args and recorded from CPEX-owned state, so it is the source of truth for "what was approved," never an LLM summary. Includes a working Keycloak CIBA channel plugin.

A policy expresses it with a sugar verb that parses to a single Step::Elicit:

plugins:
  - name: manager-approver
    kind: elicitation/ciba
    hooks: [elicit]

routes:
  - tool: approve_raise
    pre_invocation:
      - "require_approval(manager-approver, from: claim.manager, channel: \"ciba\", scope: \"args.amount <= 25000\", purpose: \"Approve raise\")"

from is who to ask (approver, resolved from the bag — may differ from the subject); scope is the args-binding the runtime checks against the live request; purpose is the audited, human-readable description.

The model: async, retry-based resume

An elicitation has three short, synchronous touch-points — the hours-long human gap lives in the channel (Keycloak CIBA), never in a blocking call:

  1. Dispatch — first arrival: register the intent, open the channel backchannel, return a correlation id.
  2. Check — on each agent retry: read status (Pending / Resolved{approved|denied} / Expired) without blocking.
  3. Validate — once resolved: verify the response is genuine (signed token, intent binding, responder identity). The runtime then layers the scope-over-args sufficiency check before honoring the approval.

While pending, the phase suspends rather than denies. Decision stays binary — a suspended phase reports Allow + Some(PendingElicitation), and the host maps that to JSON-RPC -32120 ("not complete, retry echoing this id"). The forwarding rule stays one clause: forward iff Allow AND pending.is_none(). Expiry / channel error fails closed (on_error: deny default).

What's included

Area Change
DSL verbs (apl-core) require_approval / confirm / step_up / attestation / info / review — sugar over one Step::Elicit(ElicitStep) with an ElicitKind. Fields: from, purpose, scope, timeout, channel, config_override, on_error. Parser + evaluator (dispatch/check/validate flow, pending short-circuit, scope-over-args) + route.rs resume wiring.
Elicitation hook family (cpex-core) New ElicitationHook (single hook name "elicit") + ElicitationPayload with a Dispatch/Check/Validate operation discriminator — mirrors the delegation hook layout; dispatch is free via invoke_entries::<ElicitationHook>. cpex-core keeps its own ElicitationOp/ElicitationStatusKind/ElicitationOutcomeKind (no apl-core dep); the bridge maps.
Bridge / invoker (apl-cpex) ElicitationInvoker (dispatch/check/validate) + elicitation_invoker.rs; dispatch-plan resolves an elicit entry by name; route_handler drives the pending/resume flow and surfaces elicitation.* bag attrs (id, approver, intent_id, channel) for audit.
CIBA channel plugin (builtins/plugins/elicitation-ciba, new crate) kind: elicitation/ciba — a HookHandler<ElicitationHook> implementing the three operations against Keycloak CIBA: fromlogin_hint, purposebinding_message, timeoutrequested_expiry. Intent store, config, factory. Registered via cpex-builtins.
Args binding via scope The approval is bound to the request with an APL boolean expression (args.amount <= 25000) the runtime evaluates at validate — kept in APL because Keycloak has no RFC 9396 RAR.

Notable design decisions

  • Approver ≠ subject. from resolves the party to ask from the request bag (e.g. claim.manager); the resolved identity is cross-checked against the actual responder at validate.
  • Sufficiency vs genuineness split. The channel plugin proves the response is genuine (signature / intent binding / responder); the sufficiency check (scope over live args) stays in the runtime because it is an APL expression the plugin can't evaluate.
  • One hook, three operations. A single elicit hook + payload discriminator (not three hook names) keeps registration and the dispatch plan trivial — one plugin, one entry, resolved by name like token.delegate.
  • Tri-state without widening Decision. PendingElicitation rides alongside an Allow so the deny path stays untouched; the host's one-clause forward rule and -32120 retry contract carry the suspend/resume.
  • Fail-closed. Expiry, channel error, or a failed validation deny (subject to on_error).
  • cpex-core decoupled from apl-core. The hook payload defines its own op/status/outcome enums; the apl-cpex bridge translates to/from apl-core's ElicitKind / ElicitationStatus / ElicitationOutcome.

Public API surface

  • cpex-core::elicitation: ElicitationHook, HOOK_ELICIT ("elicit"), ElicitationPayload, ElicitationOp, ElicitationStatusKind, ElicitationOutcomeKind.
  • apl-core::step: Step::Elicit, ElicitStep, ElicitKind, ElicitationInvoker (dispatch/check/validate), ElicitationDispatch, ElicitationStatus, ElicitationOutcome, ElicitationValidation, PendingElicitation, ElicitationError.
  • apl-cpex: ElicitationInvoker bridge, dispatch-plan elicitation_entries, route-handler pending/resume wiring.
  • builtins/plugins/elicitation-ciba: kind = "elicitation/ciba" (CibaApproverFactory), registered through cpex-builtins.

Testing

  • CIBA pluginbuiltins/plugins/elicitation-ciba/tests/ciba_e2e.rs (7 tests) + live_keycloak.rs (1 live integration test, ignored by default).
  • apl-cpextests/elicit_step_e2e.rs: end-to-end through a real PluginManager (dispatch → pending → resolve → validate; approve / deny / expire; args-binding). Existing suites (delegate_step_e2e, end_to_end_route, cmf_invoker_dispatch) updated for the new step/plan shape.
  • apl-core — parser / evaluator / step.rs / route.rs unit tests; tests/yaml_end_to_end.rs updated for the elicit verbs; apl-cmf/tests/end_to_end.rs touched.

Note: I have not re-run the full workspace suite on this branch — worth a cargo test --workspace / make test befor

Follow-ups

  • Site docs page for elicitation under docs/content/docs/apl/ (none in this PR).
  • Additional channel plugins beyond CIBA (Slack, in-band); capability-gating for the elicit hook; richer per-kind validation contracts.

terylt added 3 commits July 8, 2026 16:01
Signed-off-by: Teryl Taylor <terylt@ibm.com>
Signed-off-by: Teryl Taylor <terylt@ibm.com>
Signed-off-by: Teryl Taylor <terylt@ibm.com>
@araujof araujof self-assigned this Jul 14, 2026
@araujof araujof added enhancement New feature or request framework Rust labels Jul 14, 2026
@araujof araujof added this to CPEX Jul 14, 2026
@github-project-automation github-project-automation Bot moved this to Backlog in CPEX Jul 14, 2026
@araujof araujof moved this from Backlog to In progress in CPEX Jul 14, 2026
@araujof araujof added this to the 0.2.1 milestone Jul 14, 2026
@araujof
araujof marked this pull request as ready for review July 14, 2026 14:10
@araujof
araujof requested review from araujof and jonpspri as code owners July 14, 2026 14:10
@araujof araujof modified the milestones: 0.2.1, 0.2.2 Jul 14, 2026
Signed-off-by: Teryl Taylor <terylt@ibm.com>
@araujof araujof changed the title Feat: Human-in-the-loop effect for APL and elicitation plugin feat: Human-in-the-loop effect for APL and elicitation plugin Jul 15, 2026
- Fail closed when an elicitation `from` attribute reference does not
  resolve, instead of dispatching the literal to the channel as a bogus
  login_hint; literal identities still pass through. Adds tests.
- Rename `ElicitationError::Dispatch` to `Handler` and thread the
  operation name so check/validate failures no longer read as dispatch
  failures.
- Correct the stale `dispatch_elicitation` doc comment to describe the
  implemented suspend/-32120 behavior.
- Document the numeric_compare string coercion as an all-comparisons
  behavior change in the CHANGELOG, plus the elicitation feature entry.
- Add docs/apl/elicitation.md (effect example, suspend/resume model,
  CIBA config, attributes, genuineness-vs-sufficiency/JWT-trust note);
  update the effects page and APL index.

Signed-off-by: Frederico Araujo <frederico.araujo@ibm.com>

@araujof araujof left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice work! This is a very useful feature. Merging it with a few additional fixes. Further enhancements should be made in a separate PR.

I found a few issues while reviewing the PR, which I addressed on the branch.

Addressed

  1. from fail-open (auth boundary). An unresolved attribute from (e.g. claim.manager with the claim absent) no longer dispatches to a bogus login_hint — it fails closed (subject to on_error). Literal identities (emails/usernames) still pass through. Tests added.
  2. Global evaluator behavior change surfaced. The numeric_compare string→f64 coercion affects all order comparisons, not just elicitation scope; now documented in the CHANGELOG.
  3. JWT signature not verified. Acknowledged and correct given the token's provenance (client-authenticated TLS poll to the OP). The transport-trust assumption is now documented in the elicitation docs page for deployers.
  4. Docs. New docs/apl/elicitation.md (effect example, suspend/resume model, CIBA config, attributes, genuineness-vs-sufficiency note); effects page and index updated.
  5. Minor. Stale dispatch_elicitation doc comment corrected; ElicitationError::DispatchHandler with the operation named, so check/validate failures no longer read as dispatch failures.

Status

cargo test --workspace: all passing

araujof added 3 commits July 15, 2026 07:56
Resolve conflicts:
- Cargo.toml: adopt dev's 0.2.1 workspace version for all path deps,
  keeping the new cpex-plugin-elicitation-ciba entry (bumped to 0.2.1).
- route_handler.rs: keep both dev's per-site denyWith decoration and the
  elicitation pending (-32120) / peek (-32121) handling; the retry/confirm
  signals stay undecorated by design.
- CHANGELOG.md: move the #115 elicitation entries into a fresh
  [Unreleased] section above the released 0.2.1, and fix the compare links.

Signed-off-by: Frederico Araujo <frederico.araujo@ibm.com>
Satisfies the CI lint job (`cargo fmt --all -- --check`); no behavior change. Clippy (`--workspace --all-targets -D warnings`) is clean.

Signed-off-by: Frederico Araujo <frederico.araujo@ibm.com>
cargo-machete flagged both as unused — the crate surfaces errors via cpex-core's PluginError/PluginViolation and emits no tracing. Also switch its cpex-core dep to `workspace = true` (matching the other builtin plugins), which drops the stale 0.2.0 pin left by the dev merge.

Signed-off-by: Frederico Araujo <frederico.araujo@ibm.com>
@araujof
araujof merged commit 72ffad3 into dev Jul 15, 2026
9 checks passed
@github-project-automation github-project-automation Bot moved this from In progress to Done in CPEX Jul 15, 2026
@araujof
araujof deleted the feat/hil_apl branch July 15, 2026 12:20
araujof added a commit that referenced this pull request Jul 15, 2026
Bring in the human-in-the-loop (HIL) elicitation feature and the numeric
string-coercion change for order comparisons (#115), plus the new
elicitation/ciba builtin plugin.

Conflict resolution:
- Cargo.toml: keep the 0.2.2 workspace version across the internal-dep
  table and add the new cpex-plugin-elicitation-ciba entry at 0.2.2.
- CHANGELOG.md: fold dev's [Unreleased] entries (HIL elicitation; numeric
  order-comparison coercion) into the existing [0.2.2] release entry
  alongside the read_headers change, dropping the separate [Unreleased]
  section.

Workspace builds; fmt + clippy (-D warnings) clean; full test suite passes
(936); hugo docs build clean.

Signed-off-by: Frederico Araujo <frederico.araujo@ibm.com>
araujof added a commit that referenced this pull request Jul 15, 2026
…otent

The v0.2.2 release failed at `cpex-builtins` because the new
`cpex-plugin-elicitation-ciba` crate (added with the elicitation feature,
#115) was never added to the ordered publish list, so its dependent
`cpex-builtins` could not resolve it on crates.io.

- Add `cpex-plugin-elicitation-ciba` to the publish order, before
  `cpex-builtins`.
- Make the publish loop idempotent: skip a crate whose version is already
  on the index (crates.io rejects duplicate versions), so a re-run of a
  partially-published tag finishes the remaining crates instead of
  aborting on the first already-published one. Genuine errors still fail.

Signed-off-by: Frederico Araujo <frederico.araujo@ibm.com>
araujof added a commit that referenced this pull request Jul 17, 2026
* ci(release): publish cpex-plugin-elicitation-ciba; make publish idempotent

The v0.2.2 release failed at `cpex-builtins` because the new
`cpex-plugin-elicitation-ciba` crate (added with the elicitation feature,
#115) was never added to the ordered publish list, so its dependent
`cpex-builtins` could not resolve it on crates.io.

- Add `cpex-plugin-elicitation-ciba` to the publish order, before
  `cpex-builtins`.
- Make the publish loop idempotent: skip a crate whose version is already
  on the index (crates.io rejects duplicate versions), so a re-run of a
  partially-published tag finishes the remaining crates instead of
  aborting on the first already-published one. Genuine errors still fail.

Signed-off-by: Frederico Araujo <frederico.araujo@ibm.com>

* docs: add runnable CPEX tutorial and reorganize docs outline

Signed-off-by: Frederico Araujo <frederico.araujo@ibm.com>

* docs: canonicalize route form, runnable quickstart, config/testing guides, link check

Signed-off-by: Frederico Araujo <frederico.araujo@ibm.com>

* ci: fix docs link-check and exclude tutorial crate from package dry-run

Signed-off-by: Frederico Araujo <frederico.araujo@ibm.com>

* ci: gate tutorial IdP readiness with a probe; prune unused tutorial deps

Signed-off-by: Frederico Araujo <frederico.araujo@ibm.com>

---------

Signed-off-by: Frederico Araujo <frederico.araujo@ibm.com>
araujof added a commit that referenced this pull request Jul 17, 2026
* ci(release): publish cpex-plugin-elicitation-ciba; make publish idempotent

The v0.2.2 release failed at `cpex-builtins` because the new
`cpex-plugin-elicitation-ciba` crate (added with the elicitation feature,
#115) was never added to the ordered publish list, so its dependent
`cpex-builtins` could not resolve it on crates.io.

- Add `cpex-plugin-elicitation-ciba` to the publish order, before
  `cpex-builtins`.
- Make the publish loop idempotent: skip a crate whose version is already
  on the index (crates.io rejects duplicate versions), so a re-run of a
  partially-published tag finishes the remaining crates instead of
  aborting on the first already-published one. Genuine errors still fail.

Signed-off-by: Frederico Araujo <frederico.araujo@ibm.com>

* docs: add runnable CPEX tutorial and reorganize docs outline

Signed-off-by: Frederico Araujo <frederico.araujo@ibm.com>

* docs: canonicalize route form, runnable quickstart, config/testing guides, link check

Signed-off-by: Frederico Araujo <frederico.araujo@ibm.com>

* ci: fix docs link-check and exclude tutorial crate from package dry-run

Signed-off-by: Frederico Araujo <frederico.araujo@ibm.com>

* ci: gate tutorial IdP readiness with a probe; prune unused tutorial deps

Signed-off-by: Frederico Araujo <frederico.araujo@ibm.com>

---------

Signed-off-by: Frederico Araujo <frederico.araujo@ibm.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request framework Rust

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

2 participants