diff --git a/docs/dfx/chip-swimlane-profiling.md b/docs/dfx/chip-swimlane-profiling.md index 270256d3b1..7768430c1d 100644 --- a/docs/dfx/chip-swimlane-profiling.md +++ b/docs/dfx/chip-swimlane-profiling.md @@ -222,7 +222,7 @@ microseconds, downstream code sees: | Field | Meaning | | ----- | ------- | -| `task_id` | Runtime task id (`(ring_id << 32) \| local_id`); also exposed split as`ring_id` | +| `task_id` | Runtime task id (`TaskId::raw`); its high 32 bits are also exposed split off as `ring_id`, which is a ring index under `tensormap_and_ringbuffer` and an id space under `host_build_graph` | | `func_id` | Kernel function id. Always `-1` on disk; resolved post-process from `deps.json::tasks[].kernel_ids[3]` (see `swimlane_converter.resolve_func_id_from_kernel_map`) | | `core_id` / `core_type` | Physical core index and `"aic"` / `"aiv"` string | | `start_time_us` / `end_time_us` / `duration_us` | AICore execution window in microseconds | diff --git a/docs/dfx/dep-gen.md b/docs/dfx/dep-gen.md index 96548e0b08..05983f5cbc 100644 --- a/docs/dfx/dep-gen.md +++ b/docs/dfx/dep-gen.md @@ -178,11 +178,15 @@ silently lose precision if encoded as numbers. Python consumers pass these through `int(v)` which accepts either form, so the schema is JS-safe without burdening Python. -Task ids encode `(ring_id << 32) | local_id` — the same layout as -`TaskId::raw`: +Task ids are `TaskId::raw`. The low 32 bits are a local id; the high 32 bits +mean whatever the runtime that minted the record says they mean — a ring index +(`tensormap_and_ringbuffer`, `0..CHIP_MAX_RING_DEPTH-1`) or an id space +(`host_build_graph`, `0 = RING`, `1 = GRAPH_NODE`). See +`src/common/{tensormap_and_ringbuffer,host_build_graph}/task_id_encoding.h`. +Which one a record carries is a property of its runtime, not of the value: ```python -ring = (raw >> 32) & 0xFF +high = (raw >> 32) & 0xFF local = raw & 0xFFFFFFFF ``` diff --git a/problems.md b/problems.md index 14adb285dd..451b26357a 100644 --- a/problems.md +++ b/problems.md @@ -4,50 +4,81 @@ Audit of `host_build_graph` for data structures inherited from `tensormap_and_ringbuffer` whose actual hbg usage diverges from the tmr design they were shaped for. -Original audit was against `upstream/main@761fdf8d`. **Updated 2026-08-24 against -`upstream/main@3069f1aff`**, after which four of the six items are closed. Note -that #1963 renamed most hbg files (`pto_ring_buffer.h` → `ring_buffer.h`, -`pto_orchestrator.h` → `orchestrator.h`, `pto_runtime2.h` → `runtime_core.h`, -`pto_runtime2_init.cpp` → `runtime_init.cpp`, …), so paths below use the new names. +Original audit was against `upstream/main@761fdf8d`. **Updated 2026-08-25 against +`upstream/main@fc42cd1dd`**, after the L2-tensor split (#1974) and the ring removal +(#2004). #1980 retired the `PTO2` *identifiers* this audit named, so the spellings +below are the current ones; the brand prose survives in ~100 places, see +`KNOWN_ISSUES.md`. --- ## CLOSED -### 1. Single-ring shape carried as a multi-ring one — FIXED (#1965) +### 1. Single-ring shape carried as a multi-ring one — FIXED (#1965, #2004) `PTO2_MAX_RING_DEPTH` was `1` against tmr's `4`, yet the tree carried 38 -`[PTO2_MAX_RING_DEPTH]` array declarations and 21 loops that ran exactly once. -Every shared-memory operation existed twice — a scalar wrapper that filled a -one-element array, and a `_per_ring` implementation reading `[0]` (one of which -cast the array to `void` without reading it). The macro is gone from this runtime; -each operation has one scalar form. `sum_ring_heap_sizes`, which summed one element -and checked it for overflow against itself, is deleted. - -Boundaries deliberately held: the `PTO2_RING_*` knobs accept exactly what they did -before, the `RuntimeEnv` / `RUNTIME_ENV_RING_COUNT` ABI is untouched (shared with -tmr; this runtime reads slot 0), `bind_callable_to_runtime_impl`'s `extern "C"` -signature is unchanged, and the `[STALL]` grammar is byte-identical. +array declarations dimensioned by it and 21 loops that ran exactly once. #1965 +collapsed each operation to one scalar form; #2004 went further and dropped the +ring itself — a task id is now its own table index, and `task_window_mask` / +`get_slot_by_task_id` / `TaskAllocResult::slot` are gone. ### 2. `PTO2RingSet` was a one-member wrapper named "Set" — FIXED (#1965) -Held a single `PTO2TaskAllocator`, with a doc comment claiming -"PTO2_MAX_RING_DEPTH instances exist, one per scope depth". The allocator now sits -on the orchestrator directly. - ### 3. `DEP_POOL_OVERFLOW` reported for fanin exhaustion — FIXED UPSTREAM (#1963) -hbg has no dependency spill pool, yet latched a code named after tmr's. Renamed to -`SIMPLER_ERROR_FANIN_CAPACITY_EXCEEDED` — the mechanism both runtimes share, and -the words the log already printed. Value 4 held, so #1960's band contract and the -`-4` an existing caller sees are untouched. Not my change; landed independently -while #1965 was open. +Renamed to `SIMPLER_ERROR_FANIN_CAPACITY_EXCEEDED`, value 4 held. ### 4. `PTO2_DEP_POOL_SPIN_LIMIT` was a dead define — FIXED (#1965) -Defined in `ring_buffer.h`, referenced nowhere. Two struct summaries in that -file's header comment also described `FaninPool` / `DepListPool`, which the file -has not held since those pools were removed. +### 7. One byte layout shared by three unrelated structs — FIXED (#1974) + +`ChipTensor`, `TensorMapEntry` and `TensorCreateInfo` shared a byte-level layout so +three copy paths could each be a single 64-byte `memcpy`, held by 21 hand-written +`offsetof` assertions across four trees. Two of the three were shaped backwards to +satisfy it, and the entry's `memcpy` wrote `ChipTensor::buffer.size` into a +`TensorMapEntry *`. Each struct now assigns its own fields; +`TensorMapEntry::copy_tensor_create_info` is deleted (it had no callers). + +### 8. One tensor type served the boundary and both runtimes — FIXED (#1974) + +`ChipTensor` carried `owner_task_id` / `version` / `manual_dep` and two derived +caches — what the *runtime* decides about an argument, on the type a *caller* hands +in. `create_from_chip_args` recorded the mismatch as +`debug_assert(!t.manual_dep && t.version == 0)`, and `docs/buffer-abi.md` called the +type "internal" while `ChipWorker.run` took a container of them. + +Now: `ChipTensor` is 72 B of geometry plus a resolved address; +`simpler::{hbg,tmr}::Tensor` is 128 B and adds the runtime's own state; +`Runtime::set_orch_args` is the single adoption point and runs on the host in both +runtimes. `sizeof(ChipStorageTaskArgs)` fell from ~33.8 KB to 19464 B. + +### 4b. A nonzero producer ring id becomes a ring-0 dependency — FIXED + +`task_id.h` declared the encoding as `(ring_id << 32) | local_id` and named the +accessor `ring()`. **That was false for hbg**, which uses the high bits as an +id-space tag: `graph_execution.cpp` minted `TaskId::make(1, synthetic_local)` for a +Graph node. So every hbg guard reading `ring() != 0` looked like a bounds check on +a dimension that no longer exists, when it was really asking "is this a graph-node +id" — and `FaninBuilder::mark_seen` folded that question's answer into its dedup +return value, so `append_fanin_or_fail` read "not a ring task" as "not deduped +yet" and appended an edge to whatever `prod_slot` the foreign local id happened to +mask onto. + +The bug survived #2004 unchanged: it moved `mark_seen` to key on the `TaskId` and +dropped the redundant ring/slot parameters, but left the space check folded into +the dedup return value. + +`TaskId` is now an opaque 64-bit handle (`raw`, `invalid()`, `is_valid()`, `==`, +`sizeof == 8`); each runtime owns and names its own layout in +`src/common/{host_build_graph,tensormap_and_ringbuffer}/task_id_encoding.h` — +`TaskIdSpace{RING, GRAPH_NODE}` + `make_ring_task` / `make_graph_node` / +`is_ring_task` for hbg, a real `make_task_id(ring, local)` / `task_ring` for tmr. +`mark_seen` deduplicates and nothing else; `append_fanin_or_fail` rejects a +non-RING producer up front via `report_fatal(SIMPLER_ERROR_INVALID_ARGS, …)`. + +Repro: `graph_node_dependency` in +`tests/st/host_build_graph_validation/`, verified to fail (`DID NOT RAISE` — +the bogus edge was built silently) with the new guard removed. --- @@ -55,72 +86,38 @@ has not held since those pools were removed. ### 5. `on_scope_end` is an empty stub — low value -- **Evidence**: `runtime/scheduler/scheduler.h` - - ```cpp - void on_scope_end(PTO2TaskSlotState ** /*task_slot_states*/, int32_t /*count*/) {} - ``` - - still called from `end_scope` in `orchestrator_core/orchestrator.cpp`; its comment - says it is kept as a no-op so the orchestrator call site matches tmr's. -- An inline empty function costs nothing at runtime, so this is readability only. - Worth folding into the next change that touches either file, not worth its own PR. +`runtime/scheduler/scheduler.h`: +`void on_scope_end(TaskSlotState ** /*task_slot_states*/, int32_t /*count*/) {}`, +still called from `end_scope`, kept as a no-op so the orchestrator call site +matches tmr's. Inline empty function, so readability only — fold into the next +change that touches either file. ### 6. Docs name a field hbg does not have -- **Evidence**: `docs/RUNTIME_LOGIC.md` states hbg "never advances - `last_task_alive`". That identifier does not exist anywhere in the hbg tree - except that sentence. Pre-existing, one line. - -### 4b. A nonzero producer ring id becomes a ring-0 dependency - -- **Found by**: CodeRabbit on PR #1965, verified against the code. -- **Evidence**: `PTO2FaninBuilder::mark_seen` returns `false` for a producer whose - ring id is not 0, and `append_fanin_or_fail` reads `false` as "not deduped yet" - and appends the producer — with `prod_slot` already resolved against the one - shared-memory ring. A nonzero-ring producer id with a colliding local id would - create an edge to an unrelated ring-0 task. -- **Not a regression**: before #1965 the guard read - `prod_ring >= PTO2_MAX_RING_DEPTH` with that macro at `1`, which for a `uint8_t` - is the same test as `prod_ring != 0`. -- **Reachability**: `prod_ring` is `producer_task_id.ring()`. hbg builds every task - id on ring 0, and the Graph recorder already refuses an explicit dep whose - `ring() != 0`. Reaching this needs a foreign or malformed task id handed to - `set_dependencies` on the ordinary path. -- **Current state**: #1965 carries a `debug_assert(prod_ring == 0)` so UT and sim - builds trap a violation, and a comment that no longer implies rejection. The - fatal path CodeRabbit proposed (report `SIMPLER_ERROR_INVALID_ARGS` and refuse) - wants a failing repro first per `discipline.md` §3, so it needs its own change. +`docs/RUNTIME_LOGIC.md` states hbg "never advances `last_task_alive`". That +identifier does not exist anywhere in the hbg tree except that sentence. --- -## Already cleaned up by earlier work — not vestiges - -Recorded so the next audit does not re-open them. - -- `runtime/ring_buffer.h` is ~270 lines against tmr's 816: the reclaiming - allocator, its 500 ms deadlock backstop and the reclaim bookkeeping are gone, - with present-tense comments stating that nothing is reclaimed mid-run. -- The dep-pool arena region and the per-ring dep pools are gone; only comments - explaining their absence remain. -- The orchestrator and its TensorMap are host-owned rather than arena regions - (#1962), so the arena has two zones, both device-resident. - ## Deliberate, do not "fix" -- `AsyncWaitList` — live in hbg (`scheduler_dispatch.cpp` reads - `async_wait_list.count`). -- `PTO2SchedulerState` and its thirteen queue slot arrays — the scheduler really - runs on the AICPU, so the arena/offset form is required. -- `PTO2TensorMapEntry`'s 128 B two-cache-line mirror of `ChipTensor` — a shared - performance design, not a shape inherited by accident. +- `AsyncWaitList` — live in hbg (`scheduler_dispatch.cpp` reads `.count`). +- `SchedulerState` and its thirteen queue slot arrays — the scheduler runs on the + AICPU, so the arena/offset form is required. +- `TensorMapEntry`'s two-cache-line mirror of the runtime `Tensor`'s hot fields — a + shared performance design. Since #1974 it mirrors `simpler::{hbg,tmr}::Tensor`, + not the boundary type. - `ring_dep_pool` in the bind ABI — `[[maybe_unused]]` with "kept for ABI stability"; a deliberate cross-repo contract decision. +- `TaskTensor` — not a third type. A per-translation-unit alias for the runtime + being built, so kernels (identical either way, and several compiled under both) + name no runtime. The alternative was duplicating 13 MB of generated MoE kernels + into the hbg tree, where the copies would drift. ## Open sizing question (not a vestige) -`PTO2_TENSORMAP_POOL_SIZE` is `65536` in both runtimes, but tmr spends it -per-ring across 4 rings while hbg uses one ring's worth. Since #1962 that is an -explicit ~8 MB allocation per bind. With `GRAPH_MAX_NODES = 1024` and a default -task window of 16384 it is not a hard bound on either side (a capacity check backs -it), so whether hbg wants its own number is a measurable question, not a guess. +`TENSORMAP_POOL_SIZE` is `65536` in both runtimes, but tmr spends it per-ring +across 4 rings while hbg uses one ring's worth. Since #1962 that is an explicit +~8 MB allocation per bind. With `GRAPH_MAX_NODES = 1024` and a default task window +of 16384 it is not a hard bound on either side (a capacity check backs it), so +whether hbg wants its own number is a measurable question, not a guess. diff --git a/simpler_setup/tools/deps_viewer.py b/simpler_setup/tools/deps_viewer.py index d8485c8e25..80996ee667 100644 --- a/simpler_setup/tools/deps_viewer.py +++ b/simpler_setup/tools/deps_viewer.py @@ -122,7 +122,13 @@ def _normalize_small_int(v): def _node_id(task_id): - """DOT-safe node id: ``T{ring}_{local}``.""" + """DOT-safe node id: ``T{high}_{local}``. + + The high field is a ring index under ``tensormap_and_ringbuffer`` (any ring in + ``0..CHIP_MAX_RING_DEPTH-1``) and an id space under ``host_build_graph`` + (0 = RING, 1 = GRAPH_NODE), so its meaning depends on which runtime wrote the + record. It is printed verbatim either way. + """ tid = _normalize_task_id(task_id) if tid is None: return f"T_{task_id}" @@ -134,12 +140,12 @@ def _node_id(task_id): def _make_task_formatter(nodes): """Build a task-id → display-string formatter sized to the graph. - If every node lives in ring 0 the display is just ``{local}`` (the local + If every node has a 0 high field the display is just ``{local}`` (the local counter alone — no noise on workloads that never enter a manual scope). - The moment any node is in ring ≥ 1 we switch to the explicit - ``({ring}, {local})`` tuple for *every* node so the asymmetry is visible - instead of hidden (you can't have ``t0`` next to ``r1t3`` and know which - ring t0 lives in without context). + The moment any node has a nonzero one we switch to the explicit + ``({high}, {local})`` tuple for *every* node so the asymmetry is visible + instead of hidden (you can't have ``t0`` next to ``r1t3`` and know what + t0's high field is without context). """ has_multi_ring = False for n in nodes: diff --git a/simpler_setup/tools/swimlane_converter.py b/simpler_setup/tools/swimlane_converter.py index d484c066a5..40ce5e42c9 100644 --- a/simpler_setup/tools/swimlane_converter.py +++ b/simpler_setup/tools/swimlane_converter.py @@ -89,7 +89,7 @@ def _task_display_name(func_id, func_id_to_name, tdisp, *, spmd=False): def normalize_task_id_int(v): """Unsigned 64-bit task id (matches host JSON / device ``task_id.raw``). - Normalizes signed values to unsigned for ``(ring_id << 32) | local_id``. + Normalizes signed values to unsigned so the high field decodes correctly. Returns None if ``v`` is not convertible to int. """ try: @@ -104,10 +104,14 @@ def normalize_task_id_int(v): def format_task_display(task_id): """Format a task_id for human-readable labels. - Layout: 64-bit raw = (ring_id << 32) | local_id (same as runtime TaskId). + The high 32 bits are a ring index under ``tensormap_and_ringbuffer`` (any ring in + ``0..CHIP_MAX_RING_DEPTH-1``) and an id space under ``host_build_graph`` + (0 = RING, 1 = GRAPH_NODE). The short form below therefore covers tmr ring 0 and + every hbg RING task; a tmr task on ring 2 is equally ordinary and gets the long + form. Returns: - ``r{ring}t{local}`` when ring != 0 (e.g. r2t100), else ``t{local}`` for single-ring (ring 0). + ``r{high}t{local}`` when the high field != 0 (e.g. r2t100), else ``t{local}``. For invalid or non-numeric values, returns str(task_id). """ @@ -122,10 +126,11 @@ def format_task_display(task_id): def _decode_graph_node_task_id(task_id): - """Decode Scheduler-owned ring-1 Graph-node ids. + """Decode Scheduler-owned Graph-node ids. - Graph nodes use ``local=(outer_local << 10) | node_index`` while the - stream-visible outer Graph task remains on ring 0. + ``host_build_graph`` puts a materialized node in id space 1 (GRAPH_NODE) with + ``local=(outer_local << 10) | node_index``; the stream-visible outer Graph task + stays in space 0 (RING). See src/common/host_build_graph/task_id_encoding.h. """ tid = normalize_task_id_int(task_id) if tid is None or ((tid >> 32) & 0xFFFFFFFF) != 1: diff --git a/src/a2a3/runtime/host_build_graph/docs/profiling_levels.md b/src/a2a3/runtime/host_build_graph/docs/profiling_levels.md index 7534a297af..536d1221b1 100644 --- a/src/a2a3/runtime/host_build_graph/docs/profiling_levels.md +++ b/src/a2a3/runtime/host_build_graph/docs/profiling_levels.md @@ -394,7 +394,7 @@ header just like on onboard. | 4 | + Orchestrator phases (full) | At level 1 the AICore record carries the full `task_token_raw` -(`(ring_id << 32) | local_id`), read straight from +(a `TaskId::raw`; see `src/common/host_build_graph/task_id_encoding.h`), read straight from `LocalContext.async_ctx.task_token.raw` inside the AICore helper — already in cache from the dispatch payload, so no extra GM load. Identity fields the AICPU side used to write at level 1 (`func_id`, diff --git a/src/a2a3/runtime/host_build_graph/runtime/orchestrator_core/orchestrator.cpp b/src/a2a3/runtime/host_build_graph/runtime/orchestrator_core/orchestrator.cpp index 8d49941ecd..6bec0d4a2a 100644 --- a/src/a2a3/runtime/host_build_graph/runtime/orchestrator_core/orchestrator.cpp +++ b/src/a2a3/runtime/host_build_graph/runtime/orchestrator_core/orchestrator.cpp @@ -50,6 +50,7 @@ #include "dep_compute.h" #include "graph_execution.h" #include "graph_host_state.h" +#include "host_build_graph/task_id_encoding.h" #include "runtime_types.h" #include "shared_memory.h" #include "tensormap.h" @@ -1158,13 +1159,12 @@ struct FaninBuilder { int32_t *slots{nullptr}; bool mark_seen(TaskId producer_task_id) { - // Dedup only: a producer this orchestrator did not place has no entry in the - // epoch table, so it is reported as not-yet-seen and the caller appends it - // rather than rejecting it. hbg places every task on ring 0, so the assert is - // the invariant; enforcing it on the caller's side would be a new fatal path. - debug_assert(producer_task_id.ring() == 0 && "hbg places every task on ring 0"); - const int32_t prod_local = static_cast(producer_task_id.local()); - if (producer_task_id.ring() != 0 || prod_local < 0) { + // Dedup only. A negative local id cannot index the epoch table, so it is + // reported as not-yet-seen and the caller appends it rather than rejecting + // it; append_fanin_or_fail has already established that the producer is a + // ring task whose local id is a valid table entry. + const int32_t prod_local = static_cast(simpler::hbg::task_local_id(producer_task_id)); + if (prod_local < 0) { return false; } uint32_t *seen = orch->fanin_seen_epoch.get(); @@ -1177,14 +1177,36 @@ struct FaninBuilder { } }; -static bool append_fanin_or_fail( - OrchestratorState *orch, ChipTaskSlotState *prod_state, TaskId producer_task_id, FaninBuilder *fanin_builder -) { +static bool append_fanin_or_fail(OrchestratorState *orch, TaskId producer_task_id, FaninBuilder *fanin_builder) { + // Only a RING-space producer has an entry in the task table. A GRAPH_NODE id's + // low bits are a packed (outer task, node index) pair, so using them as a table + // index names an unrelated task — or, since get_slot_state_by_task_id does not + // bounds-check, no task at all. Nothing inside a Graph is reachable as a + // producer here — a recorded node hands out a RING id — so a foreign space is a + // caller error, not a case to tolerate. + // + // The table lookup lives here, after this check, rather than at the three call + // sites: that keeps the id-space invariant in one place and makes it impossible + // for a caller to form the out-of-bounds slot reference before reaching it. + if (!simpler::hbg::is_ring_task(producer_task_id)) { + orch->report_fatal( + SIMPLER_ERROR_INVALID_ARGS, __FUNCTION__, + "producer task %#llx is in id space %u, not RING; host_build_graph resolves every fanin edge against its " + "one task table", + static_cast(producer_task_id.raw), + static_cast(simpler::hbg::task_id_space(producer_task_id)) + ); + return false; + } + ChipTaskSlotState *prod_state = &orch->sm_header->tasks.get_slot_state_by_task_id( + static_cast(simpler::hbg::task_local_id(producer_task_id)) + ); // Skip a stale/reused producer slot: the cached owner id no longer resolves // to this producer (defensive — whole-graph-resident hbg does not reuse slots // at build time). A COMPLETED producer IS a real fanin edge under polling (its // completion_flags byte is set), so it is not skipped. - if (prod_state->task == nullptr || prod_state->task->task_id.local() != producer_task_id.local()) { + if (prod_state->task == nullptr || + simpler::hbg::task_local_id(prod_state->task->task_id) != simpler::hbg::task_local_id(producer_task_id)) { return true; } // Dedup by producer local id, which is also its task-table slot. @@ -1205,7 +1227,7 @@ static bool append_fanin_or_fail( orch_mark_fatal(orch, SIMPLER_ERROR_FANIN_CAPACITY_EXCEEDED); return false; } - fanin_builder->slots[fanin_builder->count++] = static_cast(producer_task_id.local()); + fanin_builder->slots[fanin_builder->count++] = static_cast(simpler::hbg::task_local_id(producer_task_id)); // Reclaim gate: record this task as a consumer of the producer. The producer // slot retires once the per-ring completed_watermark reaches this consumer id. @@ -1261,7 +1283,7 @@ static bool prepare_task( return false; } - out->task_id = TaskId::make(0, static_cast(out->alloc_result.task_id)); + out->task_id = simpler::hbg::make_ring_task(static_cast(out->alloc_result.task_id)); out->slot_state = &orch->sm_header->tasks.get_slot_state_by_task_id(out->alloc_result.task_id); out->task = &orch->sm_header->tasks.task_descriptors[out->alloc_result.task_id]; out->payload = &orch->sm_header->tasks.task_payloads[out->alloc_result.task_id]; @@ -1324,7 +1346,7 @@ static bool prepare_task( // is retirable once completed_watermark >= its own id. Each fanin edge bumps // it in append_fanin_or_fail. completion_flags for this slot were cleared // above (whole-graph-resident hbg never reuses a slot). - out->slot_state->last_consumer_local_id = static_cast(out->task_id.local()); + out->slot_state->last_consumer_local_id = static_cast(simpler::hbg::task_local_id(out->task_id)); // payload.fanin_count is set in submit_task_common's STEP 6. return true; @@ -1479,7 +1501,9 @@ static TaskOutputTensors submit_task_common( ); } - FaninBuilder fanin_builder(orch, &payload, static_cast(task_id.local()), next_fanin_seen_epoch(orch)); + FaninBuilder fanin_builder( + orch, &payload, static_cast(simpler::hbg::task_local_id(task_id)), next_fanin_seen_epoch(orch) + ); CYCLE_COUNT_LAP(g_orch_alloc_cycle); @@ -1501,10 +1525,7 @@ static TaskOutputTensors submit_task_common( if (capture_dep_graph) { dep_gen_host_graph_add_explicit_edge(dep_task_id.raw); } - SharedMemoryTaskHeader &dep_tasks = orch->sm_header->tasks; - int32_t dep_local_task_id = static_cast(dep_task_id.local()); - ChipTaskSlotState *producer_slot_state = &dep_tasks.get_slot_state_by_task_id(dep_local_task_id); - if (!append_fanin_or_fail(orch, producer_slot_state, dep_task_id, &fanin_builder)) { + if (!append_fanin_or_fail(orch, dep_task_id, &fanin_builder)) { return result; } } @@ -1516,10 +1537,7 @@ static TaskOutputTensors submit_task_common( }; auto runtime_emit = [&](TaskId producer_task_id) -> bool { - SharedMemoryTaskHeader &producer_tasks = orch->sm_header->tasks; - ChipTaskSlotState *prod_state = - &producer_tasks.get_slot_state_by_task_id(static_cast(producer_task_id.local())); - return append_fanin_or_fail(orch, prod_state, producer_task_id, &fanin_builder); + return append_fanin_or_fail(orch, producer_task_id, &fanin_builder); }; // The capture branch instantiates compute_task_fanin with a live Annotate; @@ -1791,7 +1809,7 @@ bool graph_submit_outer( orch_mark_fatal(orch, SIMPLER_ERROR_HEAP_RING_DEADLOCK); return false; } - const TaskId task_id = TaskId::make(0, static_cast(allocation.task_id)); + const TaskId task_id = simpler::hbg::make_ring_task(static_cast(allocation.task_id)); SharedMemoryTaskHeader &tasks = orch->sm_header->tasks; TaskDescriptor &task = tasks.task_descriptors[allocation.task_id]; TaskPayload &payload = tasks.task_payloads[allocation.task_id]; @@ -1817,7 +1835,7 @@ bool graph_submit_outer( orch->tensor_pool_cursor += tensor_slots; orch->scalar_pool_cursor += scalar_span; slot.task_state.store(CHIP_TASK_PENDING, std::memory_order_relaxed); - slot.last_consumer_local_id = static_cast(task_id.local()); + slot.last_consumer_local_id = static_cast(simpler::hbg::task_local_id(task_id)); slot.active_mask = ActiveMask{}; slot.task_attrs = TaskAttrs{}; slot.total_required_subtasks = 0; @@ -1841,10 +1859,11 @@ bool graph_submit_outer( ); } - FaninBuilder fanin_builder(orch, &payload, static_cast(task_id.local()), next_fanin_seen_epoch(orch)); + FaninBuilder fanin_builder( + orch, &payload, static_cast(simpler::hbg::task_local_id(task_id)), next_fanin_seen_epoch(orch) + ); auto emit = [&](TaskId producer_id) -> bool { - ChipTaskSlotState *producer = &tasks.get_slot_state_by_task_id(static_cast(producer_id.local())); - return append_fanin_or_fail(orch, producer, producer_id, &fanin_builder); + return append_fanin_or_fail(orch, producer_id, &fanin_builder); }; // An outer GRAPH task is an ordinary task, so the dependency graph // has to carry it: without this the whole Graph — and every edge into it — @@ -1967,8 +1986,9 @@ TaskOutputTensors graph_record_submit_node( // A recorded node's index equals its local task id minus the recording // baseline, so its synthetic id keeps classification and explicit-dep // arithmetic identical to the ordinary path. - const TaskId task_id = - TaskId::make(0, static_cast(recording.start_local_task_id) + static_cast(node_index)); + const TaskId task_id = simpler::hbg::make_ring_task( + static_cast(recording.start_local_task_id) + static_cast(node_index) + ); result.set_task_id(task_id); if (node_index >= GRAPH_MAX_NODES || args.has_error) { @@ -2164,8 +2184,10 @@ TaskOutputTensors graph_record_submit_node( recording.unsupported = true; } else { auto emit_inferred = [&recording, &add_fanin, node_index](TaskId producer) -> bool { - const int32_t producer_index = static_cast(producer.local()) - recording.start_local_task_id; - if (producer.ring() == 0 && producer_index >= 0 && producer_index < static_cast(node_index)) { + const int32_t producer_index = + static_cast(simpler::hbg::task_local_id(producer)) - recording.start_local_task_id; + if (simpler::hbg::is_ring_task(producer) && producer_index >= 0 && + producer_index < static_cast(node_index)) { add_fanin(static_cast(producer_index)); } return true; @@ -2176,8 +2198,9 @@ TaskOutputTensors graph_record_submit_node( } for (uint32_t i = 0; i < args.explicit_dep_count(); ++i) { const TaskId dep = args.explicit_dep(i); - const int32_t dep_index = static_cast(dep.local()) - recording.start_local_task_id; - if (!dep.is_valid() || dep.ring() != 0 || dep_index >= static_cast(node_index)) { + const int32_t dep_index = + static_cast(simpler::hbg::task_local_id(dep)) - recording.start_local_task_id; + if (!dep.is_valid() || !simpler::hbg::is_ring_task(dep) || dep_index >= static_cast(node_index)) { recording.unsupported = true; continue; } @@ -2744,7 +2767,7 @@ TaskOutputTensors OrchestratorState::alloc_tensors(const CoreTaskArgs &args) { // the run hangs. (The device watermark walk transparently steps past this // pre-set flag when a later on-device task completes.) SharedMemoryTaskHeader &done_tasks = orch->sm_header->tasks; - int32_t done_local = static_cast(prepared.task_id.local()); + int32_t done_local = static_cast(simpler::hbg::task_local_id(prepared.task_id)); done_tasks.set_completion_flag(done_local); } orch->inline_completed_tasks++; diff --git a/src/a2a3/runtime/host_build_graph/runtime/orchestrator_core/runtime_core.cpp b/src/a2a3/runtime/host_build_graph/runtime/orchestrator_core/runtime_core.cpp index 698f3bdaaa..b599217dfb 100644 --- a/src/a2a3/runtime/host_build_graph/runtime/orchestrator_core/runtime_core.cpp +++ b/src/a2a3/runtime/host_build_graph/runtime/orchestrator_core/runtime_core.cpp @@ -31,6 +31,7 @@ #include "aicpu/device_time.h" #include "common/platform_config.h" #include "common/unified_log.h" +#include "host_build_graph/task_id_encoding.h" #include "host_tensor_access.h" // simpler::hbg::Tensor-byte access for a caller that can load a device address directly. @@ -185,18 +186,20 @@ static bool wait_for_tensor_ready( // Callers branch on the returned pointer, not on `failed`: the slot is // dereferenced immediately and a null return is the only safe signal. auto resolve_producer = [&](TaskId producer) -> const ChipTaskSlotState * { - if (!producer.is_valid() || producer.ring() != 0) { + if (!producer.is_valid() || !simpler::hbg::is_ring_task(producer)) { orch.report_fatal( SIMPLER_ERROR_INVALID_ARGS, caller, - "tensor producer task %#llx has invalid ring %u; host_build_graph uses ring 0", - static_cast(producer.raw), static_cast(producer.ring()) + "tensor producer task %#llx is in id space %u, not RING; host_build_graph resolves a producer against " + "its one task table", + static_cast(producer.raw), + static_cast(simpler::hbg::task_id_space(producer)) ); failed = true; return nullptr; } auto &tasks = orch.sm_header->tasks; - int32_t local_id = static_cast(producer.local()); + int32_t local_id = static_cast(simpler::hbg::task_local_id(producer)); auto &slot = tasks.get_slot_state_by_task_id(local_id); if (slot.task == nullptr || slot.task->task_id != producer) { orch.report_fatal( @@ -211,7 +214,7 @@ static bool wait_for_tensor_ready( }; auto wait_one_producer = [&](const ChipTaskSlotState &slot) { - int32_t local_id = static_cast(slot.task->task_id.local()); + int32_t local_id = static_cast(simpler::hbg::task_local_id(slot.task->task_id)); uint64_t t0 = get_sys_cnt_aicpu(); int32_t spin_count = 0; while (slot.task_state.load(std::memory_order_acquire) < CHIP_TASK_COMPLETED) { @@ -237,7 +240,7 @@ static bool wait_for_tensor_ready( }; auto wait_one_consumers = [&](const ChipTaskSlotState &slot) { - int32_t local_id = slot.task->task_id.local(); + int32_t local_id = simpler::hbg::task_local_id(slot.task->task_id); uint64_t t0 = get_sys_cnt_aicpu(); int32_t spin_count = 0; // Polling: all consumers of this producer have retired once the per-ring diff --git a/src/a2a3/runtime/host_build_graph/runtime/runtime_types.h b/src/a2a3/runtime/host_build_graph/runtime/runtime_types.h index d78be43d3b..f0dee16f41 100644 --- a/src/a2a3/runtime/host_build_graph/runtime/runtime_types.h +++ b/src/a2a3/runtime/host_build_graph/runtime/runtime_types.h @@ -243,8 +243,9 @@ struct ChipTaskSlotState; // Forward declaration (defined below) * Fields set by Orchestrator at submission, read by Scheduler for dispatch. */ struct TaskDescriptor { - // Mixed-task identification (encodes ring_id in upper 32 bits) - TaskId task_id; // raw: (ring_id << 32) | local_id + // Task identity. See src/common/host_build_graph/task_id_encoding.h: the + // upper 32 bits are this runtime's id space, not a ring index. + TaskId task_id; // Per-slot kernel IDs (INVALID_KERNEL_ID = inactive) int32_t kernel_id[SUBTASK_SLOT_COUNT]; diff --git a/src/a2a3/runtime/host_build_graph/runtime/scheduler/scheduler.h b/src/a2a3/runtime/host_build_graph/runtime/scheduler/scheduler.h index 65cf34b583..d52d331a89 100644 --- a/src/a2a3/runtime/host_build_graph/runtime/scheduler/scheduler.h +++ b/src/a2a3/runtime/host_build_graph/runtime/scheduler/scheduler.h @@ -39,6 +39,7 @@ #include "aicpu/platform_regs.h" // get_reg_ptr / RegId for the early-dispatch doorbell #include "async_wait.h" #include "graph_execution.h" +#include "host_build_graph/task_id_encoding.h" #include "task_allocator.h" #include "runtime_types.h" #include "shared_memory.h" @@ -624,7 +625,7 @@ struct SchedulerState { // watermark >= producer.last_consumer_local_id). Whole-graph-resident hbg // has no device slot reclaim, so nothing advances a reclaim cursor here. void on_mixed_task_complete(ChipTaskSlotState &slot_state) { - const int32_t task_id = static_cast(slot_state.task->task_id.local()); + const int32_t task_id = static_cast(simpler::hbg::task_local_id(slot_state.task->task_id)); SharedMemoryTaskHeader &tasks = *task_view.tasks; slot_state.mark_completed(); // host-visible mirror (task_state = COMPLETED) diff --git a/src/a2a3/runtime/host_build_graph/runtime/tensormap.h b/src/a2a3/runtime/host_build_graph/runtime/tensormap.h index 0ad8d89a84..f08672a9ba 100644 --- a/src/a2a3/runtime/host_build_graph/runtime/tensormap.h +++ b/src/a2a3/runtime/host_build_graph/runtime/tensormap.h @@ -50,6 +50,7 @@ #include #include "common.h" +#include "host_build_graph/task_id_encoding.h" #include "profiling_config.h" #include "runtime_types.h" #include "tensor.h" @@ -564,7 +565,7 @@ struct ChipTensorMap { g_insert_count++; #endif uint32_t bucket_index = hash(addr); - auto local_id = producer_task_id.local(); + auto local_id = simpler::hbg::task_local_id(producer_task_id); const int32_t task_slot = local_id; entry->producer_task_id = producer_task_id; @@ -602,7 +603,7 @@ struct ChipTensorMap { // Update predecessor's next pointer (O(1) via prev_in_task) if (entry.prev_in_task == nullptr) { // Entry is the head of its task chain, update task_entry_heads - int32_t local_id = static_cast(entry.producer_task_id.local()); + int32_t local_id = static_cast(simpler::hbg::task_local_id(entry.producer_task_id)); const int32_t task_slot = local_id; task_entry_heads[task_slot] = entry.next_in_task; } else { diff --git a/src/a2a3/runtime/tensormap_and_ringbuffer/docs/MULTI_RING.md b/src/a2a3/runtime/tensormap_and_ringbuffer/docs/MULTI_RING.md index 10b03b14e1..8dc6d36cd3 100644 --- a/src/a2a3/runtime/tensormap_and_ringbuffer/docs/MULTI_RING.md +++ b/src/a2a3/runtime/tensormap_and_ringbuffer/docs/MULTI_RING.md @@ -27,13 +27,14 @@ Task IDs are widened from 32-bit to 64-bit to carry the ring identity: task_id.raw = (ring_id << 32) | local_id ``` -`TaskId` exposes direct accessors in `src/common/task_interface/task_id.h`: +`TaskId` itself is an opaque 64-bit handle; this runtime's encoding of it lives in +`src/common/tensormap_and_ringbuffer/task_id_encoding.h`: | API | Purpose | | --- | ------- | -| `TaskId::make(ring_id, local_id)` | Compose a 64-bit task ID (`TaskId`) | -| `task_id.ring()` | Extract `ring_id` (bits 63-32) | -| `task_id.local()` | Extract `local_id` (bits 31-0) | +| `simpler::tmr::make_task_id(ring_id, local_id)` | Compose a 64-bit task ID (`TaskId`) | +| `simpler::tmr::task_ring(task_id)` | Extract `ring_id` (bits 63-32) | +| `simpler::tmr::task_local_id(task_id)` | Extract `local_id` (bits 31-0) | | `task_id.raw` | Access the packed 64-bit encoding | Type changes: @@ -158,8 +159,8 @@ Entry validity checks and `cleanup_retired` operate per-ring: ```cpp bool entry_valid(const ChipTensorMapEntry& e) { - int32_t ring = e.producer_task_id.ring(); - int32_t local = e.producer_task_id.local(); + int32_t ring = simpler::tmr::task_ring(e.producer_task_id); + int32_t local = simpler::tmr::task_local_id(e.producer_task_id); return local >= last_task_alives[ring]; } ``` diff --git a/src/a2a3/runtime/tensormap_and_ringbuffer/host/dep_gen_replay.cpp b/src/a2a3/runtime/tensormap_and_ringbuffer/host/dep_gen_replay.cpp index 5c52ce43b9..dfcdc5e292 100644 --- a/src/a2a3/runtime/tensormap_and_ringbuffer/host/dep_gen_replay.cpp +++ b/src/a2a3/runtime/tensormap_and_ringbuffer/host/dep_gen_replay.cpp @@ -73,6 +73,7 @@ #include "data_type.h" #include "dep_compute.h" #include "task_id.h" +#include "tensormap_and_ringbuffer/task_id_encoding.h" #include "tensormap.h" #include "tensor.h" @@ -486,8 +487,8 @@ dep_gen_replay_emit_deps_json(const DepGenRecord *records, size_t num_records, c uint32_t max_local[CHIP_MAX_RING_DEPTH] = {0}; for (size_t i = 0; i < num_records; i++) { TaskId tid{records[i].task_id}; - uint8_t ring = tid.ring(); - uint32_t local = tid.local(); + uint8_t ring = simpler::tmr::task_ring(tid); + uint32_t local = simpler::tmr::task_local_id(tid); if (ring < CHIP_MAX_RING_DEPTH && local > max_local[ring]) { max_local[ring] = local; } diff --git a/src/a2a3/runtime/tensormap_and_ringbuffer/runtime/orchestrator.cpp b/src/a2a3/runtime/tensormap_and_ringbuffer/runtime/orchestrator.cpp index c38fec645e..0f3427fc2a 100644 --- a/src/a2a3/runtime/tensormap_and_ringbuffer/runtime/orchestrator.cpp +++ b/src/a2a3/runtime/tensormap_and_ringbuffer/runtime/orchestrator.cpp @@ -32,6 +32,7 @@ #include "dep_compute.h" #include "runtime_types.h" #include "shared_memory.h" +#include "tensormap_and_ringbuffer/task_id_encoding.h" #include "tensormap.h" #include "types.h" #include "tensor.h" @@ -351,8 +352,10 @@ static bool append_fanin_or_fail( // explicit) always wins, independent of the order they are reached in. prod_state->lock_fanout(); ChipTaskState pstate = prod_state->task_state.load(std::memory_order_acquire); - bool gone = prod_state->task == nullptr || prod_state->task->task_id.local() != producer_task_id.local() || - pstate == CHIP_TASK_CONSUMED; + bool gone = + prod_state->task == nullptr || + simpler::tmr::task_local_id(prod_state->task->task_id) != simpler::tmr::task_local_id(producer_task_id) || + pstate == CHIP_TASK_CONSUMED; bool already_seen = !gone && fanin_builder->mark_seen(prod_ring, prod_slot); bool claim = !gone && !already_seen; int32_t fanout_now = -1; @@ -378,7 +381,8 @@ static bool append_fanin_or_fail( if (fanout_now == CHIP_DEP_DEGREE_DEBUG_THRESHOLD + 1) { LOG_DEBUG( "dense dependency: task ring=%u id=%u fanout>%d [orch submit]", - static_cast(producer_task_id.ring()), producer_task_id.local(), CHIP_DEP_DEGREE_DEBUG_THRESHOLD + static_cast(simpler::tmr::task_ring(producer_task_id)), + simpler::tmr::task_local_id(producer_task_id), CHIP_DEP_DEGREE_DEBUG_THRESHOLD ); } // Stale/consumed producer: no edge at all. @@ -609,7 +613,7 @@ static bool prepare_task( return false; } - out->task_id = TaskId::make(ring_id, static_cast(out->alloc_result.task_id)); + out->task_id = simpler::tmr::make_task_id(ring_id, static_cast(out->alloc_result.task_id)); out->slot_state = &orch->sm_header->rings[ring_id].get_slot_state_by_slot(out->alloc_result.slot); out->task = &orch->sm_header->rings[ring_id].task_descriptors[out->alloc_result.slot]; out->payload = &orch->sm_header->rings[ring_id].task_payloads[out->alloc_result.slot]; @@ -892,7 +896,7 @@ static TaskOutputTensors submit_task_common( if (!prepare_task(orch, args, layout.total_output_size, active_mask, task_attrs, &prepared)) { return result; } - uint8_t ring_id = prepared.task_id.ring(); + uint8_t ring_id = simpler::tmr::task_ring(prepared.task_id); SchedulerState *sched = orch->scheduler; ChipRingFlowControl &fc = orch->sm_header->rings[ring_id].fc; TaskId task_id = prepared.task_id; @@ -964,9 +968,9 @@ static TaskOutputTensors submit_task_common( ); return result; } - uint8_t dep_ring_id = dep_task_id.ring(); + uint8_t dep_ring_id = simpler::tmr::task_ring(dep_task_id); SharedMemoryRingHeader &dep_ring = orch->sm_header->rings[dep_ring_id]; - int32_t dep_local_task_id = static_cast(dep_task_id.local()); + int32_t dep_local_task_id = static_cast(simpler::tmr::task_local_id(dep_task_id)); int32_t dep_last_task_alive = dep_ring.fc.last_task_alive.load(std::memory_order_acquire); if (dep_local_task_id < dep_last_task_alive) { continue; @@ -988,9 +992,10 @@ static TaskOutputTensors submit_task_common( }; auto runtime_emit = [&](TaskId producer_task_id, DepFlags kind) -> bool { - uint8_t prod_ring = producer_task_id.ring(); + uint8_t prod_ring = simpler::tmr::task_ring(producer_task_id); SharedMemoryRingHeader &producer_ring = orch->sm_header->rings[prod_ring]; - int32_t prod_slot = producer_ring.get_slot_by_task_id(static_cast(producer_task_id.local())); + int32_t prod_slot = + producer_ring.get_slot_by_task_id(static_cast(simpler::tmr::task_local_id(producer_task_id))); ChipTaskSlotState *prod_state = &producer_ring.get_slot_state_by_slot(prod_slot); return append_fanin_or_fail( orch, prod_ring, prod_slot, prod_state, producer_task_id, &fanin_builder, ring_id, kind @@ -1050,8 +1055,9 @@ static TaskOutputTensors submit_task_common( // THRESHOLD because the count lands at its final total here. if (fanin_builder.count > CHIP_DEP_DEGREE_DEBUG_THRESHOLD) { LOG_DEBUG( - "dense dependency: task ring=%u id=%u fanin>%d [orch submit]", static_cast(task_id.ring()), - task_id.local(), CHIP_DEP_DEGREE_DEBUG_THRESHOLD + "dense dependency: task ring=%u id=%u fanin>%d [orch submit]", + static_cast(simpler::tmr::task_ring(task_id)), simpler::tmr::task_local_id(task_id), + CHIP_DEP_DEGREE_DEBUG_THRESHOLD ); } payload.fanin_spill_start = fanin_builder.spill_start; @@ -1322,7 +1328,7 @@ TaskOutputTensors OrchestratorState::alloc_tensors(const CoreTaskArgs &args) { payload.init(args, outputs, prepared.alloc_result, layout); payload.fanin_actual_count = 0; payload.fanin_spill_start = 0; - payload.fanin_spill_pool = &orch->rings[prepared.task_id.ring()].fanin_pool; + payload.fanin_spill_pool = &orch->rings[simpler::tmr::task_ring(prepared.task_id)].fanin_pool; CYCLE_COUNT_LAP(g_orch_args_cycle); if (prepared.slot_state != nullptr) { diff --git a/src/a2a3/runtime/tensormap_and_ringbuffer/runtime/runtime_core.cpp b/src/a2a3/runtime/tensormap_and_ringbuffer/runtime/runtime_core.cpp index 3342386a16..0a7db6f7c0 100644 --- a/src/a2a3/runtime/tensormap_and_ringbuffer/runtime/runtime_core.cpp +++ b/src/a2a3/runtime/tensormap_and_ringbuffer/runtime/runtime_core.cpp @@ -29,6 +29,7 @@ #include "aicpu/device_time.h" #include "common/platform_config.h" // PLATFORM_PROF_SYS_CNT_FREQ (data-wait deadline) #include "common/unified_log.h" +#include "tensormap_and_ringbuffer/task_id_encoding.h" #if SIMPLER_DFX #include "aicpu/scope_stats_collector_aicpu.h" #endif @@ -114,7 +115,7 @@ static bool wait_for_tensor_ready( auto wait_one_producer = [&](const ChipTaskSlotState &slot) { uint8_t ring_id = slot.ring_id; - int32_t local_id = static_cast(slot.task->task_id.local()); + int32_t local_id = static_cast(simpler::tmr::task_local_id(slot.task->task_id)); uint64_t t0 = get_sys_cnt_aicpu(); int32_t spin_count = 0; while (slot.task_state.load(std::memory_order_acquire) < CHIP_TASK_COMPLETED) { @@ -140,7 +141,7 @@ static bool wait_for_tensor_ready( auto wait_one_consumers = [&](const ChipTaskSlotState &slot) { uint8_t ring_id = slot.ring_id; - int32_t local_id = slot.task->task_id.local(); + int32_t local_id = simpler::tmr::task_local_id(slot.task->task_id); uint64_t t0 = get_sys_cnt_aicpu(); int32_t spin_count = 0; while ((slot.fanout_refcount.load(std::memory_order_acquire) & ~FANOUT_SCOPE_BIT) < @@ -190,7 +191,9 @@ static bool wait_for_tensor_ready( auto do_wait = [&]() { // Step A: creator retention — read owner directly from tensor metadata if (owner.is_valid()) { - auto &s = orch.sm_header->rings[owner.ring()].get_slot_state_by_task_id(owner.local()); + auto &s = orch.sm_header->rings[simpler::tmr::task_ring(owner)].get_slot_state_by_task_id( + simpler::tmr::task_local_id(owner) + ); try_push(s); if (failed) return; } @@ -198,7 +201,9 @@ static bool wait_for_tensor_ready( // Step B: modifier writer lookup (OverlapMap), direct callback orch.tensor_map.lookup(tensor, [&](ChipTensorMapEntry &entry, OverlapStatus) -> bool { TaskId pid = entry.producer_task_id; - auto &s = orch.sm_header->rings[pid.ring()].get_slot_state_by_task_id(pid.local()); + auto &s = orch.sm_header->rings[simpler::tmr::task_ring(pid)].get_slot_state_by_task_id( + simpler::tmr::task_local_id(pid) + ); try_push(s); return !failed; }); diff --git a/src/a2a3/runtime/tensormap_and_ringbuffer/runtime/runtime_types.h b/src/a2a3/runtime/tensormap_and_ringbuffer/runtime/runtime_types.h index fd920a8bc9..4ef15b716d 100644 --- a/src/a2a3/runtime/tensormap_and_ringbuffer/runtime/runtime_types.h +++ b/src/a2a3/runtime/tensormap_and_ringbuffer/runtime/runtime_types.h @@ -217,7 +217,7 @@ struct DepListEntry { */ struct TaskDescriptor { // Mixed-task identification (encodes ring_id in upper 32 bits) - TaskId task_id; // raw: (ring_id << 32) | local_id + TaskId task_id; // raw: (ring_id << 32) | local_id, see task_id_encoding.h // Per-slot kernel IDs (INVALID_KERNEL_ID = inactive) int32_t kernel_id[SUBTASK_SLOT_COUNT]; diff --git a/src/a2a3/runtime/tensormap_and_ringbuffer/runtime/shared/tensormap.cpp b/src/a2a3/runtime/tensormap_and_ringbuffer/runtime/shared/tensormap.cpp index dfa35caace..a5ed7174e5 100644 --- a/src/a2a3/runtime/tensormap_and_ringbuffer/runtime/shared/tensormap.cpp +++ b/src/a2a3/runtime/tensormap_and_ringbuffer/runtime/shared/tensormap.cpp @@ -30,6 +30,7 @@ #include "common.h" #include "common/unified_log.h" +#include "tensormap_and_ringbuffer/task_id_encoding.h" // ============================================================================= // TensorMap Lookup Chain Length Statistics (compile-time toggle) @@ -256,8 +257,8 @@ int32_t ChipTensorMap::valid_count() { } void ChipTensorMap::sync_tensormap(TaskId task_id, int32_t sm_last_task_alive) { - auto ring_id = task_id.ring(); - auto local_id = task_id.local(); + auto ring_id = simpler::tmr::task_ring(task_id); + auto local_id = simpler::tmr::task_local_id(task_id); sync_validity(ring_id, sm_last_task_alive); // Only attempt cleanup when last_task_alive has actually advanced; diff --git a/src/a2a3/runtime/tensormap_and_ringbuffer/runtime/tensormap.h b/src/a2a3/runtime/tensormap_and_ringbuffer/runtime/tensormap.h index 0f0ef9533d..0d450bd385 100644 --- a/src/a2a3/runtime/tensormap_and_ringbuffer/runtime/tensormap.h +++ b/src/a2a3/runtime/tensormap_and_ringbuffer/runtime/tensormap.h @@ -46,6 +46,7 @@ #include "profiling_config.h" #include "utils/device_arena.h" #include "runtime_types.h" +#include "tensormap_and_ringbuffer/task_id_encoding.h" #include "tensor.h" // Overlap geometry types. Relocated here from tensor.h: they are used only by @@ -614,7 +615,8 @@ struct ChipTensorMap { // reused the slot (local_id + N * window) before this cleanup ran. // Free only entries produced by the retiring local_id, unlinking // each from the chain; entries from other tasks stay linked. - TaskId retired_task = TaskId::make(static_cast(ring_id), static_cast(local_id)); + TaskId retired_task = + simpler::tmr::make_task_id(static_cast(ring_id), static_cast(local_id)); ChipTensorMapEntry *cur_entry = task_entry_heads[ring_id][task_slot]; while (cur_entry != nullptr) { ChipTensorMapEntry *next_entry = cur_entry->next_in_task; // free_entry clears it @@ -659,8 +661,8 @@ struct ChipTensorMap { g_insert_count++; #endif uint32_t bucket_index = hash(addr); - auto ring_id = producer_task_id.ring(); - auto local_id = producer_task_id.local(); + auto ring_id = simpler::tmr::task_ring(producer_task_id); + auto local_id = simpler::tmr::task_local_id(producer_task_id); int32_t task_slot = local_id & (task_window_sizes[ring_id] - 1); entry->producer_task_id = producer_task_id; @@ -695,7 +697,8 @@ struct ChipTensorMap { * Check if entry is valid (producer has not retired) */ bool entry_valid(const ChipTensorMapEntry &entry) const { - return static_cast(entry.producer_task_id.local()) >= last_task_alives[entry.producer_task_id.ring()]; + return static_cast(simpler::tmr::task_local_id(entry.producer_task_id)) >= + last_task_alives[simpler::tmr::task_ring(entry.producer_task_id)]; } void remove_entry(ChipTensorMapEntry &entry) { @@ -712,8 +715,8 @@ struct ChipTensorMap { // Update predecessor's next pointer (O(1) via prev_in_task) if (entry.prev_in_task == nullptr) { // Entry is the head of its task chain, update task_entry_heads - int32_t ring_id = entry.producer_task_id.ring(); - int32_t local_id = static_cast(entry.producer_task_id.local()); + int32_t ring_id = simpler::tmr::task_ring(entry.producer_task_id); + int32_t local_id = static_cast(simpler::tmr::task_local_id(entry.producer_task_id)); int32_t task_slot = local_id & (task_window_sizes[ring_id] - 1); task_entry_heads[ring_id][task_slot] = entry.next_in_task; } else { diff --git a/src/a5/runtime/host_build_graph/docs/profiling_levels.md b/src/a5/runtime/host_build_graph/docs/profiling_levels.md index 7534a297af..536d1221b1 100644 --- a/src/a5/runtime/host_build_graph/docs/profiling_levels.md +++ b/src/a5/runtime/host_build_graph/docs/profiling_levels.md @@ -394,7 +394,7 @@ header just like on onboard. | 4 | + Orchestrator phases (full) | At level 1 the AICore record carries the full `task_token_raw` -(`(ring_id << 32) | local_id`), read straight from +(a `TaskId::raw`; see `src/common/host_build_graph/task_id_encoding.h`), read straight from `LocalContext.async_ctx.task_token.raw` inside the AICore helper — already in cache from the dispatch payload, so no extra GM load. Identity fields the AICPU side used to write at level 1 (`func_id`, diff --git a/src/a5/runtime/host_build_graph/runtime/orchestrator_core/orchestrator.cpp b/src/a5/runtime/host_build_graph/runtime/orchestrator_core/orchestrator.cpp index 8d49941ecd..6bec0d4a2a 100644 --- a/src/a5/runtime/host_build_graph/runtime/orchestrator_core/orchestrator.cpp +++ b/src/a5/runtime/host_build_graph/runtime/orchestrator_core/orchestrator.cpp @@ -50,6 +50,7 @@ #include "dep_compute.h" #include "graph_execution.h" #include "graph_host_state.h" +#include "host_build_graph/task_id_encoding.h" #include "runtime_types.h" #include "shared_memory.h" #include "tensormap.h" @@ -1158,13 +1159,12 @@ struct FaninBuilder { int32_t *slots{nullptr}; bool mark_seen(TaskId producer_task_id) { - // Dedup only: a producer this orchestrator did not place has no entry in the - // epoch table, so it is reported as not-yet-seen and the caller appends it - // rather than rejecting it. hbg places every task on ring 0, so the assert is - // the invariant; enforcing it on the caller's side would be a new fatal path. - debug_assert(producer_task_id.ring() == 0 && "hbg places every task on ring 0"); - const int32_t prod_local = static_cast(producer_task_id.local()); - if (producer_task_id.ring() != 0 || prod_local < 0) { + // Dedup only. A negative local id cannot index the epoch table, so it is + // reported as not-yet-seen and the caller appends it rather than rejecting + // it; append_fanin_or_fail has already established that the producer is a + // ring task whose local id is a valid table entry. + const int32_t prod_local = static_cast(simpler::hbg::task_local_id(producer_task_id)); + if (prod_local < 0) { return false; } uint32_t *seen = orch->fanin_seen_epoch.get(); @@ -1177,14 +1177,36 @@ struct FaninBuilder { } }; -static bool append_fanin_or_fail( - OrchestratorState *orch, ChipTaskSlotState *prod_state, TaskId producer_task_id, FaninBuilder *fanin_builder -) { +static bool append_fanin_or_fail(OrchestratorState *orch, TaskId producer_task_id, FaninBuilder *fanin_builder) { + // Only a RING-space producer has an entry in the task table. A GRAPH_NODE id's + // low bits are a packed (outer task, node index) pair, so using them as a table + // index names an unrelated task — or, since get_slot_state_by_task_id does not + // bounds-check, no task at all. Nothing inside a Graph is reachable as a + // producer here — a recorded node hands out a RING id — so a foreign space is a + // caller error, not a case to tolerate. + // + // The table lookup lives here, after this check, rather than at the three call + // sites: that keeps the id-space invariant in one place and makes it impossible + // for a caller to form the out-of-bounds slot reference before reaching it. + if (!simpler::hbg::is_ring_task(producer_task_id)) { + orch->report_fatal( + SIMPLER_ERROR_INVALID_ARGS, __FUNCTION__, + "producer task %#llx is in id space %u, not RING; host_build_graph resolves every fanin edge against its " + "one task table", + static_cast(producer_task_id.raw), + static_cast(simpler::hbg::task_id_space(producer_task_id)) + ); + return false; + } + ChipTaskSlotState *prod_state = &orch->sm_header->tasks.get_slot_state_by_task_id( + static_cast(simpler::hbg::task_local_id(producer_task_id)) + ); // Skip a stale/reused producer slot: the cached owner id no longer resolves // to this producer (defensive — whole-graph-resident hbg does not reuse slots // at build time). A COMPLETED producer IS a real fanin edge under polling (its // completion_flags byte is set), so it is not skipped. - if (prod_state->task == nullptr || prod_state->task->task_id.local() != producer_task_id.local()) { + if (prod_state->task == nullptr || + simpler::hbg::task_local_id(prod_state->task->task_id) != simpler::hbg::task_local_id(producer_task_id)) { return true; } // Dedup by producer local id, which is also its task-table slot. @@ -1205,7 +1227,7 @@ static bool append_fanin_or_fail( orch_mark_fatal(orch, SIMPLER_ERROR_FANIN_CAPACITY_EXCEEDED); return false; } - fanin_builder->slots[fanin_builder->count++] = static_cast(producer_task_id.local()); + fanin_builder->slots[fanin_builder->count++] = static_cast(simpler::hbg::task_local_id(producer_task_id)); // Reclaim gate: record this task as a consumer of the producer. The producer // slot retires once the per-ring completed_watermark reaches this consumer id. @@ -1261,7 +1283,7 @@ static bool prepare_task( return false; } - out->task_id = TaskId::make(0, static_cast(out->alloc_result.task_id)); + out->task_id = simpler::hbg::make_ring_task(static_cast(out->alloc_result.task_id)); out->slot_state = &orch->sm_header->tasks.get_slot_state_by_task_id(out->alloc_result.task_id); out->task = &orch->sm_header->tasks.task_descriptors[out->alloc_result.task_id]; out->payload = &orch->sm_header->tasks.task_payloads[out->alloc_result.task_id]; @@ -1324,7 +1346,7 @@ static bool prepare_task( // is retirable once completed_watermark >= its own id. Each fanin edge bumps // it in append_fanin_or_fail. completion_flags for this slot were cleared // above (whole-graph-resident hbg never reuses a slot). - out->slot_state->last_consumer_local_id = static_cast(out->task_id.local()); + out->slot_state->last_consumer_local_id = static_cast(simpler::hbg::task_local_id(out->task_id)); // payload.fanin_count is set in submit_task_common's STEP 6. return true; @@ -1479,7 +1501,9 @@ static TaskOutputTensors submit_task_common( ); } - FaninBuilder fanin_builder(orch, &payload, static_cast(task_id.local()), next_fanin_seen_epoch(orch)); + FaninBuilder fanin_builder( + orch, &payload, static_cast(simpler::hbg::task_local_id(task_id)), next_fanin_seen_epoch(orch) + ); CYCLE_COUNT_LAP(g_orch_alloc_cycle); @@ -1501,10 +1525,7 @@ static TaskOutputTensors submit_task_common( if (capture_dep_graph) { dep_gen_host_graph_add_explicit_edge(dep_task_id.raw); } - SharedMemoryTaskHeader &dep_tasks = orch->sm_header->tasks; - int32_t dep_local_task_id = static_cast(dep_task_id.local()); - ChipTaskSlotState *producer_slot_state = &dep_tasks.get_slot_state_by_task_id(dep_local_task_id); - if (!append_fanin_or_fail(orch, producer_slot_state, dep_task_id, &fanin_builder)) { + if (!append_fanin_or_fail(orch, dep_task_id, &fanin_builder)) { return result; } } @@ -1516,10 +1537,7 @@ static TaskOutputTensors submit_task_common( }; auto runtime_emit = [&](TaskId producer_task_id) -> bool { - SharedMemoryTaskHeader &producer_tasks = orch->sm_header->tasks; - ChipTaskSlotState *prod_state = - &producer_tasks.get_slot_state_by_task_id(static_cast(producer_task_id.local())); - return append_fanin_or_fail(orch, prod_state, producer_task_id, &fanin_builder); + return append_fanin_or_fail(orch, producer_task_id, &fanin_builder); }; // The capture branch instantiates compute_task_fanin with a live Annotate; @@ -1791,7 +1809,7 @@ bool graph_submit_outer( orch_mark_fatal(orch, SIMPLER_ERROR_HEAP_RING_DEADLOCK); return false; } - const TaskId task_id = TaskId::make(0, static_cast(allocation.task_id)); + const TaskId task_id = simpler::hbg::make_ring_task(static_cast(allocation.task_id)); SharedMemoryTaskHeader &tasks = orch->sm_header->tasks; TaskDescriptor &task = tasks.task_descriptors[allocation.task_id]; TaskPayload &payload = tasks.task_payloads[allocation.task_id]; @@ -1817,7 +1835,7 @@ bool graph_submit_outer( orch->tensor_pool_cursor += tensor_slots; orch->scalar_pool_cursor += scalar_span; slot.task_state.store(CHIP_TASK_PENDING, std::memory_order_relaxed); - slot.last_consumer_local_id = static_cast(task_id.local()); + slot.last_consumer_local_id = static_cast(simpler::hbg::task_local_id(task_id)); slot.active_mask = ActiveMask{}; slot.task_attrs = TaskAttrs{}; slot.total_required_subtasks = 0; @@ -1841,10 +1859,11 @@ bool graph_submit_outer( ); } - FaninBuilder fanin_builder(orch, &payload, static_cast(task_id.local()), next_fanin_seen_epoch(orch)); + FaninBuilder fanin_builder( + orch, &payload, static_cast(simpler::hbg::task_local_id(task_id)), next_fanin_seen_epoch(orch) + ); auto emit = [&](TaskId producer_id) -> bool { - ChipTaskSlotState *producer = &tasks.get_slot_state_by_task_id(static_cast(producer_id.local())); - return append_fanin_or_fail(orch, producer, producer_id, &fanin_builder); + return append_fanin_or_fail(orch, producer_id, &fanin_builder); }; // An outer GRAPH task is an ordinary task, so the dependency graph // has to carry it: without this the whole Graph — and every edge into it — @@ -1967,8 +1986,9 @@ TaskOutputTensors graph_record_submit_node( // A recorded node's index equals its local task id minus the recording // baseline, so its synthetic id keeps classification and explicit-dep // arithmetic identical to the ordinary path. - const TaskId task_id = - TaskId::make(0, static_cast(recording.start_local_task_id) + static_cast(node_index)); + const TaskId task_id = simpler::hbg::make_ring_task( + static_cast(recording.start_local_task_id) + static_cast(node_index) + ); result.set_task_id(task_id); if (node_index >= GRAPH_MAX_NODES || args.has_error) { @@ -2164,8 +2184,10 @@ TaskOutputTensors graph_record_submit_node( recording.unsupported = true; } else { auto emit_inferred = [&recording, &add_fanin, node_index](TaskId producer) -> bool { - const int32_t producer_index = static_cast(producer.local()) - recording.start_local_task_id; - if (producer.ring() == 0 && producer_index >= 0 && producer_index < static_cast(node_index)) { + const int32_t producer_index = + static_cast(simpler::hbg::task_local_id(producer)) - recording.start_local_task_id; + if (simpler::hbg::is_ring_task(producer) && producer_index >= 0 && + producer_index < static_cast(node_index)) { add_fanin(static_cast(producer_index)); } return true; @@ -2176,8 +2198,9 @@ TaskOutputTensors graph_record_submit_node( } for (uint32_t i = 0; i < args.explicit_dep_count(); ++i) { const TaskId dep = args.explicit_dep(i); - const int32_t dep_index = static_cast(dep.local()) - recording.start_local_task_id; - if (!dep.is_valid() || dep.ring() != 0 || dep_index >= static_cast(node_index)) { + const int32_t dep_index = + static_cast(simpler::hbg::task_local_id(dep)) - recording.start_local_task_id; + if (!dep.is_valid() || !simpler::hbg::is_ring_task(dep) || dep_index >= static_cast(node_index)) { recording.unsupported = true; continue; } @@ -2744,7 +2767,7 @@ TaskOutputTensors OrchestratorState::alloc_tensors(const CoreTaskArgs &args) { // the run hangs. (The device watermark walk transparently steps past this // pre-set flag when a later on-device task completes.) SharedMemoryTaskHeader &done_tasks = orch->sm_header->tasks; - int32_t done_local = static_cast(prepared.task_id.local()); + int32_t done_local = static_cast(simpler::hbg::task_local_id(prepared.task_id)); done_tasks.set_completion_flag(done_local); } orch->inline_completed_tasks++; diff --git a/src/a5/runtime/host_build_graph/runtime/orchestrator_core/runtime_core.cpp b/src/a5/runtime/host_build_graph/runtime/orchestrator_core/runtime_core.cpp index 698f3bdaaa..b599217dfb 100644 --- a/src/a5/runtime/host_build_graph/runtime/orchestrator_core/runtime_core.cpp +++ b/src/a5/runtime/host_build_graph/runtime/orchestrator_core/runtime_core.cpp @@ -31,6 +31,7 @@ #include "aicpu/device_time.h" #include "common/platform_config.h" #include "common/unified_log.h" +#include "host_build_graph/task_id_encoding.h" #include "host_tensor_access.h" // simpler::hbg::Tensor-byte access for a caller that can load a device address directly. @@ -185,18 +186,20 @@ static bool wait_for_tensor_ready( // Callers branch on the returned pointer, not on `failed`: the slot is // dereferenced immediately and a null return is the only safe signal. auto resolve_producer = [&](TaskId producer) -> const ChipTaskSlotState * { - if (!producer.is_valid() || producer.ring() != 0) { + if (!producer.is_valid() || !simpler::hbg::is_ring_task(producer)) { orch.report_fatal( SIMPLER_ERROR_INVALID_ARGS, caller, - "tensor producer task %#llx has invalid ring %u; host_build_graph uses ring 0", - static_cast(producer.raw), static_cast(producer.ring()) + "tensor producer task %#llx is in id space %u, not RING; host_build_graph resolves a producer against " + "its one task table", + static_cast(producer.raw), + static_cast(simpler::hbg::task_id_space(producer)) ); failed = true; return nullptr; } auto &tasks = orch.sm_header->tasks; - int32_t local_id = static_cast(producer.local()); + int32_t local_id = static_cast(simpler::hbg::task_local_id(producer)); auto &slot = tasks.get_slot_state_by_task_id(local_id); if (slot.task == nullptr || slot.task->task_id != producer) { orch.report_fatal( @@ -211,7 +214,7 @@ static bool wait_for_tensor_ready( }; auto wait_one_producer = [&](const ChipTaskSlotState &slot) { - int32_t local_id = static_cast(slot.task->task_id.local()); + int32_t local_id = static_cast(simpler::hbg::task_local_id(slot.task->task_id)); uint64_t t0 = get_sys_cnt_aicpu(); int32_t spin_count = 0; while (slot.task_state.load(std::memory_order_acquire) < CHIP_TASK_COMPLETED) { @@ -237,7 +240,7 @@ static bool wait_for_tensor_ready( }; auto wait_one_consumers = [&](const ChipTaskSlotState &slot) { - int32_t local_id = slot.task->task_id.local(); + int32_t local_id = simpler::hbg::task_local_id(slot.task->task_id); uint64_t t0 = get_sys_cnt_aicpu(); int32_t spin_count = 0; // Polling: all consumers of this producer have retired once the per-ring diff --git a/src/a5/runtime/host_build_graph/runtime/runtime_types.h b/src/a5/runtime/host_build_graph/runtime/runtime_types.h index e7c374b114..805fa02a5b 100644 --- a/src/a5/runtime/host_build_graph/runtime/runtime_types.h +++ b/src/a5/runtime/host_build_graph/runtime/runtime_types.h @@ -243,8 +243,9 @@ struct ChipTaskSlotState; // Forward declaration (defined below) * Fields set by Orchestrator at submission, read by Scheduler for dispatch. */ struct TaskDescriptor { - // Mixed-task identification (encodes ring_id in upper 32 bits) - TaskId task_id; // raw: (ring_id << 32) | local_id + // Task identity. See src/common/host_build_graph/task_id_encoding.h: the + // upper 32 bits are this runtime's id space, not a ring index. + TaskId task_id; // Per-slot kernel IDs (INVALID_KERNEL_ID = inactive) int32_t kernel_id[SUBTASK_SLOT_COUNT]; diff --git a/src/a5/runtime/host_build_graph/runtime/scheduler/scheduler.h b/src/a5/runtime/host_build_graph/runtime/scheduler/scheduler.h index 65cf34b583..d52d331a89 100644 --- a/src/a5/runtime/host_build_graph/runtime/scheduler/scheduler.h +++ b/src/a5/runtime/host_build_graph/runtime/scheduler/scheduler.h @@ -39,6 +39,7 @@ #include "aicpu/platform_regs.h" // get_reg_ptr / RegId for the early-dispatch doorbell #include "async_wait.h" #include "graph_execution.h" +#include "host_build_graph/task_id_encoding.h" #include "task_allocator.h" #include "runtime_types.h" #include "shared_memory.h" @@ -624,7 +625,7 @@ struct SchedulerState { // watermark >= producer.last_consumer_local_id). Whole-graph-resident hbg // has no device slot reclaim, so nothing advances a reclaim cursor here. void on_mixed_task_complete(ChipTaskSlotState &slot_state) { - const int32_t task_id = static_cast(slot_state.task->task_id.local()); + const int32_t task_id = static_cast(simpler::hbg::task_local_id(slot_state.task->task_id)); SharedMemoryTaskHeader &tasks = *task_view.tasks; slot_state.mark_completed(); // host-visible mirror (task_state = COMPLETED) diff --git a/src/a5/runtime/host_build_graph/runtime/scheduler/scheduler_completion.cpp b/src/a5/runtime/host_build_graph/runtime/scheduler/scheduler_completion.cpp index 7f9537c399..9c96669f8f 100644 --- a/src/a5/runtime/host_build_graph/runtime/scheduler/scheduler_completion.cpp +++ b/src/a5/runtime/host_build_graph/runtime/scheduler/scheduler_completion.cpp @@ -226,9 +226,9 @@ void SchedulerContext::complete_slot_task( if (is_pmu_enabled()) { // The slot key is the 32-bit register token AICore wrote into // dual_issue_slots[task_id & 1].task_id (the DATA_MAIN_BASE value), not - // task_id.raw — the latter carries the (ring_id << 32 | local_id) - // encoding and would never match. The task identity travels separately for - // the PmuRecord. + // task_id.raw — the latter is a 64-bit TaskId whose high bits carry this + // runtime's id space, so it would never match. The task identity travels + // separately for the PmuRecord. pmu_aicpu_complete_record( core_id, thread_idx, static_cast(expected_reg_task_id), slot_state.task->task_id.raw, slot_state.task->kernel_id[static_cast(subslot)], hank[core_id].core_type diff --git a/src/a5/runtime/host_build_graph/runtime/tensormap.h b/src/a5/runtime/host_build_graph/runtime/tensormap.h index 0ad8d89a84..f08672a9ba 100644 --- a/src/a5/runtime/host_build_graph/runtime/tensormap.h +++ b/src/a5/runtime/host_build_graph/runtime/tensormap.h @@ -50,6 +50,7 @@ #include #include "common.h" +#include "host_build_graph/task_id_encoding.h" #include "profiling_config.h" #include "runtime_types.h" #include "tensor.h" @@ -564,7 +565,7 @@ struct ChipTensorMap { g_insert_count++; #endif uint32_t bucket_index = hash(addr); - auto local_id = producer_task_id.local(); + auto local_id = simpler::hbg::task_local_id(producer_task_id); const int32_t task_slot = local_id; entry->producer_task_id = producer_task_id; @@ -602,7 +603,7 @@ struct ChipTensorMap { // Update predecessor's next pointer (O(1) via prev_in_task) if (entry.prev_in_task == nullptr) { // Entry is the head of its task chain, update task_entry_heads - int32_t local_id = static_cast(entry.producer_task_id.local()); + int32_t local_id = static_cast(simpler::hbg::task_local_id(entry.producer_task_id)); const int32_t task_slot = local_id; task_entry_heads[task_slot] = entry.next_in_task; } else { diff --git a/src/a5/runtime/tensormap_and_ringbuffer/docs/MULTI_RING.md b/src/a5/runtime/tensormap_and_ringbuffer/docs/MULTI_RING.md index 59d37baba8..88e0c24ea5 100644 --- a/src/a5/runtime/tensormap_and_ringbuffer/docs/MULTI_RING.md +++ b/src/a5/runtime/tensormap_and_ringbuffer/docs/MULTI_RING.md @@ -27,13 +27,14 @@ Task IDs are widened from 32-bit to 64-bit to carry the ring identity: task_id.raw = (ring_id << 32) | local_id ``` -`TaskId` exposes direct accessors in `src/common/task_interface/task_id.h`: +`TaskId` itself is an opaque 64-bit handle; this runtime's encoding of it lives in +`src/common/tensormap_and_ringbuffer/task_id_encoding.h`: | API | Purpose | | --- | ------- | -| `TaskId::make(ring_id, local_id)` | Compose a 64-bit task ID (`TaskId`) | -| `task_id.ring()` | Extract `ring_id` (bits 63-32) | -| `task_id.local()` | Extract `local_id` (bits 31-0) | +| `simpler::tmr::make_task_id(ring_id, local_id)` | Compose a 64-bit task ID (`TaskId`) | +| `simpler::tmr::task_ring(task_id)` | Extract `ring_id` (bits 63-32) | +| `simpler::tmr::task_local_id(task_id)` | Extract `local_id` (bits 31-0) | | `task_id.raw` | Access the packed 64-bit encoding | Type changes: @@ -158,8 +159,8 @@ Entry validity checks and `cleanup_retired` operate per-ring: ```cpp bool entry_valid(const ChipTensorMapEntry& e) { - int32_t ring = e.producer_task_id.ring(); - int32_t local = e.producer_task_id.local(); + int32_t ring = simpler::tmr::task_ring(e.producer_task_id); + int32_t local = simpler::tmr::task_local_id(e.producer_task_id); return local >= last_task_alives[ring]; } ``` diff --git a/src/a5/runtime/tensormap_and_ringbuffer/host/dep_gen_replay.cpp b/src/a5/runtime/tensormap_and_ringbuffer/host/dep_gen_replay.cpp index 5c52ce43b9..dfcdc5e292 100644 --- a/src/a5/runtime/tensormap_and_ringbuffer/host/dep_gen_replay.cpp +++ b/src/a5/runtime/tensormap_and_ringbuffer/host/dep_gen_replay.cpp @@ -73,6 +73,7 @@ #include "data_type.h" #include "dep_compute.h" #include "task_id.h" +#include "tensormap_and_ringbuffer/task_id_encoding.h" #include "tensormap.h" #include "tensor.h" @@ -486,8 +487,8 @@ dep_gen_replay_emit_deps_json(const DepGenRecord *records, size_t num_records, c uint32_t max_local[CHIP_MAX_RING_DEPTH] = {0}; for (size_t i = 0; i < num_records; i++) { TaskId tid{records[i].task_id}; - uint8_t ring = tid.ring(); - uint32_t local = tid.local(); + uint8_t ring = simpler::tmr::task_ring(tid); + uint32_t local = simpler::tmr::task_local_id(tid); if (ring < CHIP_MAX_RING_DEPTH && local > max_local[ring]) { max_local[ring] = local; } diff --git a/src/a5/runtime/tensormap_and_ringbuffer/runtime/orchestrator.cpp b/src/a5/runtime/tensormap_and_ringbuffer/runtime/orchestrator.cpp index 468eb66f90..0f3a7ae57a 100644 --- a/src/a5/runtime/tensormap_and_ringbuffer/runtime/orchestrator.cpp +++ b/src/a5/runtime/tensormap_and_ringbuffer/runtime/orchestrator.cpp @@ -32,6 +32,7 @@ #include "dep_compute.h" #include "runtime_types.h" #include "shared_memory.h" +#include "tensormap_and_ringbuffer/task_id_encoding.h" #include "tensormap.h" #include "types.h" #include "tensor.h" @@ -344,8 +345,10 @@ static bool append_fanin_or_fail( // edge so the stronger of several discovery reasons (creator vs modifier vs // explicit) always wins, independent of the order they are reached in. prod_state->lock_fanout(); - bool gone = prod_state->task == nullptr || prod_state->task->task_id.local() != producer_task_id.local() || - prod_state->task_state.load(std::memory_order_acquire) == CHIP_TASK_CONSUMED; + bool gone = + prod_state->task == nullptr || + simpler::tmr::task_local_id(prod_state->task->task_id) != simpler::tmr::task_local_id(producer_task_id) || + prod_state->task_state.load(std::memory_order_acquire) == CHIP_TASK_CONSUMED; bool already_seen = !gone && fanin_builder->mark_seen(prod_ring, prod_slot); bool claim = !gone && !already_seen; int32_t fanout_now = -1; @@ -371,7 +374,8 @@ static bool append_fanin_or_fail( if (fanout_now == CHIP_DEP_DEGREE_DEBUG_THRESHOLD + 1) { LOG_DEBUG( "dense dependency: task ring=%u id=%u fanout>%d [orch submit]", - static_cast(producer_task_id.ring()), producer_task_id.local(), CHIP_DEP_DEGREE_DEBUG_THRESHOLD + static_cast(simpler::tmr::task_ring(producer_task_id)), + simpler::tmr::task_local_id(producer_task_id), CHIP_DEP_DEGREE_DEBUG_THRESHOLD ); } // Stale/consumed producer: no edge at all. @@ -601,7 +605,7 @@ static bool prepare_task( return false; } - out->task_id = TaskId::make(ring_id, static_cast(out->alloc_result.task_id)); + out->task_id = simpler::tmr::make_task_id(ring_id, static_cast(out->alloc_result.task_id)); out->slot_state = &orch->sm_header->rings[ring_id].get_slot_state_by_slot(out->alloc_result.slot); out->task = &orch->sm_header->rings[ring_id].task_descriptors[out->alloc_result.slot]; out->payload = &orch->sm_header->rings[ring_id].task_payloads[out->alloc_result.slot]; @@ -894,7 +898,7 @@ static TaskOutputTensors submit_task_common( if (!prepare_task(orch, args, layout.total_output_size, active_mask, task_attrs, &prepared)) { return result; } - uint8_t ring_id = prepared.task_id.ring(); + uint8_t ring_id = simpler::tmr::task_ring(prepared.task_id); SchedulerState *sched = orch->scheduler; ChipRingFlowControl &fc = orch->sm_header->rings[ring_id].fc; TaskId task_id = prepared.task_id; @@ -966,9 +970,9 @@ static TaskOutputTensors submit_task_common( ); return result; } - uint8_t dep_ring_id = dep_task_id.ring(); + uint8_t dep_ring_id = simpler::tmr::task_ring(dep_task_id); SharedMemoryRingHeader &dep_ring = orch->sm_header->rings[dep_ring_id]; - int32_t dep_local_task_id = static_cast(dep_task_id.local()); + int32_t dep_local_task_id = static_cast(simpler::tmr::task_local_id(dep_task_id)); int32_t dep_last_task_alive = dep_ring.fc.last_task_alive.load(std::memory_order_acquire); if (dep_local_task_id < dep_last_task_alive) { continue; @@ -990,9 +994,10 @@ static TaskOutputTensors submit_task_common( }; auto runtime_emit = [&](TaskId producer_task_id, DepFlags kind) -> bool { - uint8_t prod_ring = producer_task_id.ring(); + uint8_t prod_ring = simpler::tmr::task_ring(producer_task_id); SharedMemoryRingHeader &producer_ring = orch->sm_header->rings[prod_ring]; - int32_t prod_slot = producer_ring.get_slot_by_task_id(static_cast(producer_task_id.local())); + int32_t prod_slot = + producer_ring.get_slot_by_task_id(static_cast(simpler::tmr::task_local_id(producer_task_id))); ChipTaskSlotState *prod_state = &producer_ring.get_slot_state_by_slot(prod_slot); return append_fanin_or_fail( orch, prod_ring, prod_slot, prod_state, producer_task_id, &fanin_builder, ring_id, kind @@ -1052,8 +1057,9 @@ static TaskOutputTensors submit_task_common( // THRESHOLD because the count lands at its final total here. if (fanin_builder.count > CHIP_DEP_DEGREE_DEBUG_THRESHOLD) { LOG_DEBUG( - "dense dependency: task ring=%u id=%u fanin>%d [orch submit]", static_cast(task_id.ring()), - task_id.local(), CHIP_DEP_DEGREE_DEBUG_THRESHOLD + "dense dependency: task ring=%u id=%u fanin>%d [orch submit]", + static_cast(simpler::tmr::task_ring(task_id)), simpler::tmr::task_local_id(task_id), + CHIP_DEP_DEGREE_DEBUG_THRESHOLD ); } payload.fanin_spill_start = fanin_builder.spill_start; @@ -1324,7 +1330,7 @@ TaskOutputTensors OrchestratorState::alloc_tensors(const CoreTaskArgs &args) { payload.init(args, outputs, prepared.alloc_result, layout); payload.fanin_actual_count = 0; payload.fanin_spill_start = 0; - payload.fanin_spill_pool = &orch->rings[prepared.task_id.ring()].fanin_pool; + payload.fanin_spill_pool = &orch->rings[simpler::tmr::task_ring(prepared.task_id)].fanin_pool; CYCLE_COUNT_LAP(g_orch_args_cycle); if (prepared.slot_state != nullptr) { diff --git a/src/a5/runtime/tensormap_and_ringbuffer/runtime/ring_buffer.h b/src/a5/runtime/tensormap_and_ringbuffer/runtime/ring_buffer.h index 6894e004f1..e48a507832 100644 --- a/src/a5/runtime/tensormap_and_ringbuffer/runtime/ring_buffer.h +++ b/src/a5/runtime/tensormap_and_ringbuffer/runtime/ring_buffer.h @@ -39,6 +39,7 @@ #include #include "runtime_types.h" +#include "tensormap_and_ringbuffer/task_id_encoding.h" #include "shared_memory.h" #include "aicpu/device_time.h" // get_sys_cnt_aicpu (deadlock wall-clock backstop) #include "common/platform_config.h" // PLATFORM_PROF_SYS_CNT_FREQ (deadlock wall-clock) @@ -70,7 +71,7 @@ constexpr uint32_t ring_mask_bit(int32_t ring_id) { inline bool reclaim_head_matches_open_task(int32_t head_task_id, uint8_t ring_id, const ChipTaskSlotState *oldest_open_task) { return oldest_open_task != nullptr && oldest_open_task->task != nullptr && - oldest_open_task->task->task_id == TaskId::make(ring_id, static_cast(head_task_id)); + oldest_open_task->task->task_id == simpler::tmr::make_task_id(ring_id, static_cast(head_task_id)); } class ReclaimPublicationRequest { diff --git a/src/a5/runtime/tensormap_and_ringbuffer/runtime/runtime_core.cpp b/src/a5/runtime/tensormap_and_ringbuffer/runtime/runtime_core.cpp index 3342386a16..0a7db6f7c0 100644 --- a/src/a5/runtime/tensormap_and_ringbuffer/runtime/runtime_core.cpp +++ b/src/a5/runtime/tensormap_and_ringbuffer/runtime/runtime_core.cpp @@ -29,6 +29,7 @@ #include "aicpu/device_time.h" #include "common/platform_config.h" // PLATFORM_PROF_SYS_CNT_FREQ (data-wait deadline) #include "common/unified_log.h" +#include "tensormap_and_ringbuffer/task_id_encoding.h" #if SIMPLER_DFX #include "aicpu/scope_stats_collector_aicpu.h" #endif @@ -114,7 +115,7 @@ static bool wait_for_tensor_ready( auto wait_one_producer = [&](const ChipTaskSlotState &slot) { uint8_t ring_id = slot.ring_id; - int32_t local_id = static_cast(slot.task->task_id.local()); + int32_t local_id = static_cast(simpler::tmr::task_local_id(slot.task->task_id)); uint64_t t0 = get_sys_cnt_aicpu(); int32_t spin_count = 0; while (slot.task_state.load(std::memory_order_acquire) < CHIP_TASK_COMPLETED) { @@ -140,7 +141,7 @@ static bool wait_for_tensor_ready( auto wait_one_consumers = [&](const ChipTaskSlotState &slot) { uint8_t ring_id = slot.ring_id; - int32_t local_id = slot.task->task_id.local(); + int32_t local_id = simpler::tmr::task_local_id(slot.task->task_id); uint64_t t0 = get_sys_cnt_aicpu(); int32_t spin_count = 0; while ((slot.fanout_refcount.load(std::memory_order_acquire) & ~FANOUT_SCOPE_BIT) < @@ -190,7 +191,9 @@ static bool wait_for_tensor_ready( auto do_wait = [&]() { // Step A: creator retention — read owner directly from tensor metadata if (owner.is_valid()) { - auto &s = orch.sm_header->rings[owner.ring()].get_slot_state_by_task_id(owner.local()); + auto &s = orch.sm_header->rings[simpler::tmr::task_ring(owner)].get_slot_state_by_task_id( + simpler::tmr::task_local_id(owner) + ); try_push(s); if (failed) return; } @@ -198,7 +201,9 @@ static bool wait_for_tensor_ready( // Step B: modifier writer lookup (OverlapMap), direct callback orch.tensor_map.lookup(tensor, [&](ChipTensorMapEntry &entry, OverlapStatus) -> bool { TaskId pid = entry.producer_task_id; - auto &s = orch.sm_header->rings[pid.ring()].get_slot_state_by_task_id(pid.local()); + auto &s = orch.sm_header->rings[simpler::tmr::task_ring(pid)].get_slot_state_by_task_id( + simpler::tmr::task_local_id(pid) + ); try_push(s); return !failed; }); diff --git a/src/a5/runtime/tensormap_and_ringbuffer/runtime/runtime_types.h b/src/a5/runtime/tensormap_and_ringbuffer/runtime/runtime_types.h index e47adf654c..b467034bae 100644 --- a/src/a5/runtime/tensormap_and_ringbuffer/runtime/runtime_types.h +++ b/src/a5/runtime/tensormap_and_ringbuffer/runtime/runtime_types.h @@ -217,7 +217,7 @@ struct DepListEntry { */ struct TaskDescriptor { // Mixed-task identification (encodes ring_id in upper 32 bits) - TaskId task_id; // raw: (ring_id << 32) | local_id + TaskId task_id; // raw: (ring_id << 32) | local_id, see task_id_encoding.h // Per-slot kernel IDs (INVALID_KERNEL_ID = inactive) int32_t kernel_id[SUBTASK_SLOT_COUNT]; diff --git a/src/a5/runtime/tensormap_and_ringbuffer/runtime/shared/tensormap.cpp b/src/a5/runtime/tensormap_and_ringbuffer/runtime/shared/tensormap.cpp index dfa35caace..a5ed7174e5 100644 --- a/src/a5/runtime/tensormap_and_ringbuffer/runtime/shared/tensormap.cpp +++ b/src/a5/runtime/tensormap_and_ringbuffer/runtime/shared/tensormap.cpp @@ -30,6 +30,7 @@ #include "common.h" #include "common/unified_log.h" +#include "tensormap_and_ringbuffer/task_id_encoding.h" // ============================================================================= // TensorMap Lookup Chain Length Statistics (compile-time toggle) @@ -256,8 +257,8 @@ int32_t ChipTensorMap::valid_count() { } void ChipTensorMap::sync_tensormap(TaskId task_id, int32_t sm_last_task_alive) { - auto ring_id = task_id.ring(); - auto local_id = task_id.local(); + auto ring_id = simpler::tmr::task_ring(task_id); + auto local_id = simpler::tmr::task_local_id(task_id); sync_validity(ring_id, sm_last_task_alive); // Only attempt cleanup when last_task_alive has actually advanced; diff --git a/src/a5/runtime/tensormap_and_ringbuffer/runtime/tensormap.h b/src/a5/runtime/tensormap_and_ringbuffer/runtime/tensormap.h index 0f0ef9533d..0d450bd385 100644 --- a/src/a5/runtime/tensormap_and_ringbuffer/runtime/tensormap.h +++ b/src/a5/runtime/tensormap_and_ringbuffer/runtime/tensormap.h @@ -46,6 +46,7 @@ #include "profiling_config.h" #include "utils/device_arena.h" #include "runtime_types.h" +#include "tensormap_and_ringbuffer/task_id_encoding.h" #include "tensor.h" // Overlap geometry types. Relocated here from tensor.h: they are used only by @@ -614,7 +615,8 @@ struct ChipTensorMap { // reused the slot (local_id + N * window) before this cleanup ran. // Free only entries produced by the retiring local_id, unlinking // each from the chain; entries from other tasks stay linked. - TaskId retired_task = TaskId::make(static_cast(ring_id), static_cast(local_id)); + TaskId retired_task = + simpler::tmr::make_task_id(static_cast(ring_id), static_cast(local_id)); ChipTensorMapEntry *cur_entry = task_entry_heads[ring_id][task_slot]; while (cur_entry != nullptr) { ChipTensorMapEntry *next_entry = cur_entry->next_in_task; // free_entry clears it @@ -659,8 +661,8 @@ struct ChipTensorMap { g_insert_count++; #endif uint32_t bucket_index = hash(addr); - auto ring_id = producer_task_id.ring(); - auto local_id = producer_task_id.local(); + auto ring_id = simpler::tmr::task_ring(producer_task_id); + auto local_id = simpler::tmr::task_local_id(producer_task_id); int32_t task_slot = local_id & (task_window_sizes[ring_id] - 1); entry->producer_task_id = producer_task_id; @@ -695,7 +697,8 @@ struct ChipTensorMap { * Check if entry is valid (producer has not retired) */ bool entry_valid(const ChipTensorMapEntry &entry) const { - return static_cast(entry.producer_task_id.local()) >= last_task_alives[entry.producer_task_id.ring()]; + return static_cast(simpler::tmr::task_local_id(entry.producer_task_id)) >= + last_task_alives[simpler::tmr::task_ring(entry.producer_task_id)]; } void remove_entry(ChipTensorMapEntry &entry) { @@ -712,8 +715,8 @@ struct ChipTensorMap { // Update predecessor's next pointer (O(1) via prev_in_task) if (entry.prev_in_task == nullptr) { // Entry is the head of its task chain, update task_entry_heads - int32_t ring_id = entry.producer_task_id.ring(); - int32_t local_id = static_cast(entry.producer_task_id.local()); + int32_t ring_id = simpler::tmr::task_ring(entry.producer_task_id); + int32_t local_id = static_cast(simpler::tmr::task_local_id(entry.producer_task_id)); int32_t task_slot = local_id & (task_window_sizes[ring_id] - 1); task_entry_heads[ring_id][task_slot] = entry.next_in_task; } else { diff --git a/src/common/host_build_graph/graph_execution.cpp b/src/common/host_build_graph/graph_execution.cpp index e42bd9638a..726d32caf4 100644 --- a/src/common/host_build_graph/graph_execution.cpp +++ b/src/common/host_build_graph/graph_execution.cpp @@ -16,6 +16,7 @@ #include #include "graph_cache.h" +#include "host_build_graph/task_id_encoding.h" namespace { @@ -438,9 +439,9 @@ GraphMaterializeResult graph_execution_materialize_slice( TaskPayload &payload = storage->payload; ChipTaskSlotState &slot = storage->slot; - const uint32_t synthetic_local = - (static_cast(outer_slot.task->task_id.local()) << 10) | static_cast(i); - task.task_id = TaskId::make(1, synthetic_local); + task.task_id = simpler::hbg::make_graph_node( + simpler::hbg::task_local_id(outer_slot.task->task_id), static_cast(i) + ); const GraphNodeDefinition &source = nodes[i]; const uint64_t node_offset = node_offsets[i]; const uint64_t output_bytes = CHIP_ALIGN_UP(static_cast(source.total_output_size), CHIP_ALIGN_SIZE); diff --git a/src/common/host_build_graph/graph_execution.h b/src/common/host_build_graph/graph_execution.h index 2d8caf5f1e..d5e1aa0e5c 100644 --- a/src/common/host_build_graph/graph_execution.h +++ b/src/common/host_build_graph/graph_execution.h @@ -18,10 +18,15 @@ #include #include +#include "host_build_graph/task_id_encoding.h" #include "runtime_types.h" #include "tensor.h" inline constexpr uint32_t GRAPH_MAX_NODES = 1024; +static_assert( + GRAPH_MAX_NODES <= (1u << simpler::hbg::GRAPH_NODE_INDEX_BITS), + "a node index must fit the low field of a GRAPH_NODE task id" +); inline constexpr int32_t GRAPH_MATERIALIZE_SLICE_NODES = 4; enum class GraphTensorSource : uint8_t { diff --git a/src/common/host_build_graph/task_id_encoding.h b/src/common/host_build_graph/task_id_encoding.h new file mode 100644 index 0000000000..566d5af64c --- /dev/null +++ b/src/common/host_build_graph/task_id_encoding.h @@ -0,0 +1,62 @@ +/* + * Copyright (c) PyPTO Contributors. + * This program is free software, you can redistribute it and/or modify it under the terms and conditions of + * CANN Open Software License Agreement Version 2.0 (the "License"). + * Please refer to the License for details. You may not use this file except in compliance with the License. + * THIS SOFTWARE IS PROVIDED ON AN "AS IS" BASIS, WITHOUT WARRANTIES OF ANY KIND, EITHER EXPRESS OR IMPLIED, + * INCLUDING BUT NOT LIMITED TO NON-INFRINGEMENT, MERCHANTABILITY, OR FITNESS FOR A PARTICULAR PURPOSE. + * See LICENSE in the root of the software repository for the full text of the License. + * ----------------------------------------------------------------------------------------------------------- + */ +/** + * How the `hbg` runtime encodes a `TaskId`. The layout is private to this runtime: + * `TaskId` itself is an opaque 64-bit handle, and `tmr` encodes something else in + * the same bits (see src/common/tensormap_and_ringbuffer/task_id_encoding.h). + */ + +#pragma once + +#include + +#include "task_id.h" + +namespace simpler::hbg { + +/** + * Which id space a task id belongs to, held in the high 32 bits. + * + * This runtime has exactly one ring, so the high bits name the *storage* an id + * resolves against rather than a ring index: + * + * RING — a task placed on the shared-memory ring. Its low bits are the + * task allocator's local id, resolvable via get_slot_by_task_id(). + * GRAPH_NODE — a node materialized inside a Graph execution. It lives in that + * execution's GraphNodeStorage, not on the ring, so its low bits + * are the packed pair below and must never be resolved against a + * ring slot. + */ +enum class TaskIdSpace : uint32_t { RING = 0, GRAPH_NODE = 1 }; + +// A graph node's low bits pack its outer GRAPH task's local id above its index +// within that graph. graph_execution.h asserts GRAPH_MAX_NODES fits the index. +inline constexpr uint32_t GRAPH_NODE_INDEX_BITS = 10; + +constexpr TaskId make_ring_task(uint32_t local_id) { + return TaskId{(static_cast(TaskIdSpace::RING) << 32) | static_cast(local_id)}; +} + +constexpr TaskId make_graph_node(uint32_t outer_local_id, uint32_t node_index) { + const uint32_t packed = (outer_local_id << GRAPH_NODE_INDEX_BITS) | node_index; + return TaskId{(static_cast(TaskIdSpace::GRAPH_NODE) << 32) | static_cast(packed)}; +} + +constexpr TaskIdSpace task_id_space(TaskId id) { return static_cast(static_cast(id.raw >> 32)); } + +constexpr bool is_ring_task(TaskId id) { return task_id_space(id) == TaskIdSpace::RING; } + +// The low 32 bits. A ring local id for a RING task; the packed pair above for a +// GRAPH_NODE, which is why callers that resolve a ring slot must gate on +// is_ring_task() first. +constexpr uint32_t task_local_id(TaskId id) { return static_cast(id.raw & 0xFFFFFFFFu); } + +} // namespace simpler::hbg diff --git a/src/common/platform/include/common/chip_swimlane_profiling.h b/src/common/platform/include/common/chip_swimlane_profiling.h index 5b6ccc4534..93e29580ae 100644 --- a/src/common/platform/include/common/chip_swimlane_profiling.h +++ b/src/common/platform/include/common/chip_swimlane_profiling.h @@ -138,11 +138,11 @@ static_assert(sizeof(ChipSwimlaneAicpuTaskRecord) == 32, "ChipSwimlaneAicpuTaskR * * Two identity fields with different roles: * - * - `task_token_raw` — the task identity `(ring_id << 32) | local_id`. - * Per-task unique. AICore reads it from + * - `task_token_raw` — the task identity, a `TaskId::raw` in whatever layout + * the minting runtime uses. Per-task unique. AICore reads it from * `LocalContext.async_ctx.task_token.raw` (already in the dispatch * payload's cache line). The host pulls it from here as the canonical - * task id + ring decoder at ALL levels — the AICPU record carries no + * task id at ALL levels — the AICPU record carries no * identity after the slim-down (only dispatch/finish timestamps and the * reg_task_id join key), so AICore is the single source of truth for * task identity. NOT a join key on its own: SPMD `block_num > num_cores`, @@ -608,7 +608,7 @@ static_assert(sizeof(ChipSwimlaneAicpuSchedPhaseRecord) == 64, "ChipSwimlaneAicp struct ChipSwimlaneAicpuOrchPhaseRecord { uint64_t start_time; // Submit start timestamp uint64_t end_time; // Submit end timestamp - uint64_t task_id; // Full TaskId encoding (ring_id << 32) | local_id + uint64_t task_id; // TaskId::raw, in the minting runtime's layout uint32_t submit_idx; // Monotonic submit counter uint32_t _pad; // 32B alignment padding }; diff --git a/src/common/platform/include/common/dep_gen.h b/src/common/platform/include/common/dep_gen.h index fa9be0a035..cfeba7d7f9 100644 --- a/src/common/platform/include/common/dep_gen.h +++ b/src/common/platform/include/common/dep_gen.h @@ -104,7 +104,7 @@ enum DepGenRecordFlags : uint32_t { * blob covers exactly two cache lines instead of straddling three. */ struct DepGenRecord { - uint64_t task_id; // TaskId encoding (ring_id << 32) | local_id + uint64_t task_id; // TaskId::raw, in the minting runtime's layout uint32_t flags; // DepGenRecordFlags bitmask uint16_t tensor_count; // number of valid ChipTensor slots uint16_t explicit_dep_count; // number of valid explicit_dep slots diff --git a/src/common/task_interface/task_id.h b/src/common/task_interface/task_id.h index 39849ff0b7..3da040607b 100644 --- a/src/common/task_interface/task_id.h +++ b/src/common/task_interface/task_id.h @@ -21,26 +21,25 @@ #include /** - * TaskId: 64-bit encoding shared by both runtimes. + * TaskId: an opaque 64-bit task handle. * - * raw encoding: (ring_id << 32) | local_id + * The bit layout belongs to the runtime that mints the id, and the two runtimes + * encode different things in the high bits — a ring index in `tmr`, an id space in + * `hbg`. Neither is declared here; code that mints or decodes an id includes its + * own runtime's encoding header: + * src/common/host_build_graph/task_id_encoding.h + * src/common/tensormap_and_ringbuffer/task_id_encoding.h * - * ring_id: which ring layer (0..CHIP_MAX_RING_DEPTH-1) - * local_id: per-ring monotonic counter + * What every holder may rely on: the handle is 8 bytes, copyable, comparable for + * identity, and has one reserved sentinel. * * Invalid sentinel: raw == UINT64_MAX (no valid task has this encoding). */ struct TaskId { uint64_t raw; - static constexpr TaskId make(uint8_t ring_id, uint32_t local_id) { - return TaskId{(static_cast(ring_id) << 32) | static_cast(local_id)}; - } - static constexpr TaskId invalid() { return TaskId{UINT64_MAX}; } - constexpr uint8_t ring() const { return static_cast(raw >> 32); } - constexpr uint32_t local() const { return static_cast(raw & 0xFFFFFFFFu); } constexpr bool is_valid() const { return raw != UINT64_MAX; } constexpr bool is_invalid() const { return raw == UINT64_MAX; } diff --git a/src/common/tensormap_and_ringbuffer/task_id_encoding.h b/src/common/tensormap_and_ringbuffer/task_id_encoding.h new file mode 100644 index 0000000000..bf18daf343 --- /dev/null +++ b/src/common/tensormap_and_ringbuffer/task_id_encoding.h @@ -0,0 +1,42 @@ +/* + * Copyright (c) PyPTO Contributors. + * This program is free software, you can redistribute it and/or modify it under the terms and conditions of + * CANN Open Software License Agreement Version 2.0 (the "License"). + * Please refer to the License for details. You may not use this file except in compliance with the License. + * THIS SOFTWARE IS PROVIDED ON AN "AS IS" BASIS, WITHOUT WARRANTIES OF ANY KIND, EITHER EXPRESS OR IMPLIED, + * INCLUDING BUT NOT LIMITED TO NON-INFRINGEMENT, MERCHANTABILITY, OR FITNESS FOR A PARTICULAR PURPOSE. + * See LICENSE in the root of the software repository for the full text of the License. + * ----------------------------------------------------------------------------------------------------------- + */ +/** + * How the `tmr` runtime encodes a `TaskId`. The layout is private to this runtime: + * `TaskId` itself is an opaque 64-bit handle, and `hbg` encodes something else in + * the same bits (see src/common/host_build_graph/task_id_encoding.h). + */ + +#pragma once + +#include + +#include "task_id.h" + +namespace simpler::tmr { + +/** + * raw = (ring_id << 32) | local_id + * + * ring_id: which ring layer the task was placed on (0..CHIP_MAX_RING_DEPTH-1) + * local_id: that ring's monotonic task counter + * + * Every task this runtime mints lives on a ring, so the pair fully identifies a + * ring slot: `rings[ring_id].get_slot_by_task_id(local_id)`. + */ +constexpr TaskId make_task_id(uint8_t ring_id, uint32_t local_id) { + return TaskId{(static_cast(ring_id) << 32) | static_cast(local_id)}; +} + +constexpr uint8_t task_ring(TaskId id) { return static_cast(id.raw >> 32); } + +constexpr uint32_t task_local_id(TaskId id) { return static_cast(id.raw & 0xFFFFFFFFu); } + +} // namespace simpler::tmr diff --git a/tests/st/host_build_graph_validation/kernels/orchestration/validation_orch.cpp b/tests/st/host_build_graph_validation/kernels/orchestration/validation_orch.cpp index e0af30294b..34d30a6d24 100644 --- a/tests/st/host_build_graph_validation/kernels/orchestration/validation_orch.cpp +++ b/tests/st/host_build_graph_validation/kernels/orchestration/validation_orch.cpp @@ -14,6 +14,8 @@ #include "orchestration_api.h" // NOLINT(build/include_subdir) +#include "host_build_graph/task_id_encoding.h" + #define FUNC_NOOP_AIC 0 #define FUNC_NOOP_AIV0 1 #define FUNC_NOOP_AIV1 2 @@ -40,10 +42,21 @@ void submit_overflowing_mix_task() { simpler::hbg::Tensor tensor_with_unbound_owner(const simpler::hbg::Tensor &external) { simpler::hbg::Tensor forged = external; - forged.owner_task_id = TaskId::make(0, 17); + forged.owner_task_id = simpler::hbg::make_ring_task(17); return forged; } +// A GRAPH_NODE id names storage inside a Graph execution, not a ring slot, so it +// can never be a fanin producer. Declaring one as an explicit dependency is the +// caller error append_fanin_or_fail rejects. +void submit_task_depending_on_graph_node() { + const TaskId deps[1] = {simpler::hbg::make_graph_node(/*outer_local_id=*/1, /*node_index=*/0)}; + CoreTaskArgs args; + args.launch_spec.set_block_num(1); + args.set_dependencies(deps, 1); + rt_submit_aiv_task(FUNC_NOOP_AIV0, args); +} + } // namespace extern "C" { @@ -71,6 +84,9 @@ __attribute__((visibility("default"))) void aicpu_orchestration_entry(const Chip case 3: set_tensor_data(tensor_with_unbound_owner(external), 1, index, 7); return; + case 4: + submit_task_depending_on_graph_node(); + return; default: rt_report_fatal( SIMPLER_ERROR_INVALID_ARGS, "unknown validation case %llu", static_cast(case_id) diff --git a/tests/st/host_build_graph_validation/test_host_build_graph_validation.py b/tests/st/host_build_graph_validation/test_host_build_graph_validation.py index 5ee15a45f2..f44d16f1a8 100644 --- a/tests/st/host_build_graph_validation/test_host_build_graph_validation.py +++ b/tests/st/host_build_graph_validation/test_host_build_graph_validation.py @@ -38,6 +38,7 @@ "mixed_subtask_overflow": 1, "unbound_owner_read": 2, "unbound_owner_write": 3, + "graph_node_dependency": 4, } diff --git a/tests/ut/cpp/a2a3/test_aicore_completion_mailbox.cpp b/tests/ut/cpp/a2a3/test_aicore_completion_mailbox.cpp index 838999769d..325d208b38 100644 --- a/tests/ut/cpp/a2a3/test_aicore_completion_mailbox.cpp +++ b/tests/ut/cpp/a2a3/test_aicore_completion_mailbox.cpp @@ -32,7 +32,10 @@ namespace { -TaskId make_token(uint32_t local) { return TaskId::make(/*ring=*/0, local); } +// The mailbox stores a token and compares it for identity; it never decodes one, +// and the encoding belongs to whichever runtime minted it, so any distinct 64-bit +// value serves here. +TaskId make_token(uint32_t local) { return TaskId{local}; } AICoreCompletionMailbox *fresh_mailbox() { // ~256KB heap allocation — avoid stack/BSS pressure across tests. diff --git a/tests/ut/cpp/a2a3/test_hbg_submit_poison.cpp b/tests/ut/cpp/a2a3/test_hbg_submit_poison.cpp index 5d47d8d683..0ea592002b 100644 --- a/tests/ut/cpp/a2a3/test_hbg_submit_poison.cpp +++ b/tests/ut/cpp/a2a3/test_hbg_submit_poison.cpp @@ -33,6 +33,7 @@ #include "utils/device_arena.h" #include "orchestrator.h" #include "shared_memory.h" +#include "host_build_graph/task_id_encoding.h" namespace { @@ -139,7 +140,7 @@ TEST_F(HbgSubmitPoisonTest, EveryDeviceReadFieldIsWrittenOverPoison) { const ChipTaskSlotState &st = tasks.slot_states[local]; // Descriptor: the task id is written to this exact local id. - EXPECT_EQ(desc.task_id.local(), static_cast(local)); + EXPECT_EQ(simpler::hbg::task_local_id(desc.task_id), static_cast(local)); // task_state is written at submit (reset_for_reuse skips it): PENDING for a // dispatchable task, COMPLETED for a pre-completed hidden-alloc. Either way a // real enum, never poison. @@ -163,8 +164,8 @@ TEST_F(HbgSubmitPoisonTest, EveryDeviceReadFieldIsWrittenOverPoison) { } // Field-specific coverage on the real task: tensors, scalar, packed output buffer. - const TaskDescriptor &root_desc = tasks.task_descriptors[root.task_id().local()]; - const TaskPayload &root_pl = tasks.task_payloads[root.task_id().local()]; + const TaskDescriptor &root_desc = tasks.task_descriptors[simpler::hbg::task_local_id(root.task_id())]; + const TaskPayload &root_pl = tasks.task_payloads[simpler::hbg::task_local_id(root.task_id())]; EXPECT_EQ(root_pl.tensor_count, 1); EXPECT_EQ(root_pl.scalar_count, 1); EXPECT_EQ(root_pl.dump_metadata.dump_arg_mask, (uint64_t{1} << 0) | (uint64_t{1} << 1)); @@ -175,7 +176,7 @@ TEST_F(HbgSubmitPoisonTest, EveryDeviceReadFieldIsWrittenOverPoison) { EXPECT_EQ(root_desc.kernel_id[static_cast(SubtaskSlot::AIV0)], 0); // The consumer's fanin is written: two duplicate deps dedupe to one. - const TaskPayload &cons_pl = tasks.task_payloads[consumer.task_id().local()]; + const TaskPayload &cons_pl = tasks.task_payloads[simpler::hbg::task_local_id(consumer.task_id())]; EXPECT_EQ(cons_pl.fanin_count, 1); - EXPECT_EQ(cons_pl.fanin_data()[0], static_cast(root.task_id().local())); + EXPECT_EQ(cons_pl.fanin_data()[0], static_cast(simpler::hbg::task_local_id(root.task_id()))); } diff --git a/tests/ut/cpp/a2a3/test_hbg_tensormap.cpp b/tests/ut/cpp/a2a3/test_hbg_tensormap.cpp index ea09905b3e..76cba1d205 100644 --- a/tests/ut/cpp/a2a3/test_hbg_tensormap.cpp +++ b/tests/ut/cpp/a2a3/test_hbg_tensormap.cpp @@ -25,6 +25,7 @@ #include #include "tensormap.h" +#include "host_build_graph/task_id_encoding.h" namespace { @@ -65,9 +66,9 @@ class HbgTensorMapTest : public ::testing::Test { // inserts remain visible until dependency computation explicitly removes one. TEST_F(HbgTensorMapTest, EveryProducerOfARegionStaysVisible) { simpler::hbg::Tensor t = make_test_tensor(0x1000, 256); - tmap.insert(t, TaskId::make(0, 0)); - tmap.insert(t, TaskId::make(0, 1)); - tmap.insert(t, TaskId::make(0, 2)); + tmap.insert(t, simpler::hbg::make_ring_task(0)); + tmap.insert(t, simpler::hbg::make_ring_task(1)); + tmap.insert(t, simpler::hbg::make_ring_task(2)); EXPECT_EQ(tmap.valid_count(), 3); TestLookupResult result; @@ -77,9 +78,9 @@ TEST_F(HbgTensorMapTest, EveryProducerOfARegionStaysVisible) { for (const auto &e : result.entries) { producers.push_back(e.entry->producer_task_id); } - EXPECT_NE(std::find(producers.begin(), producers.end(), TaskId::make(0, 0)), producers.end()); - EXPECT_NE(std::find(producers.begin(), producers.end(), TaskId::make(0, 1)), producers.end()); - EXPECT_NE(std::find(producers.begin(), producers.end(), TaskId::make(0, 2)), producers.end()); + EXPECT_NE(std::find(producers.begin(), producers.end(), simpler::hbg::make_ring_task(0)), producers.end()); + EXPECT_NE(std::find(producers.begin(), producers.end(), simpler::hbg::make_ring_task(1)), producers.end()); + EXPECT_NE(std::find(producers.begin(), producers.end(), simpler::hbg::make_ring_task(2)), producers.end()); } // Two tasks whose local ids alias to the same task slot both keep their entries; @@ -87,8 +88,8 @@ TEST_F(HbgTensorMapTest, EveryProducerOfARegionStaysVisible) { TEST_F(HbgTensorMapTest, SlotAliasingTasksBothKeepTheirEntries) { simpler::hbg::Tensor t = make_test_tensor(0x1000, 256); // Task 0 and task 0 + WINDOW_SIZE share slot 0 (local_id & (WINDOW_SIZE-1)). - tmap.insert(t, TaskId::make(0, 0)); - tmap.insert(t, TaskId::make(0, WINDOW_SIZE)); + tmap.insert(t, simpler::hbg::make_ring_task(0)); + tmap.insert(t, simpler::hbg::make_ring_task(WINDOW_SIZE)); EXPECT_EQ(tmap.valid_count(), 2); TestLookupResult result; @@ -98,8 +99,10 @@ TEST_F(HbgTensorMapTest, SlotAliasingTasksBothKeepTheirEntries) { for (const auto &e : result.entries) { producers.push_back(e.entry->producer_task_id); } - EXPECT_NE(std::find(producers.begin(), producers.end(), TaskId::make(0, 0)), producers.end()); - EXPECT_NE(std::find(producers.begin(), producers.end(), TaskId::make(0, WINDOW_SIZE)), producers.end()); + EXPECT_NE(std::find(producers.begin(), producers.end(), simpler::hbg::make_ring_task(0)), producers.end()); + EXPECT_NE( + std::find(producers.begin(), producers.end(), simpler::hbg::make_ring_task(WINDOW_SIZE)), producers.end() + ); } // Without an explicit semantic removal, direct inserts consume one pool entry @@ -111,7 +114,7 @@ TEST_F(HbgTensorMapTest, PoolOccupancyOnlyGrows) { EXPECT_EQ(tmap.free_entries(), POOL_SIZE); for (int32_t i = 0; i < 8; i++) { - tmap.insert(make_test_tensor(0x1000 + 0x100 * i, 64), TaskId::make(0, i)); + tmap.insert(make_test_tensor(0x1000 + 0x100 * i, 64), simpler::hbg::make_ring_task(i)); EXPECT_EQ(tmap.current_used(), i + 1); EXPECT_EQ(tmap.free_entries(), POOL_SIZE - (i + 1)); } @@ -124,9 +127,9 @@ TEST_F(HbgTensorMapTest, PoolOccupancyOnlyGrows) { TEST_F(HbgTensorMapTest, ResetLeavesTheMapAsFreshlyInitialized) { simpler::hbg::Tensor t = make_test_tensor(0x1000, 256); for (int32_t i = 0; i < 8; i++) { - tmap.insert(make_test_tensor(0x1000 + 0x100 * i, 64), TaskId::make(0, i)); + tmap.insert(make_test_tensor(0x1000 + 0x100 * i, 64), simpler::hbg::make_ring_task(i)); } - tmap.insert(t, TaskId::make(0, 9)); + tmap.insert(t, simpler::hbg::make_ring_task(9)); ASSERT_EQ(tmap.current_used(), 9); tmap.reset(); @@ -141,12 +144,12 @@ TEST_F(HbgTensorMapTest, ResetLeavesTheMapAsFreshlyInitialized) { // And it is usable, not merely empty: the second body's producer is the only one a // lookup can reach. - tmap.insert(t, TaskId::make(0, 3)); + tmap.insert(t, simpler::hbg::make_ring_task(3)); EXPECT_EQ(tmap.current_used(), 1); TestLookupResult second_body; run_lookup(tmap, t, second_body); ASSERT_EQ(second_body.count, 1); - EXPECT_EQ(second_body.entries[0].entry->producer_task_id, TaskId::make(0, 3)); + EXPECT_EQ(second_body.entries[0].entry->producer_task_id, simpler::hbg::make_ring_task(3)); } // Reset is reachable any number of times, including on a map that was never inserted @@ -160,11 +163,11 @@ TEST_F(HbgTensorMapTest, ResetIsIdempotentAndKeepsReservedSizes) { // A task id that aliases slot 0 still lands in a working chain after two resets. simpler::hbg::Tensor t = make_test_tensor(0x2000, 128); - tmap.insert(t, TaskId::make(0, WINDOW_SIZE)); + tmap.insert(t, simpler::hbg::make_ring_task(WINDOW_SIZE)); TestLookupResult result; run_lookup(tmap, t, result); ASSERT_EQ(result.count, 1); - EXPECT_EQ(result.entries[0].entry->producer_task_id, TaskId::make(0, WINDOW_SIZE)); + EXPECT_EQ(result.entries[0].entry->producer_task_id, simpler::hbg::make_ring_task(WINDOW_SIZE)); } // Filling the pool drives free_entries() to zero. No device-completion watermark @@ -172,7 +175,7 @@ TEST_F(HbgTensorMapTest, ResetIsIdempotentAndKeepsReservedSizes) { // waiting for asynchronous reclaim that HBG does not have. TEST_F(HbgTensorMapTest, ExhaustedPoolStaysExhausted) { for (int32_t i = 0; i < POOL_SIZE; i++) { - tmap.insert(make_test_tensor(0x10000 + 0x100 * i, 64), TaskId::make(0, i % WINDOW_SIZE)); + tmap.insert(make_test_tensor(0x10000 + 0x100 * i, 64), simpler::hbg::make_ring_task(i % WINDOW_SIZE)); } EXPECT_EQ(tmap.current_used(), POOL_SIZE); EXPECT_EQ(tmap.free_entries(), 0); diff --git a/tests/ut/cpp/a2a3/test_orchestrator_fanin.cpp b/tests/ut/cpp/a2a3/test_orchestrator_fanin.cpp index a3aaa6b205..d1f5201fcd 100644 --- a/tests/ut/cpp/a2a3/test_orchestrator_fanin.cpp +++ b/tests/ut/cpp/a2a3/test_orchestrator_fanin.cpp @@ -18,6 +18,7 @@ #include "utils/device_arena.h" #include "orchestrator.h" #include "shared_memory.h" +#include "tensormap_and_ringbuffer/task_id_encoding.h" class OrchestratorFaninTest : public ::testing::Test { protected: @@ -81,9 +82,13 @@ TEST_F(OrchestratorFaninTest, DuplicateExplicitProducerAddsOneFanin) { ASSERT_TRUE(consumer.task_id().is_valid()); auto &producer_slot = - sm_handle->header->rings[producer.task_id().ring()].get_slot_state_by_task_id(producer.task_id().local()); + sm_handle->header->rings[simpler::tmr::task_ring(producer.task_id())].get_slot_state_by_task_id( + simpler::tmr::task_local_id(producer.task_id()) + ); auto &consumer_slot = - sm_handle->header->rings[consumer.task_id().ring()].get_slot_state_by_task_id(consumer.task_id().local()); + sm_handle->header->rings[simpler::tmr::task_ring(consumer.task_id())].get_slot_state_by_task_id( + simpler::tmr::task_local_id(consumer.task_id()) + ); ASSERT_NE(consumer_slot.payload, nullptr); EXPECT_EQ(consumer_slot.payload->fanin_actual_count, 1); @@ -113,7 +118,9 @@ TEST_F(OrchestratorFaninTest, ExplicitWaitDepProducesWaitOnlyEdge) { ASSERT_TRUE(consumer.task_id().is_valid()); auto &consumer_slot = - sm_handle->header->rings[consumer.task_id().ring()].get_slot_state_by_task_id(consumer.task_id().local()); + sm_handle->header->rings[simpler::tmr::task_ring(consumer.task_id())].get_slot_state_by_task_id( + simpler::tmr::task_local_id(consumer.task_id()) + ); ASSERT_NE(consumer_slot.payload, nullptr); ASSERT_EQ(consumer_slot.payload->fanin_actual_count, 1); EXPECT_EQ(consumer_slot.payload->fanin_inline_edges[0].flags(), DEP_WAIT); @@ -136,9 +143,13 @@ TEST_F(OrchestratorFaninTest, DuplicateProducerOrAccumulatesFlags) { ASSERT_TRUE(consumer.task_id().is_valid()); auto &producer_slot = - sm_handle->header->rings[producer.task_id().ring()].get_slot_state_by_task_id(producer.task_id().local()); + sm_handle->header->rings[simpler::tmr::task_ring(producer.task_id())].get_slot_state_by_task_id( + simpler::tmr::task_local_id(producer.task_id()) + ); auto &consumer_slot = - sm_handle->header->rings[consumer.task_id().ring()].get_slot_state_by_task_id(consumer.task_id().local()); + sm_handle->header->rings[simpler::tmr::task_ring(consumer.task_id())].get_slot_state_by_task_id( + simpler::tmr::task_local_id(consumer.task_id()) + ); ASSERT_NE(consumer_slot.payload, nullptr); ASSERT_EQ(consumer_slot.payload->fanin_actual_count, 1); EXPECT_EQ(consumer_slot.payload->fanin_inline_edges[0].flags(), DEP_WAIT | DEP_RETAIN); @@ -177,13 +188,17 @@ TEST_F(OrchestratorFaninTest, DuplicateProducerInSpillRegionDedups) { ASSERT_TRUE(consumer.task_id().is_valid()); auto &consumer_slot = - sm_handle->header->rings[consumer.task_id().ring()].get_slot_state_by_task_id(consumer.task_id().local()); + sm_handle->header->rings[simpler::tmr::task_ring(consumer.task_id())].get_slot_state_by_task_id( + simpler::tmr::task_local_id(consumer.task_id()) + ); ASSERT_NE(consumer_slot.payload, nullptr); TaskPayload *payload = consumer_slot.payload; EXPECT_EQ(payload->fanin_actual_count, kProducers); // duplicate folded, not 66 TaskId dup = producers.back().task_id(); - auto &dup_slot = sm_handle->header->rings[dup.ring()].get_slot_state_by_task_id(dup.local()); + auto &dup_slot = sm_handle->header->rings[simpler::tmr::task_ring(dup)].get_slot_state_by_task_id( + simpler::tmr::task_local_id(dup) + ); EXPECT_EQ(dup_slot.fanout_count, FANOUT_SCOPE_BIT + 1); // one pin, not two // The first spilled edge is the duplicated producer; its flags OR-folded to @@ -204,7 +219,9 @@ TEST_F(OrchestratorFaninTest, AllCompletedFastPathReleasesWaitOnlyPin) { TaskOutputTensors producer = orch.submit_dummy_task(producer_args); ASSERT_TRUE(producer.task_id().is_valid()); auto &producer_slot = - sm_handle->header->rings[producer.task_id().ring()].get_slot_state_by_task_id(producer.task_id().local()); + sm_handle->header->rings[simpler::tmr::task_ring(producer.task_id())].get_slot_state_by_task_id( + simpler::tmr::task_local_id(producer.task_id()) + ); // COMPLETED but not consumed (the open scope still pins it): the consumer takes // the all-completed fast path. producer_slot.task_state.store(CHIP_TASK_COMPLETED, std::memory_order_release); @@ -235,7 +252,8 @@ TEST_F(OrchestratorFaninTest, SubmitPathHeapDeadlockLogReportsRingAndRealHeapSta ASSERT_TRUE(first.task_id().is_valid()); auto &ring = sm_handle->header->rings[1]; - auto &first_slot = ring.get_slot_state_by_task_id(static_cast(first.task_id().local())); + auto &first_slot = + ring.get_slot_state_by_task_id(static_cast(simpler::tmr::task_local_id(first.task_id()))); orch.end_scope(); first_slot.task_state.store(CHIP_TASK_COMPLETED, std::memory_order_release); sched.check_and_handle_consumed(first_slot); diff --git a/tests/ut/cpp/a2a3/test_tensormap.cpp b/tests/ut/cpp/a2a3/test_tensormap.cpp index febd247e70..abf4179f52 100644 --- a/tests/ut/cpp/a2a3/test_tensormap.cpp +++ b/tests/ut/cpp/a2a3/test_tensormap.cpp @@ -31,6 +31,7 @@ #include "utils/device_arena.h" #include "orchestration_api.h" #include "tensormap.h" +#include "tensormap_and_ringbuffer/task_id_encoding.h" // ============================================================================= // Helpers @@ -156,7 +157,7 @@ TEST_F(TensorMapTest, HashBoundedByBucketCount) { TEST_F(TensorMapTest, InsertThenLookupFindsProducer) { simpler::tmr::Tensor t = make_test_tensor(0x1000, 256); - TaskId tid = TaskId::make(0, 0); + TaskId tid = simpler::tmr::make_task_id(0, 0); tmap.insert(t, tid); TestLookupResult result; @@ -175,8 +176,8 @@ TEST_F(TensorMapTest, LookupEmptyReturnsZero) { TEST_F(TensorMapTest, InsertMultipleSameBuffer) { simpler::tmr::Tensor t1 = make_test_tensor(0x1000, 256); simpler::tmr::Tensor t2 = make_test_tensor(0x1000, 128); - TaskId tid1 = TaskId::make(0, 0); - TaskId tid2 = TaskId::make(0, 1); + TaskId tid1 = simpler::tmr::make_task_id(0, 0); + TaskId tid2 = simpler::tmr::make_task_id(0, 1); tmap.insert(t1, tid1); tmap.insert(t2, tid2); @@ -190,18 +191,18 @@ TEST_F(TensorMapTest, InsertMultipleSameBuffer) { TEST_F(TensorMapTest, InsertDifferentBuffersNoCollision) { simpler::tmr::Tensor t1 = make_test_tensor(0x1000, 256); simpler::tmr::Tensor t2 = make_test_tensor(0x2000, 256); - tmap.insert(t1, TaskId::make(0, 0)); - tmap.insert(t2, TaskId::make(0, 1)); + tmap.insert(t1, simpler::tmr::make_task_id(0, 0)); + tmap.insert(t2, simpler::tmr::make_task_id(0, 1)); TestLookupResult r1; run_lookup(tmap, t1, r1); EXPECT_EQ(r1.count, 1); - EXPECT_EQ(r1.entries[0].entry->producer_task_id, TaskId::make(0, 0)); + EXPECT_EQ(r1.entries[0].entry->producer_task_id, simpler::tmr::make_task_id(0, 0)); TestLookupResult r2; run_lookup(tmap, t2, r2); EXPECT_EQ(r2.count, 1); - EXPECT_EQ(r2.entries[0].entry->producer_task_id, TaskId::make(0, 1)); + EXPECT_EQ(r2.entries[0].entry->producer_task_id, simpler::tmr::make_task_id(0, 1)); } // ============================================================================= @@ -213,7 +214,7 @@ TEST_F(TensorMapTest, OverlapFastPathCovered) { // Consumer covers producer -> COVERED simpler::tmr::Tensor producer = make_test_tensor(0x1000, 256); simpler::tmr::Tensor consumer = make_test_tensor(0x1000, 512); - tmap.insert(producer, TaskId::make(0, 0)); + tmap.insert(producer, simpler::tmr::make_task_id(0, 0)); TestLookupResult result; run_lookup(tmap, consumer, result); @@ -226,7 +227,7 @@ TEST_F(TensorMapTest, OverlapFastPathOther) { // Consumer does NOT cover producer -> OTHER simpler::tmr::Tensor producer = make_test_tensor(0x1000, 512); simpler::tmr::Tensor consumer = make_test_tensor(0x1000, 256); - tmap.insert(producer, TaskId::make(0, 0)); + tmap.insert(producer, simpler::tmr::make_task_id(0, 0)); TestLookupResult result; run_lookup(tmap, consumer, result); @@ -236,7 +237,7 @@ TEST_F(TensorMapTest, OverlapFastPathOther) { TEST_F(TensorMapTest, OverlapFastPathExactMatch) { simpler::tmr::Tensor t = make_test_tensor(0x1000, 256); - tmap.insert(t, TaskId::make(0, 0)); + tmap.insert(t, simpler::tmr::make_task_id(0, 0)); TestLookupResult result; run_lookup(tmap, t, result); @@ -259,7 +260,7 @@ TEST_F(TensorMapTest, OverlapSlowPathNoOverlap) { uint32_t con_offsets[] = {128, 0}; simpler::tmr::Tensor consumer = base.view(con_shapes, con_offsets); - tmap.insert(producer, TaskId::make(0, 0)); + tmap.insert(producer, simpler::tmr::make_task_id(0, 0)); TestLookupResult result; run_lookup(tmap, consumer, result); @@ -277,7 +278,7 @@ TEST_F(TensorMapTest, OverlapSlowPathPartialOverlap) { uint32_t con_offsets[] = {64, 0}; simpler::tmr::Tensor consumer = base.view(con_shapes, con_offsets); - tmap.insert(producer, TaskId::make(0, 0)); + tmap.insert(producer, simpler::tmr::make_task_id(0, 0)); TestLookupResult result; run_lookup(tmap, consumer, result); @@ -296,7 +297,7 @@ TEST_F(TensorMapTest, OverlapSlowPathCovered) { uint32_t con_offsets[] = {0, 0}; simpler::tmr::Tensor consumer = base.view(con_shapes, con_offsets); - tmap.insert(producer, TaskId::make(0, 0)); + tmap.insert(producer, simpler::tmr::make_task_id(0, 0)); TestLookupResult result; run_lookup(tmap, consumer, result); @@ -313,7 +314,7 @@ TEST_F(TensorMapTest, VersionMismatchReturnsOther) { simpler::tmr::Tensor producer = make_test_tensor(0x1000, 256, 1, 0); simpler::tmr::Tensor consumer = make_test_tensor(0x1000, 256, 1, 1); - tmap.insert(producer, TaskId::make(0, 0)); + tmap.insert(producer, simpler::tmr::make_task_id(0, 0)); TestLookupResult result; run_lookup(tmap, consumer, result); @@ -327,8 +328,8 @@ TEST_F(TensorMapTest, VersionMismatchReturnsOther) { TEST_F(TensorMapTest, StaleEntriesSkippedDuringLookup) { simpler::tmr::Tensor t = make_test_tensor(0x1000, 256); - tmap.insert(t, TaskId::make(0, 0)); - tmap.insert(t, TaskId::make(0, 1)); + tmap.insert(t, simpler::tmr::make_task_id(0, 0)); + tmap.insert(t, simpler::tmr::make_task_id(0, 1)); // Advance validity to skip task 0 tmap.sync_validity(0, 1); @@ -336,14 +337,14 @@ TEST_F(TensorMapTest, StaleEntriesSkippedDuringLookup) { TestLookupResult result; run_lookup(tmap, t, result); ASSERT_EQ(result.count, 1); - EXPECT_EQ(result.entries[0].entry->producer_task_id, TaskId::make(0, 1)); + EXPECT_EQ(result.entries[0].entry->producer_task_id, simpler::tmr::make_task_id(0, 1)); } TEST_F(TensorMapTest, StaleEntriesNotTruncatedAcrossRings) { simpler::tmr::Tensor t = make_test_tensor(0x1000, 256); // Ring 0, task 0 and Ring 1, task 0 -> same bucket - tmap.insert(t, TaskId::make(0, 0)); - tmap.insert(t, TaskId::make(1, 0)); + tmap.insert(t, simpler::tmr::make_task_id(0, 0)); + tmap.insert(t, simpler::tmr::make_task_id(1, 0)); // Invalidate ring 0 only tmap.sync_validity(0, 1); @@ -352,7 +353,7 @@ TEST_F(TensorMapTest, StaleEntriesNotTruncatedAcrossRings) { run_lookup(tmap, t, result); // Ring 1 task 0 still valid, ring 0 task 0 invalidated ASSERT_EQ(result.count, 1); - EXPECT_EQ(result.entries[0].entry->producer_task_id, TaskId::make(1, 0)); + EXPECT_EQ(result.entries[0].entry->producer_task_id, simpler::tmr::make_task_id(1, 0)); } // ============================================================================= @@ -361,9 +362,9 @@ TEST_F(TensorMapTest, StaleEntriesNotTruncatedAcrossRings) { TEST_F(TensorMapTest, CleanupRetiredRemovesEntriesForRetiredTasks) { simpler::tmr::Tensor t = make_test_tensor(0x1000, 256); - tmap.insert(t, TaskId::make(0, 0)); - tmap.insert(t, TaskId::make(0, 1)); - tmap.insert(t, TaskId::make(0, 2)); + tmap.insert(t, simpler::tmr::make_task_id(0, 0)); + tmap.insert(t, simpler::tmr::make_task_id(0, 1)); + tmap.insert(t, simpler::tmr::make_task_id(0, 2)); EXPECT_EQ(tmap.valid_count(), 3); // Cleanup tasks [0, 2) on ring 0 @@ -374,13 +375,13 @@ TEST_F(TensorMapTest, CleanupRetiredRemovesEntriesForRetiredTasks) { TestLookupResult result; run_lookup(tmap, t, result); ASSERT_EQ(result.count, 1); - EXPECT_EQ(result.entries[0].entry->producer_task_id, TaskId::make(0, 2)); + EXPECT_EQ(result.entries[0].entry->producer_task_id, simpler::tmr::make_task_id(0, 2)); } TEST_F(TensorMapTest, CleanupRetiredPreservesOtherRings) { simpler::tmr::Tensor t = make_test_tensor(0x1000, 256); - tmap.insert(t, TaskId::make(0, 0)); - tmap.insert(t, TaskId::make(1, 0)); + tmap.insert(t, simpler::tmr::make_task_id(0, 0)); + tmap.insert(t, simpler::tmr::make_task_id(1, 0)); tmap.cleanup_retired(0, 0, 1); @@ -389,12 +390,12 @@ TEST_F(TensorMapTest, CleanupRetiredPreservesOtherRings) { TestLookupResult result; run_lookup(tmap, t, result); ASSERT_EQ(result.count, 1); - EXPECT_EQ(result.entries[0].entry->producer_task_id, TaskId::make(1, 0)); + EXPECT_EQ(result.entries[0].entry->producer_task_id, simpler::tmr::make_task_id(1, 0)); } TEST_F(TensorMapTest, CleanupRetiredFreesEntriesToPool) { simpler::tmr::Tensor t = make_test_tensor(0x1000, 256); - tmap.insert(t, TaskId::make(0, 0)); + tmap.insert(t, simpler::tmr::make_task_id(0, 0)); EXPECT_EQ(tmap.free_num, 0); EXPECT_EQ(tmap.next_entry_idx, 1); @@ -403,7 +404,7 @@ TEST_F(TensorMapTest, CleanupRetiredFreesEntriesToPool) { EXPECT_EQ(tmap.free_num, 1) << "Cleaned entry should be in free list"; // New insert should reuse free entry instead of allocating fresh - tmap.insert(t, TaskId::make(0, 1)); + tmap.insert(t, simpler::tmr::make_task_id(0, 1)); EXPECT_EQ(tmap.free_num, 0); EXPECT_EQ(tmap.next_entry_idx, 1) << "Should reuse freed entry, not allocate new"; } @@ -415,8 +416,8 @@ TEST_F(TensorMapTest, CleanupRetiredFreesEntriesToPool) { TEST_F(TensorMapTest, CleanupRetiredSparesLaterTaskReusingSlot) { simpler::tmr::Tensor t = make_test_tensor(0x1000, 256); // Task 0 and task 0 + WINDOW_SIZE share slot 0 (local_id & (WINDOW_SIZE-1)). - tmap.insert(t, TaskId::make(0, 0)); - tmap.insert(t, TaskId::make(0, WINDOW_SIZE)); + tmap.insert(t, simpler::tmr::make_task_id(0, 0)); + tmap.insert(t, simpler::tmr::make_task_id(0, WINDOW_SIZE)); ASSERT_EQ(tmap.valid_count(), 2); // Retire only task 0. @@ -427,7 +428,7 @@ TEST_F(TensorMapTest, CleanupRetiredSparesLaterTaskReusingSlot) { TestLookupResult result; run_lookup(tmap, t, result); ASSERT_EQ(result.count, 1); - EXPECT_EQ(result.entries[0].entry->producer_task_id, TaskId::make(0, WINDOW_SIZE)); + EXPECT_EQ(result.entries[0].entry->producer_task_id, simpler::tmr::make_task_id(0, WINDOW_SIZE)); } // ============================================================================= @@ -436,9 +437,9 @@ TEST_F(TensorMapTest, CleanupRetiredSparesLaterTaskReusingSlot) { TEST_F(TensorMapTest, MultiRingIndependentLookup) { simpler::tmr::Tensor t = make_test_tensor(0x1000, 256); - tmap.insert(t, TaskId::make(0, 5)); - tmap.insert(t, TaskId::make(1, 3)); - tmap.insert(t, TaskId::make(2, 7)); + tmap.insert(t, simpler::tmr::make_task_id(0, 5)); + tmap.insert(t, simpler::tmr::make_task_id(1, 3)); + tmap.insert(t, simpler::tmr::make_task_id(2, 7)); TestLookupResult result; run_lookup(tmap, t, result); @@ -451,7 +452,7 @@ TEST_F(TensorMapTest, MultiRingIndependentLookup) { TestLookupResult result2; run_lookup(tmap, t, result2); EXPECT_EQ(result2.count, 1); - EXPECT_EQ(result2.entries[0].entry->producer_task_id, TaskId::make(1, 3)); + EXPECT_EQ(result2.entries[0].entry->producer_task_id, simpler::tmr::make_task_id(1, 3)); } // ============================================================================= @@ -462,7 +463,7 @@ TEST_F(TensorMapTest, LookupReturnsAllMatches) { simpler::tmr::Tensor t = make_test_tensor(0x1000, 256); // Insert 20 entries for the same buffer (was capped at 16 before #669) for (int i = 0; i < 20; i++) { - tmap.insert(t, TaskId::make(0, i)); + tmap.insert(t, simpler::tmr::make_task_id(0, i)); } TestLookupResult result; @@ -478,28 +479,28 @@ TEST_F(TensorMapTest, PoolExhaustionAsserts) { // With pool_size=64, inserting 64 entries should work, 65th should fail for (int i = 0; i < POOL_SIZE; i++) { simpler::tmr::Tensor t = make_test_tensor(0x1000 + i * 0x100, 256); - tmap.insert(t, TaskId::make(0, i)); + tmap.insert(t, simpler::tmr::make_task_id(0, i)); } EXPECT_EQ(tmap.next_entry_idx, POOL_SIZE); EXPECT_EQ(tmap.free_num, 0); // 65th insert should trigger always_assert (pool overflow) simpler::tmr::Tensor overflow = make_test_tensor(0x9000, 256); - EXPECT_THROW(tmap.insert(overflow, TaskId::make(0, POOL_SIZE)), std::runtime_error); + EXPECT_THROW(tmap.insert(overflow, simpler::tmr::make_task_id(0, POOL_SIZE)), std::runtime_error); } TEST_F(TensorMapTest, FreeListRecycling) { simpler::tmr::Tensor t = make_test_tensor(0x1000, 256); // Insert and cleanup 10 entries for (int i = 0; i < 10; i++) { - tmap.insert(t, TaskId::make(0, i)); + tmap.insert(t, simpler::tmr::make_task_id(0, i)); } tmap.cleanup_retired(0, 0, 10); EXPECT_EQ(tmap.free_num, 10); // Re-insert should use free list for (int i = 10; i < 20; i++) { - tmap.insert(t, TaskId::make(0, i)); + tmap.insert(t, simpler::tmr::make_task_id(0, i)); } EXPECT_EQ(tmap.free_num, 0); EXPECT_EQ(tmap.next_entry_idx, 10) << "No new pool entries consumed when free list available"; @@ -512,7 +513,7 @@ TEST_F(TensorMapTest, FreeListRecycling) { TEST_F(TensorMapTest, PerTaskEntryListTracksMultipleOutputs) { simpler::tmr::Tensor t1 = make_test_tensor(0x1000, 256); simpler::tmr::Tensor t2 = make_test_tensor(0x2000, 128); - TaskId tid = TaskId::make(0, 5); + TaskId tid = simpler::tmr::make_task_id(0, 5); tmap.insert(t1, tid); tmap.insert(t2, tid); @@ -530,9 +531,9 @@ TEST_F(TensorMapTest, PerTaskEntryListTracksMultipleOutputs) { TEST_F(TensorMapTest, RemoveMiddleEntryPreservesChain) { simpler::tmr::Tensor t = make_test_tensor(0x1000, 256); - TaskId tid0 = TaskId::make(0, 0); - TaskId tid1 = TaskId::make(0, 1); - TaskId tid2 = TaskId::make(0, 2); + TaskId tid0 = simpler::tmr::make_task_id(0, 0); + TaskId tid1 = simpler::tmr::make_task_id(0, 1); + TaskId tid2 = simpler::tmr::make_task_id(0, 2); tmap.insert(t, tid0); tmap.insert(t, tid1); @@ -547,7 +548,7 @@ TEST_F(TensorMapTest, RemoveMiddleEntryPreservesChain) { std::set found_locals; for (int i = 0; i < result.count; i++) { - found_locals.insert(result.entries[i].entry->producer_task_id.local()); + found_locals.insert(simpler::tmr::task_local_id(result.entries[i].entry->producer_task_id)); } EXPECT_TRUE(found_locals.count(0)); EXPECT_TRUE(found_locals.count(2)); @@ -558,9 +559,9 @@ TEST_F(TensorMapTest, RemoveMiddleEntryPreservesChain) { // ============================================================================= TEST(TaskIdTest, MakeAndDecode) { - auto tid = TaskId::make(3, 42); - EXPECT_EQ(tid.ring(), 3); - EXPECT_EQ(tid.local(), 42u); + auto tid = simpler::tmr::make_task_id(3, 42); + EXPECT_EQ(simpler::tmr::task_ring(tid), 3); + EXPECT_EQ(simpler::tmr::task_local_id(tid), 42u); } TEST(TaskIdTest, InvalidSentinel) { @@ -570,21 +571,21 @@ TEST(TaskIdTest, InvalidSentinel) { } TEST(TaskIdTest, Equality) { - auto a = TaskId::make(1, 100); - auto b = TaskId::make(1, 100); - auto c = TaskId::make(2, 100); + auto a = simpler::tmr::make_task_id(1, 100); + auto b = simpler::tmr::make_task_id(1, 100); + auto c = simpler::tmr::make_task_id(2, 100); EXPECT_EQ(a, b); EXPECT_NE(a, c); } TEST(TaskIdTest, RingIdMaxValue) { - auto tid = TaskId::make(255, 0); - EXPECT_EQ(tid.ring(), 255); - EXPECT_EQ(tid.local(), 0u); + auto tid = simpler::tmr::make_task_id(255, 0); + EXPECT_EQ(simpler::tmr::task_ring(tid), 255); + EXPECT_EQ(simpler::tmr::task_local_id(tid), 0u); } TEST(TaskIdTest, LocalIdMaxValue) { - auto tid = TaskId::make(0, UINT32_MAX); - EXPECT_EQ(tid.ring(), 0); - EXPECT_EQ(tid.local(), UINT32_MAX); + auto tid = simpler::tmr::make_task_id(0, UINT32_MAX); + EXPECT_EQ(simpler::tmr::task_ring(tid), 0); + EXPECT_EQ(simpler::tmr::task_local_id(tid), UINT32_MAX); } diff --git a/tests/ut/cpp/a5/test_aicore_completion_mailbox.cpp b/tests/ut/cpp/a5/test_aicore_completion_mailbox.cpp index 7d848e21c6..6bc026eb4e 100644 --- a/tests/ut/cpp/a5/test_aicore_completion_mailbox.cpp +++ b/tests/ut/cpp/a5/test_aicore_completion_mailbox.cpp @@ -29,7 +29,10 @@ namespace { -TaskId make_token(uint32_t local) { return TaskId::make(/*ring=*/0, local); } +// The mailbox stores a token and compares it for identity; it never decodes one, +// and the encoding belongs to whichever runtime minted it, so any distinct 64-bit +// value serves here. +TaskId make_token(uint32_t local) { return TaskId{local}; } AICoreCompletionMailbox *fresh_mailbox() { void *raw = ::operator new(sizeof(AICoreCompletionMailbox)); diff --git a/tests/ut/cpp/a5/test_hbg_submit_poison.cpp b/tests/ut/cpp/a5/test_hbg_submit_poison.cpp index 5d47d8d683..0ea592002b 100644 --- a/tests/ut/cpp/a5/test_hbg_submit_poison.cpp +++ b/tests/ut/cpp/a5/test_hbg_submit_poison.cpp @@ -33,6 +33,7 @@ #include "utils/device_arena.h" #include "orchestrator.h" #include "shared_memory.h" +#include "host_build_graph/task_id_encoding.h" namespace { @@ -139,7 +140,7 @@ TEST_F(HbgSubmitPoisonTest, EveryDeviceReadFieldIsWrittenOverPoison) { const ChipTaskSlotState &st = tasks.slot_states[local]; // Descriptor: the task id is written to this exact local id. - EXPECT_EQ(desc.task_id.local(), static_cast(local)); + EXPECT_EQ(simpler::hbg::task_local_id(desc.task_id), static_cast(local)); // task_state is written at submit (reset_for_reuse skips it): PENDING for a // dispatchable task, COMPLETED for a pre-completed hidden-alloc. Either way a // real enum, never poison. @@ -163,8 +164,8 @@ TEST_F(HbgSubmitPoisonTest, EveryDeviceReadFieldIsWrittenOverPoison) { } // Field-specific coverage on the real task: tensors, scalar, packed output buffer. - const TaskDescriptor &root_desc = tasks.task_descriptors[root.task_id().local()]; - const TaskPayload &root_pl = tasks.task_payloads[root.task_id().local()]; + const TaskDescriptor &root_desc = tasks.task_descriptors[simpler::hbg::task_local_id(root.task_id())]; + const TaskPayload &root_pl = tasks.task_payloads[simpler::hbg::task_local_id(root.task_id())]; EXPECT_EQ(root_pl.tensor_count, 1); EXPECT_EQ(root_pl.scalar_count, 1); EXPECT_EQ(root_pl.dump_metadata.dump_arg_mask, (uint64_t{1} << 0) | (uint64_t{1} << 1)); @@ -175,7 +176,7 @@ TEST_F(HbgSubmitPoisonTest, EveryDeviceReadFieldIsWrittenOverPoison) { EXPECT_EQ(root_desc.kernel_id[static_cast(SubtaskSlot::AIV0)], 0); // The consumer's fanin is written: two duplicate deps dedupe to one. - const TaskPayload &cons_pl = tasks.task_payloads[consumer.task_id().local()]; + const TaskPayload &cons_pl = tasks.task_payloads[simpler::hbg::task_local_id(consumer.task_id())]; EXPECT_EQ(cons_pl.fanin_count, 1); - EXPECT_EQ(cons_pl.fanin_data()[0], static_cast(root.task_id().local())); + EXPECT_EQ(cons_pl.fanin_data()[0], static_cast(simpler::hbg::task_local_id(root.task_id()))); } diff --git a/tests/ut/cpp/a5/test_orchestrator_fanin.cpp b/tests/ut/cpp/a5/test_orchestrator_fanin.cpp index 6ea756e409..0616252269 100644 --- a/tests/ut/cpp/a5/test_orchestrator_fanin.cpp +++ b/tests/ut/cpp/a5/test_orchestrator_fanin.cpp @@ -18,6 +18,7 @@ #include "utils/device_arena.h" #include "orchestrator.h" #include "shared_memory.h" +#include "tensormap_and_ringbuffer/task_id_encoding.h" class OrchestratorFaninTest : public ::testing::Test { protected: @@ -93,9 +94,13 @@ TEST_F(OrchestratorFaninTest, DuplicateExplicitProducerAddsOneFanin) { ASSERT_TRUE(consumer.task_id().is_valid()); auto &producer_slot = - sm_handle->header->rings[producer.task_id().ring()].get_slot_state_by_task_id(producer.task_id().local()); + sm_handle->header->rings[simpler::tmr::task_ring(producer.task_id())].get_slot_state_by_task_id( + simpler::tmr::task_local_id(producer.task_id()) + ); auto &consumer_slot = - sm_handle->header->rings[consumer.task_id().ring()].get_slot_state_by_task_id(consumer.task_id().local()); + sm_handle->header->rings[simpler::tmr::task_ring(consumer.task_id())].get_slot_state_by_task_id( + simpler::tmr::task_local_id(consumer.task_id()) + ); ASSERT_NE(consumer_slot.payload, nullptr); EXPECT_EQ(consumer_slot.payload->fanin_actual_count, 1); @@ -125,7 +130,9 @@ TEST_F(OrchestratorFaninTest, ExplicitWaitDepProducesWaitOnlyEdge) { ASSERT_TRUE(consumer.task_id().is_valid()); auto &consumer_slot = - sm_handle->header->rings[consumer.task_id().ring()].get_slot_state_by_task_id(consumer.task_id().local()); + sm_handle->header->rings[simpler::tmr::task_ring(consumer.task_id())].get_slot_state_by_task_id( + simpler::tmr::task_local_id(consumer.task_id()) + ); ASSERT_NE(consumer_slot.payload, nullptr); ASSERT_EQ(consumer_slot.payload->fanin_actual_count, 1); EXPECT_EQ(consumer_slot.payload->fanin_inline_edges[0].flags(), DEP_WAIT); @@ -148,9 +155,13 @@ TEST_F(OrchestratorFaninTest, DuplicateProducerOrAccumulatesFlags) { ASSERT_TRUE(consumer.task_id().is_valid()); auto &producer_slot = - sm_handle->header->rings[producer.task_id().ring()].get_slot_state_by_task_id(producer.task_id().local()); + sm_handle->header->rings[simpler::tmr::task_ring(producer.task_id())].get_slot_state_by_task_id( + simpler::tmr::task_local_id(producer.task_id()) + ); auto &consumer_slot = - sm_handle->header->rings[consumer.task_id().ring()].get_slot_state_by_task_id(consumer.task_id().local()); + sm_handle->header->rings[simpler::tmr::task_ring(consumer.task_id())].get_slot_state_by_task_id( + simpler::tmr::task_local_id(consumer.task_id()) + ); ASSERT_NE(consumer_slot.payload, nullptr); ASSERT_EQ(consumer_slot.payload->fanin_actual_count, 1); EXPECT_EQ(consumer_slot.payload->fanin_inline_edges[0].flags(), DEP_WAIT | DEP_RETAIN); @@ -189,13 +200,17 @@ TEST_F(OrchestratorFaninTest, DuplicateProducerInSpillRegionDedups) { ASSERT_TRUE(consumer.task_id().is_valid()); auto &consumer_slot = - sm_handle->header->rings[consumer.task_id().ring()].get_slot_state_by_task_id(consumer.task_id().local()); + sm_handle->header->rings[simpler::tmr::task_ring(consumer.task_id())].get_slot_state_by_task_id( + simpler::tmr::task_local_id(consumer.task_id()) + ); ASSERT_NE(consumer_slot.payload, nullptr); TaskPayload *payload = consumer_slot.payload; EXPECT_EQ(payload->fanin_actual_count, kProducers); // duplicate folded, not 66 TaskId dup = producers.back().task_id(); - auto &dup_slot = sm_handle->header->rings[dup.ring()].get_slot_state_by_task_id(dup.local()); + auto &dup_slot = sm_handle->header->rings[simpler::tmr::task_ring(dup)].get_slot_state_by_task_id( + simpler::tmr::task_local_id(dup) + ); EXPECT_EQ(dup_slot.fanout_count, FANOUT_SCOPE_BIT + 1); // one pin, not two // The first spilled edge is the duplicated producer; its flags OR-folded to @@ -216,7 +231,9 @@ TEST_F(OrchestratorFaninTest, AllCompletedFastPathReleasesWaitOnlyPin) { TaskOutputTensors producer = orch.submit_dummy_task(producer_args); ASSERT_TRUE(producer.task_id().is_valid()); auto &producer_slot = - sm_handle->header->rings[producer.task_id().ring()].get_slot_state_by_task_id(producer.task_id().local()); + sm_handle->header->rings[simpler::tmr::task_ring(producer.task_id())].get_slot_state_by_task_id( + simpler::tmr::task_local_id(producer.task_id()) + ); // COMPLETED but not consumed (the open scope still pins it): the consumer takes // the all-completed fast path. producer_slot.task_state.store(CHIP_TASK_COMPLETED, std::memory_order_release); @@ -247,7 +264,8 @@ TEST_F(OrchestratorFaninTest, SubmitPathHeapDeadlockLogReportsRingAndRealHeapSta ASSERT_TRUE(first.task_id().is_valid()); auto &ring = sm_handle->header->rings[1]; - auto &first_slot = ring.get_slot_state_by_task_id(static_cast(first.task_id().local())); + auto &first_slot = + ring.get_slot_state_by_task_id(static_cast(simpler::tmr::task_local_id(first.task_id()))); orch.end_scope(); first_slot.task_state.store(CHIP_TASK_COMPLETED, std::memory_order_release); sched.check_and_handle_consumed(first_slot); diff --git a/tests/ut/cpp/a5/test_tensormap.cpp b/tests/ut/cpp/a5/test_tensormap.cpp index 97c34f498c..0b59859812 100644 --- a/tests/ut/cpp/a5/test_tensormap.cpp +++ b/tests/ut/cpp/a5/test_tensormap.cpp @@ -31,6 +31,7 @@ #include "utils/device_arena.h" #include "orchestration_api.h" #include "tensormap.h" +#include "tensormap_and_ringbuffer/task_id_encoding.h" // ============================================================================= // Helpers @@ -157,7 +158,7 @@ TEST_F(TensorMapTest, HashBoundedByBucketCount) { TEST_F(TensorMapTest, InsertThenLookupFindsProducer) { simpler::tmr::Tensor t = make_test_tensor(0x1000, 256); - TaskId tid = TaskId::make(0, 0); + TaskId tid = simpler::tmr::make_task_id(0, 0); tmap.insert(t, tid); TestLookupResult result; @@ -176,8 +177,8 @@ TEST_F(TensorMapTest, LookupEmptyReturnsZero) { TEST_F(TensorMapTest, InsertMultipleSameBuffer) { simpler::tmr::Tensor t1 = make_test_tensor(0x1000, 256); simpler::tmr::Tensor t2 = make_test_tensor(0x1000, 128); - TaskId tid1 = TaskId::make(0, 0); - TaskId tid2 = TaskId::make(0, 1); + TaskId tid1 = simpler::tmr::make_task_id(0, 0); + TaskId tid2 = simpler::tmr::make_task_id(0, 1); tmap.insert(t1, tid1); tmap.insert(t2, tid2); @@ -191,18 +192,18 @@ TEST_F(TensorMapTest, InsertMultipleSameBuffer) { TEST_F(TensorMapTest, InsertDifferentBuffersNoCollision) { simpler::tmr::Tensor t1 = make_test_tensor(0x1000, 256); simpler::tmr::Tensor t2 = make_test_tensor(0x2000, 256); - tmap.insert(t1, TaskId::make(0, 0)); - tmap.insert(t2, TaskId::make(0, 1)); + tmap.insert(t1, simpler::tmr::make_task_id(0, 0)); + tmap.insert(t2, simpler::tmr::make_task_id(0, 1)); TestLookupResult r1; run_lookup(tmap, t1, r1); EXPECT_EQ(r1.count, 1); - EXPECT_EQ(r1.entries[0].entry->producer_task_id, TaskId::make(0, 0)); + EXPECT_EQ(r1.entries[0].entry->producer_task_id, simpler::tmr::make_task_id(0, 0)); TestLookupResult r2; run_lookup(tmap, t2, r2); EXPECT_EQ(r2.count, 1); - EXPECT_EQ(r2.entries[0].entry->producer_task_id, TaskId::make(0, 1)); + EXPECT_EQ(r2.entries[0].entry->producer_task_id, simpler::tmr::make_task_id(0, 1)); } // ============================================================================= @@ -214,7 +215,7 @@ TEST_F(TensorMapTest, OverlapFastPathCovered) { // Consumer covers producer -> COVERED simpler::tmr::Tensor producer = make_test_tensor(0x1000, 256); simpler::tmr::Tensor consumer = make_test_tensor(0x1000, 512); - tmap.insert(producer, TaskId::make(0, 0)); + tmap.insert(producer, simpler::tmr::make_task_id(0, 0)); TestLookupResult result; run_lookup(tmap, consumer, result); @@ -227,7 +228,7 @@ TEST_F(TensorMapTest, OverlapFastPathOther) { // Consumer does NOT cover producer -> OTHER simpler::tmr::Tensor producer = make_test_tensor(0x1000, 512); simpler::tmr::Tensor consumer = make_test_tensor(0x1000, 256); - tmap.insert(producer, TaskId::make(0, 0)); + tmap.insert(producer, simpler::tmr::make_task_id(0, 0)); TestLookupResult result; run_lookup(tmap, consumer, result); @@ -237,7 +238,7 @@ TEST_F(TensorMapTest, OverlapFastPathOther) { TEST_F(TensorMapTest, OverlapFastPathExactMatch) { simpler::tmr::Tensor t = make_test_tensor(0x1000, 256); - tmap.insert(t, TaskId::make(0, 0)); + tmap.insert(t, simpler::tmr::make_task_id(0, 0)); TestLookupResult result; run_lookup(tmap, t, result); @@ -260,7 +261,7 @@ TEST_F(TensorMapTest, OverlapSlowPathNoOverlap) { uint32_t con_offsets[] = {128, 0}; simpler::tmr::Tensor consumer = base.view(con_shapes, con_offsets); - tmap.insert(producer, TaskId::make(0, 0)); + tmap.insert(producer, simpler::tmr::make_task_id(0, 0)); TestLookupResult result; run_lookup(tmap, consumer, result); @@ -278,7 +279,7 @@ TEST_F(TensorMapTest, OverlapSlowPathPartialOverlap) { uint32_t con_offsets[] = {64, 0}; simpler::tmr::Tensor consumer = base.view(con_shapes, con_offsets); - tmap.insert(producer, TaskId::make(0, 0)); + tmap.insert(producer, simpler::tmr::make_task_id(0, 0)); TestLookupResult result; run_lookup(tmap, consumer, result); @@ -297,7 +298,7 @@ TEST_F(TensorMapTest, OverlapSlowPathCovered) { uint32_t con_offsets[] = {0, 0}; simpler::tmr::Tensor consumer = base.view(con_shapes, con_offsets); - tmap.insert(producer, TaskId::make(0, 0)); + tmap.insert(producer, simpler::tmr::make_task_id(0, 0)); TestLookupResult result; run_lookup(tmap, consumer, result); @@ -314,7 +315,7 @@ TEST_F(TensorMapTest, VersionMismatchReturnsOther) { simpler::tmr::Tensor producer = make_test_tensor(0x1000, 256, 1, 0); simpler::tmr::Tensor consumer = make_test_tensor(0x1000, 256, 1, 1); - tmap.insert(producer, TaskId::make(0, 0)); + tmap.insert(producer, simpler::tmr::make_task_id(0, 0)); TestLookupResult result; run_lookup(tmap, consumer, result); @@ -328,8 +329,8 @@ TEST_F(TensorMapTest, VersionMismatchReturnsOther) { TEST_F(TensorMapTest, StaleEntriesSkippedDuringLookup) { simpler::tmr::Tensor t = make_test_tensor(0x1000, 256); - tmap.insert(t, TaskId::make(0, 0)); - tmap.insert(t, TaskId::make(0, 1)); + tmap.insert(t, simpler::tmr::make_task_id(0, 0)); + tmap.insert(t, simpler::tmr::make_task_id(0, 1)); // Advance validity to skip task 0 tmap.sync_validity(0, 1); @@ -337,14 +338,14 @@ TEST_F(TensorMapTest, StaleEntriesSkippedDuringLookup) { TestLookupResult result; run_lookup(tmap, t, result); ASSERT_EQ(result.count, 1); - EXPECT_EQ(result.entries[0].entry->producer_task_id, TaskId::make(0, 1)); + EXPECT_EQ(result.entries[0].entry->producer_task_id, simpler::tmr::make_task_id(0, 1)); } TEST_F(TensorMapTest, StaleEntriesNotTruncatedAcrossRings) { simpler::tmr::Tensor t = make_test_tensor(0x1000, 256); // Ring 0, task 0 and Ring 1, task 0 -> same bucket - tmap.insert(t, TaskId::make(0, 0)); - tmap.insert(t, TaskId::make(1, 0)); + tmap.insert(t, simpler::tmr::make_task_id(0, 0)); + tmap.insert(t, simpler::tmr::make_task_id(1, 0)); // Invalidate ring 0 only tmap.sync_validity(0, 1); @@ -353,7 +354,7 @@ TEST_F(TensorMapTest, StaleEntriesNotTruncatedAcrossRings) { run_lookup(tmap, t, result); // Ring 1 task 0 still valid, ring 0 task 0 invalidated ASSERT_EQ(result.count, 1); - EXPECT_EQ(result.entries[0].entry->producer_task_id, TaskId::make(1, 0)); + EXPECT_EQ(result.entries[0].entry->producer_task_id, simpler::tmr::make_task_id(1, 0)); } // ============================================================================= @@ -362,9 +363,9 @@ TEST_F(TensorMapTest, StaleEntriesNotTruncatedAcrossRings) { TEST_F(TensorMapTest, CleanupRetiredRemovesEntriesForRetiredTasks) { simpler::tmr::Tensor t = make_test_tensor(0x1000, 256); - tmap.insert(t, TaskId::make(0, 0)); - tmap.insert(t, TaskId::make(0, 1)); - tmap.insert(t, TaskId::make(0, 2)); + tmap.insert(t, simpler::tmr::make_task_id(0, 0)); + tmap.insert(t, simpler::tmr::make_task_id(0, 1)); + tmap.insert(t, simpler::tmr::make_task_id(0, 2)); EXPECT_EQ(tmap.valid_count(), 3); // Cleanup tasks [0, 2) on ring 0 @@ -375,13 +376,13 @@ TEST_F(TensorMapTest, CleanupRetiredRemovesEntriesForRetiredTasks) { TestLookupResult result; run_lookup(tmap, t, result); ASSERT_EQ(result.count, 1); - EXPECT_EQ(result.entries[0].entry->producer_task_id, TaskId::make(0, 2)); + EXPECT_EQ(result.entries[0].entry->producer_task_id, simpler::tmr::make_task_id(0, 2)); } TEST_F(TensorMapTest, CleanupRetiredPreservesOtherRings) { simpler::tmr::Tensor t = make_test_tensor(0x1000, 256); - tmap.insert(t, TaskId::make(0, 0)); - tmap.insert(t, TaskId::make(1, 0)); + tmap.insert(t, simpler::tmr::make_task_id(0, 0)); + tmap.insert(t, simpler::tmr::make_task_id(1, 0)); tmap.cleanup_retired(0, 0, 1); @@ -390,12 +391,12 @@ TEST_F(TensorMapTest, CleanupRetiredPreservesOtherRings) { TestLookupResult result; run_lookup(tmap, t, result); ASSERT_EQ(result.count, 1); - EXPECT_EQ(result.entries[0].entry->producer_task_id, TaskId::make(1, 0)); + EXPECT_EQ(result.entries[0].entry->producer_task_id, simpler::tmr::make_task_id(1, 0)); } TEST_F(TensorMapTest, CleanupRetiredFreesEntriesToPool) { simpler::tmr::Tensor t = make_test_tensor(0x1000, 256); - tmap.insert(t, TaskId::make(0, 0)); + tmap.insert(t, simpler::tmr::make_task_id(0, 0)); EXPECT_EQ(tmap.free_num, 0); EXPECT_EQ(tmap.next_entry_idx, 1); @@ -404,7 +405,7 @@ TEST_F(TensorMapTest, CleanupRetiredFreesEntriesToPool) { EXPECT_EQ(tmap.free_num, 1) << "Cleaned entry should be in free list"; // New insert should reuse free entry instead of allocating fresh - tmap.insert(t, TaskId::make(0, 1)); + tmap.insert(t, simpler::tmr::make_task_id(0, 1)); EXPECT_EQ(tmap.free_num, 0); EXPECT_EQ(tmap.next_entry_idx, 1) << "Should reuse freed entry, not allocate new"; } @@ -416,8 +417,8 @@ TEST_F(TensorMapTest, CleanupRetiredFreesEntriesToPool) { TEST_F(TensorMapTest, CleanupRetiredSparesLaterTaskReusingSlot) { simpler::tmr::Tensor t = make_test_tensor(0x1000, 256); // Task 0 and task 0 + WINDOW_SIZE share slot 0 (local_id & (WINDOW_SIZE-1)). - tmap.insert(t, TaskId::make(0, 0)); - tmap.insert(t, TaskId::make(0, WINDOW_SIZE)); + tmap.insert(t, simpler::tmr::make_task_id(0, 0)); + tmap.insert(t, simpler::tmr::make_task_id(0, WINDOW_SIZE)); ASSERT_EQ(tmap.valid_count(), 2); // Retire only task 0. @@ -428,7 +429,7 @@ TEST_F(TensorMapTest, CleanupRetiredSparesLaterTaskReusingSlot) { TestLookupResult result; run_lookup(tmap, t, result); ASSERT_EQ(result.count, 1); - EXPECT_EQ(result.entries[0].entry->producer_task_id, TaskId::make(0, WINDOW_SIZE)); + EXPECT_EQ(result.entries[0].entry->producer_task_id, simpler::tmr::make_task_id(0, WINDOW_SIZE)); } // ============================================================================= @@ -437,9 +438,9 @@ TEST_F(TensorMapTest, CleanupRetiredSparesLaterTaskReusingSlot) { TEST_F(TensorMapTest, MultiRingIndependentLookup) { simpler::tmr::Tensor t = make_test_tensor(0x1000, 256); - tmap.insert(t, TaskId::make(0, 5)); - tmap.insert(t, TaskId::make(1, 3)); - tmap.insert(t, TaskId::make(2, 7)); + tmap.insert(t, simpler::tmr::make_task_id(0, 5)); + tmap.insert(t, simpler::tmr::make_task_id(1, 3)); + tmap.insert(t, simpler::tmr::make_task_id(2, 7)); TestLookupResult result; run_lookup(tmap, t, result); @@ -452,7 +453,7 @@ TEST_F(TensorMapTest, MultiRingIndependentLookup) { TestLookupResult result2; run_lookup(tmap, t, result2); EXPECT_EQ(result2.count, 1); - EXPECT_EQ(result2.entries[0].entry->producer_task_id, TaskId::make(1, 3)); + EXPECT_EQ(result2.entries[0].entry->producer_task_id, simpler::tmr::make_task_id(1, 3)); } // ============================================================================= @@ -463,7 +464,7 @@ TEST_F(TensorMapTest, LookupReturnsAllMatches) { simpler::tmr::Tensor t = make_test_tensor(0x1000, 256); // Insert 20 entries for the same buffer (was capped at 16 before #669) for (int i = 0; i < 20; i++) { - tmap.insert(t, TaskId::make(0, i)); + tmap.insert(t, simpler::tmr::make_task_id(0, i)); } TestLookupResult result; @@ -479,28 +480,28 @@ TEST_F(TensorMapTest, PoolExhaustionAsserts) { // With pool_size=64, inserting 64 entries should work, 65th should fail for (int i = 0; i < POOL_SIZE; i++) { simpler::tmr::Tensor t = make_test_tensor(0x1000 + i * 0x100, 256); - tmap.insert(t, TaskId::make(0, i)); + tmap.insert(t, simpler::tmr::make_task_id(0, i)); } EXPECT_EQ(tmap.next_entry_idx, POOL_SIZE); EXPECT_EQ(tmap.free_num, 0); // 65th insert should trigger always_assert (pool overflow) simpler::tmr::Tensor overflow = make_test_tensor(0x9000, 256); - EXPECT_THROW(tmap.insert(overflow, TaskId::make(0, POOL_SIZE)), std::runtime_error); + EXPECT_THROW(tmap.insert(overflow, simpler::tmr::make_task_id(0, POOL_SIZE)), std::runtime_error); } TEST_F(TensorMapTest, FreeListRecycling) { simpler::tmr::Tensor t = make_test_tensor(0x1000, 256); // Insert and cleanup 10 entries for (int i = 0; i < 10; i++) { - tmap.insert(t, TaskId::make(0, i)); + tmap.insert(t, simpler::tmr::make_task_id(0, i)); } tmap.cleanup_retired(0, 0, 10); EXPECT_EQ(tmap.free_num, 10); // Re-insert should use free list for (int i = 10; i < 20; i++) { - tmap.insert(t, TaskId::make(0, i)); + tmap.insert(t, simpler::tmr::make_task_id(0, i)); } EXPECT_EQ(tmap.free_num, 0); EXPECT_EQ(tmap.next_entry_idx, 10) << "No new pool entries consumed when free list available"; @@ -513,7 +514,7 @@ TEST_F(TensorMapTest, FreeListRecycling) { TEST_F(TensorMapTest, PerTaskEntryListTracksMultipleOutputs) { simpler::tmr::Tensor t1 = make_test_tensor(0x1000, 256); simpler::tmr::Tensor t2 = make_test_tensor(0x2000, 128); - TaskId tid = TaskId::make(0, 5); + TaskId tid = simpler::tmr::make_task_id(0, 5); tmap.insert(t1, tid); tmap.insert(t2, tid); @@ -531,9 +532,9 @@ TEST_F(TensorMapTest, PerTaskEntryListTracksMultipleOutputs) { TEST_F(TensorMapTest, RemoveMiddleEntryPreservesChain) { simpler::tmr::Tensor t = make_test_tensor(0x1000, 256); - TaskId tid0 = TaskId::make(0, 0); - TaskId tid1 = TaskId::make(0, 1); - TaskId tid2 = TaskId::make(0, 2); + TaskId tid0 = simpler::tmr::make_task_id(0, 0); + TaskId tid1 = simpler::tmr::make_task_id(0, 1); + TaskId tid2 = simpler::tmr::make_task_id(0, 2); tmap.insert(t, tid0); tmap.insert(t, tid1); @@ -548,7 +549,7 @@ TEST_F(TensorMapTest, RemoveMiddleEntryPreservesChain) { std::set found_locals; for (int i = 0; i < result.count; i++) { - found_locals.insert(result.entries[i].entry->producer_task_id.local()); + found_locals.insert(simpler::tmr::task_local_id(result.entries[i].entry->producer_task_id)); } EXPECT_TRUE(found_locals.count(0)); EXPECT_TRUE(found_locals.count(2)); @@ -559,9 +560,9 @@ TEST_F(TensorMapTest, RemoveMiddleEntryPreservesChain) { // ============================================================================= TEST(TaskIdTest, MakeAndDecode) { - auto tid = TaskId::make(3, 42); - EXPECT_EQ(tid.ring(), 3); - EXPECT_EQ(tid.local(), 42u); + auto tid = simpler::tmr::make_task_id(3, 42); + EXPECT_EQ(simpler::tmr::task_ring(tid), 3); + EXPECT_EQ(simpler::tmr::task_local_id(tid), 42u); } TEST(TaskIdTest, InvalidSentinel) { @@ -571,21 +572,21 @@ TEST(TaskIdTest, InvalidSentinel) { } TEST(TaskIdTest, Equality) { - auto a = TaskId::make(1, 100); - auto b = TaskId::make(1, 100); - auto c = TaskId::make(2, 100); + auto a = simpler::tmr::make_task_id(1, 100); + auto b = simpler::tmr::make_task_id(1, 100); + auto c = simpler::tmr::make_task_id(2, 100); EXPECT_EQ(a, b); EXPECT_NE(a, c); } TEST(TaskIdTest, RingIdMaxValue) { - auto tid = TaskId::make(255, 0); - EXPECT_EQ(tid.ring(), 255); - EXPECT_EQ(tid.local(), 0u); + auto tid = simpler::tmr::make_task_id(255, 0); + EXPECT_EQ(simpler::tmr::task_ring(tid), 255); + EXPECT_EQ(simpler::tmr::task_local_id(tid), 0u); } TEST(TaskIdTest, LocalIdMaxValue) { - auto tid = TaskId::make(0, UINT32_MAX); - EXPECT_EQ(tid.ring(), 0); - EXPECT_EQ(tid.local(), UINT32_MAX); + auto tid = simpler::tmr::make_task_id(0, UINT32_MAX); + EXPECT_EQ(simpler::tmr::task_ring(tid), 0); + EXPECT_EQ(simpler::tmr::task_local_id(tid), UINT32_MAX); } diff --git a/tests/ut/cpp/a5/test_wiring.cpp b/tests/ut/cpp/a5/test_wiring.cpp index 001a24b541..ad48cbae1a 100644 --- a/tests/ut/cpp/a5/test_wiring.cpp +++ b/tests/ut/cpp/a5/test_wiring.cpp @@ -32,6 +32,7 @@ #include "orchestrator.h" #include "utils/device_arena.h" #include "scheduler/scheduler.h" +#include "tensormap_and_ringbuffer/task_id_encoding.h" // ============================================================================= // Fixture: sets up runtime state with shared memory and provides helpers @@ -124,7 +125,7 @@ TEST(ReclaimHeadMatchTest, ComparesExactRingAndLocalTaskId) { TaskDescriptor descriptor{}; ChipTaskSlotState slot{}; slot.task = &descriptor; - descriptor.task_id = TaskId::make(0, 16); + descriptor.task_id = simpler::tmr::make_task_id(0, 16); EXPECT_FALSE(reclaim_head_matches_open_task(0, 0, &slot)); EXPECT_TRUE(reclaim_head_matches_open_task(16, 0, &slot)); diff --git a/tests/ut/cpp/common/test_hbg_graph_cache.cpp b/tests/ut/cpp/common/test_hbg_graph_cache.cpp index e2ab7a2f9a..fca60c2b83 100644 --- a/tests/ut/cpp/common/test_hbg_graph_cache.cpp +++ b/tests/ut/cpp/common/test_hbg_graph_cache.cpp @@ -26,6 +26,7 @@ #include "graph_execution.h" #include "runtime_status/error_names.h" #include "scheduler/scheduler.h" +#include "host_build_graph/task_id_encoding.h" namespace { @@ -453,7 +454,7 @@ TEST(GraphExecutionReplay, ResubmissionRebuildsFromDefinition) { ASSERT_NE(execution, nullptr); TaskDescriptor outer_task{}; - outer_task.task_id = TaskId::make(1, 7); + outer_task.task_id = simpler::hbg::make_ring_task(7); outer_task.packed_buffer_base = heap.base(); outer_task.packed_buffer_end = heap.end(); ChipTaskSlotState outer_slot{}; @@ -477,7 +478,7 @@ TEST(GraphExecutionReplay, ResubmissionRebuildsFromDefinition) { graph_execution_mark_completed(*execution); execution->retired_nodes.store(2, std::memory_order_release); - outer_task.task_id = TaskId::make(1, 8); + outer_task.task_id = simpler::hbg::make_ring_task(8); // Poison every field the rebuild is responsible for restoring. A replay that // preserved any of them would leave the poison observable. @@ -501,7 +502,7 @@ TEST(GraphExecutionReplay, ResubmissionRebuildsFromDefinition) { EXPECT_EQ(node.payload.scalar_data()[0], 99U); EXPECT_EQ(execution->node_at(1).payload.scalar_data()[0], 18U); EXPECT_EQ(node.payload.tensor_data()[0].version, 0); - EXPECT_EQ(node.task.task_id, TaskId::make(1, (8U << 10U))); + EXPECT_EQ(node.task.task_id, simpler::hbg::make_graph_node(/*outer_local_id=*/8, /*node_index=*/0)); EXPECT_EQ(node.task.packed_buffer_base, heap.base()); EXPECT_EQ(node.payload.tensor_data()[0].buffer.addr, reinterpret_cast(second_boundary.data())); EXPECT_EQ(execution->node_at(1).payload.tensor_data()[0].buffer.addr, reinterpret_cast(heap.base() + 16)); @@ -529,7 +530,7 @@ TEST(GraphExecutionReplay, MaterializesBoundaryScalarPoolWiderThanTaskPayload) { ASSERT_NE(execution, nullptr); TaskDescriptor outer_task{}; - outer_task.task_id = TaskId::make(1, 7); + outer_task.task_id = simpler::hbg::make_ring_task(7); outer_task.packed_buffer_base = heap.base(); outer_task.packed_buffer_end = heap.end(); ChipTaskSlotState outer_slot{}; @@ -709,7 +710,7 @@ TEST(GraphExecutionProgress, InternalNodeResolutionIsNotAHostCompletion) { AsyncWaitList wait_list{}; wait_list.entries[0].slot_state = &node.slot; - wait_list.entries[0].task_token = TaskId::make(0, 1); + wait_list.entries[0].task_token = simpler::hbg::make_ring_task(1); wait_list.entries[0].normal_done = true; wait_list.count = 1; @@ -736,7 +737,7 @@ TEST(GraphExecutionMaterialize, DirtyStorageYieldsValidExecution) { ASSERT_NE(execution, nullptr); TaskDescriptor outer_task{}; - outer_task.task_id = TaskId::make(1, 5); + outer_task.task_id = simpler::hbg::make_ring_task(5); outer_task.packed_buffer_base = heap.base(); outer_task.packed_buffer_end = heap.end(); ChipTaskSlotState outer_slot{}; diff --git a/tests/ut/cpp/common/test_hbg_graph_submit_failure.cpp b/tests/ut/cpp/common/test_hbg_graph_submit_failure.cpp index 652420d522..cedb0caf16 100644 --- a/tests/ut/cpp/common/test_hbg_graph_submit_failure.cpp +++ b/tests/ut/cpp/common/test_hbg_graph_submit_failure.cpp @@ -28,6 +28,7 @@ #include "shared_memory.h" #include "task_interface/assert_compat.h" #include "utils/device_arena.h" +#include "host_build_graph/task_id_encoding.h" class HbgGraphSubmitFailureTest : public ::testing::Test { protected: @@ -107,7 +108,7 @@ TEST_F(HbgGraphSubmitFailureTest, InFlightGraphInvocationsReserveHeapOnlyAtCommi EXPECT_FALSE(second.recording); EXPECT_FALSE(second.execute_block); ASSERT_TRUE(second.task_id.is_valid()); - EXPECT_EQ(second.task_id.local(), first.task_id.local() + 1); + EXPECT_EQ(simpler::hbg::task_local_id(second.task_id), simpler::hbg::task_local_id(first.task_id) + 1); EXPECT_EQ(orch.task_allocator.heap_top(), 0u); EXPECT_EQ(graph_host_upload_count(*graph_state), 2u); @@ -238,8 +239,8 @@ TEST_F(HbgGraphSubmitFailureTest, WorkerRecordsWhileMainThreadSubmitsSameHashShe EXPECT_FALSE(third.execute_block); ASSERT_TRUE(second.task_id.is_valid()); ASSERT_TRUE(third.task_id.is_valid()); - EXPECT_EQ(second.task_id.local(), first.task_id.local() + 1); - EXPECT_EQ(third.task_id.local(), first.task_id.local() + 2); + EXPECT_EQ(simpler::hbg::task_local_id(second.task_id), simpler::hbg::task_local_id(first.task_id) + 1); + EXPECT_EQ(simpler::hbg::task_local_id(third.task_id), simpler::hbg::task_local_id(first.task_id) + 2); EXPECT_EQ(orch.task_allocator.heap_top(), 0u) << "no shell may take heap before commit"; orch.graph_commit(); @@ -435,7 +436,7 @@ TEST_F(HbgGraphSubmitFailureTest, CachedGraphUsesFinalTaskWindowSlot) { EXPECT_FALSE(replay.execute_block); ASSERT_TRUE(replay.task_id.is_valid()); - EXPECT_EQ(replay.task_id.local(), static_cast(allocator.capacity() - 1)); + EXPECT_EQ(simpler::hbg::task_local_id(replay.task_id), static_cast(allocator.capacity() - 1)); EXPECT_EQ(allocator.active_count(), allocator.capacity()); EXPECT_EQ(allocator.active_count(), allocator.capacity()); EXPECT_EQ(sm_handle->header->orch_error_code.load(std::memory_order_acquire), SIMPLER_ERROR_NONE); @@ -743,7 +744,7 @@ TEST_F(HbgGraphSubmitFailureTest, AnOrdinaryAllocationInterleavesWithADeferredSh EXPECT_GT(orch.task_allocator.heap_top(), heap_after_ordinary) << "the shell's block sits above the ordinary task's, not before it"; SharedMemoryTaskHeader &tasks = sm_handle->header->tasks; - const int32_t shell_slot = static_cast(graph.task_id.local()); + const int32_t shell_slot = static_cast(simpler::hbg::task_local_id(graph.task_id)); const TaskDescriptor *shell = tasks.slot_states[shell_slot].task.get(); ASSERT_NE(shell, nullptr); ASSERT_NE(shell->packed_buffer_base, nullptr); @@ -894,7 +895,7 @@ TEST_F(HbgGraphSubmitFailureTest, AHiddenAllocTaskLeavesItsDispatchPredicateDefi ASSERT_TRUE(outputs.task_id().is_valid()); ASSERT_FALSE(orch.fatal); - const uint64_t slot = outputs.task_id().local(); + const uint64_t slot = simpler::hbg::task_local_id(outputs.task_id()); ASSERT_LT(slot, static_cast(kPoisonedSlots)) << "the submitted slot must be one this test poisoned"; EXPECT_EQ(payloads[slot].predicate.op, PredicateOp::NONE); EXPECT_EQ(payloads[slot].predicate.addr, 0u); diff --git a/tests/ut/cpp/common/test_hbg_mailbox_init.cpp b/tests/ut/cpp/common/test_hbg_mailbox_init.cpp index 19b778ac7b..5225b685eb 100644 --- a/tests/ut/cpp/common/test_hbg_mailbox_init.cpp +++ b/tests/ut/cpp/common/test_hbg_mailbox_init.cpp @@ -27,7 +27,10 @@ namespace { -TaskId make_token(uint32_t local) { return TaskId::make(/*ring=*/0, local); } +// The mailbox stores a token and compares it for identity; it never decodes one, +// and the encoding belongs to whichever runtime minted it, so any distinct 64-bit +// value serves here. +TaskId make_token(uint32_t local) { return TaskId{local}; } // A mailbox on memory holding a prior generation's bytes, which is what the // pooled arena hands the AICPU: the region is never uploaded, so nothing zeroes diff --git a/tests/ut/cpp/common/test_hbg_slot_claim.cpp b/tests/ut/cpp/common/test_hbg_slot_claim.cpp index a90deec3fc..2ab93a423b 100644 --- a/tests/ut/cpp/common/test_hbg_slot_claim.cpp +++ b/tests/ut/cpp/common/test_hbg_slot_claim.cpp @@ -27,6 +27,7 @@ #include "orchestrator.h" #include "shared_memory.h" #include "utils/device_arena.h" +#include "host_build_graph/task_id_encoding.h" class HbgSlotClaimTest : public ::testing::Test { protected: @@ -177,7 +178,7 @@ TEST_F(HbgSlotClaimTest, CachedGraphReplayClaimsAPoisonedSlot) { poison_slot(1); const GraphScopeResult replay = orch.graph_begin(0x51ADC1A2, boundary_args, 0x1736); ASSERT_TRUE(replay.task_id.is_valid()); - ASSERT_EQ(replay.task_id.local(), 1u); + ASSERT_EQ(simpler::hbg::task_local_id(replay.task_id), 1u); expect_slot_pristine(1); } diff --git a/tests/ut/cpp/common/test_hbg_sm_compaction.cpp b/tests/ut/cpp/common/test_hbg_sm_compaction.cpp index b47794ecae..bc4447fe6c 100644 --- a/tests/ut/cpp/common/test_hbg_sm_compaction.cpp +++ b/tests/ut/cpp/common/test_hbg_sm_compaction.cpp @@ -26,6 +26,7 @@ #include "graph_execution.h" #include "shared_memory.h" +#include "host_build_graph/task_id_encoding.h" namespace { @@ -86,7 +87,7 @@ class Mirror { // orchestrator's bump cursors hand them out, and gets content that identifies // the slot so the compaction can be checked element by element. for (uint64_t i = 0; i < SUBMITTED; ++i) { - descriptors()[i].task_id = TaskId::make(0, static_cast(i)); + descriptors()[i].task_id = simpler::hbg::make_ring_task(static_cast(i)); payloads()[i].tensor_count = TENSORS_PER_TASK; payloads()[i].scalar_count = SCALARS_PER_TASK; payloads()[i].fanin_count = FANIN_PER_TASK; @@ -109,7 +110,7 @@ class Mirror { completion_flags()[i].store(static_cast(i & 1), std::memory_order_relaxed); } // A slot past the submitted prefix, to prove it does not travel. - descriptors()[SUBMITTED].task_id = TaskId::make(0, 0xBEEF); + descriptors()[SUBMITTED].task_id = simpler::hbg::make_ring_task(0xBEEF); } // Overwrite the three fields that can hold a graph-heap address with ones out @@ -223,7 +224,7 @@ TEST(HbgSmCompaction, CarriesEveryLiveSlotsContent) { Compacted compacted(mirror); for (uint64_t i = 0; i < SUBMITTED; ++i) { - EXPECT_EQ(compacted.descriptors()[i].task_id.local(), i) << "slot " << i; + EXPECT_EQ(simpler::hbg::task_local_id(compacted.descriptors()[i].task_id), i) << "slot " << i; EXPECT_EQ(compacted.payload_at(i)->tensor_count, TENSORS_PER_TASK) << "slot " << i; EXPECT_EQ(compacted.payload_at(i)->tensor_data()[0].buffer.addr, 0x1000 + i * 0x10) << "slot " << i; EXPECT_EQ(compacted.slot_states()[i].last_consumer_local_id, static_cast(i)) << "slot " << i; diff --git a/tests/ut/cpp/common/test_scope_deadlock_detection.cpp b/tests/ut/cpp/common/test_scope_deadlock_detection.cpp index 90b675b2af..6ec63e1c5a 100644 --- a/tests/ut/cpp/common/test_scope_deadlock_detection.cpp +++ b/tests/ut/cpp/common/test_scope_deadlock_detection.cpp @@ -15,6 +15,7 @@ #include #include "ring_buffer.h" +#include "tensormap_and_ringbuffer/task_id_encoding.h" // CMake compiles this source against both the a2a3 and a5 runtime objects. namespace { @@ -25,7 +26,7 @@ constexpr int32_t POOL_CAPACITY = 8; void make_head_match_old_structural_predicate( ChipTaskSlotState &head, TaskDescriptor &descriptor, uint8_t ring_id, uint32_t local_task_id ) { - descriptor.task_id = TaskId::make(ring_id, local_task_id); + descriptor.task_id = simpler::tmr::make_task_id(ring_id, local_task_id); head.task = &descriptor; head.task_state.store(CHIP_TASK_COMPLETED, std::memory_order_release); head.fanout_count = FANOUT_SCOPE_BIT;