DO NOT MERGE: #34154: test(java25): runnable demos for carrier scheduling and compact object headers

[fabrizzio-dotCMS]

What

Seven runnable main demos under dotCMS/src/test/java/com/dotcms/jdk/, built for the Java 21→25 tech talk (#34154). They are the evidence half of the talk: each one turns a claim about the JVM into something an audience can watch happen, on this codebase's own classes and its own incidents — with one exception, §6, which is teaching material for the same topic rather than evidence.

They cover three of the talk's topics: carrier scheduling and pinning (Loom), object layout (JEP 519), and the stream APIs (JEP 485 plus custom collectors).

No production code is touched. Nothing new runs in CI beyond compiling — the classes are named *Demo, so surefire skips them.

1. VirtualThreadCarrierTimelineDemo — you can watch the carriers

One row per carrier, one column per 100 ms tick, showing which of six virtual threads was mounted at that instant.

mode=cpu    (never releases)          mode=sleep    (releases on every sleep)
carrier-1 | 111111111111111111        carrier-1 | 1.4.2.6.3.1.5.2.
carrier-2 | 222222222222222222        carrier-2 | 2.5.3.1.4.6.2.3.
2 of 6 tasks ever got a carrier       6 of 6 tasks got a carrier
canary: NEVER RAN                     canary: ran in 1 ms

Two things make it possible without JFR or an agent:

• The carrier is observable from toString(). VirtualThread.toString() appends @ForkJoinPool-1-worker-N only while mounted (.../runnable@ForkJoinPool-1-worker-1 vs .../timed_waiting). The poller has to be a platform thread — run it on a virtual one with -Dmonitor=virtual and the tool freezes mid-drawing, which is the same failure everything else is suffering.
• Blocking file I/O is reproducible without NFS. A FIFO with no writer blocks in read() indefinitely. -Dmode=file comes out identical to a CPU hog — 2 of 6 tasks mounted, 0 hand-offs, the canary never runs in 4 s — while -Dmode=socket frees both carriers immediately.

That last contrast is the point, and it is the shape of #37038 (fixed in #37041): JEP 491 stopped synchronized from pinning; it did not make file I/O unmount. Confusing the two killed content indexing until the pod was restarted.

2. VirtualThreadCarrierStarvationDemo — the blast radius

Ten uninterruptible tasks, submitted either to a virtual-thread executor or a fixed pool of two platform threads. Both start exactly 2 of 10. The difference is what happens to everything else:

mode=vt        tasks that STARTED: 2 of 10   (no '>>>' — the innocent virtual thread was starved too)
mode=platform  tasks that STARTED: 2 of 10   >>> the INNOCENT virtual thread RAN

With platform threads the starvation is contained in your pool under your bound. With virtual threads the scarce resource is the carrier pool — availableProcessors() wide, global to the JVM — so an unrelated task that never touched that executor starves as well.

3. VirtualThreadYieldVsParkDemo

The same two carriers, separating yielding from parking, for the part of the talk that explains why the scheduler is cooperative and nothing reclaims a carrier by force.

4. CompactObjectHeadersDemo — measuring JEP 519 on our own records

-XX:+UseCompactObjectHeaders is read once at JVM startup, so no test can compare both settings — which is exactly why CacheSizingUtilTest was rewritten to assert layout-independent invariants when the flag went in (78ab65083d, #35931, which had to delete its hardcoded byte counts in the same commit). This demo therefore relaunches itself in two child JVMs and prints both columns:

shape                   12-byte hdr     8-byte hdr      saved
java.lang.Object              16 B           8 B        8 B   header only, no fields
ContentSearchHit              40 B          32 B        8 B   real record, 5 refs + 1 float
SiteSearchHit                 40 B          40 B        0 B   real record, 6 refs + 1 float
realistic hit               3176 B        2992 B      184 B   + its own 20-field _source map

Read the two hit rows together: SiteSearchHit carries one more reference than ContentSearchHit, and with 12-byte headers that field costs 0 bytes — 12+24=36 rounds up to 40, so it lands in padding already paid for. With 8-byte headers the short shape lands exactly on 32 and the same field now costs 8. Compact headers do not simply make objects 4 bytes smaller; they change which refactors are worth doing. The SearchHit split (#36899) returned nothing before the flag and returns 8 bytes per hit after it.

The last row keeps that honest, and is the row to end on: against a hit carrying its own _source, the split is worth 0.25% while the flag itself is worth 5.8% — because the flag shrinks every object in the graph, the map nodes and strings and char arrays, not the record you were looking at.

How it measures — two independent instruments, neither a sizing API reporting on itself:

1. ThreadMXBean.getCurrentThreadAllocatedBytes() around the fill loop (the headline). The JVM counts the bytes it handed out, so it is exact and does not wait on the collector. Verified reproducible: identical output at 200k, 500k and 1M instances per shape.
2. Used-heap delta as a cross-check, with the backing array allocated before the baseline so its own reference slots are excluded. Printed unrounded next to the counter, noise and all — it answers the fair question the counter cannot, that the bytes are not merely un-allocated but genuinely not resident.

An earlier revision used the heap delta alone and is why both are here now: it reported the realistic hit row as 585 B at 25k instances and 3160 B at 50k. The allocation counter does not have that failure mode.

It runs two ways and gives the same numbers: standalone with java <path>.java (falling back to local twin records, which it says in the output), or with -cp dotCMS/target/classes against the real ContentSearchHit / SiteSearchHit.

5. StreamGatherersDemo — the extension point the middle of a stream never had

A Stream's intermediate vocabulary closed in Java 8. Collectors lets you write your own terminal operation; nothing let you write your own intermediate one. That matters because the existing ones are amnesic — map sees one element and produces one, filter sees one and decides keep-or-drop — so anything needing state between elements (batching, sliding windows, running totals, collapsing runs, stopping on a condition) had to leave the stream entirely.

windowFixed(3)        [[idx1, idx2, idx3], [idx4, idx5, idx6], [idx7]]
                                                               ^^^^^^^ the short tail, emitted for free
scan (running total)  [10, 30, 60, 100]     <- one output per input
fold (single value)   [100]                 <- one output in total
collapsingRuns()      [OK, ERROR, OK]       <- from [OK, OK, OK, ERROR, ERROR, OK]
untilError()          [OK, OK, OK]          <- returning false ends the stream

Two things this lands on, both already written by hand here:

• OSIndexAPIImpl.getIndexAlias (~846-855) does stream().map(...).collect(toList()) and immediately Lists.partition(physicalNames, ALIAS_LOOKUP_BATCH_SIZE) — the whole list exists solely so it can be cut up. Lists.partition appears in a dozen more places.
• PopulateContentletAsJSONUtil (~466-471 and ~332-334) splits the batch flush across two places that must agree: the accumulator flushes at MAX_BATCH_SIZE, and the caller must remember if (!paramsInsert.isEmpty()) doInsertBatch(...) for the remainder. Duplicated again for updates. That lone [idx7] above is the flush you cannot forget.

mapConcurrent is where this topic meets Loom — each element on a virtual thread, under a bound you choose, encounter order preserved:

sequential             1246 ms
mapConcurrent(4)        347 ms
same order as input   true
ran on virtual threads true

That is the JDK's answer to "parallelise the I/O in this stream", which parallelStream() never did well: it borrows the common ForkJoinPool, sized for CPU work and shared process-wide.

6. SimplestGathererDidactic — the same API taken apart, one step at a time

StreamGatherersDemo above shows what gatherers are for. This one is its teaching companion and shows how to write one, starting from a gatherer that does nothing at all:

static Gatherer<String, Void, String> passThrough() {
    return Gatherer.of((state, element, downstream) -> {
        downstream.push(element);
        return true;
    });
}

That is already the whole API. The four stateless methods are the same gatherer one line apart, which makes the only real decision visible — how many times you call push:

source         [ana, beto, caro]
passThrough    [ana, beto, caro]                       push once           -> like map's identity
onlyLong       [beto, caro]                            push zero or once   -> behaves like filter
upperCase      [ANA, BETO, CARO]                       push something else -> behaves like map
twice          [ana, ana, beto, beto, caro, caro]      push more than once -> behaves like flatMap

So a gatherer subsumes the operations we already had. What makes it new is the first parameter, state, which the four above ignore — and the three that follow do not:

numbered       [1. ana, 2. beto, 3. caro]                        <- output depends on how many came before
whenChanged    [ok, error, ok]            from [ok,ok,ok,error,error,ok]
occurrence     [ana (1), beto (1), ana (2), caro (1), ana (3), beto (2)]

numbered() cannot be written with map: the result depends on how many elements went past first. whenChanged() cannot be written with filter: a predicate sees one element and nothing about its neighbours. The javadoc records the three rules that go with state — it must be a mutable object, never an int (a lambda cannot capture a mutable local); Gatherer.ofSequential is what supplies it; and the integrator should return downstream.push(...) rather than return true, so a downstream that has stopped asking propagates back up.

7. CustomCollectorDemo — when a custom Collector is worth writing

Anatomy first (supplier / accumulator / combiner / finisher, and why A and R differ), then two traps worth seeing once:

• IDENTITY_FINISH silently skips the finisher. Declaring it alongside a finisher does not fail — the stream just never calls it, so a method promising an immutable list hands back a plain mutable ArrayList. Verified: the demo mutates the result and nothing complains.
• Copy and view are not interchangeable. List.copyOf copies and rejects null; Collections.unmodifiableList wraps and accepts it. Stream.toList() is immutable and null-tolerant while Collectors.toUnmodifiableList() throws NPE — a difference that surfaces wherever an unset content field arrives as null.

The one that is worth writing merges into a map and reports which keys collided — the answer Collectors.toMap cannot give, since it can only throw (losing every other collision) or take a merge function and discard the loser in silence:

merged                {title=Home v2, body=..., author=ana}
conflicts             [title]
in parallel           {...} conflicts=[title]   <- the combiner detects them too

  toMap(k, v)         IllegalStateException   <- and it cannot say how many others clashed
  toMap(k, v, merge)  {author=ana, body=..., title=Home v2}   <- did anything clash? no way to know

All four pieces do real work there, including the combiner, which surfaces collisions neither parallel half saw alone. The javadoc also records the counterexample: the three-way IndexPolicy split in ContentletIndexAPIImpl.addContentToIndex uses CollectionsUtils.partition with positional get(0)/get(1)/get(2), and wants Collectors.groupingBy — not a custom collector.

Testing

All seven were run end to end on JDK 25.0.2 and the module's test sources compile with the real build (./mvnw test-compile -pl :dotcms-core --am -Dmaven.build.cache.enabled=false, exit 0; every .class file lands in target/test-classes).

java dotCMS/src/test/java/com/dotcms/jdk/VirtualThreadCarrierTimelineDemo.java     # -Dmode=file|socket|sleep|cpu
java -Dmode=vt dotCMS/src/test/java/com/dotcms/jdk/VirtualThreadCarrierStarvationDemo.java
java -Dmode=busy dotCMS/src/test/java/com/dotcms/jdk/VirtualThreadYieldVsParkDemo.java
java dotCMS/src/test/java/com/dotcms/jdk/CompactObjectHeadersDemo.java             # -Dn=1000000
java dotCMS/src/test/java/com/dotcms/jdk/StreamGatherersDemo.java
java dotCMS/src/test/java/com/dotcms/jdk/SimplestGathererDidactic.java
java dotCMS/src/test/java/com/dotcms/jdk/CustomCollectorDemo.java

Notes for the reviewer

• System.out is deliberate and is called out in each javadoc. The console output is the artifact being shown to an audience, and the demos must run with java Foo.java on a bare JDK with no dotCMS on the classpath, where Logger does not exist. The Logger-only Critical Rule targets production code.
• Not JUnit tests, on purpose. The CPU tasks ignore interrupts and could never be reclaimed inside a shared surefire JVM, and the compact-headers comparison needs two JVMs. They are main demos compiled by the real build so they cannot silently rot, and named *Demo so surefire skips them. They register in no suite by design.
• Nothing here is meant to merge as product code — this is talk material kept in-tree so the compiler keeps it honest.

🤖 Generated with Claude Code

Related to: #34154

This PR fixes: #34154

[claude]

Claude finished @fabrizzio-dotCMS's task in 1m 59s —— View job

---

Review: PR #37117 — runnable JDK demos

Read all seven files. These are main demos under src/test/, deliberately not JUnit, with System.out and the two-JVM relaunch called out and justified in each javadoc — none of that is a finding (the Logger/Config rules target production code). Two things are worth fixing before the talk, both about correctness of what the demo shows.

New Issues

• 🟡 Medium: dotCMS/src/test/java/com/dotcms/jdk/VirtualThreadCarrierTimelineDemo.java:137-140 — In compare mode the socket and file phases are commented out:

  runPhase("sleep");
  //runPhase("socket");
  //runPhase("file");
  runPhase("cpu");

  But the class javadoc (line 76: "compare (default) all four phases: sleep, socket, file, cpu"), the surrounding comment (lines 133-136, still describing why the order is sleep→socket→file→cpu), and the PR description all say the default runs the full sequence. The file-vs-socket contrast is the centerpiece of the Reindex mapping guard leaks semaphore permits, permanently killing content indexing (regression in 26.07.13-01) #37038 lesson ("-Dmode=file comes out identical to a CPU hog") — with these lines commented, the default run no longer demonstrates it (still reachable via -Dmode=file / -Dmode=socket). Looks like leftover debugging. Either restore the two calls or update the javadoc/comment/PR text to match what compare actually does. Fix this →

• 🟡 Medium: dotCMS/src/test/java/com/dotcms/jdk/CompactObjectHeadersDemo.java:379-380 — splitSaving mixes the two JVM columns:

  final long splitSaving = classic.getOrDefault("SiteSearchHit", 0L)
          - compact.getOrDefault("ContentSearchHit", 0L);

  It subtracts a compact-header size from a classic-header size. The printed sentence ("splitting the two record shapes apart returns N B") is about the saving under the flag, which should be compact SiteSearchHit − compact ContentSearchHit (= 40−32 = 8). The current expression yields 8 only because SiteSearchHit measures 40 in both columns for this data; if compact headers ever shrank SiteSearchHit, the number would be wrong. For talk material where these figures are the point, compute both operands from the compact map (or explicitly from classic for the "before the flag" side you're contrasting against). Fix this →

Everything else checks out: the Acc.put collision detection uses containsKey before put (correctly counting a prior-null value as a clash), the parallel mergingReportingConflicts path surfaces the collision through either the accumulator or the combiner regardless of how the 4-element stream splits, the IDENTITY_FINISH trap demo relies on genuine JDK behavior, and the gatherer short-circuit (return false) / ofSequential state examples are accurate. No security, transaction, or replay concerns — no production code, no DB, no I/O beyond the demos' own sockets/FIFO which are cleaned up.

• Read all 7 demo files
• Analyze for bugs, conventions, correctness
• Post review
  · branch issue-34154-loom-demos