Supabase switchover readiness: tests, parity fixes, mixed-mode auth, CI fixture

.github/prompts/compare-master.prompt.md

@@ -0,0 +1,7 @@
+We are working on the branch "Migration". We are doing a big refactoring, but we should be getting the same results as we had on Master.
+
+The code from Master is available via git worktree, found at C:\dev\BloomLibrary.worktrees\master\.
+
+I have started the server from master on http://localhost:5179/, so you can open the page there and compare html, screenshots, console messages, and network requests using chrome-devtools-mcp.
+
+Now, run your own copy of vite dev server on this current ("Migration") branch, and compare the following page. Compare the page output, the console, if necessary the network log. Find out what we need to fix in our refactoring so that we get the same results as Master.

.github/workflows/supabase-integration.yml

@@ -0,0 +1,79 @@
+name: Supabase Integration Tests
+
+# Spins up a throwaway local Supabase stack (see supabase/config.toml), loads a
+# scrubbed SQL fixture (src/data-layer/test/fixtures/supabase-fixture.sql), and
+# runs the gated RUN_SUPABASE_TESTS read-path integration suites against it.
+# Nothing here touches any real/production backend.
+
+on:
+    push:
+        branches:
+            - master
+            - release
+            - embed
+            - SupabaseMigration
+    pull_request:
+    workflow_dispatch:
+
+jobs:
+    supabase-integration:
+        runs-on: ubuntu-latest
+
+        steps:
+            - name: Checkout repository
+              uses: actions/checkout@v4
+
+            # Same Node+pnpm toolchain setup as build-and-deploy.yml (the repo
+            # switched from yarn to pnpm via corepack's packageManager field).
+            - name: Install Vite+ (manages Node.js and pnpm)
+              uses: voidzero-dev/setup-vp@2dec1e33f4ab2c6d5bce1b0c4607961bb1a3f7a1 # v1.12.0
+              with:
+                  version: 0.1.24
+                  node-version-file: .node-version
+                  cache: true
+
+            - name: Install Supabase CLI
+              uses: supabase/setup-cli@v1
+              with:
+                  version: latest
+
+            - name: Start Supabase (db, rest, api, auth only)
+              # Unneeded services are disabled in supabase/config.toml, so a plain
+              # start brings up just Postgres, PostgREST, Kong, and auth.
+              run: supabase start
+
+            - name: Load scrubbed fixture and reload PostgREST
+              env:
+                  DB_URL: postgresql://postgres:postgres@127.0.0.1:44322/postgres
+              run: |
+                  psql "$DB_URL" -v ON_ERROR_STOP=1 -f src/data-layer/test/fixtures/supabase-fixture.sql
+                  # Nudge PostgREST to pick up the freshly created tables/RPC.
+                  psql "$DB_URL" -c "NOTIFY pgrst, 'reload schema';"
+
+            - name: Wait for PostgREST to serve the loaded data
+              env:
+                  # Standard supabase local demo anon key (same one baked into
+                  # SupabaseConnection.ts); safe to hard-code, it is not a secret.
+                  ANON_KEY: eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJzdXBhYmFzZS1kZW1vIiwicm9sZSI6ImFub24iLCJleHAiOjE5ODM4MTI5OTZ9.CRXP1A7WOeoJeXxjNni43kdQwgnWNReilDMblYTn_I0
+              run: |
+                  for i in $(seq 1 30); do
+                      body=$(curl -s "http://127.0.0.1:44321/rest/v1/books?select=id&limit=1" \
+                          -H "apikey: $ANON_KEY" -H "Authorization: Bearer $ANON_KEY" || true)
+                      if echo "$body" | grep -q '"id"'; then
+                          echo "PostgREST is serving books."
+                          exit 0
+                      fi
+                      echo "Waiting for PostgREST schema reload ($i/30)... last response: $body"
+                      sleep 2
+                  done
+                  echo "PostgREST never served the fixture data" >&2
+                  exit 1
+
+            - name: Install dependencies
+              run: vp install --frozen-lockfile
+
+            - name: Run Supabase read integration tests
+              run: >
+                  RUN_SUPABASE_TESTS=true npx vitest run
+                  src/data-layer/test/SupabaseRead.integration.test.ts
+                  src/data-layer/test/SupabaseRead.more.integration.test.ts

README.md

@@ -23,6 +23,79 @@ To run the unit tests, do `vp run test`
 ### Pointing to Prod, Dev, or Local
 BloomLibrary talks to a [Parse](https://parseplatform.org/) server to get the list of books. This can be the Production Parse server, or the Development Parse server, or a locally hosted Parse server. You can manually change which server it talks to if needed. See `ParseServerConnection.ts`.
 
+## Supabase (local database)
+
+We are migrating the database from Parse Server to Supabase (Postgres). On the
+`SupabaseMigration` branch, the anonymous read path (grids, search, book detail,
+language/topic menus) can run against a **local** Supabase filled with a sample
+of real production books. Auth/writes still require Parse.
+
+### One-time setup (Windows)
+
+1. Install [Podman](https://podman.io/) and create its VM
+   (Docker Desktop also works, but Podman is our documented path):
+
+   ```powershell
+   winget install RedHat.Podman
+   podman machine init
+   podman machine set --rootful   # rootless port forwarding doesn't reach the Windows host
+   podman machine start
+   ```
+
+2. Clone [bloom-core-supabase](https://github.com/BloomBooks/bloom-core-supabase)
+   (e.g. to `D:\bloom-core-supabase`) and in it run `pnpm install`.
+
+3. Start the local Supabase stack and create the schema. If Docker Desktop is
+   also installed, point the CLI at Podman's pipe first:
+
+   ```powershell
+   $env:DOCKER_HOST = "npipe:////./pipe/podman-machine-default"
+   pnpm exec supabase start -x logflare,vector
+   pnpm exec supabase db reset      # applies migrations + seed
+   ```
+
+   Note: the local ports are 44321 (API), 44322 (DB), 44323 (Studio) — not the
+   Supabase defaults; see that repo's README for why (Windows excluded port
+   ranges) and for other gotchas.
+
+4. Import ~100 real books from production Parse (idempotent; re-run to refresh):
+
+   ```powershell
+   pnpm --filter @bloom/sync-tool import-sample
+   ```
+
+### Running blorg against it (each session)
+
+```powershell
+podman machine start                             # after a reboot
+cd D:\bloom-core-supabase
+$env:DOCKER_HOST = "npipe:////./pipe/podman-machine-default"
+pnpm exec supabase start -x logflare,vector
+```
+
+then here:
+
+```
+VITE_DATA_LAYER_IMPL=supabase yarn dev
+```
+
+Parse remains the default when the env var is unset. Collections still come
+from Contentful, stats from api.bloomlibrary.org, thumbnails/artifacts from
+production S3 — only the book database is local.
+
+### Supabase integration tests
+
+With the local stack running and sample data imported:
+
+```
+RUN_SUPABASE_TESTS=true yarn vitest run src/data-layer/test/SupabaseRead.integration.test.ts
+```
+
+These assert against live query behavior (including that filters actually
+constrain results — pure unit tests with a mocked client can't catch
+PostgREST serialization or dropped-filter bugs). Implementation notes and
+known v0 divergences: `src/data-layer/implementations/supabase/README.md`.
+
 ## Localization
 
 See details in `src/translations/README.md`.

SWITCHOVER-READINESS.md

@@ -0,0 +1,63 @@
+# Supabase switchover readiness — blorg
+
+Goal: prove blorg is 100% ready to flip `VITE_DATA_LAYER_IMPL=supabase` for the
+**anonymous browsing** scope (the declared scope of the current Supabase data
+layer — see `src/data-layer/implementations/supabase/README.md`). Auth, writes,
+and moderation remain on Parse until the backend's auth milestone.
+
+Derived from a 3-way gap analysis (data-layer parity matrix, whole-repo Parse
+sweep, bloom-core-supabase backend inventory) on 2026-07-18.
+
+## A. Read-path parity gaps (blorg + backend)
+
+| # | Item | Status | Where |
+|---|------|--------|-------|
+| A1 | Search relevance ranking (Parse $text/$score vs ilike-AND + newest-first) | ⚠ deferred by decision 2026-07-18 — accepted degraded for switchover test period; revisit after data-migration milestone | both repos |
+| A2 | Non-canonical `topic:` filters silently return nothing (Parse regex-ORs) | ✅ closed — `match_topic_tags` RPC (bloom-core-supabase migration 20260718000000) resolves non-canonical topics to tag names; SupabaseBookQueryBuilder requires "any of" them (overlaps), mirroring Parse's anchored-single / substring-multi case-insensitive semantics; unit + integration tests added | both repos |
+| A3 | `tags.category` column missing → `TagModel.category` always undefined | ✅ accepted — verified zero UI consumers of TagModel.category in anon scope (only Contentful page fields use `.category`) | — |
+| A4 | `sendConcernEmail` throws under Supabase ("Report this book" is anon-reachable) | ✅ send-concern-email edge function built (bloom-core-supabase branch `send-concern-email`, unmerged); Supabase client impl wired to it; mixed mode (D1) meanwhile routes the live path via Parse | both repos |
+| A5 | `anyOfThese`/`derivedFrom` union IDs client-side then `.in("id",…)` — scale risk | ✅ verified at 699 local books: broad search ~150ms, anyOfThese ~120ms (75f566b); re-check after full data sync | blorg |
+| A6 | Wildcard tag inside any-of list fails closed | ⚠ accepted (no known caller); covered by unit tests where reachable | blorg |
+
+## B. Test safety net
+
+| # | Item | Status |
+|---|------|--------|
+| B1 | Unit tests for `SupabaseBookQueryBuilder` (~646 lines, riskiest layer) | ✅ 60 tests, CI-safe (30a03fa) |
+| B2 | Unit tests for `SupabaseBookMapper` | ✅ 9 tests (30a03fa) |
+| B3 | Integration suite breadth (real collection filter shapes, guards for A6) | ✅ 21 gated tests across 2 files (75f566b); known gap: derivedFrom publisher-negation branch lacks real-data coverage in the sample |
+| B4 | Contract tests runnable in CI (local stack in GH Actions; db repo CI already resets a stack) | ❌ open |
+| B5 | Runtime smoke test: browse the app with `VITE_DATA_LAYER_IMPL=supabase` against local stack | ✅ passed 2026-07-18 (home/search/detail/language/topic; zero data-layer failures) |
+
+## C. Parse usage outside the data layer (switchover blockers)
+
+| # | Item | Status |
+|---|------|--------|
+| C1 | `export/freeLearningIO.ts` raw Parse REST call w/ hardcoded prod app id | ✅ routed through book repository (81e88ff); accepted narrowing: inCirculation strictly-true vs legacy true-or-unset |
+| C2 | Bloom API auth bridge (`connection/ApiConnection.ts`) reads session token from a singleton login no longer populates — confirmed live bug | ✅ fixed (8f2265c); follow-up: same dead-singleton reads remain in LibraryQueries/LibraryQueryHooks/LibraryUpdates (see C4) |
+| C3 | Stats path posts Parse query DSL (`$regex`, `$score`) to api.bloomlibrary.org | ❌ open — needs server-side plan; document as external dependency |
+| C4 | Dead/duplicate Parse plumbing (`connection/ParseServerConnection.ts` dead fns, `LibraryUpdates.updateBook`, duplicated connection config) | ✅ both files deleted; live calls rewired to data-layer ParseConnection (01b37a1) |
+| C5 | Book-navigation interceptor service worker (`src/book-navigation-interceptor-sw.js`, new from master 2026-07-18 merge) does anonymous Parse REST reads (`classes/books` by bookInstanceId via `ParseConnectionConfig`) — a service worker can't easily share the app's data-layer | ❌ open — anon-read scope, so it IS a switchover blocker; needs a Supabase query path (or an impl-switched fetch) inside the SW. Discovered while integrating master; not yet decided/scheduled |
+
+## D. Guardrails for out-of-scope paths
+
+| # | Item | Status |
+|---|------|--------|
+| D1 | Under supabase impl: login/write UI must not route into throwing stubs (decide: hide, disable, or keep Parse-backed) | ✅ decided 2026-07-18 — mixed mode: keep auth/user Parse-backed. Supabase registration binds ParseAuthenticationService/ParseUserRepository under the Supabase impl keys (`implementations/supabase/index.ts`); getBloomApiHeaders() therefore carries the real Parse session token. Covered by DataLayer.test.ts |
+| D2 | Analytics interface unwired on both sides (Parse side unregistered) — confirm nothing calls `getAnalyticsService()` before wiring | ✅ verified (zero callers) |
+| D3 | Moderator grid column sorting not honored by the Supabase read path (`SupabaseBookRepository.getBooksForGrid` hardcodes `BookOrderingScheme.Default`; `applyOrdering` only maps ordering *schemes*, not arbitrary grid columns) | ⚠ known gap, moderator-scope only. The Parse path was fixed to thread `query.sorting` through (`constructParseSortOrder`). Moderator writes are Parse-backed under mixed mode (HybridBookRepository delegates `updateBook`/`deleteBook`/`saveArtifactVisibility` to Parse), so the whole moderator grid/edit path still round-trips through Parse; column sorting on a future Supabase moderator path would need a column→db-column ordering map. Out of anon-browsing scope. |
+| D4 | Bulk-edit reads capped by PostgREST `max_rows` under the Supabase read path: BulkChangeFunctions / getAllGridDataAndExportCsv ask for effectively-unlimited rows, but PostgREST clamps to 1000 (supabase/config.toml), so a moderator bulk operation over more matches would silently process only the first page. Parse path unaffected. | ⚠ known gap, moderator-scope only (same acceptance basis as D3: the moderator write path is Parse-backed under mixed mode and out of anon-browsing scope). A Supabase moderator milestone needs paged reads here. Flagged by Devin round 6, 2026-07-18 |
+
+## E. Backend dependencies (bloom-core-supabase)
+
+| # | Item | Status |
+|---|------|--------|
+| E1 | `fs` file/thumbnail serving | ✅ live in production |
+| E2 | `social` OpenGraph previews | ⚠ code done; production worker routing pending |
+| E3 | Read schema (books/languages/tags/users + RLS anon read) | ✅ local milestone done |
+| E4 | Full production data sync (watermark + tombstones) | ❌ future milestone (explicitly after this readiness work) |
+| E5 | FTS/search-string derivation triggers | ❌ open (blocked on A1 design) |
+
+"100% ready" = all A/B/C items closed or explicitly accepted, D decided, and the
+switchover still gated on E4 (data migration), which is the next milestone after
+this one.