From c978fe6b0cfdf1093556d23431c71654c5876d26 Mon Sep 17 00:00:00 2001 From: Hoon Date: Fri, 10 Jul 2026 15:02:36 +0900 Subject: [PATCH 1/4] chore: rename CI workflow from ci-v1 to ci-v2 --- .cursor/rules/script-style.mdc | 4 ++-- .github/workflows/{ci-v1.yml => ci-v2.yml} | 0 README.md | 4 ++-- docs/MSA_MIGRATION.md | 4 ++-- 4 files changed, 6 insertions(+), 6 deletions(-) rename .github/workflows/{ci-v1.yml => ci-v2.yml} (100%) diff --git a/.cursor/rules/script-style.mdc b/.cursor/rules/script-style.mdc index c827ae05..8513ec24 100644 --- a/.cursor/rules/script-style.mdc +++ b/.cursor/rules/script-style.mdc @@ -1,6 +1,6 @@ --- description: Bash script, CD workflow, and deploy JSON formatting conventions -globs: scripts/**/*.sh,.github/workflows/ci-v1.yml,.github/workflows/cd-v1.yml,infra/azure/parameters*.json +globs: scripts/**/*.sh,.github/workflows/ci-v2.yml,.github/workflows/cd-v1.yml,infra/azure/parameters*.json alwaysApply: false --- @@ -57,7 +57,7 @@ Set `LOG_PREFIX` **before** sourcing `common.sh`. Keep `#!/bin/sh`, POSIX-only, no bash `common.sh`. English comments, `set -e`, minimal headers. -## CI and CD workflows ([`ci-v1.yml`](.github/workflows/ci-v1.yml), [`cd-v1.yml`](.github/workflows/cd-v1.yml)) +## CI and CD workflows ([`ci-v2.yml`](.github/workflows/ci-v2.yml), [`cd-v1.yml`](.github/workflows/cd-v1.yml)) Shared conventions: diff --git a/.github/workflows/ci-v1.yml b/.github/workflows/ci-v2.yml similarity index 100% rename from .github/workflows/ci-v1.yml rename to .github/workflows/ci-v2.yml diff --git a/README.md b/README.md index 9bc2115f..2639b1ac 100644 --- a/README.md +++ b/README.md @@ -123,7 +123,7 @@ Service boundaries and MSA migration plan: [docs/SERVICE_BOUNDARIES.md](docs/SER * **GitHub Actions** - * CI on pull requests and pushes to `main` / `develop` — [`.github/workflows/ci-v1.yml`](.github/workflows/ci-v1.yml) + * CI on pull requests and pushes to `main` / `develop` — [`.github/workflows/ci-v2.yml`](.github/workflows/ci-v2.yml) * Jobs: .NET build, Rust tests, web lint/test/build, service integration tests (Testcontainers), media harness E2E (Compose + Azurite) * Local parity: `./scripts/run-all-tests.sh` (integration + harness + Rust + web; add `--skip-harness` for faster runs) @@ -463,7 +463,7 @@ Integration tests and harness E2E both use Docker Compose, but they work differe Use `scripts/ci/docker-test.sh` or `--profile test` for integration tests. Do **not** use `docker-dotnet.sh` / `sdk` — the `test` service mounts `/var/run/docker.sock` for Testcontainers. -**CI parity:** [`.github/workflows/ci-v1.yml`](.github/workflows/ci-v1.yml). +**CI parity:** [`.github/workflows/ci-v2.yml`](.github/workflows/ci-v2.yml). #### Quick start — integration tests only diff --git a/docs/MSA_MIGRATION.md b/docs/MSA_MIGRATION.md index c2c24264..c18f1fce 100644 --- a/docs/MSA_MIGRATION.md +++ b/docs/MSA_MIGRATION.md @@ -26,13 +26,13 @@ OpenAPI/Swagger for all domain services is **v2.0.0** (document name `v2`), set | HTTP routes | Unversioned (`/api/posts`, …) | No breaking contract change for clients | | OpenAPI / Swagger | **v2.0.0** | Per-service docs at `/api` in dev/Docker | | Redis queue/event payloads | `SchemaVersion = 1` | Unchanged — bump only when payload shape breaks | -| CI/CD workflows (`ci-v1.yml`) | Pipeline v1 | Unrelated to API semver | +| CI/CD workflows (`ci-v2.yml`) | Pipeline v2 | Unrelated to API semver | Path-based versioning (`/api/v2/*`) is not planned unless a future release requires side-by-side contracts. --- -## CI ladder (`ci-v1.yml`) +## CI ladder (`ci-v2.yml`) Build once, test many times — harness reuses compiled artifacts instead of rebuilding inside the harness job. From 48b78f0beef596ce1a03adb230fd626bdc1690d0 Mon Sep 17 00:00:00 2001 From: Hoon Date: Fri, 10 Jul 2026 15:49:17 +0900 Subject: [PATCH 2/4] feat: drive Azure Container Apps from parameters.prod.json via Bicep --- infra/README.md | 10 +- infra/azure/README.md | 188 +++--- infra/azure/main.bicep | 469 +++++---------- infra/azure/modules/container-app.bicep | 25 +- infra/azure/modules/migrate-job.bicep | 26 +- .../monitoring/prometheus/prometheus-start.sh | 69 ++- infra/azure/parameters.dev.json | 80 ++- infra/azure/parameters.prod.json | 548 ++++++++++++++---- infra/nginx/nginx.production.conf | 4 +- scripts/azure-deploy-infra.sh | 45 +- 10 files changed, 853 insertions(+), 611 deletions(-) diff --git a/infra/README.md b/infra/README.md index 540db06a..654512fc 100644 --- a/infra/README.md +++ b/infra/README.md @@ -216,16 +216,16 @@ infra/azure/monitoring/ | `grafana:3000` (host) | `tangle-study-grafana` (external FQDN) | | `postgres-exporter` → compose `db` | `tangle-study-postgres-exporter` → **Neon** | | `redis-exporter` → compose `redis` | `tangle-study-redis-exporter` → internal Redis | -| `api:8080/metrics` | `tangle-study-api/metrics` (short name; ACA ingress port 80) | +| `gateway` / services `:8080/metrics` | `tangle-study-gateway` / `tangle-study-/metrics` (short name; ACA ingress port 80) | | `rust-worker-*:9090/metrics` | `tangle-study-worker-*` (short name; ACA ingress port 80) | CD ([`scripts/cd/azure-cd-build-push.sh`](../scripts/cd/azure-cd-build-push.sh)) builds and pushes -`tangle-study-prometheus` and `tangle-study-grafana` alongside api/web/worker. Deploy -([`azure-cd-deploy-image.sh`](../scripts/cd/azure-cd-deploy-image.sh)) sets GHCR images plus runtime -env from [`azure-aca-urls.sh`](../scripts/cd/libs/azure-aca-urls.sh) (e.g. `PROMETHEUS_URL=http://tangle-study-prometheus`, +`tangle-study-prometheus` and `tangle-study-grafana` alongside gateway/services/web/workers. Deploy +([`azure-cd-deploy-bicep.sh`](../scripts/cd/azure-cd-deploy-bicep.sh)) applies images and env from +[`parameters.prod.json`](azure/parameters.prod.json) (e.g. `PROMETHEUS_URL=http://tangle-study-prometheus`, `REDIS_URL=redis://tangle-study-redis`). Secrets: `METRICS_SCRAPE_SECRET`, `GRAFANA_ADMIN_PASSWORD`, and Neon `POSTGRES_CONNECTION_STRING` -(postgres-exporter DSN derived at inject time). See [infra/azure/README.md](azure/README.md#monitoring-on-aca). +(postgres-exporter DSN derived at Bicep deploy time). See [infra/azure/README.md](azure/README.md#monitoring-on-aca). Grafana login on Azure: `admin` / `GRAFANA_ADMIN_PASSWORD`. Resolve URL: diff --git a/infra/azure/README.md b/infra/azure/README.md index 4651d0a0..9b50a679 100644 --- a/infra/azure/README.md +++ b/infra/azure/README.md @@ -2,6 +2,8 @@ Bicep templates for Tangle on **Azure Container Apps**, tuned for a **study / low-cost** setup. +**Single source of truth:** [`parameters.prod.json`](parameters.prod.json) defines every Container App (image, env, secret refs, ingress, replicas) and every EF migrate job. [`main.bicep`](main.bicep) loops over that map; CD ([`cd-v2.yml`](../../.github/workflows/cd-v2.yml)) runs Bicep on every deploy. + ## What we avoid (no free tier) | Service | Alternative | @@ -24,25 +26,34 @@ Bicep templates for Tangle on **Azure Container Apps**, tuned for a **study / lo ``` infra/azure/ - main.bicep # Per-environment stack + main.bicep # Data-driven stack (loops containerApps + migrateJobs) parameters.dev.json # Optional local experiments (tangle-study-dev) - parameters.prod.json # Production (tangle-study-prod) — CD target + parameters.prod.json # Production (tangle-study-prod) — CD target / SSoT monitoring/ - prometheus/ # Custom GHCR image (ACA scrape config) + prometheus/ # Custom GHCR image (ACA scrape config for MSA services) grafana/ # Custom GHCR image (bundles infra/grafana provisioning) modules/ infra-container.bicep # Redis (internal TCP ingress on :6379) storage.bicep # Blob only app-insights.bicep # Application Insights (linked to Log Analytics) - container-app.bicep # API, web, workers, monitoring (GHCR pull) - migrate-job.bicep + container-app.bicep # Gateway, services, web, workers, monitoring (GHCR pull) + migrate-job.bicep # Per-service EF migrate jobs ... ``` ## Manual deploy (production) +Pass the same secure values CD uses (or leave them empty for placeholder-only bootstrap): + ```bash -chmod +x scripts/azure-deploy-infra.sh +POSTGRES_CONNECTION_STRING='...' \ +BLOB_CONNECTION_STRING='...' \ +JWT_SECRET='...' \ +WORKER_CALLBACK_SECRET='...' \ +METRICS_SCRAPE_SECRET='...' \ +GRAFANA_ADMIN_PASSWORD='...' \ +GATEWAY_SECRET='...' \ +INTERNAL_SERVICE_SECRET='...' \ ./scripts/azure-deploy-infra.sh prod ``` @@ -60,7 +71,14 @@ Public GHCR images need **no** registry username/password on Container Apps. CD builds and pushes: -- `ghcr.io//tangle-study/tangle-study-api:` +- `ghcr.io//tangle-study/tangle-study-gateway:` +- `ghcr.io//tangle-study/tangle-study-users:` +- `ghcr.io//tangle-study/tangle-study-media:` +- `ghcr.io//tangle-study/tangle-study-chat:` +- `ghcr.io//tangle-study/tangle-study-location:` +- `ghcr.io//tangle-study/tangle-study-community:` +- `ghcr.io//tangle-study/tangle-study-group:` +- `ghcr.io//tangle-study/tangle-study-social:` - `ghcr.io//tangle-study/tangle-study-web:` - `ghcr.io//tangle-study/tangle-study-worker-media:` - `ghcr.io//tangle-study/tangle-study-worker-chat:` @@ -68,172 +86,98 @@ CD builds and pushes: - `ghcr.io//tangle-study/tangle-study-prometheus:` - `ghcr.io//tangle-study/tangle-study-grafana:` -Exporter images (`postgres-exporter`, `redis-exporter`) use public Docker Hub tags from `docker/versions.prod.env` via Bicep parameters. +Exporter images (`postgres-exporter`, `redis-exporter`) use public Docker Hub tags from `parameters.prod.json` → `infra`. Set `containerRegistry` in `parameters.prod.json` to match your org. `usePlaceholderImages: true` allows infra deploy before custom images exist — -it swaps `api`/`web`/`worker` images for a public placeholder and sets +it swaps gateway/service/web/worker images for a public placeholder and sets `targetPort: 80` (matching the placeholder's exposed port) instead of the -real app port (`8080`). Prometheus/Grafana always use their upstream images -regardless of this flag. +real app port (`8080`). Prometheus/Grafana use their upstream images when +this flag is true. -**This is a one-time bootstrap switch, not a steady-state setting.** It only -matters for the manual `azure-deploy-infra.sh` path (see -[CD vs manual deploy](#cd-vs-manual-deploy) below) — once the stack has been -bootstrapped and real images exist in GHCR, set it to `false` and leave it -there. Routine CD deploys never read this flag at all. +**Bootstrap switch only.** CD (`cd-v2.yml`) always passes `usePlaceholderImages=false` +after pushing real images. Manual `azure-deploy-infra.sh` defaults to `true`. ## Internal networking -Cross-app URLs in CD use **ACA short Container App names** (no `targetPort`, no internal FQDN). Centralized in [`scripts/cd/libs/azure-aca-urls.sh`](../../scripts/cd/libs/azure-aca-urls.sh) and injected by [`azure-cd-deploy-image.sh`](../../scripts/cd/azure-cd-deploy-image.sh). +Cross-app URLs live in [`parameters.prod.json`](parameters.prod.json) as ACA **short Container App names** (no `targetPort`, no internal FQDN). Bicep applies them on every deploy. ### ACA short names: do not append `targetPort` Within a Container Apps Environment, HTTP ingress apps can be reached by **short app name** -(e.g. `tangle-study-api`). This behaves differently from Docker Compose, where `api:8080` +(e.g. `tangle-study-gateway`). This behaves differently from Docker Compose, where `gateway:8080` works because the service listens on that port directly. On ACA, the short name resolves to the **ingress front door** (port **80**), which forwards -to the container's `targetPort` (8080 for our API). If you append `:8080` to the short name, +to the container's `targetPort` (8080 for .NET services). If you append `:8080` to the short name, the client connects to the **pod IP on port 8080** instead of the ingress — and that port is not exposed on the pod network, so the connection **times out**. | Caller URL | Result | |------------|--------| -| `http://tangle-study-api/health` | Works — ingress :80 → container :8080 | -| `http://tangle-study-api:8080/health` | **Timeout** — hits pod IP :8080 directly | -| `https://tangle-study-api.internal./health` | Works — internal FQDN on :443 | - -**Symptoms when misconfigured** (`TANGLE_API_UPSTREAM=tangle-study-api:8080`): +| `http://tangle-study-gateway/health` | Works — ingress :80 → container :8080 | +| `http://tangle-study-gateway:8080/health` | **Timeout** — hits pod IP :8080 directly | +| `https://tangle-study-gateway.internal./health` | Works — internal FQDN on :443 | -- Direct curl from web container to `http://tangle-study-api/health` → **200 Healthy** -- Curl through nginx (`http://127.0.0.1/health`) → **504 Gateway Timeout** -- Public smoke test (`https:///health`) → **504** -- Nginx error log: `upstream timed out ... while connecting to upstream` to `100.100.x.x:8080` - -**Correct config:** set `TANGLE_API_UPSTREAM` to the short name **only** (no port): +**Correct web config** (in `parameters.prod.json`): ```json -"TANGLE_API_UPSTREAM": "tangle-study-api", -"TANGLE_API_HOST": "tangle-study-api" +"TANGLE_API_UPSTREAM": "tangle-study-gateway", +"TANGLE_API_HOST": "tangle-study-gateway" ``` Nginx renders this at container start ([`infra/nginx/docker-entrypoint.sh`](../nginx/docker-entrypoint.sh)). -Changing the env var requires a **new web revision** — a running container keeps the old -upstream until redeployed. - -**Verify from the web container** (after exec): - -```bash -# Wrong upstream still baked in? -grep -A1 'upstream tangle_api' /etc/nginx/conf.d/default.conf - -# Direct to API (bypasses nginx) — should work either way -curl -sf http://tangle-study-api/health - -# Through nginx — fails until upstream omits :8080 -curl -sv http://127.0.0.1/health -``` - -**Fix in prod:** - -```bash -az containerapp update -n tangle-study-web -g tangle-study-prod \ - --set-env-vars TANGLE_API_UPSTREAM=tangle-study-api TANGLE_API_HOST=tangle-study-api -``` - -Or re-run CD [`scripts/cd/azure-cd-deploy-image.sh`](../../scripts/cd/azure-cd-deploy-image.sh), which reads -[`parameters.prod.json`](parameters.prod.json). - -For **HTTP ingress** apps (API, Prometheus, workers, exporters, web→API), use the short Container App name -**without a port** (e.g. `tangle-study-api`, `http://tangle-study-prometheus`). Internal FQDNs on :443 also work but CD does not use them. | App | Hostname (within environment) | |-----|------------------------------| | Postgres | **Neon** (external; not in ACA) | | Redis | `tangle-study-redis` (TCP; port 6379 implicit) | -| API | `tangle-study-api` (HTTP ingress short name; port 80 implicit) | -| Prometheus | `tangle-study-prometheus` (HTTP ingress short name) | +| Gateway | `tangle-study-gateway` (HTTP ingress; web upstream) | +| Users / Media / Chat / Location / Community / Group / Social | `tangle-study-` | +| Prometheus | `tangle-study-prometheus` | | Workers / exporters | `tangle-study-worker-*`, `tangle-study-postgres-exporter`, `tangle-study-redis-exporter` | -| Web | public FQDN → proxies to API | +| Web | public FQDN → proxies to gateway | | Grafana | public FQDN (external ingress) | -Postgres connection string is injected by CD from GitHub secret `POSTGRES_CONNECTION_STRING` (API + migrate job). Postgres-exporter gets a derived `postgresql://` DSN for Neon. Redis host/URL is set at CD deploy (`Redis__ConnectionString=tangle-study-redis`, `REDIS_URL=redis://tangle-study-redis`, `REDIS_ADDR=tangle-study-redis`). +Postgres connection string is a secure Bicep param from GitHub secret `POSTGRES_CONNECTION_STRING` (all DB services + migrate jobs). Postgres-exporter gets a derived `postgresql://` DSN. Redis short names are set in `parameters.prod.json`. ## CD vs manual deploy -Two independent deploy paths exist, and they do **not** share state at -runtime — keeping this straight matters, because an env var set by one path -is invisible to the other. - -- **Manual (`azure-deploy-infra.sh` → `main.bicep`)**: one-time bootstrap. - Provisions the Container Apps Environment, Redis, storage, monitoring, - and seeds the *initial* container env vars/secrets (including a computed - `TANGLE_API_UPSTREAM` for web, based on `usePlaceholderImages`). This is - meant for first-time setup or infra-only changes (module edits), and is - not re-run automatically by CD. -- **CD (`main` branch / Deploy workflow → `scripts/cd/azure-cd-deploy-image.sh`)**: - the routine deploy path. Builds and pushes GHCR images, waits for - propagation, then calls `az containerapp update --set-env-vars` per app - using **`parameters.prod.json` → `containerApps..env` as the single - source of truth**. It does **not** re-run Bicep, so any env var that only - exists in `main.bicep` — and isn't also listed in - `parameters.prod.json` — will never be updated by CD and can silently - drift from what Bicep would compute (this bit us once with - `TANGLE_API_UPSTREAM` pointing at the wrong port after switching - `usePlaceholderImages`). - -**Practical rule:** if a container app's env var can change (ports, -hostnames, feature flags), define it explicitly in `parameters.prod.json`'s -`containerApps` block, not just in `main.bicep`. Treat `main.bicep`'s -computed env values (like `apiAppUpstream`) as bootstrap defaults only — -`parameters.prod.json` overrides them on every subsequent CD run. - -Because CD never re-runs Bicep, a full clean deploy through CD alone is -always safe with `usePlaceholderImages: false` — CD builds and pushes real -images before touching any Container App, regardless of what that flag says. -The flag only matters if you're re-running the manual bootstrap script. +Both paths run the **same** `main.bicep` against `parameters.prod.json`. Secrets are `@secure()` Bicep params (never stored in the JSON file). + +- **Manual (`azure-deploy-infra.sh`)**: bootstrap or infra-only experiments. Defaults `usePlaceholderImages=true`. Pass secure env vars when you want real secret values. +- **CD (`cd-v2.yml` → [`azure-cd-deploy-bicep.sh`](../../scripts/cd/azure-cd-deploy-bicep.sh))**: routine path. Builds/pushes GHCR images, then `az deployment group create` with `imageTag=`, `usePlaceholderImages=false`, and GitHub Environment secrets. Then runs per-service migrate jobs and smoke tests. + +**Practical rule:** add or change an app only in `parameters.prod.json` (`containerApps` / `migrateJobs`). Do not hardcode new apps in `main.bicep`. ## Monitoring on ACA | Container App | Ingress | Notes | |---------------|---------|-------| -| `tangle-study-postgres-exporter` | internal :9187 | Scrapes Neon (`DATA_SOURCE_NAME` from CD) | +| `tangle-study-postgres-exporter` | internal :9187 | Scrapes Neon (`DATA_SOURCE_NAME` from Bicep) | | `tangle-study-redis-exporter` | internal :9121 | Scrapes internal Redis | -| `tangle-study-prometheus` | internal | Custom GHCR image; scrapes API, workers, exporters (short names) | +| `tangle-study-prometheus` | internal | Custom GHCR image; scrapes gateway + 7 services + workers + exporters | | `tangle-study-grafana` | **external** :3000 | Custom GHCR image; login `admin` / `GRAFANA_ADMIN_PASSWORD` | **CD path:** [`scripts/cd/azure-cd-build-push.sh`](../../scripts/cd/azure-cd-build-push.sh) builds -`tangle-study-prometheus` and `tangle-study-grafana` from [`monitoring/`](monitoring/) (bundles -[`infra/grafana/provisioning/`](../grafana/provisioning/) and recording rules). Deploy sets image + -env from [`parameters.prod.json`](parameters.prod.json); [`azure-cd-deploy-image.sh`](../../scripts/cd/azure-cd-deploy-image.sh) -injects cross-app URLs from [`azure-aca-urls.sh`](../../scripts/cd/libs/azure-aca-urls.sh) (e.g. -`PROMETHEUS_URL=http://tangle-study-prometheus`, `REDIS_URL=redis://tangle-study-redis`). Do **not** deploy vanilla `prom/prometheus` or `grafana/grafana` — they skip the ACA entrypoints and provisioning. +`tangle-study-prometheus` and `tangle-study-grafana` from [`monitoring/`](monitoring/). +Do **not** deploy vanilla `prom/prometheus` or `grafana/grafana` — they skip the ACA entrypoints and provisioning. Grafana bundles dashboards and alerts from [`infra/grafana/provisioning/`](../grafana/provisioning/). See [infra/README.md](../README.md) for metric and alert details. Short names must omit `targetPort` — see [ACA short names](#aca-short-names-do-not-append-targetport). ## After Bicep deploy -> **Clean deploy order matters.** For a brand-new stack, run the manual -> `azure-deploy-infra.sh` bootstrap once (with `usePlaceholderImages: true` -> only if GHCR images don't exist yet) to create the Container Apps -> Environment, Redis, storage, and monitoring shells. After that, all -> routine deploys — including future clean redeploys of app containers — -> go through the **Deploy** GitHub Actions workflow, which builds and -> pushes GHCR images first and then updates each Container App directly -> from `parameters.prod.json` (see [CD vs manual deploy](#cd-vs-manual-deploy)). -> The manual script is not part of this routine loop and should not need -> to be re-run unless infra itself (Bicep modules) changes. +> For a brand-new stack, run `azure-deploy-infra.sh prod` once (placeholders OK if GHCR images do not exist yet). After that, routine deploys go through **Deploy** (`cd-v2.yml`), which builds images then re-runs Bicep from `parameters.prod.json`. 1. Create a Neon project and database; copy the Npgsql connection string. 2. Copy storage account connection string → GitHub secret `BLOB_CONNECTION_STRING`. -3. Set GitHub Environment **`prod`** secrets (see [DEPLOYMENT.md](../../docs/DEPLOYMENT.md)), including `POSTGRES_CONNECTION_STRING` and `GRAFANA_ADMIN_PASSWORD`. -4. Merge to **`main`** (or run **Deploy** workflow) — CD pushes GHCR images and updates Container Apps. -5. Migrate runs automatically via `scripts/cd/azure-cd-migrate.sh` in the deploy workflow. -6. Smoke tests run via `scripts/cd/azure-cd-smoke.sh` (`/health` + SPA shell). +3. Set GitHub Environment **`prod`** secrets (see [DEPLOYMENT.md](../../docs/DEPLOYMENT.md)), including `POSTGRES_CONNECTION_STRING`, `GATEWAY_SECRET`, `INTERNAL_SERVICE_SECRET`, and `GRAFANA_ADMIN_PASSWORD`. +4. Merge to **`main`** (or run **Deploy** workflow) — CD pushes GHCR images and runs Bicep. +5. Migrate runs automatically via `scripts/cd/azure-cd-migrate.sh` (one job per DB-owning service). +6. Smoke tests run via `scripts/cd/azure-cd-smoke.sh` (`/health` via web → gateway + SPA shell). -If upgrading from an older stack with `tangle-study-postgres`, delete the orphaned Container App and `postgres-data` file share after redeploying Bicep. +If upgrading from an older stack with `tangle-study-api` / `tangle-study-migrate`, delete those orphaned resources after the MSA cutover deploy. ## Resource groups @@ -244,16 +188,16 @@ If upgrading from an older stack with `tangle-study-postgres`, delete the orphan ## Redis on ACA (trade-offs) -API and workers are **stateless**; Redis holds shared ephemeral state (cache, SignalR backplane, Streams, live positions). This is the correct pattern for multi-replica ACA. +API services and workers are **stateless**; Redis holds shared ephemeral state (cache, SignalR backplane, Streams, live positions). This is the correct pattern for multi-replica ACA. | Risk | Impact | |------|--------| | No persistence volume on ACA Redis | Restart wipes cache/streams/live positions; Neon is source of truth | -| Single replica SPOF | Redis down → API `/health` fails, queues/SignalR stop | +| Single replica SPOF | Redis down → health checks / queues / SignalR stop | | Workers scale to 0 | Streams backlog while cold; data lost only if Redis also restarts | | No TLS on internal Redis | OK within CAE; revisit if moving to managed Redis | -Scale-to-zero workers (`workerMinReplicas: 0`) save cost; Redis and monitoring stay at `minReplicas: 1`. +Scale-to-zero workers (`minReplicas: 0` in `parameters.prod.json`) save cost; Redis and monitoring stay at `minReplicas: 1`. ## Why not Postgres on ACA @@ -271,6 +215,6 @@ Azure Container Apps is a good fit for **stateless** app containers. The first p For a learning deployment with no data-migration requirement, **Neon** (free tier, SSL, no ACA networking) replaces self-hosted Postgres. Redis remains on ACA because it holds **ephemeral** state only; Postgres is the **source of truth** and belongs in managed storage. -**Current wiring:** GitHub secret `POSTGRES_CONNECTION_STRING` → CD inject into API and migrate job; `tangle-study-postgres-exporter` scrapes Neon via a derived DSN (see [Monitoring on ACA](#monitoring-on-aca) above). +**Current wiring:** GitHub secret `POSTGRES_CONNECTION_STRING` → Bicep secure param into all DB services and migrate jobs; `tangle-study-postgres-exporter` scrapes Neon via a derived DSN (see [Monitoring on ACA](#monitoring-on-aca) above). -**Upgrading from an old stack:** delete orphaned `tangle-study-postgres` Container App and `postgres-data` file share after redeploying Bicep (see [After Bicep deploy](#after-bicep-deploy)). \ No newline at end of file +**Upgrading from an old stack:** delete orphaned `tangle-study-postgres` Container App and `postgres-data` file share after redeploying Bicep (see [After Bicep deploy](#after-bicep-deploy)). diff --git a/infra/azure/main.bicep b/infra/azure/main.bicep index 2b958378..124a0b54 100644 --- a/infra/azure/main.bicep +++ b/infra/azure/main.bicep @@ -23,21 +23,51 @@ param registryUsername string = '' @secure() param registryPassword string = '' -param redisImage string = 'redis:8-alpine' +@description('Pinned infra image tags (postgres, redis, exporters, upstream prometheus/grafana).') +param infra object = {} -param prometheusImage string = 'prom/prometheus:v3.12.0' +@description('Pinned build-base image tags (dotnet, node, nginx, rust, debian). Kept for parameters.prod.json single source of truth; unused by Bicep runtime.') +#disable-next-line no-unused-params +param buildImages object = {} -param grafanaImage string = 'grafana/grafana:13.0.2' +@description('Container Apps map from parameters.prod.json (single source of truth).') +param containerApps object = {} -param postgresExporterImage string = 'prometheuscommunity/postgres-exporter:v0.19.1' +@description('EF migrate jobs from parameters.prod.json.') +param migrateJobs array = [] -param redisExporterImage string = 'oliver006/redis_exporter:v1.86.0' +@secure() +param postgresConnectionString string = '' + +@secure() +param postgresExporterDsn string = '' + +@secure() +param blobConnectionString string = '' + +@secure() +param jwtSecret string = '' + +@secure() +param workerCallbackSecret string = '' + +@secure() +param metricsScrapeSecret string = '' + +@secure() +param placesApiKey string = '' + +@secure() +param applicationInsightsConnectionString string = '' -param monitoringMinReplicas int = 1 +@secure() +param grafanaAdminPassword string = '' + +@secure() +param gatewaySecret string = '' -param apiMinReplicas int = 1 -param webMinReplicas int = 1 -param workerMinReplicas int = 0 +@secure() +param internalServiceSecret string = '' param tags object = { project: 'tangle-study' @@ -46,28 +76,37 @@ param tags object = { var namePrefix = '${baseName}${environment}' var placeholderImage = 'mcr.microsoft.com/k8se/quickstart:latest' -var apiImage = usePlaceholderImages ? placeholderImage : '${containerRegistry}/tangle-study-api:${imageTag}' -var webImage = usePlaceholderImages ? placeholderImage : '${containerRegistry}/tangle-study-web:${imageTag}' -var workerChatImage = usePlaceholderImages ? placeholderImage : '${containerRegistry}/tangle-study-worker-chat:${imageTag}' -var workerMediaImage = usePlaceholderImages ? placeholderImage : '${containerRegistry}/tangle-study-worker-media:${imageTag}' -var workerLocationImage = usePlaceholderImages ? placeholderImage : '${containerRegistry}/tangle-study-worker-location:${imageTag}' -var resolvedPrometheusImage = usePlaceholderImages ? prometheusImage : '${containerRegistry}/tangle-study-prometheus:${imageTag}' -var resolvedGrafanaImage = usePlaceholderImages ? grafanaImage : '${containerRegistry}/tangle-study-grafana:${imageTag}' - -var workerMetricsSecretEnvVars = [ - { name: 'metrics-secret', envName: 'METRICS_SCRAPE_SECRET' } -] -var apiSecretEnvVars = [ - { name: 'postgres-conn', envName: 'ConnectionStrings__DefaultConnection' } - { name: 'blob-conn', envName: 'Media__ConnectionString' } - { name: 'jwt-secret', envName: 'Jwt__Secret' } - { name: 'worker-callback', envName: 'Media__WorkerCallbackSecret' } - { name: 'metrics-secret', envName: 'Metrics__ScrapeSecret' } - { name: 'places-api-key', envName: 'Places__ApiKey' } - { name: 'appinsights-conn', envName: 'APPLICATIONINSIGHTS_CONNECTION_STRING' } +// Secret name → value map for containerApps.secretEnvVars and migrate jobs. +var secretValues = { + 'postgres-conn': postgresConnectionString + 'postgres-dsn': postgresExporterDsn + 'blob-conn': blobConnectionString + 'jwt-secret': jwtSecret + 'worker-callback': workerCallbackSecret + 'metrics-secret': metricsScrapeSecret + 'places-api-key': placesApiKey + 'appinsights-conn': applicationInsightsConnectionString + 'grafana-admin-password': grafanaAdminPassword + 'gateway-secret': gatewaySecret + 'internal-service-secret': internalServiceSecret +} + +var customImageTypes = [ + 'gateway' + 'users' + 'media' + 'chat' + 'location' + 'community' + 'group' + 'social' + 'web' + 'worker' ] +var appEntries = items(containerApps) + module logAnalytics 'modules/log-analytics.bicep' = { name: 'log-analytics' params: { @@ -108,29 +147,16 @@ module containerAppsEnv 'modules/container-apps-env.bicep' = { } } -// Redis TCP ingress: short app name within the CAE (port 6379 implicit). -var redisInternalHost = 'tangle-study-redis' -var redisConnectionString = redisInternalHost -var redisUrl = 'redis://${redisInternalHost}' -var apiAppHost = 'tangle-study-api' - -// ACA HTTP ingress: short app names route on port 80 (ingress front door), not -// the container targetPort. Do not append :8080 — callers time out on pod IP. -// NOTE: Bootstrap-only (azure-deploy-infra.sh). CD reads TANGLE_API_UPSTREAM -// from parameters.prod.json; keep both in sync. -var apiAppUpstream = apiAppHost -var apiAppBaseUrl = 'http://${apiAppHost}' -var prometheusInternalUrl = 'http://tangle-study-prometheus' - -module redis 'modules/infra-container.bicep' = { +// Redis is infrastructure (TCP ingress); always provisioned when present in containerApps. +module redis 'modules/infra-container.bicep' = if (contains(containerApps, 'tangle-study-redis')) { name: 'infra-redis' params: { name: 'tangle-study-redis' location: location managedEnvironmentId: containerAppsEnv.outputs.id - containerImage: redisImage - minReplicas: 1 - maxReplicas: 1 + containerImage: infra.?redis.?image ?? 'redis:8-alpine' + minReplicas: containerApps['tangle-study-redis'].?minReplicas ?? 1 + maxReplicas: containerApps['tangle-study-redis'].?maxReplicas ?? 1 tcpProbePort: 6379 tags: tags envVars: [] @@ -138,288 +164,83 @@ module redis 'modules/infra-container.bicep' = { } } -module api 'modules/container-app.bicep' = { - name: 'container-app-api' - params: { - name: 'tangle-study-api' - location: location - managedEnvironmentId: containerAppsEnv.outputs.id - containerImage: apiImage - targetPort: usePlaceholderImages ? 80 : 8080 - enableIngress: true - externalIngress: false - minReplicas: apiMinReplicas - maxReplicas: 3 - healthCheckPath: usePlaceholderImages ? '' : '/health' - registryLoginServer: 'ghcr.io' - registryUsername: registryUsername - registryPassword: registryPassword - tags: tags - envVars: [ - { name: 'ASPNETCORE_ENVIRONMENT', value: 'Production' } - { name: 'ASPNETCORE_URLS', value: 'http://+:8080' } - { name: 'Redis__Enabled', value: 'true' } - { name: 'Redis__ConnectionString', value: redisConnectionString } - { name: 'Media__ContainerName', value: storage.outputs.containerName } - { name: 'Media__PublicBlobEndpoint', value: storage.outputs.blobEndpoint } - { name: 'Metrics__RequireScrapeSecret', value: 'true' } - ] - secretEnvVars: apiSecretEnvVars - } -} - -module web 'modules/container-app.bicep' = { - name: 'container-app-web' - params: { - name: 'tangle-study-web' - location: location - managedEnvironmentId: containerAppsEnv.outputs.id - containerImage: webImage - targetPort: 80 - enableIngress: true - externalIngress: true - ingressTransport: 'auto' - minReplicas: webMinReplicas - maxReplicas: 3 - healthCheckPath: '' - registryLoginServer: 'ghcr.io' - registryUsername: registryUsername - registryPassword: registryPassword - tags: tags - envVars: [ - { name: 'TANGLE_API_UPSTREAM', value: apiAppUpstream } - ] - secretEnvVars: [] - } -} - -module workerChat 'modules/container-app.bicep' = { - name: 'container-app-worker-chat' - params: { - name: 'tangle-study-worker-chat' - location: location - managedEnvironmentId: containerAppsEnv.outputs.id - containerImage: workerChatImage - targetPort: 9090 - enableIngress: true - externalIngress: false - minReplicas: workerMinReplicas - maxReplicas: 2 - registryLoginServer: 'ghcr.io' - registryUsername: registryUsername - registryPassword: registryPassword - tags: tags - envVars: [ - { name: 'REDIS_URL', value: redisUrl } - { name: 'WORKER_STREAM_PREFIX', value: 'tangle:queue:' } - { name: 'WORKER_STREAM_KEY', value: 'chat.message.created' } - { name: 'WORKER_CONSUMER_GROUP', value: 'tangle-study-workers' } - { name: 'WORKER_METRICS_PORT', value: '9090' } - { name: 'RUST_LOG', value: 'info' } - ] - secretEnvVars: workerMetricsSecretEnvVars - } -} - -module workerMedia 'modules/container-app.bicep' = { - name: 'container-app-worker-media' - params: { - name: 'tangle-study-worker-media' - location: location - managedEnvironmentId: containerAppsEnv.outputs.id - containerImage: workerMediaImage - targetPort: 9090 - enableIngress: true - externalIngress: false - minReplicas: workerMinReplicas - maxReplicas: 2 - registryLoginServer: 'ghcr.io' - registryUsername: registryUsername - registryPassword: registryPassword - tags: tags - envVars: [ - { name: 'REDIS_URL', value: redisUrl } - { name: 'WORKER_STREAM_PREFIX', value: 'tangle:queue:' } - { name: 'WORKER_STREAM_KEY', value: 'media.uploaded' } - { name: 'WORKER_CONSUMER_GROUP', value: 'tangle-study-workers' } - { name: 'API_BASE_URL', value: apiAppBaseUrl } - { name: 'MEDIA_CONTAINER_NAME', value: storage.outputs.containerName } - { name: 'WORKER_METRICS_PORT', value: '9090' } - { name: 'RUST_LOG', value: 'info' } - ] - secretEnvVars: concat(workerMetricsSecretEnvVars, [ - { name: 'blob-conn', envName: 'AZURE_STORAGE_CONNECTION_STRING' } - { name: 'worker-callback', envName: 'WORKER_CALLBACK_SECRET' } - ]) - } -} - -module workerLocation 'modules/container-app.bicep' = { - name: 'container-app-worker-location' - params: { - name: 'tangle-study-worker-location' - location: location - managedEnvironmentId: containerAppsEnv.outputs.id - containerImage: workerLocationImage - targetPort: 9090 - enableIngress: true - externalIngress: false - minReplicas: workerMinReplicas - maxReplicas: 2 - registryLoginServer: 'ghcr.io' - registryUsername: registryUsername - registryPassword: registryPassword - tags: tags - envVars: [ - { name: 'REDIS_URL', value: redisUrl } - { name: 'WORKER_STREAM_PREFIX', value: 'tangle:queue:' } - { name: 'WORKER_STREAM_KEY', value: 'location.cluster' } - { name: 'WORKER_CONSUMER_GROUP', value: 'tangle-study-workers' } - { name: 'API_BASE_URL', value: apiAppBaseUrl } - { name: 'WORKER_METRICS_PORT', value: '9090' } - { name: 'RUST_LOG', value: 'info' } - ] - secretEnvVars: workerMetricsSecretEnvVars - } -} - -module postgresExporter 'modules/container-app.bicep' = { - name: 'container-app-postgres-exporter' - params: { - name: 'tangle-study-postgres-exporter' - location: location - managedEnvironmentId: containerAppsEnv.outputs.id - containerImage: postgresExporterImage - targetPort: 9187 - enableIngress: true - externalIngress: false - minReplicas: monitoringMinReplicas - maxReplicas: 1 - healthCheckPath: '' - registryLoginServer: '' - registryUsername: '' - registryPassword: '' - tags: tags - envVars: [] - secretEnvVars: [ - { name: 'postgres-dsn', envName: 'DATA_SOURCE_NAME' } - ] - } -} - -module redisExporter 'modules/container-app.bicep' = { - name: 'container-app-redis-exporter' - params: { - name: 'tangle-study-redis-exporter' - location: location - managedEnvironmentId: containerAppsEnv.outputs.id - containerImage: redisExporterImage - targetPort: 9121 - enableIngress: true - externalIngress: false - minReplicas: monitoringMinReplicas - maxReplicas: 1 - healthCheckPath: '' - registryLoginServer: '' - registryUsername: '' - registryPassword: '' - tags: tags - envVars: [ - { name: 'REDIS_ADDR', value: redisInternalHost } - ] - secretEnvVars: [] - } -} - -module prometheus 'modules/container-app.bicep' = { - name: 'container-app-prometheus' - params: { - name: 'tangle-study-prometheus' - location: location - managedEnvironmentId: containerAppsEnv.outputs.id - containerImage: resolvedPrometheusImage - targetPort: 9090 - enableIngress: true - externalIngress: false - minReplicas: monitoringMinReplicas - maxReplicas: 1 - healthCheckPath: '' - registryLoginServer: 'ghcr.io' - registryUsername: registryUsername - registryPassword: registryPassword - tags: tags - envVars: [] - secretEnvVars: [ - { name: 'metrics-secret', envName: 'METRICS_SCRAPE_SECRET' } - ] +module apps 'modules/container-app.bicep' = [ + for item in appEntries: if (item.value.type != 'redis') { + name: 'app-${item.key}' + params: { + name: item.key + location: location + managedEnvironmentId: containerAppsEnv.outputs.id + containerImage: !empty(item.value.?image) + ? (usePlaceholderImages && contains(customImageTypes, item.value.type) + ? placeholderImage + : (usePlaceholderImages && item.value.type == 'prometheus' + ? infra.prometheus.image + : (usePlaceholderImages && item.value.type == 'grafana' + ? infra.grafana.image + : '${containerRegistry}/${item.value.image}:${imageTag}'))) + : (!empty(item.value.?infraImage) + ? infra[item.value.infraImage].image + : placeholderImage) + targetPort: usePlaceholderImages && contains(customImageTypes, item.value.type) + ? 80 + : (item.value.?targetPort ?? 8080) + enableIngress: true + externalIngress: item.value.?externalIngress ?? false + ingressTransport: item.value.?ingressTransport ?? 'auto' + minReplicas: item.value.?minReplicas ?? 1 + maxReplicas: item.value.?maxReplicas ?? 3 + healthCheckPath: usePlaceholderImages && contains(customImageTypes, item.value.type) + ? '' + : (item.value.?healthCheckPath ?? '') + registryLoginServer: !empty(item.value.?image) ? 'ghcr.io' : '' + registryUsername: registryUsername + registryPassword: registryPassword + tags: tags + env: item.value.?env ?? {} + secretEnvVars: item.value.?secretEnvVars ?? [] + secretValues: secretValues + extraEnvVars: item.value.type == 'media' ? [ + { name: 'Media__PublicBlobEndpoint', value: storage.outputs.blobEndpoint } + { name: 'Media__ContainerName', value: storage.outputs.containerName } + ] : [] + } } -} - -module grafana 'modules/container-app.bicep' = { - name: 'container-app-grafana' - params: { - name: 'tangle-study-grafana' - location: location - managedEnvironmentId: containerAppsEnv.outputs.id - containerImage: resolvedGrafanaImage - targetPort: 3000 - enableIngress: true - externalIngress: true - minReplicas: monitoringMinReplicas - maxReplicas: 1 - healthCheckPath: '' - registryLoginServer: 'ghcr.io' - registryUsername: registryUsername - registryPassword: registryPassword - tags: tags - envVars: [ - { name: 'GF_SECURITY_ADMIN_USER', value: 'admin' } - { name: 'GF_USERS_ALLOW_SIGN_UP', value: 'false' } - { name: 'PROMETHEUS_URL', value: prometheusInternalUrl } - ] - secretEnvVars: [ - { name: 'grafana-admin-password', envName: 'GF_SECURITY_ADMIN_PASSWORD' } - ] - } -} +] -module migrateJob 'modules/migrate-job.bicep' = { - name: 'migrate-job' - params: { - name: 'tangle-study-migrate' - location: location - managedEnvironmentId: containerAppsEnv.outputs.id - containerImage: apiImage - registryLoginServer: 'ghcr.io' - registryUsername: registryUsername - registryPassword: registryPassword - tags: tags - envVars: [ - { name: 'ASPNETCORE_ENVIRONMENT', value: 'Production' } - ] - secretEnvVars: [ - { name: 'postgres-conn', envName: 'ConnectionStrings__DefaultConnection' } - ] +module migrate 'modules/migrate-job.bicep' = [ + for job in migrateJobs: { + name: 'migrate-${job.name}' + params: { + name: job.name + location: location + managedEnvironmentId: containerAppsEnv.outputs.id + containerImage: usePlaceholderImages + ? placeholderImage + : '${containerRegistry}/${job.image}:${imageTag}' + registryLoginServer: 'ghcr.io' + registryUsername: registryUsername + registryPassword: registryPassword + tags: tags + command: job.command + env: { + ASPNETCORE_ENVIRONMENT: 'Production' + } + secretEnvVars: [ + { + name: 'postgres-conn' + envName: 'ConnectionStrings__DefaultConnection' + } + ] + secretValues: secretValues + } } -} +] -output webUrl string = 'https://${web.outputs.fqdn}' -output grafanaUrl string = 'https://${grafana.outputs.fqdn}' -output prometheusInternalUrl string = prometheusInternalUrl -output redisAppName string = redis.outputs.name +output prometheusInternalUrl string = 'http://tangle-study-prometheus' +output redisAppName string = contains(containerApps, 'tangle-study-redis') ? 'tangle-study-redis' : '' output blobEndpoint string = storage.outputs.blobEndpoint output appInsightsConnectionString string = appInsights.outputs.connectionString output containerAppsEnvironmentId string = containerAppsEnv.outputs.id -output migrateJobName string = migrateJob.outputs.name -output containerAppNames object = { - api: api.outputs.name - web: web.outputs.name - redis: redis.outputs.name - workerChat: workerChat.outputs.name - workerMedia: workerMedia.outputs.name - workerLocation: workerLocation.outputs.name - postgresExporter: postgresExporter.outputs.name - redisExporter: redisExporter.outputs.name - prometheus: prometheus.outputs.name - grafana: grafana.outputs.name -} +output migrateJobNames array = [for job in migrateJobs: job.name] +output containerAppNames array = [for item in appEntries: item.key] diff --git a/infra/azure/modules/container-app.bicep b/infra/azure/modules/container-app.bicep index 1f9cb13a..493b0a94 100644 --- a/infra/azure/modules/container-app.bicep +++ b/infra/azure/modules/container-app.bicep @@ -11,15 +11,20 @@ param minReplicas int = 1 param maxReplicas int = 3 param cpu string = '0.5' param memory string = '1Gi' -param envVars array = [] -@description('Secret refs: { name, envName, value? }. Omit value for CD-injected secrets.') +param env object = {} +@description('Secret refs: { name, envName }. Values come from secretValues by name.') param secretEnvVars array = [] +@description('Map of secret name → value (deduped).') +@secure() +param secretValues object = {} param registryLoginServer string = '' param registryUsername string = '' @secure() param registryPassword string = '' @description('HTTP path for liveness/readiness. Leave empty to skip probes (placeholder images).') param healthCheckPath string = '' +@description('Extra env vars merged after env (e.g. computed blob endpoint).') +param extraEnvVars array = [] param tags object = {} var hasRegistry = !empty(registryLoginServer) && !empty(registryUsername) && !empty(registryPassword) @@ -54,9 +59,17 @@ var registrySecrets = hasRegistry ? [ } ] : [] -var appSecretDefinitions = [for item in secretEnvVars: { - name: item.name - value: item.?value ?? 'pending-deploy' +var envVars = [for item in items(env): { + name: item.key + value: string(item.value) +}] + +var allSecretNames = [for item in secretEnvVars: item.name] +var uniqueSecretNames = empty(allSecretNames) ? [] : union(allSecretNames, allSecretNames) + +var appSecretDefinitions = [for secretName in uniqueSecretNames: { + name: secretName + value: secretValues[?secretName] ?? 'pending-deploy' }] var secretEnvMappings = [for item in secretEnvVars: { @@ -100,7 +113,7 @@ resource app 'Microsoft.App/containerApps@2026-01-01' = { cpu: json(cpu) memory: memory } - env: concat(envVars, secretEnvMappings) + env: concat(envVars, extraEnvVars, secretEnvMappings) probes: probes } ] diff --git a/infra/azure/modules/migrate-job.bicep b/infra/azure/modules/migrate-job.bicep index a4ce257e..575600c5 100644 --- a/infra/azure/modules/migrate-job.bicep +++ b/infra/azure/modules/migrate-job.bicep @@ -5,8 +5,12 @@ param managedEnvironmentId string param containerImage string param cpu string = '0.5' param memory string = '1Gi' -param envVars array = [] +param env object = {} param secretEnvVars array = [] +@secure() +param secretValues object = {} +@description('Container command, e.g. ["dotnet", "Users.dll", "--migrate"].') +param command array = ['dotnet', 'Users.dll', '--migrate'] param registryLoginServer string = '' param registryUsername string = '' @secure() @@ -22,9 +26,17 @@ var registrySecrets = hasRegistry ? [ } ] : [] -var appSecretDefinitions = [for item in secretEnvVars: { - name: item.name - value: item.?value ?? 'pending-deploy' +var envVars = [for item in items(env): { + name: item.key + value: string(item.value) +}] + +var allSecretNames = [for item in secretEnvVars: item.name] +var uniqueSecretNames = empty(allSecretNames) ? [] : union(allSecretNames, allSecretNames) + +var appSecretDefinitions = [for secretName in uniqueSecretNames: { + name: secretName + value: secretValues[?secretName] ?? 'pending-deploy' }] var secretEnvMappings = [for item in secretEnvVars: { @@ -61,11 +73,7 @@ resource job 'Microsoft.App/jobs@2024-03-01' = { memory: memory } env: concat(envVars, secretEnvMappings) - command: [ - 'dotnet' - 'Api.dll' - '--migrate' - ] + command: command } ] } diff --git a/infra/azure/monitoring/prometheus/prometheus-start.sh b/infra/azure/monitoring/prometheus/prometheus-start.sh index b7401ef5..30786379 100644 --- a/infra/azure/monitoring/prometheus/prometheus-start.sh +++ b/infra/azure/monitoring/prometheus/prometheus-start.sh @@ -1,5 +1,5 @@ #!/bin/sh -# Generate ACA scrape configs using short app names (see scripts/cd/libs/azure-aca-urls.sh). +# Generate ACA scrape configs using short app names (MSA services + workers + exporters). set -e : "${METRICS_SCRAPE_SECRET:?METRICS_SCRAPE_SECRET is required}" @@ -8,9 +8,72 @@ mkdir -p /etc/prometheus/scrape cat > /etc/prometheus/scrape/aca.yml < containerApps.tangle-study-web.env). -# ACA: use short app name only (tangle-study-api), NOT tangle-study-api:8080. -# See infra/azure/README.md#aca-http-short-names-do-not-append-targetport. +# ACA: use short app name only (tangle-study-gateway), NOT tangle-study-gateway:8080. +# See infra/azure/README.md#aca-short-names-do-not-append-targetport. # Only these two are passed to envsubst; all other $vars below are native # nginx variables and must NOT use ${...} syntax or envsubst will blank them. diff --git a/scripts/azure-deploy-infra.sh b/scripts/azure-deploy-infra.sh index e8623cc2..8f1521a5 100755 --- a/scripts/azure-deploy-infra.sh +++ b/scripts/azure-deploy-infra.sh @@ -1,12 +1,16 @@ #!/usr/bin/env bash # Deploy Tangle Azure infrastructure with Bicep (study / free-tier friendly). # -# Postgres: Neon (external) — connection string injected by CD, not Bicep. +# Postgres: Neon (external) — connection string injected as a secure Bicep param. # Redis + monitoring exporters run as Container Apps; app images from GHCR. +# parameters.prod.json is the single source of truth for containerApps + migrateJobs. # # Prerequisites: Azure CLI (`az`), logged in (`az login`). # # Production (matches GitHub Environment `prod` / CD on main): +# POSTGRES_CONNECTION_STRING=... BLOB_CONNECTION_STRING=... JWT_SECRET=... \ +# WORKER_CALLBACK_SECRET=... METRICS_SCRAPE_SECRET=... GRAFANA_ADMIN_PASSWORD=... \ +# GATEWAY_SECRET=... INTERNAL_SERVICE_SECRET=... \ # ./scripts/azure-deploy-infra.sh prod # set -euo pipefail @@ -16,13 +20,12 @@ cd "$ROOT" LOG_PREFIX="[DEPLOY][INFRA]" # shellcheck source=scripts/shared/common.sh source "$ROOT/scripts/shared/common.sh" -# shellcheck source=scripts/ci/libs/versions-prod-env.sh -source "$ROOT/scripts/ci/libs/versions-prod-env.sh" -load_versions_prod_env "$ROOT" TARGET="${1:-}" LOCATION="${AZURE_LOCATION:-eastus}" SUBSCRIPTION="${AZURE_SUBSCRIPTION_ID:-}" +IMAGE_TAG="${IMAGE_TAG:-latest}" +USE_PLACEHOLDER_IMAGES="${USE_PLACEHOLDER_IMAGES:-true}" if [[ -n "$SUBSCRIPTION" ]]; then az account set --subscription "$SUBSCRIPTION" @@ -38,9 +41,16 @@ Usage: ./scripts/azure-deploy-infra.sh Environment variables: AZURE_SUBSCRIPTION_ID Optional subscription override AZURE_LOCATION Azure region (default: eastus) - COMPOSE_ENV_FILE Optional — pinned infra tags (default: docker/versions.prod.env) + IMAGE_TAG Container image tag (default: latest) + USE_PLACEHOLDER_IMAGES true|false (default: true for manual bootstrap) GHCR_REGISTRY_USERNAME Optional — prefer GitHub secret + CD inject for private GHCR GHCR_REGISTRY_PASSWORD Optional — GitHub PAT with read:packages + +Secure params (required for a usable stack; optional for placeholder-only bootstrap): + POSTGRES_CONNECTION_STRING, BLOB_CONNECTION_STRING, JWT_SECRET, + WORKER_CALLBACK_SECRET, METRICS_SCRAPE_SECRET, GRAFANA_ADMIN_PASSWORD, + GATEWAY_SECRET, INTERNAL_SERVICE_SECRET + PLACES_API_KEY, APPLICATIONINSIGHTS_CONNECTION_STRING (optional) EOF } @@ -48,12 +58,27 @@ deploy_env() { local rg="$1" local parameters_file="$2" + [[ -f "$parameters_file" ]] || fail "missing parameter file: $parameters_file" + + local postgres_dsn="" + if [[ -n "${POSTGRES_CONNECTION_STRING:-}" ]]; then + postgres_dsn="$(python3 "$ROOT/scripts/cd/libs/parse_postgres_conn.py" <<< "$POSTGRES_CONNECTION_STRING")" + fi + local extra_params=( - --parameters "redisImage=${REDIS_IMAGE}" - --parameters "prometheusImage=${PROMETHEUS_IMAGE}" - --parameters "grafanaImage=${GRAFANA_IMAGE}" - --parameters "postgresExporterImage=${POSTGRES_EXPORTER_IMAGE}" - --parameters "redisExporterImage=${REDIS_EXPORTER_IMAGE}" + --parameters "imageTag=${IMAGE_TAG}" + --parameters "usePlaceholderImages=${USE_PLACEHOLDER_IMAGES}" + --parameters "postgresConnectionString=${POSTGRES_CONNECTION_STRING:-}" + --parameters "postgresExporterDsn=${postgres_dsn}" + --parameters "blobConnectionString=${BLOB_CONNECTION_STRING:-}" + --parameters "jwtSecret=${JWT_SECRET:-}" + --parameters "workerCallbackSecret=${WORKER_CALLBACK_SECRET:-}" + --parameters "metricsScrapeSecret=${METRICS_SCRAPE_SECRET:-}" + --parameters "placesApiKey=${PLACES_API_KEY:-}" + --parameters "applicationInsightsConnectionString=${APPLICATIONINSIGHTS_CONNECTION_STRING:-}" + --parameters "grafanaAdminPassword=${GRAFANA_ADMIN_PASSWORD:-}" + --parameters "gatewaySecret=${GATEWAY_SECRET:-}" + --parameters "internalServiceSecret=${INTERNAL_SERVICE_SECRET:-}" ) if [[ -n "${GHCR_REGISTRY_USERNAME:-}" && -n "${GHCR_REGISTRY_PASSWORD:-}" ]]; then From baef51182b7924907895adea4ca2d35b7b8a63ab Mon Sep 17 00:00:00 2001 From: Hoon Date: Fri, 10 Jul 2026 15:49:23 +0900 Subject: [PATCH 3/4] feat: replace CD image inject/deploy scripts with Bicep-based cd-v2 workflow --- .github/workflows/{cd-v1.yml => cd-v2.yml} | 32 ++-- scripts/cd/azure-cd-build-push.sh | 19 +++ scripts/cd/azure-cd-deploy-bicep.sh | 94 +++++++++++ scripts/cd/azure-cd-deploy-image.sh | 154 ------------------ scripts/cd/azure-cd-inject-secrets.sh | 144 ---------------- scripts/cd/azure-cd-migrate.sh | 145 ++++++++++------- scripts/cd/azure-cd-wait-image.sh | 19 ++- scripts/cd/libs/azure-aca-urls.sh | 10 +- .../cd/libs/azure-container-apps-readiness.sh | 2 +- workers/config/workers.yml | 2 +- 10 files changed, 231 insertions(+), 390 deletions(-) rename .github/workflows/{cd-v1.yml => cd-v2.yml} (89%) create mode 100644 scripts/cd/azure-cd-deploy-bicep.sh delete mode 100755 scripts/cd/azure-cd-deploy-image.sh delete mode 100755 scripts/cd/azure-cd-inject-secrets.sh diff --git a/.github/workflows/cd-v1.yml b/.github/workflows/cd-v2.yml similarity index 89% rename from .github/workflows/cd-v1.yml rename to .github/workflows/cd-v2.yml index a34ceb52..58e7cf9c 100644 --- a/.github/workflows/cd-v1.yml +++ b/.github/workflows/cd-v2.yml @@ -25,7 +25,7 @@ jobs: deploy: name: Deploy to production runs-on: ubuntu-latest - timeout-minutes: 60 + timeout-minutes: 90 environment: prod if: > @@ -90,7 +90,7 @@ jobs: run: bash scripts/cd/azure-cd-build-push.sh ############################################ - # 4. IMAGE PROPAGATION WAIT + # 5. IMAGE PROPAGATION WAIT ############################################ - name: Wait for GHCR image propagation env: @@ -99,7 +99,7 @@ jobs: run: bash scripts/cd/azure-cd-wait-image.sh ############################################ - # 5. AZURE LOGIN + # 6. AZURE LOGIN ############################################ - name: Azure login (OIDC) uses: azure/login@v3 @@ -109,36 +109,32 @@ jobs: subscription-id: ${{ vars.AZURE_SUBSCRIPTION_ID }} ############################################ - # 6. SECRETS + # 7. BICEP DEPLOY ############################################ - - name: Inject secrets + - name: Deploy infrastructure and apps (Bicep) env: AZURE_RESOURCE_GROUP: ${{ vars.AZURE_RESOURCE_GROUP }} + IMAGE_TAG: ${{ steps.meta.outputs.sha }} + USE_PLACEHOLDER_IMAGES: "false" + POSTGRES_CONNECTION_STRING: ${{ secrets.POSTGRES_CONNECTION_STRING }} BLOB_CONNECTION_STRING: ${{ secrets.BLOB_CONNECTION_STRING }} JWT_SECRET: ${{ secrets.JWT_SECRET }} WORKER_CALLBACK_SECRET: ${{ secrets.WORKER_CALLBACK_SECRET }} METRICS_SCRAPE_SECRET: ${{ secrets.METRICS_SCRAPE_SECRET }} - POSTGRES_CONNECTION_STRING: ${{ secrets.POSTGRES_CONNECTION_STRING }} GRAFANA_ADMIN_PASSWORD: ${{ secrets.GRAFANA_ADMIN_PASSWORD }} + GATEWAY_SECRET: ${{ secrets.GATEWAY_SECRET }} + INTERNAL_SERVICE_SECRET: ${{ secrets.INTERNAL_SERVICE_SECRET }} APPLICATIONINSIGHTS_CONNECTION_STRING: ${{ secrets.APPLICATIONINSIGHTS_CONNECTION_STRING }} PLACES_API_KEY: ${{ secrets.PLACES_API_KEY }} - run: bash scripts/cd/azure-cd-inject-secrets.sh - - ############################################ - # 7. DEPLOY IMAGES - ############################################ - - name: Deploy images on container apps - env: - AZURE_RESOURCE_GROUP: ${{ vars.AZURE_RESOURCE_GROUP }} - CONTAINER_REGISTRY: ${{ steps.meta.outputs.registry }} - IMAGE_TAG: ${{ steps.meta.outputs.sha }} + GHCR_REGISTRY_USERNAME: ${{ secrets.GHCR_REGISTRY_USERNAME }} + GHCR_REGISTRY_PASSWORD: ${{ secrets.GHCR_REGISTRY_PASSWORD }} JWT_EXPIRY_MINUTES: ${{ vars.JWT_EXPIRY_MINUTES }} - run: bash scripts/cd/azure-cd-deploy-image.sh + run: bash scripts/cd/azure-cd-deploy-bicep.sh ############################################ # 8. MIGRATION ############################################ - - name: Run EF migrate + - name: Run EF migrate jobs if: github.event_name == 'workflow_run' || inputs.skip_migrate != true env: AZURE_RESOURCE_GROUP: ${{ vars.AZURE_RESOURCE_GROUP }} diff --git a/scripts/cd/azure-cd-build-push.sh b/scripts/cd/azure-cd-build-push.sh index 6c9859dc..e16dc578 100755 --- a/scripts/cd/azure-cd-build-push.sh +++ b/scripts/cd/azure-cd-build-push.sh @@ -50,6 +50,14 @@ bash "$ROOT/scripts/ci/dotnet-publish.sh" log_step "BUILD IMAGES" IMAGES=( + tangle-study-gateway + tangle-study-users + tangle-study-media + tangle-study-chat + tangle-study-location + tangle-study-community + tangle-study-group + tangle-study-social tangle-study-web tangle-study-worker-media tangle-study-worker-chat @@ -59,6 +67,14 @@ IMAGES=( ) declare -A DOCKERFILE_MAP=( + [tangle-study-gateway]="services/Gateway/Dockerfile.runtime" + [tangle-study-users]="services/Users/Dockerfile.runtime" + [tangle-study-media]="services/Media/Dockerfile.runtime" + [tangle-study-chat]="services/Chat/Dockerfile.runtime" + [tangle-study-location]="services/Location/Dockerfile.runtime" + [tangle-study-community]="services/Community/Dockerfile.runtime" + [tangle-study-group]="services/Group/Dockerfile.runtime" + [tangle-study-social]="services/Social/Dockerfile.runtime" [tangle-study-web]="clients/web/Dockerfile" [tangle-study-worker-media]="workers/docker/Dockerfile.runtime.media" [tangle-study-worker-chat]="workers/docker/Dockerfile.runtime.chat" @@ -72,6 +88,9 @@ set_build_args() { BUILD_ARGS=() case "$image" in + tangle-study-gateway|tangle-study-users|tangle-study-media|tangle-study-chat|tangle-study-location|tangle-study-community|tangle-study-group|tangle-study-social) + BUILD_ARGS+=("--build-arg" "DOTNET_ASPNET_IMAGE=$DOTNET_ASPNET") + ;; tangle-study-web) BUILD_ARGS+=( "--build-arg" "NODE_IMAGE=$NODE_IMG" diff --git a/scripts/cd/azure-cd-deploy-bicep.sh b/scripts/cd/azure-cd-deploy-bicep.sh new file mode 100644 index 00000000..c7efe909 --- /dev/null +++ b/scripts/cd/azure-cd-deploy-bicep.sh @@ -0,0 +1,94 @@ +#!/usr/bin/env bash +# Idempotent Azure Container Apps deploy via Bicep. +# parameters.prod.json is the single source of truth for apps, env, and migrate jobs. +# Secure values are passed as Bicep @secure() parameters (never written into the JSON file). +# +# Usage (CD): +# AZURE_RESOURCE_GROUP=tangle-study-prod IMAGE_TAG= \ +# POSTGRES_CONNECTION_STRING=... BLOB_CONNECTION_STRING=... ... \ +# bash scripts/cd/azure-cd-deploy-bicep.sh +# +set -euo pipefail + +ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)" +cd "$ROOT" +LOG_PREFIX="[DEPLOY][INFRA]" +# shellcheck source=scripts/shared/common.sh +source "$ROOT/scripts/shared/common.sh" + +require_env AZURE_RESOURCE_GROUP +require_env IMAGE_TAG +require_env POSTGRES_CONNECTION_STRING +require_env BLOB_CONNECTION_STRING +require_env JWT_SECRET +require_env WORKER_CALLBACK_SECRET +require_env METRICS_SCRAPE_SECRET +require_env GRAFANA_ADMIN_PASSWORD +require_env GATEWAY_SECRET +require_env INTERNAL_SERVICE_SECRET + +PARAM_FILE="${PARAM_FILE:-infra/azure/parameters.prod.json}" +TEMPLATE_FILE="${TEMPLATE_FILE:-infra/azure/main.bicep}" +USE_PLACEHOLDER_IMAGES="${USE_PLACEHOLDER_IMAGES:-false}" + +[[ -f "$PARAM_FILE" ]] || fail "missing parameter file: $PARAM_FILE" +[[ -f "$TEMPLATE_FILE" ]] || fail "missing template file: $TEMPLATE_FILE" + +############################################ +log_step "RESOLVE DERIVED SECRETS" + +POSTGRES_EXPORTER_DSN="$(python3 "$ROOT/scripts/cd/libs/parse_postgres_conn.py" <<< "$POSTGRES_CONNECTION_STRING")" +[[ -n "$POSTGRES_EXPORTER_DSN" ]] || fail "failed to derive postgres exporter DSN" + +if [[ -z "${APPLICATIONINSIGHTS_CONNECTION_STRING:-}" ]]; then + log_info "fetching application insights connection string" + APPLICATIONINSIGHTS_CONNECTION_STRING="$(az monitor app-insights component show \ + --app "${APP_INSIGHTS_NAME:-tanglestudyprod-appi}" \ + --resource-group "$AZURE_RESOURCE_GROUP" \ + --query connectionString -o tsv 2>/dev/null || true)" +fi + +APPLICATIONINSIGHTS_CONNECTION_STRING="${APPLICATIONINSIGHTS_CONNECTION_STRING:-}" + +############################################ +log_step "DEPLOY BICEP" + +log_info "rg=$AZURE_RESOURCE_GROUP template=$TEMPLATE_FILE param-file=$PARAM_FILE imageTag=$IMAGE_TAG" + +extra_params=( + --parameters "imageTag=${IMAGE_TAG}" + --parameters "usePlaceholderImages=${USE_PLACEHOLDER_IMAGES}" + --parameters "postgresConnectionString=${POSTGRES_CONNECTION_STRING}" + --parameters "postgresExporterDsn=${POSTGRES_EXPORTER_DSN}" + --parameters "blobConnectionString=${BLOB_CONNECTION_STRING}" + --parameters "jwtSecret=${JWT_SECRET}" + --parameters "workerCallbackSecret=${WORKER_CALLBACK_SECRET}" + --parameters "metricsScrapeSecret=${METRICS_SCRAPE_SECRET}" + --parameters "placesApiKey=${PLACES_API_KEY:-}" + --parameters "applicationInsightsConnectionString=${APPLICATIONINSIGHTS_CONNECTION_STRING}" + --parameters "grafanaAdminPassword=${GRAFANA_ADMIN_PASSWORD}" + --parameters "gatewaySecret=${GATEWAY_SECRET}" + --parameters "internalServiceSecret=${INTERNAL_SERVICE_SECRET}" +) + +if [[ -n "${GHCR_REGISTRY_USERNAME:-}" && -n "${GHCR_REGISTRY_PASSWORD:-}" ]]; then + extra_params+=( + --parameters "registryUsername=${GHCR_REGISTRY_USERNAME}" + --parameters "registryPassword=${GHCR_REGISTRY_PASSWORD}" + ) +fi + +if [[ -n "${JWT_EXPIRY_MINUTES:-}" ]]; then + log_info "JWT_EXPIRY_MINUTES is set (${JWT_EXPIRY_MINUTES}); ensure parameters.prod.json Jwt__ExpiryMinutes matches or update users env after deploy" +fi + +az deployment group create \ + --resource-group "$AZURE_RESOURCE_GROUP" \ + --template-file "$TEMPLATE_FILE" \ + --parameters "@${PARAM_FILE}" \ + "${extra_params[@]}" \ + --output table + +############################################ +log_step "BICEP DEPLOY COMPLETE" +log_info "status=completed imageTag=$IMAGE_TAG" diff --git a/scripts/cd/azure-cd-deploy-image.sh b/scripts/cd/azure-cd-deploy-image.sh deleted file mode 100755 index 299915a0..00000000 --- a/scripts/cd/azure-cd-deploy-image.sh +++ /dev/null @@ -1,154 +0,0 @@ -#!/usr/bin/env bash -set -euo pipefail - -ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)" -LOG_PREFIX="[DEPLOY]" -# shellcheck source=scripts/shared/common.sh -source "$ROOT/scripts/shared/common.sh" - -require_env AZURE_RESOURCE_GROUP - -PARAM_FILE="${PARAM_FILE:-infra/azure/parameters.prod.json}" - -[[ -f "$PARAM_FILE" ]] || fail "parameter file not found: $PARAM_FILE" - -CONTAINER_REGISTRY="${CONTAINER_REGISTRY:-$(jq -r '.parameters.containerRegistry.value // empty' "$PARAM_FILE")}" -IMAGE_TAG="${IMAGE_TAG:-$(jq -r '.parameters.imageTag.value // empty' "$PARAM_FILE")}" - -log_step "INTEGRATED APPLICATION DEPLOYMENT START" -log_info "resource-group=$AZURE_RESOURCE_GROUP param-file=$PARAM_FILE" - -# shellcheck source=scripts/cd/libs/azure-aca-urls.sh -source "$ROOT/scripts/cd/libs/azure-aca-urls.sh" -log_info "aca-api-url=${ACA_API_HTTP_URL} aca-prometheus-url=${ACA_PROMETHEUS_HTTP_URL} aca-redis=${ACA_REDIS_HOST}" - -append_monitoring_env() { - local app_name="$1" - case "$app_name" in - tangle-study-grafana) - env_args+=("PROMETHEUS_URL=${ACA_PROMETHEUS_HTTP_URL}") - ;; - esac -} - -append_api_env() { - local app_name="$1" - local arg filtered=() - - [[ "$app_name" == "tangle-study-api" ]] || return 0 - - env_args+=("Redis__ConnectionString=${ACA_REDIS_ADDR}") - - [[ -n "${JWT_EXPIRY_MINUTES:-}" ]] || return 0 - - for arg in "${env_args[@]}"; do - [[ "$arg" == Jwt__ExpiryMinutes=* ]] && continue - filtered+=("$arg") - done - env_args=("${filtered[@]}") - env_args+=("Jwt__ExpiryMinutes=${JWT_EXPIRY_MINUTES}") -} - -append_worker_env() { - local app_name="$1" - - case "$app_name" in - tangle-study-worker-*) - ;; - *) - return 0 - ;; - esac - - env_args+=("REDIS_URL=${ACA_REDIS_URL}") - - case "$app_name" in - tangle-study-worker-media|tangle-study-worker-location) - env_args+=("API_BASE_URL=${ACA_API_HTTP_URL}") - ;; - esac -} - -append_infra_env() { - local app_name="$1" - case "$app_name" in - tangle-study-redis-exporter) - env_args+=("REDIS_ADDR=${ACA_REDIS_ADDR}") - ;; - esac -} - -############################################ -log_step "PROCESSING CONTAINER APPS" - -jq -c '.parameters.containerApps | to_entries[]' "$PARAM_FILE" \ - | while read -r app; do - name=$(echo "$app" | jq -r '.key') - log_step "DEPLOYING: $name" - - image=$(echo "$app" | jq -r '.value.image // empty') - infra_key=$(echo "$app" | jq -r '.value.infraImage // empty') - ref="" - - if [[ -n "$image" ]]; then - [[ -n "$CONTAINER_REGISTRY" && "$CONTAINER_REGISTRY" != "null" ]] || fail "[$name] containerRegistry is missing or null" - [[ -n "$IMAGE_TAG" && "$IMAGE_TAG" != "null" ]] || fail "[$name] imageTag is missing or null" - ref="${CONTAINER_REGISTRY}/${image}:${IMAGE_TAG}" - elif [[ -n "$infra_key" ]]; then - ref=$(jq -r --arg k "$infra_key" '.parameters.infra.value[$k].image // empty' "$PARAM_FILE") - [[ -n "$ref" && "$ref" != "null" ]] || fail "[$name] infra image not found for key: $infra_key" - fi - - env_json=$(echo "$app" | jq -c '.value.env // {}') - env_args=() - - while IFS="=" read -r k v; do - if [[ -n "$k" ]]; then - env_args+=("${k}=${v}") - fi - done < <(echo "$env_json" | jq -r 'to_entries[] | "\(.key)=\(.value)"') - - append_monitoring_env "$name" - append_api_env "$name" - append_worker_env "$name" - append_infra_env "$name" - - cmd=("az" "containerapp" "update" "--name" "$name" "--resource-group" "$AZURE_RESOURCE_GROUP" "--output" "none") - - if [[ -n "$ref" ]]; then - log_info "target image: $ref" - cmd+=("--image" "$ref") - fi - - if [[ ${#env_args[@]} -gt 0 ]]; then - log_info "target envs: ${#env_args[@]} variables detected" - cmd+=("--set-env-vars" "${env_args[@]}") - fi - - if [[ -n "$ref" || ${#env_args[@]} -gt 0 ]]; then - log_info "deploying single-shot revision" - "${cmd[@]}" - log_info "successfully deployed: $name" - else - log_warn "no image or env configuration found; skipping: $name" - fi - done - -############################################ -log_step "DEPLOYING: tangle-study-migrate" - -MIGRATE_IMAGE="${CONTAINER_REGISTRY}/tangle-study-api:${IMAGE_TAG}" - -log_info "target image: $MIGRATE_IMAGE" - -az containerapp job update \ - --name tangle-study-migrate \ - --resource-group "$AZURE_RESOURCE_GROUP" \ - --image "$MIGRATE_IMAGE" \ - --output none - -log_info "successfully deployed job: tangle-study-migrate" - -############################################ -log_step "DEPLOYMENT SUMMARY" -log_info "status=completed" diff --git a/scripts/cd/azure-cd-inject-secrets.sh b/scripts/cd/azure-cd-inject-secrets.sh deleted file mode 100755 index efe3a81c..00000000 --- a/scripts/cd/azure-cd-inject-secrets.sh +++ /dev/null @@ -1,144 +0,0 @@ -#!/usr/bin/env bash -set -euo pipefail - -ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)" -LOG_PREFIX="[DEPLOY]" -# shellcheck source=scripts/shared/common.sh -source "$ROOT/scripts/shared/common.sh" - -############################################ -log_step "SECRET INJECTION START" - -log_info "resource-group=$AZURE_RESOURCE_GROUP" -log_info "target=azure-container-apps" - -############################################ -log_step "VALIDATING REQUIRED ENVIRONMENT VARIABLES" - -require_env AZURE_RESOURCE_GROUP -require_env POSTGRES_CONNECTION_STRING -require_env BLOB_CONNECTION_STRING -require_env JWT_SECRET -require_env WORKER_CALLBACK_SECRET -require_env METRICS_SCRAPE_SECRET -require_env GRAFANA_ADMIN_PASSWORD - -log_info "environment validation: OK" - -############################################ -log_step "RESOLVING APPLICATION INSIGHTS" - -if [[ -z "${APPLICATIONINSIGHTS_CONNECTION_STRING:-}" ]]; then - log_info "fetching application insights connection string" - APPLICATIONINSIGHTS_CONNECTION_STRING="$(az monitor app-insights component show \ - --app "${APP_INSIGHTS_NAME:-tanglestudyprod-appi}" \ - --resource-group "$AZURE_RESOURCE_GROUP" \ - --query connectionString -o tsv)" -fi - -[[ -n "$APPLICATIONINSIGHTS_CONNECTION_STRING" ]] || fail "app insights resolution failed" -log_info "application insights: RESOLVED" - -############################################ -log_step "BUILDING SECRET PAYLOAD" - -POSTGRES_EXPORTER_DSN="$(python3 "$ROOT/scripts/cd/libs/parse_postgres_conn.py" <<< "$POSTGRES_CONNECTION_STRING")" - -log_info "postgres-dsn: GENERATED" - -############################################ -log_step "APPLYING SECRETS (tangle-study-api)" - -api_secrets=( - "postgres-conn=$POSTGRES_CONNECTION_STRING" - "blob-conn=$BLOB_CONNECTION_STRING" - "jwt-secret=$JWT_SECRET" - "worker-callback=$WORKER_CALLBACK_SECRET" - "metrics-secret=$METRICS_SCRAPE_SECRET" - "appinsights-conn=$APPLICATIONINSIGHTS_CONNECTION_STRING" - "places-api-key=${PLACES_API_KEY:-}" -) - -az containerapp secret set \ - --name tangle-study-api \ - --resource-group "$AZURE_RESOURCE_GROUP" \ - --secrets "${api_secrets[@]}" \ - --output none - -log_info "secrets applied: tangle-study-api" - -############################################ -log_step "APPLYING SECRETS (postgres-exporter)" - -az containerapp secret set \ - --name tangle-study-postgres-exporter \ - --resource-group "$AZURE_RESOURCE_GROUP" \ - --secrets "postgres-dsn=$POSTGRES_EXPORTER_DSN" \ - --output none - -log_info "secrets applied: tangle-study-postgres-exporter" - -############################################ -log_step "APPLYING SECRETS (prometheus)" - -az containerapp secret set \ - --name tangle-study-prometheus \ - --resource-group "$AZURE_RESOURCE_GROUP" \ - --secrets "metrics-secret=$METRICS_SCRAPE_SECRET" \ - --output none - -log_info "secrets applied: tangle-study-prometheus" - -############################################ -log_step "APPLYING SECRETS (grafana)" - -az containerapp secret set \ - --name tangle-study-grafana \ - --resource-group "$AZURE_RESOURCE_GROUP" \ - --secrets "grafana-admin-password=$GRAFANA_ADMIN_PASSWORD" \ - --output none - -log_info "secrets applied: tangle-study-grafana" - -############################################ -log_step "APPLYING SECRETS (workers)" - -apply_worker_secrets() { - local app="$1"; shift - az containerapp secret set \ - --name "$app" \ - --resource-group "$AZURE_RESOURCE_GROUP" \ - --secrets "$@" \ - --output none - log_info "secrets applied: $app" -} - -apply_worker_secrets tangle-study-worker-chat \ - "metrics-secret=$METRICS_SCRAPE_SECRET" - -apply_worker_secrets tangle-study-worker-location \ - "metrics-secret=$METRICS_SCRAPE_SECRET" \ - "worker-callback=$WORKER_CALLBACK_SECRET" - -apply_worker_secrets tangle-study-worker-media \ - "metrics-secret=$METRICS_SCRAPE_SECRET" \ - "blob-conn=$BLOB_CONNECTION_STRING" \ - "worker-callback=$WORKER_CALLBACK_SECRET" - -############################################ -log_step "APPLYING SECRETS (tangle-study-migrate JOB)" - -az containerapp job secret set \ - --name tangle-study-migrate \ - --resource-group "$AZURE_RESOURCE_GROUP" \ - --secrets "postgres-conn=$POSTGRES_CONNECTION_STRING" \ - --output none - -log_info "secrets applied: tangle-study-migrate (job)" - -############################################ -log_step "SECRET INJECTION SUMMARY" - -log_info "all-secrets-applied=true" -log_info "next-step=run update-image.sh / deploy.sh to roll out a new revision referencing these secrets" -log_info "injection-status=COMPLETED" \ No newline at end of file diff --git a/scripts/cd/azure-cd-migrate.sh b/scripts/cd/azure-cd-migrate.sh index db02244c..7f36cae9 100755 --- a/scripts/cd/azure-cd-migrate.sh +++ b/scripts/cd/azure-cd-migrate.sh @@ -1,5 +1,6 @@ #!/usr/bin/env bash -# EF Core Container Apps Job runner: start migration job, poll status, dump logs on failure. +# EF Core Container Apps Job runner: start each migrate job from parameters.prod.json, +# poll status, dump logs on failure. # # Usage: AZURE_RESOURCE_GROUP=tangle-study-prod bash scripts/cd/azure-cd-migrate.sh # @@ -14,83 +15,103 @@ source "$ROOT/scripts/cd/libs/container-app-job-logs.sh" require_env AZURE_RESOURCE_GROUP -JOB_NAME="${MIGRATE_JOB_NAME:-tangle-study-migrate}" +PARAM_FILE="${PARAM_FILE:-infra/azure/parameters.prod.json}" TIMEOUT="${MIGRATE_TIMEOUT_SEC:-600}" RG="$AZURE_RESOURCE_GROUP" +[[ -f "$PARAM_FILE" ]] || fail "missing parameter file: $PARAM_FILE" + +mapfile -t JOB_NAMES < <(jq -r '.parameters.migrateJobs.value[].name' "$PARAM_FILE") +[[ ${#JOB_NAMES[@]} -gt 0 ]] || fail "no migrateJobs found in $PARAM_FILE" + dump_failure() { - local execution_name="$1" + local job_name="$1" + local execution_name="$2" - log_error "dumping-job-logs execution=$execution_name" + log_error "dumping-job-logs job=$job_name execution=$execution_name" dump_container_app_job_logs \ - "$JOB_NAME" \ + "$job_name" \ "$RG" \ "$execution_name" \ - "$JOB_NAME" \ + "$job_name" \ 80 || true } -############################################ -log_step "START MIGRATION JOB" - -log_info "job=$JOB_NAME rg=$RG timeout=${TIMEOUT}s" +run_one_job() { + local job_name="$1" -EXECUTION_NAME="$(az containerapp job start \ - --name "$JOB_NAME" \ - --resource-group "$RG" \ - --query name \ - --output tsv)" - -log_info "execution_started name=$EXECUTION_NAME" - -############################################ -log_step "POLL JOB STATUS" + ############################################ + log_step "START MIGRATION JOB" + log_info "job=$job_name rg=$RG timeout=${TIMEOUT}s" -deadline=$((SECONDS + TIMEOUT)) -status="Running" - -while (( SECONDS < deadline )); do - status="$(az containerapp job execution show \ - --name "$JOB_NAME" \ + local execution_name + execution_name="$(az containerapp job start \ + --name "$job_name" \ --resource-group "$RG" \ - --job-execution-name "$EXECUTION_NAME" \ - --query properties.status \ - --output tsv 2>/dev/null || echo "Running")" - - log_info "execution_status=$status" - - case "$status" in - Succeeded) - log_step "MIGRATION SUCCESS" - log_info "execution=$EXECUTION_NAME status=Succeeded" - exit 0 - ;; - - Failed|Stopped) - log_step "MIGRATION FAILED" - log_error "execution=$EXECUTION_NAME status=$status" - - az containerapp job execution show \ - --name "$JOB_NAME" \ - --resource-group "$RG" \ - --job-execution-name "$EXECUTION_NAME" \ - --query "{name:name,status:properties.status,startTime:properties.startTime,endTime:properties.endTime}" \ - --output json 2>/dev/null | redact_log_stream >&2 || true - - dump_failure "$EXECUTION_NAME" - exit 1 - ;; - esac - - sleep 10 -done + --query name \ + --output tsv)" + + log_info "execution_started name=$execution_name" + + ############################################ + log_step "POLL JOB STATUS" + + local deadline=$((SECONDS + TIMEOUT)) + local status="Running" + + while (( SECONDS < deadline )); do + status="$(az containerapp job execution show \ + --name "$job_name" \ + --resource-group "$RG" \ + --job-execution-name "$execution_name" \ + --query properties.status \ + --output tsv 2>/dev/null || echo "Running")" + + log_info "job=$job_name execution_status=$status" + + case "$status" in + Succeeded) + log_info "migration success job=$job_name execution=$execution_name" + return 0 + ;; + + Failed|Stopped) + log_error "migration failed job=$job_name execution=$execution_name status=$status" + + az containerapp job execution show \ + --name "$job_name" \ + --resource-group "$RG" \ + --job-execution-name "$execution_name" \ + --query "{name:name,status:properties.status,startTime:properties.startTime,endTime:properties.endTime}" \ + --output json 2>/dev/null | redact_log_stream >&2 || true + + dump_failure "$job_name" "$execution_name" + return 1 + ;; + esac + + sleep 10 + done + + log_error "migration timeout job=$job_name execution=$execution_name timeout=${TIMEOUT}s" + dump_failure "$job_name" "$execution_name" + return 1 +} ############################################ -log_step "MIGRATION TIMEOUT" - -log_error "execution=$EXECUTION_NAME status=TIMEOUT timeout=${TIMEOUT}s" +log_step "RUN ALL MIGRATE JOBS" +log_info "jobs=${#JOB_NAMES[@]}" + +failed=0 +for job_name in "${JOB_NAMES[@]}"; do + if ! run_one_job "$job_name"; then + failed=1 + fi +done -dump_failure "$EXECUTION_NAME" +[[ $failed -eq 0 ]] || fail "one or more migrate jobs failed" -exit 1 +############################################ +log_step "MIGRATION SUCCESS" +log_info "all-jobs-succeeded=true" diff --git a/scripts/cd/azure-cd-wait-image.sh b/scripts/cd/azure-cd-wait-image.sh index 7b76bc88..a9c458e5 100755 --- a/scripts/cd/azure-cd-wait-image.sh +++ b/scripts/cd/azure-cd-wait-image.sh @@ -9,16 +9,19 @@ source "$ROOT/scripts/shared/common.sh" require_env CONTAINER_REGISTRY require_env IMAGE_TAG +PARAM_FILE="${PARAM_FILE:-infra/azure/parameters.prod.json}" +[[ -f "$PARAM_FILE" ]] || fail "missing parameter file: $PARAM_FILE" + log_step "GHCR CONSISTENCY GATE" -IMAGES=( - tangle-study-web - tangle-study-worker-media - tangle-study-worker-chat - tangle-study-worker-location - tangle-study-prometheus - tangle-study-grafana -) +mapfile -t IMAGES < <(jq -r ' + .parameters.containerApps.value + | to_entries[] + | select(.value.image != null) + | .value.image +' "$PARAM_FILE") + +[[ ${#IMAGES[@]} -gt 0 ]] || fail "no containerApps with image found in $PARAM_FILE" wait_image() { local image="$1" diff --git a/scripts/cd/libs/azure-aca-urls.sh b/scripts/cd/libs/azure-aca-urls.sh index 70afe1fe..7cc53e11 100644 --- a/scripts/cd/libs/azure-aca-urls.sh +++ b/scripts/cd/libs/azure-aca-urls.sh @@ -5,12 +5,18 @@ # Do not append :8080, :9090, etc. — that hits the pod IP and times out. # # Redis (TCP): use the short name only; clients default to port 6379. +# +# MSA cutover: edge traffic goes to tangle-study-gateway (not the removed monolith api). -ACA_API_HOST="${ACA_API_HOST:-tangle-study-api}" +ACA_GATEWAY_HOST="${ACA_GATEWAY_HOST:-tangle-study-gateway}" ACA_PROMETHEUS_HOST="${ACA_PROMETHEUS_HOST:-tangle-study-prometheus}" ACA_REDIS_HOST="${ACA_REDIS_HOST:-tangle-study-redis}" -ACA_API_HTTP_URL="${ACA_API_HTTP_URL:-http://${ACA_API_HOST}}" +ACA_GATEWAY_HTTP_URL="${ACA_GATEWAY_HTTP_URL:-http://${ACA_GATEWAY_HOST}}" ACA_PROMETHEUS_HTTP_URL="${ACA_PROMETHEUS_HTTP_URL:-http://${ACA_PROMETHEUS_HOST}}" ACA_REDIS_URL="${ACA_REDIS_URL:-redis://${ACA_REDIS_HOST}}" ACA_REDIS_ADDR="${ACA_REDIS_ADDR:-${ACA_REDIS_HOST}}" + +# Back-compat aliases for older scripts that still reference ACA_API_*. +ACA_API_HOST="${ACA_API_HOST:-$ACA_GATEWAY_HOST}" +ACA_API_HTTP_URL="${ACA_API_HTTP_URL:-$ACA_GATEWAY_HTTP_URL}" diff --git a/scripts/cd/libs/azure-container-apps-readiness.sh b/scripts/cd/libs/azure-container-apps-readiness.sh index ee89458c..3b75f0b9 100644 --- a/scripts/cd/libs/azure-container-apps-readiness.sh +++ b/scripts/cd/libs/azure-container-apps-readiness.sh @@ -425,7 +425,7 @@ probe_web_api_health_via_exec() { else cat "$output_file" >&2 fi - log_warn "ACA cross-app HTTP routing is per caller — re-run scripts/cd/azure-cd-deploy-image.sh or scripts/cd/azure-cd-smoke.sh to verify TANGLE_API_UPSTREAM." >&2 + log_warn "ACA cross-app HTTP routing is per caller — re-run scripts/cd/azure-cd-deploy-bicep.sh or scripts/cd/azure-cd-smoke.sh to verify TANGLE_API_UPSTREAM." >&2 log_warn "nginx proxy_set_header Host must match the upstream hostname (see infra/nginx/docker-entrypoint.sh)." >&2 rm -f "$output_file" return 1 diff --git a/workers/config/workers.yml b/workers/config/workers.yml index 9c3f9ca6..27a0e25a 100644 --- a/workers/config/workers.yml +++ b/workers/config/workers.yml @@ -5,7 +5,7 @@ # # Local Docker Compose values: workers/config/compose-workers.yml (included by docker-compose.yml). # Secrets (WORKER_CALLBACK_SECRET, AZURE_STORAGE_CONNECTION_STRING) are never committed for prod; -# Azure CD injects them via scripts/cd/azure-cd-inject-secrets.sh. +# Azure CD passes them as Bicep @secure() params via scripts/cd/azure-cd-deploy-bicep.sh. version: 1 From 6fb8eb3294e51a832ebf6b728c15326d7cd7ef6c Mon Sep 17 00:00:00 2001 From: Hoon Date: Fri, 10 Jul 2026 15:49:28 +0900 Subject: [PATCH 4/4] docs: update deployment and MSA guides for cd-v2 and MSA Container Apps --- .cursor/rules/script-style.mdc | 9 +-- docs/DEPLOYMENT.md | 143 +++++++++++++++------------------ docs/MSA_MIGRATION.md | 59 +++++--------- 3 files changed, 88 insertions(+), 123 deletions(-) diff --git a/.cursor/rules/script-style.mdc b/.cursor/rules/script-style.mdc index 8513ec24..dd2e0765 100644 --- a/.cursor/rules/script-style.mdc +++ b/.cursor/rules/script-style.mdc @@ -1,12 +1,12 @@ --- description: Bash script, CD workflow, and deploy JSON formatting conventions -globs: scripts/**/*.sh,.github/workflows/ci-v2.yml,.github/workflows/cd-v1.yml,infra/azure/parameters*.json +globs: scripts/**/*.sh,.github/workflows/ci-v2.yml,.github/workflows/cd-v2.yml,infra/azure/parameters*.json alwaysApply: false --- # Script, CI/CD workflow, and deploy JSON style -Reference entrypoint: [`scripts/cd/azure-cd-inject-secrets.sh`](scripts/cd/azure-cd-inject-secrets.sh). +Reference entrypoint: [`scripts/cd/azure-cd-deploy-bicep.sh`](scripts/cd/azure-cd-deploy-bicep.sh). ## Bash entrypoints (`scripts/**/*.sh`) @@ -39,8 +39,7 @@ log_step "SECTION NAME IN ALL CAPS" | Scripts | Prefix | |---------|--------| | CD build / wait / migrate / smoke | `[DEPLOY][BUILD]`, `[DEPLOY][IMAGE]`, `[DEPLOY][MIGRATE]`, `[DEPLOY][SMOKE]` | -| CD deploy / secrets | `[DEPLOY]` | -| Infra Bicep deploy | `[DEPLOY][INFRA]` | +| Infra / CD Bicep deploy | `[DEPLOY][INFRA]` | | CI tests / harness / versions | `[CI][TEST]`, `[CI][HARNESS]`, `[CI][VERSIONS]`, `[CI][CACHE]` | | Dev / local tooling | `[DEV]`, `[DOTNET]`, `[MIGRATE]`, `[TEST]` | @@ -57,7 +56,7 @@ Set `LOG_PREFIX` **before** sourcing `common.sh`. Keep `#!/bin/sh`, POSIX-only, no bash `common.sh`. English comments, `set -e`, minimal headers. -## CI and CD workflows ([`ci-v2.yml`](.github/workflows/ci-v2.yml), [`cd-v1.yml`](.github/workflows/cd-v1.yml)) +## CI and CD workflows ([`ci-v2.yml`](.github/workflows/ci-v2.yml), [`cd-v2.yml`](.github/workflows/cd-v2.yml)) Shared conventions: diff --git a/docs/DEPLOYMENT.md b/docs/DEPLOYMENT.md index c6ebda54..f1ea9dcb 100644 --- a/docs/DEPLOYMENT.md +++ b/docs/DEPLOYMENT.md @@ -1,10 +1,10 @@ # Deployment -Azure Container Apps deployment. Secrets are injected from **GitHub Environment secrets** at deploy time via [`cd-v1.yml`](../.github/workflows/cd-v1.yml). +Azure Container Apps deployment. Secrets are passed as Bicep `@secure()` parameters from **GitHub Environment secrets** at deploy time via [`cd-v2.yml`](../.github/workflows/cd-v2.yml). App topology and env live in [`parameters.prod.json`](../infra/azure/parameters.prod.json). **Local development** uses Docker Compose with the **gateway-centric stack** (gateway, users, and all domain services). See [README](../README.md) and [MSA_MIGRATION.md](MSA_MIGRATION.md). -> **Note:** `services/Api/` (legacy monolith) was removed from the repo. Azure Bicep, parameters, migrate job, and several CD scripts still reference `tangle-study-api` until gateway/users + domain Container Apps cutover. CD image build no longer produces `tangle-study-api`. +> **Note:** `services/Api/` (legacy monolith) was removed. Production CD deploys gateway + domain services from `parameters.prod.json`. Delete orphaned `tangle-study-api` / `tangle-study-migrate` resources after the first MSA cutover if they still exist in the resource group. --- @@ -71,11 +71,12 @@ Study-friendly stack — **no managed Redis or ACR** on Azure (no useful free ti | Compose (local) | Azure | | ------------------------------------ | ------------------------------------------- | -| `api` | `tangle-study-api` Container App (monolith) | -| `media` | **Not deployed yet** — still in monolith on Azure | +| `gateway` | `tangle-study-gateway` Container App | +| `users` / `media` / `chat` / … | `tangle-study-` Container Apps | | `nginx` / web | `tangle-study-web` Container App | | `db` | **Neon** (external Postgres) | | `redis` | `tangle-study-redis` Container App | +| `rust-worker-*` | `tangle-study-worker-*` Container Apps | | `prometheus` / `grafana` / exporters | Monitoring Container Apps on ACA | | `azurite` | Storage account (blob) | | Built images | **GHCR** (`ghcr.io//tangle-study/...`) | @@ -148,16 +149,15 @@ Create a Neon project + database and set `POSTGRES_CONNECTION_STRING` in GitHub. ### 4. CD pipeline -After CI passes on `main`, [cd-v1.yml](../.github/workflows/cd-v1.yml): +After CI passes on `main`, [cd-v2.yml](../.github/workflows/cd-v2.yml): -1. Compiles once (`dotnet-publish.sh`, `build-workers-release.sh`), then builds and pushes runtime images to GHCR: `tangle-study-api`, `tangle-study-web`, `tangle-study-worker-media`, `tangle-study-worker-chat`, `tangle-study-worker-location`, plus monitoring images (`[azure-cd-build-push.sh](../scripts/cd/azure-cd-build-push.sh)`) +1. Compiles once (`dotnet-publish.sh`, `build-workers-release.sh`), then builds and pushes runtime images to GHCR: gateway, users, media, chat, location, community, group, social, web, workers, plus monitoring (`[azure-cd-build-push.sh](../scripts/cd/azure-cd-build-push.sh)`) 2. Waits for GHCR image propagation (`[azure-cd-wait-image.sh](../scripts/cd/azure-cd-wait-image.sh)`) -3. Injects secrets into Container Apps (Neon, monitoring, JWT, blob, etc.) (`[azure-cd-inject-secrets.sh](../scripts/cd/azure-cd-inject-secrets.sh)`) -4. Updates app images and env from `[parameters.prod.json](../infra/azure/parameters.prod.json)` (`[azure-cd-deploy-image.sh](../scripts/cd/azure-cd-deploy-image.sh)`) — includes `TANGLE_API_UPSTREAM` for web -5. Runs `tangle-study-migrate` job (`[azure-cd-migrate.sh](../scripts/cd/azure-cd-migrate.sh)`) -6. Smoke tests (`/health` proxied to API + SPA shell) (`[azure-cd-smoke.sh](../scripts/cd/azure-cd-smoke.sh)`) +3. Runs Bicep (`[azure-cd-deploy-bicep.sh](../scripts/cd/azure-cd-deploy-bicep.sh)`) with `@parameters.prod.json`, `imageTag=`, and secure secrets — provisions/updates all Container Apps and migrate jobs +4. Runs per-service migrate jobs (`[azure-cd-migrate.sh](../scripts/cd/azure-cd-migrate.sh)`) +5. Smoke tests (`/health` proxied to gateway + SPA shell) (`[azure-cd-smoke.sh](../scripts/cd/azure-cd-smoke.sh)`) -Manual deploy: **Actions → CD-v1 → Run workflow** (uses `prod` environment). +Manual deploy: **Actions → Deploy → Run workflow** (uses `prod` environment). Optional: add **required reviewers** on the `prod` environment in GitHub for approval gates. @@ -194,49 +194,35 @@ Store these on GitHub Environment `production`. The deploy workflow maps each se ### Legacy monolith API (removed) -The `services/Api/` project and `tangle-study-api` image were removed from the repo. Azure Bicep/parameters below still describe the old monolith deploy — update during gateway/users ACA cutover. Until then, treat this section as historical. +The `services/Api/` project and `tangle-study-api` image were removed from the repo. Production CD (`cd-v2.yml`) deploys the MSA Container Apps from [`parameters.prod.json`](../infra/azure/parameters.prod.json). +### Shared secrets (MSA) | GitHub secret | Container App env var | Required | Notes | | ------------------------------------------ | ----------------------------------------------------------------------------- | -------- | --------------------------------------------------------------------------------------------------------------------- | -| `BLOB_CONNECTION_STRING` | `Media__ConnectionString` | Yes | Azure Storage account connection string | -| `JWT_SECRET` | `Jwt__Secret` | Yes | Min 32 chars; overrides `security.yml` placeholder | -| `JWT_EXPIRY_MINUTES` (GitHub **variable**) | `Jwt__ExpiryMinutes` | No | Token lifetime in minutes; default `15` from `[parameters.prod.json](../infra/azure/parameters.prod.json)` when unset | -| `WORKER_CALLBACK_SECRET` | `Media__WorkerCallbackSecret` | Yes | Shared with media worker for internal callbacks | -| `INTERNAL_SERVICE_SECRET` (planned) | `InternalAccess__Secret` (Api), `MediaClient__InternalSecret` (Api) | When media on ACA | Monolith ↔ media `X-Internal-Secret`; already set in Compose dev | -| `METRICS_SCRAPE_SECRET` | `Metrics__ScrapeSecret` (API), `METRICS_SCRAPE_SECRET` (Prometheus + workers) | Yes | Required when `Metrics:RequireScrapeSecret` is true | -| `POSTGRES_CONNECTION_STRING` | `ConnectionStrings__DefaultConnection` (API + migrate) | Yes | Neon Npgsql connection string from console | +| `BLOB_CONNECTION_STRING` | `Media__ConnectionString` (media), `AZURE_STORAGE_CONNECTION_STRING` (worker-media) | Yes | Azure Storage account connection string | +| `JWT_SECRET` | `Jwt__Secret` (users) | Yes | Min 32 chars; overrides `security.yml` placeholder | +| `JWT_EXPIRY_MINUTES` (GitHub **variable**) | `Jwt__ExpiryMinutes` (users) | No | Default `15` from `[parameters.prod.json](../infra/azure/parameters.prod.json)` when unset | +| `WORKER_CALLBACK_SECRET` | `Media__WorkerCallbackSecret`, `WorkerCallback__Secret`, `WORKER_CALLBACK_SECRET` | Yes | Shared with media/location workers for internal callbacks | +| `GATEWAY_SECRET` | `Gateway__Secret` (gateway), `GatewayIdentity__Secret` (services) | Yes | Trusted `X-Gateway-Secret` between gateway and services | +| `INTERNAL_SERVICE_SECRET` | `InternalAccess__Secret` + `*Client__InternalSecret` / `Users__InternalSecret` | Yes | Shared `X-Internal-Secret` for `/internal/*` service-to-service calls | +| `METRICS_SCRAPE_SECRET` | `Metrics__ScrapeSecret` / `METRICS_SCRAPE_SECRET` | Yes | Required when `Metrics:RequireScrapeSecret` is true | +| `POSTGRES_CONNECTION_STRING` | `ConnectionStrings__DefaultConnection` (DB services + migrate jobs) | Yes | Neon Npgsql connection string from console | | `GRAFANA_ADMIN_PASSWORD` | `GF_SECURITY_ADMIN_PASSWORD` (Grafana) | Yes | External Grafana Container App admin password | -| `PLACES_API_KEY` | `Places__ApiKey` | No | Google Places / Geocoding; leave empty to disable search | +| `PLACES_API_KEY` | `Places__ApiKey` (location) | No | Google Places / Geocoding; leave empty to disable search | ### Media (`services/Media`) -**Compose:** deployed by default (see [Local Compose (media extracted)](#local-compose-media-extracted)). +**Compose + Azure:** deployed via `parameters.prod.json` → `tangle-study-media` (and `tangle-study-migrate-media`). Secrets are listed under [Shared secrets (MSA)](#shared-secrets-msa). Non-secret wiring: -**Azure:** not in Bicep/CD yet. When added, map secrets and wire strangler routing: +| Source | Container App env var | Example | +|--------|----------------------|---------| +| Bicep storage output | `Media__PublicBlobEndpoint` | `https://….blob.core.windows.net/` | +| Bicep storage output | `Media__ContainerName` | `tangle-media` | +| `parameters.prod.json` | `Redis__ConnectionString` | `tangle-study-redis` | +| `parameters.prod.json` | `Users__BaseUrl` | `http://tangle-study-users` | -| GitHub secret | Container App env var | Required | Notes | -| ---------------------------- | ---------------------------------- | -------- | --------------------------------------------------------------------- | -| `JWT_SECRET` | `Jwt__Secret` | Yes | **Same value as monolith** — validates tokens issued by Api login | -| `BLOB_CONNECTION_STRING` | `Media__ConnectionString` | Yes | Azure Storage account connection string | -| `WORKER_CALLBACK_SECRET` | `Media__WorkerCallbackSecret` | Yes | Shared with `worker-media` for `PATCH /internal/media/.../processed` | -| `POSTGRES_CONNECTION_STRING` | `ConnectionStrings__DefaultConnection` | Yes | Shared Neon; media uses `media` schema — run Media EF migrate job | -| `METRICS_SCRAPE_SECRET` | `Metrics__ScrapeSecret` | Yes | When `Metrics:RequireScrapeSecret` is true | -| `INTERNAL_SERVICE_SECRET` | `InternalAccess__Secret` | Yes | Shared `X-Internal-Secret` for `/internal/*`; same value on callers (`MediaClient__InternalSecret`, etc.) | - -Non-secrets at deploy time: - -| Variable / Bicep output | Container App env var | Example | -| ---------------------------- | --------------------------- | --------------------------------------------- | -| — | `Monolith__BaseUrl` | `http://tangle-study-api` (ACA short name) | -| `MEDIA_PUBLIC_BLOB_ENDPOINT` | `Media__PublicBlobEndpoint` | `https://tanglestaging.blob.core.windows.net` | -| — | `Redis__ConnectionString` | `tangle-study-redis` (ACA short name) | - -Queue stream prefix comes from each service's `*-config.yml` (`Redis:WorkQueueStreamPrefix`); override with `Redis__WorkQueueStreamPrefix` if needed. - -**Web nginx (Azure cutover):** point `nginx.production.conf` at gateway/users + domain Container Apps (local Compose uses a single gateway upstream). Until then, production keeps serving `/api/*` from the monolith. - -**Worker:** change `API_BASE_URL` on `tangle-study-worker-media` from `http://tangle-study-api` to `http://tangle-study-media` when the media Container App exists. +Worker media uses `API_BASE_URL=http://tangle-study-media` from `parameters.prod.json`. Web nginx proxies to `tangle-study-gateway`. Non-secrets (`Jwt:Issuer`, `Jwt:Audience`, limits, queue stream prefix) stay in Gateway/Users `security.yml` and each service's `*-config.yml` baked into the image. @@ -268,7 +254,7 @@ Non-secret config can be GitHub **variables** or Bicep parameters: | Variable | Env var | Default | Notes | | -------- | --------------------- | ------------------ | -------------------------------------------------------------- | -| — | `TANGLE_API_UPSTREAM` | `tangle-study-api` | Internal API short name (ACA HTTP ingress port 80; no `:8080`) | +| — | `TANGLE_API_UPSTREAM` | `tangle-study-gateway` | Internal gateway short name (ACA HTTP ingress port 80; no `:8080`) | Build the web image with `--build-arg NGINX_CONF=nginx.production.conf` for Azure (no Azurite proxy). See `[infra/nginx/nginx.production.conf](../infra/nginx/nginx.production.conf)`. @@ -283,14 +269,14 @@ Build the web image with `--build-arg NGINX_CONF=nginx.production.conf` for Azur | `METRICS_SCRAPE_SECRET` | `METRICS_SCRAPE_SECRET` | Yes | Protects `/metrics`; Prometheus sends `X-Metrics-Secret` | -Redis URL and API base URL are set at CD deploy time. **Today (Azure):** `API_BASE_URL=http://tangle-study-api` for media and location workers (monolith still owns those callbacks on Azure). **After ACA cutover:** media worker uses `http://tangle-study-media`; location worker uses `http://tangle-study-location`; chat worker uses `http://tangle-study-chat`. +Redis URL and service base URLs are set in `parameters.prod.json` and applied by Bicep. Workers use `API_BASE_URL=http://tangle-study-` (media → media, location → location, chat → chat). Per-worker settings in `[parameters.prod.json](../infra/azure/parameters.prod.json)`: | Variable | Env var | Example | | ---------------------- | ---------------------- | ------------------------------------------------------------ | -| `API_BASE_URL` | `API_BASE_URL` | Compose: `http://media:8080` (media worker). Azure today: `http://tangle-study-api`; target: `http://tangle-study-media` | +| `API_BASE_URL` | `API_BASE_URL` | Compose: `http://media:8080` (media worker). Azure: `http://tangle-study-media` | | `WORKER_STREAM_KEY` | `WORKER_STREAM_KEY` | `media.uploaded`, `chat.message.created`, `location.cluster` | | `MEDIA_CONTAINER_NAME` | `MEDIA_CONTAINER_NAME` | `tangle-media` | @@ -301,7 +287,7 @@ Per-worker settings in `[parameters.prod.json](../infra/azure/parameters.prod.js ## Database migrations -Production and staging **do not** apply migrations on service startup. Migrations run via `./scripts/migrate.sh` (local) or the Azure migrate job (stale — still references removed `tangle-study-api` image until cutover). +Production and staging **do not** apply migrations on service startup. Migrations run via `./scripts/migrate.sh` (local) or the per-service Azure migrate jobs listed in `parameters.prod.json` → `migrateJobs` (`azure-cd-migrate.sh`). ### Command (local) @@ -328,16 +314,16 @@ ConnectionStrings__DefaultConnection="$POSTGRES_CONNECTION_STRING" \ ### CD step (Container Apps Job) -The deploy workflow runs migrations automatically **after** images are deployed (the job needs the new API image on GHCR and the migrate job updated first): +The deploy workflow runs migrations automatically **after** Bicep applies the new images (each migrate job needs its service image on GHCR): -1. Build and push images to GHCR (API image tag matches app deploy). -2. Inject secrets and deploy all Container App images from `[parameters.prod.json](../infra/azure/parameters.prod.json)` — including the API revision and the migrate job image (`[azure-cd-deploy-image.sh](../scripts/cd/azure-cd-deploy-image.sh)`). -3. Start `tangle-study-migrate` (`[azure-cd-migrate.sh](../scripts/cd/azure-cd-migrate.sh)`); proceed only if execution status is `Succeeded`. +1. Build and push images to GHCR (service image tags match app deploy). +2. Run Bicep from `[parameters.prod.json](../infra/azure/parameters.prod.json)` (`[azure-cd-deploy-bicep.sh](../scripts/cd/azure-cd-deploy-bicep.sh)`) — apps, secrets, and migrate job images. +3. Start each job in `migrateJobs` (`[azure-cd-migrate.sh](../scripts/cd/azure-cd-migrate.sh)`); proceed only if every execution status is `Succeeded`. 4. Smoke test via web ingress (`[azure-cd-smoke.sh](../scripts/cd/azure-cd-smoke.sh)`). Skip on manual runs: **Deploy → Run workflow → Skip EF migrate job**. -Development/Docker still auto-migrate on API startup for local convenience. +Development/Docker still auto-migrate on service startup for local convenience. --- @@ -358,8 +344,8 @@ Container Apps platform logs go to the same Log Analytics workspace. Bicep deploys a **monitoring stack** on Container Apps (Compose parity). **CD** builds custom GHCR images (`tangle-study-prometheus`, `tangle-study-grafana`) that bundle provisioning from `[infra/grafana/provisioning/](../infra/grafana/provisioning/)` and ACA scrape entrypoints from -`[infra/azure/monitoring/](../infra/azure/monitoring/)`. Deploy injects cross-app URLs from -`[azure-aca-urls.sh](../scripts/cd/libs/azure-aca-urls.sh)` via `[azure-cd-deploy-image.sh](../scripts/cd/azure-cd-deploy-image.sh)` +`[infra/azure/monitoring/](../infra/azure/monitoring/)`. Cross-app URLs come from +`[parameters.prod.json](../infra/azure/parameters.prod.json)` via `[azure-cd-deploy-bicep.sh](../scripts/cd/azure-cd-deploy-bicep.sh)` (e.g. `PROMETHEUS_URL=http://tangle-study-prometheus`, `REDIS_URL=redis://tangle-study-redis`). HTTP short app names must omit `targetPort` ([ACA short names](../infra/azure/README.md#aca-short-names-do-not-append-targetport)). @@ -385,7 +371,7 @@ Local Prometheus/Grafana under `[infra/](../infra/)` remain for Docker Compose ( ### Post-deploy smoke tests -After migrate, [cd-v1.yml](../.github/workflows/cd-v1.yml) runs `[scripts/cd/azure-cd-smoke.sh](../scripts/cd/azure-cd-smoke.sh)`: +After migrate, [cd-v2.yml](../.github/workflows/cd-v2.yml) runs `[scripts/cd/azure-cd-smoke.sh](../scripts/cd/azure-cd-smoke.sh)`: 1. Waits for **API and web** Container App revisions to reach `healthState=Healthy` 2. `GET https:///` — SPA shell loads @@ -532,22 +518,25 @@ Only then proceed to [MSA extraction](MSA_MIGRATION.md#extraction-order). or (`sslmode=verify-ca` or `verify-full` also accepted.) 2. Confirm the connection string format (Npgsql or URI) matches Neon console output before setting the GitHub secret. -3. Re-inject and re-run migrate: +3. Re-run Bicep deploy (with corrected secrets) and migrate: ```bash AZURE_RESOURCE_GROUP=tangle-study-prod \ + IMAGE_TAG='...' \ POSTGRES_CONNECTION_STRING='...' \ BLOB_CONNECTION_STRING='...' \ JWT_SECRET='...' \ WORKER_CALLBACK_SECRET='...' \ METRICS_SCRAPE_SECRET='...' \ GRAFANA_ADMIN_PASSWORD='...' \ - bash scripts/cd/azure-cd-inject-secrets.sh + GATEWAY_SECRET='...' \ + INTERNAL_SERVICE_SECRET='...' \ + bash scripts/cd/azure-cd-deploy-bicep.sh AZURE_RESOURCE_GROUP=tangle-study-prod bash scripts/cd/azure-cd-migrate.sh ``` - Or trigger **Actions → CD-v1 → Run workflow**. + Or trigger **Actions → Deploy → Run workflow**. -CD now rejects malformed strings in `[scripts/cd/azure-cd-inject-secrets.sh](../scripts/cd/azure-cd-inject-secrets.sh)` before injecting secrets. Migrate job logs are dumped automatically on failure in `[scripts/cd/azure-cd-migrate.sh](../scripts/cd/azure-cd-migrate.sh)`. +`[azure-cd-deploy-bicep.sh](../scripts/cd/azure-cd-deploy-bicep.sh)` derives the postgres-exporter DSN and rejects empty required secrets. Migrate job logs are dumped automatically on failure in `[scripts/cd/azure-cd-migrate.sh](../scripts/cd/azure-cd-migrate.sh)`. If the password appeared in Log Analytics, rotate it in the Neon console. @@ -559,43 +548,43 @@ If the password appeared in Log Analytics, rotate it in the Neon console. **Symptom:** API logs show `StackExchange.Redis.ConnectionMultiplexer.Connect` failure at startup; revision crash-loops (older builds) or `/health` reports Redis **Unhealthy**. -**Cause:** The Redis **container can be healthy** while **cross-app TCP** from `tangle-study-api` to `tangle-study-redis` fails. Common reasons: +**Cause:** The Redis **container can be healthy** while **cross-app TCP** from a service (e.g. `tangle-study-users`) to `tangle-study-redis` fails. Common reasons: - `tangle-study-redis` deployed **without internal TCP ingress** - Ingress metadata looks correct but ACA Envoy routing is **stale** (same class as historical Postgres TCP issues) -The API connection string uses the **short app name** `tangle-study-redis` (port 6379 implicit; set by CD). +Service connection strings use the **short app name** `tangle-study-redis` (port 6379 implicit; set in `parameters.prod.json`). -**Fix:** Verify Redis ingress and API env, then re-deploy and re-run smoke: +**Fix:** Verify Redis ingress and service env, then re-deploy and re-run smoke: ```bash az containerapp show -n tangle-study-redis -g tangle-study-prod \ --query "properties.configuration.ingress.{transport:transport,targetPort:targetPort,fqdn:fqdn}" -o yaml -az containerapp show -n tangle-study-api -g tangle-study-prod \ +az containerapp show -n tangle-study-users -g tangle-study-prod \ --query "properties.template.containers[0].env[?name=='Redis__ConnectionString']" -o yaml -AZURE_RESOURCE_GROUP=tangle-study-prod bash scripts/cd/azure-cd-deploy-image.sh +AZURE_RESOURCE_GROUP=tangle-study-prod IMAGE_TAG='...' bash scripts/cd/azure-cd-deploy-bicep.sh AZURE_RESOURCE_GROUP=tangle-study-prod ./scripts/cd/azure-cd-smoke.sh ``` Manual cross-app probe from your laptop (use short name `tangle-study-redis`): ```bash -az containerapp exec -n tangle-study-api -g tangle-study-prod \ - --container tangle-study-api \ +az containerapp exec -n tangle-study-users -g tangle-study-prod \ + --container tangle-study-users \ --command "timeout 5 bash -c 'echo > /dev/tcp/tangle-study-redis/6379' && echo OK || echo FAIL" ``` From CI or a non-interactive shell, wrap exec: ```bash -script -q -c "az containerapp exec -n tangle-study-api -g tangle-study-prod \ - --container tangle-study-api \ +script -q -c "az containerapp exec -n tangle-study-users -g tangle-study-prod \ + --container tangle-study-users \ --command \"timeout 5 bash -c 'echo > /dev/tcp/tangle-study-redis/6379' && echo OK || echo FAIL\"" /dev/null ``` -`Redis__ConnectionString` includes StackExchange.Redis options (`abortConnect=false`, connect timeouts). The API sets `AbortOnConnectFail=false` so a transient Redis blip does not crash-loop; `/health` shows **Unhealthy** until Redis is reachable. +`Redis__ConnectionString` includes StackExchange.Redis options (`abortConnect=false`, connect timeouts). Services set `AbortOnConnectFail=false` so a transient Redis blip does not crash-loop; `/health` shows **Unhealthy** until Redis is reachable. Fresh infra deploys pick up TCP ingress from `[infra/azure/modules/infra-container.bicep](../infra/azure/modules/infra-container.bicep)` when `tcpProbePort` is set (6379 for Redis). @@ -688,27 +677,27 @@ SMOKE_SKIP_API_WAIT=1 AZURE_RESOURCE_GROUP=tangle-study-prod ./scripts/cd/azure- **Symptom:** `[azure-cd-smoke.sh](../scripts/cd/azure-cd-smoke.sh)` fails; web nginx logs show `GET /health HTTP/1.1" 404`. -**Cause:** Web nginx proxies `/health` to the internal API (`TANGLE_API_UPSTREAM`, default `tangle-study-api`). ACA HTTP ingress listens on port 80 for short app names — do **not** append `:8080` (connects to pod IP and times out). Infra deployed with **placeholder images** may also drift `targetPort` until CD reconciles env from `[parameters.prod.json](../infra/azure/parameters.prod.json)`. +**Cause:** Web nginx proxies `/health` to the gateway (`TANGLE_API_UPSTREAM`, `tangle-study-gateway`). ACA HTTP ingress listens on port 80 for short app names — do **not** append `:8080` (connects to pod IP and times out). Infra deployed with **placeholder images** may also drift `targetPort` until CD reconciles env from `[parameters.prod.json](../infra/azure/parameters.prod.json)`. -**Fix:** Re-run deploy-image to reconcile `TANGLE_API_UPSTREAM` from `[parameters.prod.json](../infra/azure/parameters.prod.json)`, then smoke: +**Fix:** Re-run Bicep deploy to reconcile `TANGLE_API_UPSTREAM` from `[parameters.prod.json](../infra/azure/parameters.prod.json)`, then smoke: ```bash -AZURE_RESOURCE_GROUP=tangle-study-prod bash scripts/cd/azure-cd-deploy-image.sh +AZURE_RESOURCE_GROUP=tangle-study-prod IMAGE_TAG='...' bash scripts/cd/azure-cd-deploy-bicep.sh AZURE_RESOURCE_GROUP=tangle-study-prod ./scripts/cd/azure-cd-smoke.sh ``` Verify: ```bash -az containerapp show -n tangle-study-api -g tangle-study-prod \ - --query "properties.configuration.ingress.targetPort" -o tsv # expect 8080 +az containerapp show -n tangle-study-web -g tangle-study-prod \ + --query "properties.configuration.ingress.targetPort" -o tsv # expect 80 az containerapp show -n tangle-study-web -g tangle-study-prod \ --query "properties.template.containers[0].env[?name=='TANGLE_API_UPSTREAM']" -o yaml -# expect value: tangle-study-api +# expect value: tangle-study-gateway ``` -If `/health` still fails with **503** or `Unhealthy`, the API is up but a dependency check failed (Postgres, Redis) — check `tangle-study-api` logs. +If `/health` still fails with **503** or `Unhealthy`, the gateway is up but a dependency check failed (Postgres, Redis) — check `tangle-study-gateway` / `tangle-study-users` logs. --- diff --git a/docs/MSA_MIGRATION.md b/docs/MSA_MIGRATION.md index c18f1fce..20d6a407 100644 --- a/docs/MSA_MIGRATION.md +++ b/docs/MSA_MIGRATION.md @@ -81,7 +81,7 @@ Tier 4 (needs compose-build) **Runtime images:** CI/CD and harness use `docker-compose.runtime.yml` (slim Dockerfiles COPY prebuilt binaries). Local `docker compose up --build` still uses multi-stage Dockerfiles for ad-hoc dev. -**CD** mirrors CI compile step: `azure-cd-build-push.sh` runs `dotnet-publish.sh` + `build-workers-release.sh`, then pushes runtime images (web, workers, monitoring). **`tangle-study-api` removed from repo** — Azure Bicep/parameters still reference it until cutover. +**CD** mirrors CI compile step: `azure-cd-build-push.sh` runs `dotnet-publish.sh` + `build-workers-release.sh`, then pushes runtime images (gateway, users, media, chat, location, community, group, social, web, workers, monitoring). Azure Bicep loops `parameters.prod.json` → `containerApps` / `migrateJobs` on every deploy (`cd-v2.yml`). ### Adding the next service @@ -187,9 +187,9 @@ Postgres: one instance — `public` schema (monolith), `media` schema (media-ser | Item | Notes | |------|-------| -| Azure media Container App | CD + Bicep/parameters | -| `nginx.production.conf` media upstream | Strangler for prod | -| Worker `API_BASE_URL` on Azure | Point at media-service, not monolith | +| First prod CD cutover | Run `cd-v2.yml` once with `GATEWAY_SECRET` + `INTERNAL_SERVICE_SECRET` set; delete orphaned `tangle-study-api` / `tangle-study-migrate` if present | + +Azure Container Apps for media, chat, location, community, group, social, users, and gateway are defined in [`parameters.prod.json`](../infra/azure/parameters.prod.json) and provisioned by Bicep on every CD run. Worker `API_BASE_URL` values point at the matching service short names. `nginx.production.conf` upstream is `tangle-study-gateway` via web env. **Dev data:** uploads live in `media."MediaAssets"`. Legacy `public."MediaAssets"` is gone — reset the Compose DB volume if you need a clean slate. @@ -222,13 +222,9 @@ Postgres: one instance — `public` schema (monolith), `media` schema (media-ser | Monolith cleanup (`Domain/Chat`, public chat tables) | Done | | `Chat.Tests` + monolith boundary tests (historical) | Done | -### Still open (before `main` cutover) +### Azure cutover -| Item | Notes | -|------|-------| -| Azure chat Container App | CD + Bicep/parameters | -| `nginx.production.conf` chat upstream | Strangler for prod | -| Worker `API_BASE_URL` on Azure | Point at chat-service, not monolith | +Defined in [`parameters.prod.json`](../infra/azure/parameters.prod.json) (Container App + migrate job). See [Step 1 — Still open](#still-open-before-main-cutover). **Dev data:** chat rows live in `chat` schema. Legacy `public."Chat*"` tables are dropped by `RemoveMonolithChatTables` — reset the Compose DB volume if you need a clean slate. @@ -278,13 +274,9 @@ Postgres: one instance — `public` schema (monolith), `media`, `chat`, and `loc | Monolith cleanup (`Domain/Location`, public location tables) | Done | | `Location.Tests` + monolith boundary tests (historical) | Done | -### Still open (before `main` cutover) +### Azure cutover -| Item | Notes | -|------|-------| -| Azure location Container App | CD + Bicep/parameters | -| `nginx.production.conf` location upstream | Strangler for prod | -| Worker `API_BASE_URL` on Azure | Point at location-service, not monolith | +Defined in [`parameters.prod.json`](../infra/azure/parameters.prod.json) (Container App + migrate job). See [Step 1 — Still open](#still-open-before-main-cutover). **Dev data:** pins and sessions live in `location` schema. Legacy `public."MapPins"` / `public."LocationSessions"` are dropped by `RemoveMonolithLocationTables` — reset the Compose DB volume if you need a clean slate. @@ -322,12 +314,9 @@ community ──ILocationClient──► location Postgres: `community` schema (`Posts`, `Comments`). Legacy `public."Posts"` / `public."Comments"` dropped by `RemoveMonolithCommunityTables`. -### Still open (before `main` cutover) +### Azure cutover -| Item | Notes | -|------|-------| -| Azure community Container App | CD + Bicep/parameters | -| `nginx.production.conf` community upstream | Strangler for prod | +Defined in [`parameters.prod.json`](../infra/azure/parameters.prod.json) (Container App + migrate job). See [Step 1 — Still open](#still-open-before-main-cutover). ## Step 5 — group-service (`develop`: done) @@ -348,12 +337,9 @@ group ──ILocationClient──► location (end-sessions) Postgres: `group` schema. Legacy `public` group tables dropped by `RemoveMonolithGroupTables` **without copying rows**. Local Compose stacks must wipe the Postgres volume (`docker compose down -v`) when upgrading an existing DB; see [GROUP.md](../services/Group/GROUP.md). -### Still open (before `main` cutover) +### Azure cutover -| Item | Notes | -|------|-------| -| Azure group Container App | CD + Bicep/parameters | -| `nginx.production.conf` group upstream | Strangler for prod | +Defined in [`parameters.prod.json`](../infra/azure/parameters.prod.json) (Container App + migrate job). See [Step 1 — Still open](#still-open-before-main-cutover). ## Step 6 — social-service (friendships + user-blocks) (`develop`: done) @@ -370,14 +356,11 @@ social ──IUserClient──► users (users, nicknames, friends-list visibili Postgres: `social` schema (`Friendships`, `FriendRequests`, `UserBlocks`). Legacy `public` social tables dropped by `RemoveMonolithSocialTables` **without copying rows**. Local Compose stacks must wipe the Postgres volume (`docker compose down -v`) when upgrading an existing DB; see [SOCIAL.md](../services/Social/SOCIAL.md). -### Still open (before `main` cutover) +### Azure cutover -| Item | Notes | -|------|-------| -| Azure social Container App | CD + Bicep/parameters | -| `nginx.production.conf` social upstream | Strangler for prod | +Defined in [`parameters.prod.json`](../infra/azure/parameters.prod.json) (Container App + migrate job). See [Step 1 — Still open](#still-open-before-main-cutover). -## Step 7 — users-service + gateway (`develop`: Compose done; Azure open) +## Step 7 — users-service + gateway (`develop`: Compose done; Azure in parameters) Users and gateway are **separate deployables** (not combined). Users owns identity; gateway owns routing and JWT validation. **Compose dev stack is complete**; Azure/CD and `nginx.production.conf` remain open. @@ -425,15 +408,9 @@ users ──I*Client──► community, media, chat, group, social, location (u | Local Prometheus/Grafana scrape targets | Done (users, media, chat, location, community, group, social) | | Stack harness through gateway | Done (`run-stack-harness.sh` starts full module-filtered stack) | -### Still open (before `main` cutover) - -| Item | Notes | -|------|-------| -| Azure users + gateway Container Apps | CD + Bicep/parameters | -| `nginx.production.conf` gateway upstream | Strangler for prod | -| Azure domain Container Apps (media, chat, location, community, group, social) | CD + Bicep/parameters per service | +### Azure cutover -Azure/nginx.production cutover deferred. See [Strangler edge](#strangler-edge) below. +Users + gateway Container Apps, YARP cluster addresses, and web → `tangle-study-gateway` upstream are in [`parameters.prod.json`](../infra/azure/parameters.prod.json). Domain services and workers are co-deployed by the same Bicep pass. See [Step 1 — Still open](#still-open-before-main-cutover). --- @@ -442,6 +419,6 @@ Azure/nginx.production cutover deferred. See [Strangler edge](#strangler-edge) b | Environment | Edge | Routing | |-------------|------|---------| | **Compose (`develop`)** | `infra/nginx/nginx.conf` | All `/api/*`, `/hubs/*`, `/internal/*` → `gateway:8080`; gateway YARP routes to users, media, chat, location, community, group, social | -| **Azure (`main`)** | `infra/nginx/nginx.production.conf` | Still monolith-only until cutover | +| **Azure (`main`)** | `infra/nginx/nginx.production.conf` | All `/api/*`, `/hubs/*`, `/health` → `tangle-study-gateway` (via `TANGLE_API_UPSTREAM`) | Gateway validates JWT once; services receive identity via `X-User-Id` (trusted gateway secret). Login/JWT issuance stays on users-service.