Real-time fraud feature store on Databricks with Lakebase PostgreSQL

Purpose: show how to score a card swipe in milliseconds using fresh per-account risk features, without recomputing them from raw transactions on every request.

At swipe time, a fraud app needs a fast answer to: "What do we already know about this account?" This project builds that path end to end:

1. Card transactions live in the Databricks lakehouse.
2. A materialized view rolls them up into per-account risk features (velocity, geo-spread, a simple risk score).
3. A synced table copies those features into Lakebase Postgres for sub-millisecond point lookups by account_id.
4. A small scoring app looks up the features and returns APPROVE / REVIEW / DECLINE.

Everything is packaged as a Databricks Asset Bundle. Clone it, point it at your workspace, and follow the Quickstart below.

---

Architecture

   Card transactions (Delta: transactions)            <- offline, lakehouse
              |
              v
   Materialized View: account_fraud_features          <- Lakeflow feature pipeline
   (per-account velocity / geo-spread / risk score)      features are *built* here
              |
              v
   Synced Table: account_fraud_features_online        <- Lakebase Postgres
   (primary key = account_id)                            features are *served* here
              |
              v
   Fraud-scoring app: point-lookup features by account_id, in milliseconds

Why a materialized view and a synced table?

They do different jobs:

• The materialized view precomputes features in Databricks so you don't re-aggregate raw transactions on every swipe.
• The synced table is a Postgres copy optimized for fast primary-key lookups by the scoring app. Databricks is great for building features; Lakebase is great for serving them at request time.

Materialized view (account_fraud_features): per-account features, including the stored risk_score, built from historical transactions.

Synced table (account_fraud_features_online): the Lakebase Postgres copy the scoring app looks up by account_id. Status is Online, source is the MV, primary key is account_id, sync mode is Snapshot.

A scheduled Workflow keeps the online copy fresh: it refreshes the MV, then syncs to Lakebase. The sync job keeps the Postgres copy fresh after the MV refreshes.

[ Task 1: refresh MV ]  --depends_on-->  [ Task 2: sync MV to Lakebase ]

---

What it builds

Object	Type	Purpose
transactions	Materialized view	Synthetic, deterministic card-transaction source data
account_fraud_features	Materialized view	Where features are built (per-account aggregates in Databricks)
account_fraud_features_online	Lakebase synced table	Where features are served (low-latency Postgres copy for the app)
fraud_feature_sync_job	Workflow	Refresh the MV, then sync so Postgres stays fresh

Sample of the served features (500 accounts):

account_id	txn_count	distinct_merchants	distinct_countries	foreign_txn_count	max_amount	risk_score
372	10	6	5	6	1933.18	1.00
271	10	5	5	7	1897.18	1.00
182	10	6	2	3	1810.93	0.52
32	10	8	4	3	1463.55	0.49

---

Prerequisites

• A Databricks workspace with Unity Catalog, serverless Lakeflow pipelines, and Lakebase enabled. Works on Databricks Free Edition.
• Databricks CLI, logged in:

  databricks auth login --profile <profile>

• A catalog you can create a schema in. Bundle default is catalog main, schema fraud_feature_store.
  • Free Edition: catalog is workspace. Pass --var catalog=workspace on every bundle deploy / bundle run below (or edit databricks.yml).
  • Other workspaces: main is usually fine. Omit --var catalog=....

Set these once so the rest of the commands stay copy-pasteable:

PROFILE=<profile>
CATALOG=workspace          # Free Edition. Use main on most other workspaces.

---

Quickstart

1. Deploy the bundle

Creates the feature pipeline and the refresh/sync workflow.

databricks bundle deploy -p $PROFILE -t dev --var catalog=$CATALOG

2. Build the features

databricks bundle run feature_pipeline -p $PROFILE -t dev --var catalog=$CATALOG

This creates transactions and account_fraud_features in $CATALOG.fraud_feature_store.

3. Create a Lakebase project and sync the features

Create a Lakebase Autoscaling project (skip if you already have one):

databricks postgres create-project fraud-feature-store -p $PROFILE \
  --json '{"spec": {"display_name": "Fraud Feature Store"}}'

Then create the synced table (Snapshot, PK account_id):

PROFILE=$PROFILE CATALOG=$CATALOG SCHEMA=fraud_feature_store \
  PROJECT=fraud-feature-store ./scripts/create_synced_table.sh

Or do the same in the UI: Catalog Explorer → account_fraud_features → Create synced table. Destination $CATALOG.fraud_feature_store.account_fraud_features_online, primary key account_id, mode Snapshot, create a new pipeline.

Wait until the synced table status is Online.

4. Keep it fresh with the Workflow

Copy the synced table's sync pipeline ID (Overview → Data ingest, or the script's get-synced-table command). Set it in databricks.yml (variables.sync_pipeline_id), then:

databricks bundle deploy -p $PROFILE -t dev --var catalog=$CATALOG
databricks bundle run fraud_feature_sync_job -p $PROFILE -t dev --var catalog=$CATALOG

Task 1 refreshes the MV, Task 2 syncs it to Lakebase. The sync job keeps the Postgres copy fresh after the MV refreshes.

5. The fraud-scoring app (interactive)

A small Streamlit app in app/ makes it tangible: swipe a card (account, amount, country, merchant) and it does a primary-key point lookup against the Lakebase synced table, blends the stored risk features with the live transaction, and returns APPROVE / REVIEW / DECLINE, with the lookup latency to show off the sub-millisecond serving path.

APPROVE. Account 32, US, $250. Stored risk is 0.49 and there is no live signal, so the swipe is approved.

REVIEW. Account 125, US, $250. Stored risk is 0.79. That is high enough for review, but DECLINE needs a live signal (foreign country or unusual amount).

DECLINE. Account 18, GB, $150. A foreign swipe on top of a high stored risk score declines the transaction.

Deploy it as a Databricks App:

databricks apps create fraud-scoring -p $PROFILE
databricks sync ./app /Workspace/Users/<you>/fraud-scoring-app -p $PROFILE
databricks apps deploy fraud-scoring \
  --source-code-path /Workspace/Users/<you>/fraud-scoring-app -p $PROFILE

Then wire it to Lakebase. Skip any of these and the swipe lookup will fail.

1. Attach the database resource. App → Edit → Resources → Add → Database → your Lakebase project + databricks_postgres, permission Can connect. That injects PGHOST / PGPORT / PGDATABASE / PGUSER / PGSSLMODE. There is no PGPASSWORD. Lakebase uses a short-lived Databricks OAuth token, which the app mints from the SDK at request time (lakebase.py).
2. Grant the app's service principal read access. The app connects as the SP; the Postgres role name is the app's client id (shown on the app page). In the Lakebase project SQL editor:

   GRANT USAGE  ON SCHEMA fraud_feature_store
     TO "<app-sp-client-id>";
   GRANT SELECT ON fraud_feature_store.account_fraud_features_online
     TO "<app-sp-client-id>";

3. FEATURES_TABLE is already set in app/app.yaml to fraud_feature_store.account_fraud_features_online (the Postgres schema.table, not public). Change it only if you used a different schema, then sync and redeploy the app.

Try the same swipes as the screenshots: account 32 / US / $250 (APPROVE), 125 / US / $250 (REVIEW), 18 / GB / $150 (DECLINE).

Or run locally (uses your own Databricks auth to mint the token):

cd app
pip install -r requirements.txt
export PGHOST=<project-endpoint-host> PGUSER=<you@example.com> PGDATABASE=databricks_postgres
export FEATURES_TABLE=fraud_feature_store.account_fraud_features_online
streamlit run app.py

---

Repo layout

.
├── databricks.yml                      # Asset Bundle: variables + targets
├── LICENSE                             # MIT
├── images/                             # README screenshots (MV, synced table, scoring app)
├── resources/
│   ├── feature_pipeline.pipeline.yml   # the feature pipeline (transactions + MV)
│   └── sync_job.job.yml                # the workflow: refresh MV → sync to Lakebase
├── src/
│   ├── transactions.sql                # synthetic card-transaction source
│   └── account_fraud_features.sql      # the materialized view served online
├── scripts/
│   ├── create_synced_table.sh          # create the Lakebase synced table (CLI)
│   └── teardown.sh                     # safely de-provision everything (dry-run by default)
└── app/
    ├── app.py                          # Streamlit "swipe a card" UI
    ├── scoring.py                      # real-time APPROVE/REVIEW/DECLINE logic
    ├── lakebase.py                     # Lakebase Postgres connection + lookup
    ├── requirements.txt
    └── app.yaml                        # Databricks Apps entry point

---

Notes

• Sync modes. This demo uses Snapshot (full refresh each run), the simplest fit for "run after the MV in a Workflow." Triggered (incremental) and Continuous (always-on) are alternatives.
• Determinism. Source data is generated deterministically (hash-salted) so refreshes are idempotent. Raise num_transactions to grow the dataset.
• Cost & cleanup. The Databricks App runs a container continuously while deployed (the main ongoing cost); the pipelines bill only per run; your Lakebase project bills for its own compute. To de-provision everything safely (dry-run by default):

  # preview what would be removed
  PROFILE=<profile> CATALOG=<cat> SCHEMA=<schema> SYNC_PIPELINE_ID=<id> \
    ./scripts/teardown.sh
  # actually remove it (APP_ACTION=stop pauses the app; =delete removes it)
  CONFIRM=yes APP_ACTION=delete PROFILE=<profile> CATALOG=<cat> SCHEMA=<schema> \
    SYNC_PIPELINE_ID=<id> ./scripts/teardown.sh

  Stop/pause your Lakebase project from the Lakebase UI separately.

License

MIT. See LICENSE.