Skip to content

Read the Piebald usage archive from beside its database - #295

Merged
mike1858 merged 1 commit into
mainfrom
feat/piebald-usage-archive-beside-db
Oct 30, 2026
Merged

mike1858 merged 1 commit into
mainfrom
feat/piebald-usage-archive-beside-db

Conversation

@mike1858

@mike1858 mike1858 commented Oct 30, 2026 •

Copy link
Copy Markdown
Member

Why

#294 taught the Piebald analyzer to keep counting deleted chats from usage archives in <state dir>/splitrail/piebald-archive/. In practice that put part of Piebald's history somewhere else. Copying, moving, or backing up Piebald's database (app.db with its -wal and -shm) no longer carried everything Splitrail counts. Backup jobs that snapshot app.db skipped the archives without saying so. An archive is part of its database's history, so it should travel with the database.

#294 hasn't shipped in a release yet (the latest is v3.12.0), so the location can change now without a migration path for released users.

What changed

  • The archive is now one file beside the database it belongs to, named after it: app.db becomes app.db.usage-archive (usage_archive_path). On Linux that's ~/.local/share/piebald/app.db.usage-archive; macOS and Windows follow Piebald's own data directory.
  • parse_source merges the archive that sits beside whatever database it parses. This replaces the check that merged archives only into this machine's database (is_default_database). A test fixture or a copy of another installation brings its own archive or none, so it can't pick up this machine's, by construction rather than by comparing paths.
  • Only that exact path is read. A .partial file beside it (an archive being rebuilt) and other databases in the folder are ignored, so writers rebuild the archive and rename it into place.
  • Merging is unchanged. Archived turns are de-duplicated against the live parse by global_hash, the live copy wins, and an unreadable archive is skipped with a warning.
  • The README section "Keeping usage from deleted Piebald chats" now gives the new location per OS.

Unchanged: with no archive beside the database, parse_source returns exactly what it returned before. The watcher still treats only app.db itself as data. The archive is rewritten right before a deletion, and the deletion changes app.db, so a refresh follows anyway (now noted in a comment).

Validation

  • cargo fmt --check
  • cargo clippy --locked --all-targets -- -D warnings
  • cargo test --locked: 537 passed. The archive tests now put the archive beside the live fixture and parse it through PiebaldAnalyzer::parse_source, so the path rule is exercised together with the merge. They cover:
    • a deleted chat's usage kept with identical price, date, and project
    • turns both live and archived counted once
    • a reused integer message ID
    • an unreadable archive
    • a .partial file and other databases ignored
    • a missing archive
    • the path rule itself
  • On a real installation, three archives written under the old layout were merged into app.db.usage-archive (42 chats, 12,865 messages). I compared splitrail stats from the Keep counting usage of deleted Piebald chats from archives #294 build, reading the old folder, with this build, reading the file beside app.db:
    • both reported 8,500 Piebald conversations
    • all 287 earlier days were identical
    • today differed only by turns written between the two runs
    • after the old folder was removed, the totals stayed the same

Summary by CodeRabbit

  • New Features
    • Usage history can now be archived alongside each Piebald database, including copied databases. Archived turns are included when viewing history, while turns already present in the database are counted only once.
    • Incomplete archives are not read; they become available after completion.
  • Documentation
    • Updated archive instructions to explain where archives are stored, how they relate to database copies, and how completed archives are named.

Usage archives of deleted Piebald chats lived in Splitrail's state
directory, apart from the database whose history they hold, so copying
or backing up app.db (with -wal and -shm) left them behind. The archive
is now one file beside the database, named after it
(app.db.usage-archive), and parse_source merges the archive beside
whatever database it parses. A fixture or another installation's copy
therefore brings its own archive or none, which replaces the
default-database path check.

Only that exact path is read, so a .partial rebuild or another database
in the folder is ignored. Merging, de-duplication by global_hash, and
skipping an unreadable archive are unchanged.
@coderabbitai

coderabbitai Bot commented Oct 30, 2026 •

Copy link
Copy Markdown

Review in Change Stack →

Caution

Review failed

The pull request is closed.

Walkthrough

Piebald usage archives now use one .usage-archive file beside each database. The analyzer merges archive messages with live messages, while missing or unreadable archives add no messages. The README and tests describe and cover the sibling-file behavior.

Changes: Piebald usage archives

Layer / File(s) Summary
Sibling archive lookup and merge
README.md, src/analyzers/piebald.rs
The analyzer derives one archive path by appending .usage-archive to the database path. It skips missing or unreadable archives and appends only messages whose global_hash is not present in the live database. The README documents the sibling archive and partial-file name.
Source parsing and archive coverage
src/analyzers/piebald.rs, src/analyzers/piebald_schema_tests.rs
parse_source applies archive merging to each source database. Tests cover deleted-chat history, duplicate messages, missing or unreadable archives, and exclusion of partial or unrelated nearby files.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Change: Bug fix

Possibly related PRs

  • Piebald-AI/splitrail#294: Adds the archive feature that this change updates, including archive lookup and merge behavior in the analyzer and tests.

Suggested reviewers: jimyag

Merge Risk: Minimal · up to 10b89

Piebald usage archives now live beside each database and are merged with its live history. Released users have no old-layout archives to lose, so there is no actionable merge-blocking risk.

Finishing Touches
📝 Generate docstrings
  • Commit to this branch
  • Create a new PR
🧪 Generate unit tests (beta)
  • Commit to this branch
  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@mike1858
mike1858 enabled auto-merge (squash) October 30, 2026 21:06
@mike1858
mike1858 merged commit 11bf8ca into main Oct 30, 2026
6 of 7 checks passed
@mike1858
mike1858 deleted the feat/piebald-usage-archive-beside-db branch October 30, 2026 21:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant