Skip to content

Support single-use matches during import deduplication - #219

Open
ryanduguid wants to merge 1 commit into
beancount:masterfrom
ryanduguid:support-single-use-deduplication
Open

ryanduguid wants to merge 1 commit into
beancount:masterfrom
ryanduguid:support-single-use-deduplication

Conversation

@ryanduguid

@ryanduguid ryanduguid commented Oct 9, 2026 •

Copy link
Copy Markdown

Summary

Add optional single-use matching for importers whose otherwise identical source records represent separate occurrences. This lets an importer retain an extra bank record when only one matching occurrence exists in the journal. Related to #179.

mark_duplicate_entries(entries, existing, window, compare, match_once=True)

The default remains many-to-one. The option allocates each eligible existing object once per call, excludes already-commented targets from overlapping files, and preserves target provenance. Matching uses the existing date window and greedy input order, selecting the last available match. The importer docstring shows how to opt in.

Evidence

  • Before: with one journal fee, both matching synthetic bank records are marked against the same journal line, leaving zero live imports.
    After: the opt-in mode retains the extra occurrence. A CLI regression with two overlapping files retains two live imports, comments three duplicates with exact provenance, and reconciles the closing AUD bank balance.
  • Linux CPython 3.14.8 / Beancount 3.2.3: python -m pytest beangulp passes 174 tests; configured Ruff 0.12.12 passes. All 18 focused tests pass on Windows. Ten example unit tests and the CSV, OFX and investment importer examples pass on the final files.

Risk

This is a policy proposal for importers that know their source preserves occurrence counts. Repeated representations with the same unique source ID usually need the default. Greedy allocation may leave an ambiguous duplicate live. Premarked inputs retain and reserve their directive target; truthy boolean markers raise ValueError in the opt-in mode.

Unverified

The full Windows suite has the same 16 failing tests before and after this change. make test/test-all, the full Python and platform matrix, the example importer workflows, hosted CI and performance on large inputs were not run. Whether real bank exports preserve occurrence identity was not checked.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant