Skip to content

[MNT] Similarity search cleanup#3222

Merged
TonyBagnall merged 35 commits into
mainfrom
simsearch_revamp
Jul 10, 2026
Merged

[MNT] Similarity search cleanup#3222
TonyBagnall merged 35 commits into
mainfrom
simsearch_revamp

Conversation

@baraline

@baraline baraline commented Jan 3, 2026

Copy link
Copy Markdown
Member

What does this implement/fix? Explain your changes.

Following the decisions made after #2473 and multiple discussions, this PR aims at clarifying the goal and interface of the similarity search module, notably by removing all components related to motif discovery, in order to not mix both intents even if they are related down the line.

This introduced a unique interface for all similarity search estimator, and focus on two use cases :

  • Similarity search with subsequences
  • Similarity search with whole series.

Both cases take as input a 3D array (a time series collection) during fit and a 2D array (a subsequence or a whole series) during predict. The predict function returns two arrays indexes, distances that contain the indexes of the matches in the collection, as well as their distances to the 2D series.

The goal is to validate this structure to be able to move the module out of experimental in the near future.

Content of the PR :

Module restructure

  • aeon.similarity_search reorganized from series / collection into two task-based submodules: subsequence (find a short query inside long series) and whole_series (find nearest full series in a collection).
  • Motif search (STOMP) removed; the module now focuses purely on nearest-neighbor search.

Estimators

  • subsequence: MASS kept, DummySimilaritySearch renamed to NaiveSubsequenceSearch; new BaseSubsequenceSearch and BaseDistanceProfileSearch base classes.
  • whole_series: new NaiveSeriesSearch, and new SimHashIndexANN approximate-NN index replacing the old RandomProjectionCosineLSH.

API & internals

  • Unified fit(X) / predict(X, k) interface returning (indexes, distances) across all searchers.
  • Hardened input validation (query shape/length, n_jobs checks) and improved error messages.
  • Naive searchers reuse aeon.distances kernels; optimized top-k extraction and exclusion-mask helpers; multithreading via n_jobs.

Testing

  • New test suites for the new base classes and estimators; mock similarity searchers and the estimator-check harness updated to the new interface.

Docs & notebooks

  • API reference, getting started, and examples pages updated to the new structure.
  • New simhash_index.ipynb notebook (theory + benchmarks, with illustration); existing similarity-search and distance-profile notebooks reworked and re-run.

Any other comments?

The code related to motif search has been deleted to simplify the intent and structure of the similarity search module. But a new motif search module would be a welcome addition later.

PR checklist

For all contributions
  • I've added myself to the list of contributors. Alternatively, you can use the @all-contributors bot to do this for you after the PR has been merged.
  • The PR title starts with either [ENH], [MNT], [DOC], [BUG], [REF], [DEP] or [GOV] indicating whether the PR topic is related to enhancement, maintenance, documentation, bugs, refactoring, deprecation or governance.
For new estimators and functions
  • I've added the estimator/function to the online API documentation.
  • (OPTIONAL) I've added myself as a __maintainer__ at the top of relevant files and want to be contacted regarding its maintenance. Unmaintained files may be removed. This is for the full file, and you should not add yourself if you are just making minor changes or do not want to help maintain its contents.
For developers with write access
  • (OPTIONAL) I've updated aeon's CODEOWNERS to receive notifications about future changes to these files.
Notes :

Might need to add a "predict_batch" to handle collections in predict, as some algorithms can offer optimisation for predicting on collections. By default, it loops on the predict method for series, but child classes can override it for optimisations. This will be handled in a future PR.

@review-notebook-app

Copy link
Copy Markdown

Check out this pull request on  ReviewNB

See visual diffs & provide feedback on Jupyter Notebooks.


Powered by ReviewNB

@aeon-actions-bot aeon-actions-bot Bot added examples Example notebook related maintenance Continuous integration, unit testing & package distribution similarity search Similarity search package testing Testing related issue or pull request labels Jan 3, 2026
@aeon-actions-bot

Copy link
Copy Markdown
Contributor

Thank you for contributing to aeon

I have added the following labels to this PR based on the title: [ maintenance ].
I have added the following labels to this PR based on the changes made: [ examples, similarity search, testing ]. Feel free to change these if they do not properly represent the PR.

The Checks tab will show the status of our automated tests. You can click on individual test runs in the tab or "Details" in the panel below to see more information if there is a failure.

If our pre-commit code quality check fails, any trivial fixes will automatically be pushed to your PR unless it is a draft.

Don't hesitate to ask questions on the aeon Slack channel if you have any.

PR CI actions

These checkboxes will add labels to enable/disable CI functionality for this PR. This may not take effect immediately, and a new commit may be required to run the new configuration.

  • Run pre-commit checks for all files
  • Run mypy typecheck tests
  • Run all pytest tests and configurations
  • Run all notebook example tests
  • Run numba-disabled codecov tests
  • Stop automatic pre-commit fixes (always disabled for drafts)
  • Disable numba cache loading
  • Regenerate expected results for testing
  • Push an empty commit to re-run CI checks

baraline and others added 20 commits March 27, 2026 12:37
…nterface

Also corrects MASS capability:multithreading tag to False (it has no
threading machinery), which unblocks check_multithreading_param.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
… clarify whole_series base

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…timator

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…h BruteForce class

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…otebooks

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…e, debug and optimize top k and mask, re-use distance module functions, harden input validation.
@baraline baraline marked this pull request as ready for review July 3, 2026 12:57
@baraline

baraline commented Jul 3, 2026

Copy link
Copy Markdown
Member Author

CI Error unrelated to the changes

@TonyBagnall

Copy link
Copy Markdown
Contributor

just wondering whether k could be used in fit to avoid calculations?

@TonyBagnall

Copy link
Copy Markdown
Contributor

forgive my ignorance, but the obvious use case for me is searching for a subseries in a single long series? I guess that just collapses to passing (1,n_channels, n_timepoints)?

@TonyBagnall

Copy link
Copy Markdown
Contributor

agsain, just general thoughts on a first look through, but could we generalise this to any aeon distance function? At least for the naive exact searchers?

@TonyBagnall TonyBagnall added the codecov actions Run the codecov action on a PR label Jul 5, 2026
@baraline

baraline commented Jul 6, 2026

Copy link
Copy Markdown
Member Author

For the naive search sure, i can pass a distance arg that fetch distance modules funcs.

@baraline

baraline commented Jul 6, 2026

Copy link
Copy Markdown
Member Author

Regarding k in fit, I ́m not sure I understand what you had in mind, you can’t precompute distances without a query. Or are you talking about approximate methods ?

@TonyBagnall

Copy link
Copy Markdown
Contributor

Regarding k in fit, I ́m not sure I understand what you had in mind, you can’t precompute distances without a query. Or are you talking about approximate methods ?

I think I was just wrong :)

TonyBagnall
TonyBagnall previously approved these changes Jul 6, 2026

@TonyBagnall TonyBagnall left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like it, I think this is an improvement. Like the coverage too :) Im happy to approve it. Chatgpt had some niggles when I got it to read, only one I think maybe should go in is this

  1. SimHashIndexANN should validate integer hyperparameters more strictly.

At the moment n_tables and n_bits_per_table appear to be checked by value comparisons, but not by type. That means floats and bools can pass the early validation and then fail later in less helpful places, for example when constructing projection shapes or iterating over bit positions. I would explicitly require positive integer values, excluding bools, e.g.

if (
    not isinstance(self.n_tables, (int, np.integer))
    or isinstance(self.n_tables, bool)
    or self.n_tables < 1
):
    raise ValueError(...)

if (
    not isinstance(self.n_bits_per_table, (int, np.integer))
    or isinstance(self.n_bits_per_table, bool)
    or not 1 <= self.n_bits_per_table <= 64
):
    raise ValueError(...)

@baraline

baraline commented Jul 6, 2026

Copy link
Copy Markdown
Member Author

Agreed, will make the adjustment with this and the distance arg for naive search.

@baraline

Copy link
Copy Markdown
Member Author

All done @TonyBagnall

@TonyBagnall TonyBagnall left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fantastic, good improvement

@TonyBagnall TonyBagnall merged commit ec38246 into main Jul 10, 2026
22 checks passed
@TonyBagnall TonyBagnall deleted the simsearch_revamp branch July 10, 2026 23:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

codecov actions Run the codecov action on a PR examples Example notebook related maintenance Continuous integration, unit testing & package distribution similarity search Similarity search package testing Testing related issue or pull request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants