Repository navigation
Granular control of PISA fast-histogram threading and more robust numba thread setup - #939
Merged
Merged
Conversation
…or granular histogram threading control
…pand globals readme
JanWeldert
approved these changes
May 28, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Changes
numbawith correct number of threads already uponpisapackage import (see Is it merely a coincidence that certain modules' numba operations respectPISA_NUM_THREADS? #785)fast_histogramhistogramming, settingPISA_HIST_THREADINGnow takes precedence overPISA_NUM_THREADSandPISA_TARGET, in order to enable granular histogram threading control.PISA_NUM_THREADS > 1. Instead, the user needs to opt in explicitly by settingPISA_HIST_THREADING='auto'(recovers current default) or any integer.Motivation
Fixes Is it merely a coincidence that certain modules' numba operations respect
PISA_NUM_THREADS? #785Multi-threaded histogramming can
applyofutils.hist(see timing data below, and through thread contention potentally/seemingly also parallel numba computations made by other services) and therebyAs a result, significant gain from multi-threaded (in particular) oscillation calculations is partially compensated by the performance hit on histogramming the events in the sample—to the tune of ~ 2 minutes when going from 1 -> 12 threads.
Timing study
analysed script (no octant check to speed it up—still retaining hundreds of template calculations)
results
get_outputsHere we see that histogram threading always introduces overhead, whether relative to numba parallel or the baseline: the
ThreadPoolExecutoroverhead in_threaded_fh_histogramdditself is the problem.Following are two sets of neutrino
Pipelineprofile reports for more details, for configurations 2 and 4 above.