Skip to content

Use real component dtype when writing statistics for complex data - #306

Merged
gshiroma merged 37 commits into
isce-framework:developfrom
gshiroma:fix_gcov_off_diag_stats_dtype
Sep 16, 2026
Merged

gshiroma merged 37 commits into
isce-framework:developfrom
gshiroma:fix_gcov_off_diag_stats_dtype

Conversation

@gshiroma

Copy link
Copy Markdown
Contributor

This PR updates BaseL2WriterSingleInput to ensure statistics computed from complex-valued data (e.g., GCOV off-diagonal terms) are written using the corresponding real-valued dtype.

gshiroma and others added 30 commits July 3, 2025 11:30
@gshiroma
gshiroma requested a review from nemo794 June 18, 2026 05:43
@nemo794

nemo794 commented Aug 20, 2026

Copy link
Copy Markdown
Contributor

Adding more context to this PR description:

Currently, GCOV off-diagonal complex-valued datasets (e.g., HHHV with complex64 dtype), the statistics attributes (min_real_value, mean_real_value, min_imag_value, etc.) are incorrectly being stored as complex-valued (e.g., complex64) with the actual metric saved to the real component. (See screenshot)

However, for a complex64 dataset (composed of float32 + float32), statistics attributes should be stored as float32, not complex64, since the statistics themselves are real-valued scalars. This was inconsistent with RSLC and GSLC products, where statistics for complex-valued datasets are correctly stored as float-valued.

This PR addresses that inconsistency.

Screenshot 2026-06-17 at 6 44 36 PM

@nemo794 nemo794 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @gshiroma !

I'm pretty sure you tested this PR awhile back. Please attach a screenshot of the output product, as a sanity check to confirm that these fields are being populated correctly now?

See #306 (comment) for an example screenshot of what the metadata looked like previously. Thanks!

@gshiroma gshiroma added this to the R05.03.0 milestone Aug 27, 2026
@gshiroma

gshiroma commented Sep 8, 2026

Copy link
Copy Markdown
Contributor Author

Thanks @gshiroma !

I'm pretty sure you tested this PR awhile back. Please attach a screenshot of the output product, as a sanity check to confirm that these fields are being populated correctly now?

See #306 (comment) for an example screenshot of what the metadata looked like previously. Thanks!

Yes, I tested this PR previously. Here’s a before/after comparison of the statistics attributes for the geocoded complex dataset crosstalk/txHorizontalCrosspol.

Before: The statistics attributes were incorrectly stored as complex64:

>>> x = h5py.File('output_gcov/gcov.h5')['/science/LSAR/GCOV/metadata/calibrationInformation/crosstalk/txHorizontalCrosspol']
>>> print([x.attrs[d].dtype for d in [
...     'mean_real_value', 'min_real_value', 'max_real_value',
...     'sample_stddev_real', 'mean_imag_value', 'min_imag_value',
...     'max_imag_value', 'sample_stddev_imag']])
[dtype('complex64'), dtype('complex64'), dtype('complex64'), dtype('complex64'),
 dtype('complex64'), dtype('complex64'), dtype('complex64'), dtype('complex64')]

After: The statistics attributes are now correctly stored as float32:

>>> y = h5py.File('output_gcov/gcov_fix_stats_dtype.h5')['/science/LSAR/GCOV/metadata/calibrationInformation/crosstalk/txHorizontalCrosspol']
>>> print([y.attrs[d].dtype for d in [
...     'mean_real_value', 'min_real_value', 'max_real_value',
...     'sample_stddev_real', 'mean_imag_value', 'min_imag_value',
...     'max_imag_value', 'sample_stddev_imag']])
[dtype('float32'), dtype('float32'), dtype('float32'), dtype('float32'),
 dtype('float32'), dtype('float32'), dtype('float32'), dtype('float32')]

So the real and imaginary statistics are now being populated with the expected data type.

@hfattahi hfattahi left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Thanks for confirming that you have tested the PR.

@bhawkins bhawkins left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@gshiroma
gshiroma merged commit 0efd009 into isce-framework:develop Sep 16, 2026
8 checks passed
@gshiroma
gshiroma deleted the fix_gcov_off_diag_stats_dtype branch September 16, 2026 21:38
Tyler-g-hudson added a commit that referenced this pull request Sep 23, 2026
commit 15c2c9b
Author: Seongsu Jeong <27862199+seongsujeong@users.noreply.github.com>
Date:   Wed Sep 23 13:27:49 2026 -0700

    Improvement for TEC correction (#369)

    * add switch to control TEC correction's behavior

    * refine the TOP TEC selection logic; add option to polynomial fitting to TEC profile

    * TEC polynomial fittinf

    * fix GSLC schema

    * add TEC correction paramters to GCOV and InSAR workflows; refactor topside TEC rejection logic

    * applying comments from PR review

    * Apply suggestion from PR review

    * add RANSAC implementation, address comments from PR review

    * turnon ignore_invalid_topside_tec

    * Check if TEC file exists in the path provided in runconfig

    * test core for MAD thresholding

    * MAD parameter tuning

    * Apply `b_error=false` in zero-filled TEC correction LUTs

    * `sigma` -> `num_sigma`; expose `num_sigma` in runconfig; revise docstring

    * remove irrelevant comment

    * fix typo

    * revise docstring

    * `_mad_polyfit()` -> `_smooth_mad_polyfit()`

    * Return the original TEC values when thresholdong or smoothing is not possible

    * revise docstring

    * rename function and variable

commit 2cab1cd
Author: Brian Hawkins <1729052+bhawkins@users.noreply.github.com>
Date:   Wed Sep 23 13:20:11 2026 -0500

    Rename mask attributes (#382)

    * Rename RSLC

    * Rename attributes in GCOV writer

    * Rename attributes in GSLC writer

    * Update GCOV and GSLC PIX XML schema.

    * Fix capitalization of GSLC XML spec

    * Fix copy/paste typos

    Co-authored-by: Gustavo H. X. Shiroma <52007211+gshiroma@users.noreply.github.com>

    ---------

    Co-authored-by: Gustavo H. X. Shiroma <52007211+gshiroma@users.noreply.github.com>

commit 07d65b3
Author: xhuang-jpl <118782850+xhuang-jpl@users.noreply.github.com>
Date:   Wed Sep 23 09:19:58 2026 -0700

    Add polarization-specific `validDataMask` datasets to InSAR product (#379)

    * sm r3.4

    * change the SM commit id for R4.0.2

    * update the insar mask

    * update the subswath mask

    * accommodate the old RSLC

    * fix the insar unit test failures

    * fix the geocode_insar for the pixelOffsets in the RUNW product

    * remove unused codes

    * fix two major bugs from s-sasaki-earthsea-wizard

    * update the mask name and description

    * fix the comment from PR review

    * upate the comments

    * update the blocks

    * add the fill value

    * compute the validPixelFraction

    * fix the offsets None fill value

    * prefer the valid mask to the subswath mask

    * remove the _chunk_lines and make it same with the block_lines

    * update the attribute name

    ---------

    Co-authored-by: Xiaodong Huang <xhuang@nisar-adt-dev-3.jpl.nasa.gov>

commit 5072085
Author: Jungkyo Jung <56169931+oberonia78@users.noreply.github.com>
Date:   Tue Sep 22 16:19:49 2026 -0700

    Fix insar-mask showing wrong values in reference and secondary subswaths (#381)

    * fix a bug in insar mask showing wrong values in subswath

    * fix typo and remove redundant lines

    * update variable name and add more descriptions

    ---------

    Co-authored-by: Jungkyo Jung <jungkyoj@nisar-adt-dev-3.jpl.nasa.gov>

commit 35e4829
Author: Gustavo H. X. Shiroma <52007211+gshiroma@users.noreply.github.com>
Date:   Mon Sep 21 16:26:10 2026 -0700

    Add  the `fill_value` parameter to `GeocodeCov` CPU module (#352)

    * Add an option in the GCOV workflow to assign values to regions with missing radar samples

    * Add parameter  to the  module

    * Add parameter  to the  module

    * Add parameter  to the  module

    * Add parameter  to the  module

    * Set layover/shadow mask layer fill value to 255.

    * Set layover/shadow mask layer fill value to 255.

    * update data type

    * ensure that unprocessed blocks are also filled with `fill_value`

    * ensure that unprocessed blocks are also filled with `fill_value`

    * ensure that unprocessed blocks are also filled with `fill_value`

    * update docstrings

    * print warning messages

    * improve unit test

    * update warning message

    * fix GDAL data types for integer-valued data

    * Update GeocodeCov complex fill values to NaN + NaN.j rather than NaN + 0.j

    * Update GeocodeCov complex fill values to NaN + NaN.j rather than NaN + 0.j

    * Update GeocodeCov complex fill values to NaN + NaN.j rather than NaN + 0.j

    * Update GeocodeCov complex fill values to NaN + NaN.j rather than NaN + 0.j

    * Update GeocodeCov complex fill values to NaN + NaN.j rather than NaN + 0.j

    * Update GeocodeCov complex fill values to NaN + NaN.j rather than NaN + 0.j

    * Use camel case for C++ function name

    * Update tests/python/extensions/pybind/geocode/geocodeCov.py

    Co-authored-by: Brian Hawkins <1729052+bhawkins@users.noreply.github.com>

    * improve warning messages

    * improve warning messages

    * Set fill value of GCOV and GSLC `inputDataExceptionMask` to `65535`

    ---------

    Co-authored-by: Brian Hawkins <1729052+bhawkins@users.noreply.github.com>

commit e915b19
Author: Jungkyo Jung <56169931+oberonia78@users.noreply.github.com>
Date:   Mon Sep 21 10:08:50 2026 -0700

    Support multiple preprocessing mask types for unwrap (#288)

    * enable multi item for unwrap mask

    * change default

    ---------

    Co-authored-by: Jungkyo Jung <jungkyoj@nisar-adt-dev-5.jpl.nasa.gov>

commit 0c775d7
Author: Gustavo H. X. Shiroma <52007211+gshiroma@users.noreply.github.com>
Date:   Thu Sep 17 13:15:12 2026 -0700

    Update GCOV and GSLC writer to geocode the `inputDataExceptionMask` (`UInt16`) dataset (#378)

    * disable polarimetric symmetrization by default

    * revert changes to `symmetrize_cross_pol_channels`

    * Update GCOV and GSLC specification XMLs

    * Revert changes to the GCOV and GSLC specification XMLs

    * Update GCOV and GSLC writers to handle integer 16 masks

    * add fill_value to `inputDataExceptionMask` and copy attributes from input H5 dataset

    * add fill_value to `inputDataExceptionMask` and copy attributes from input H5 dataset

    * add fill_value to `inputDataExceptionMask` and copy attributes from input H5 dataset

    * add fill_value to `inputDataExceptionMask` and copy attributes from input H5 dataset

    * improve warning messages

    * rename inputDataExceptionMask attributes `maskValidPixelFraction` to `rslcValidPixelFraction`

    * rename inputDataExceptionMask attributes `maskValidPixelFraction` to `rslcValidPixelFraction`

    * Update GCOV and GSLC product specs XMLs

    * Update GCOV and GSLC product specs XMLs

    * Update python/packages/nisar/products/writers/BaseL2WriterSingleInput.py

    Co-authored-by: Seongsu Jeong <27862199+seongsujeong@users.noreply.github.com>

    ---------

    Co-authored-by: Seongsu Jeong <27862199+seongsujeong@users.noreply.github.com>

commit 828ab91
Author: Gustavo H. X. Shiroma <52007211+gshiroma@users.noreply.github.com>
Date:   Wed Sep 16 10:42:18 2026 -0700

    Ensure `_FillValue` attributes in metadata cubes are stored using the same data type as their corresponding datasets. (#309)

    * disable polarimetric symmetrization by default

    * revert changes to `symmetrize_cross_pol_channels`

    * Update GCOV and GSLC specification XMLs

    * Revert changes to the GCOV and GSLC specification XMLs

    * ensure attribute _FillValue is stored with the same data type as the corresponding H5 dataset

    * improve docstrings

    * Ensure `fill_value` has the same data type as the parameter `dtype`

    * Ensure `fill_value` has the same data type as the parameter `dtype`

commit 3aeb5a4
Author: Gustavo H. X. Shiroma <52007211+gshiroma@users.noreply.github.com>
Date:   Wed Sep 16 09:17:18 2026 -0700

    Remove grid mapping projection from L1 products (#310)

    * disable polarimetric symmetrization by default

    * revert changes to `symmetrize_cross_pol_channels`

    * Update GCOV and GSLC specification XMLs

    * Revert changes to the GCOV and GSLC specification XMLs

    * save the grid_mapping attribute only when generating L2 products

    * Add comments and docstrings

    * improve docstrings

commit 81a47c7
Author: Gustavo H. X. Shiroma <52007211+gshiroma@users.noreply.github.com>
Date:   Wed Sep 16 09:15:21 2026 -0700

    Ensure that the GCOV and GSLC HDF5 dataset noiseEquivalentBackscatter is stored as float32 (#307)

    * disable polarimetric symmetrization by default

    * revert changes to `symmetrize_cross_pol_channels`

    * Update GCOV and GSLC specification XMLs

    * Revert changes to the GCOV and GSLC specification XMLs

    * Ensure that the GCOV and GSLC HDF5 dataset noiseEquivalentBackscatter is stored as float32

    * Ensure that the GCOV and GSLC HDF5 dataset noiseEquivalentBackscatter is stored as float32

    * Ensure that the GCOV and GSLC HDF5 dataset noiseEquivalentBackscatter is stored as float32

commit 0efd009
Author: Gustavo H. X. Shiroma <52007211+gshiroma@users.noreply.github.com>
Date:   Wed Sep 16 09:14:32 2026 -0700

    Use real component dtype when writing statistics for complex data (#306)

    * disable polarimetric symmetrization by default

    * revert changes to `symmetrize_cross_pol_channels`

    * Update GCOV and GSLC specification XMLs

    * Revert changes to the GCOV and GSLC specification XMLs

    * Use real-valued component when writing real-valued statistics of complex data

    * Use real-valued component when writing real-valued statistics of complex data

    * improve comments

    * add parameter `to_stats_format_function` to the function `_get_attribute_dict()`

    * Improve docstrings

commit 72f8e57
Author: Hirad Ghaemi <92554775+rad-eng-59@users.noreply.github.com>
Date:   Tue Sep 15 18:11:58 2026 -0700

    Caltone Frequency Extraction and Scaling (#323)

    * Allow parsing caltone freq from L0B in focus.py if not in runconfig
    * Default caltone freq to null and add description in focus.yaml
    * Scale rx channel imbalance by complex caltone offset over all rxs
    * Use LNA phase and Caltone mag to build complex scalar in RX DBF
    * Fix a typo in docstring
    * Add a check for range of valid caltone frequency when parsing
    * Define hardcoded Caltone norm factor as a module-level constant
    * Adjust caltone norm based on rcid=240 used as reference

commit 355f6b3
Author: Jungkyo Jung <56169931+oberonia78@users.noreply.github.com>
Date:   Tue Sep 15 14:18:34 2026 -0700

    qFSP reduction for ionosphere (#333)

    * qFSP reduction for ionosphere

    * minor changes

    * refine default and scheme

    * add docstring

    * reoganize order

    * more continuous boundary in artifact regions

    * add docstring and comments

    * simplify options

    * replace polyfit with offsets_polyfit

    * move functions from isce to nisar

    * restore comment line

    * fix some docstrings based on comments

    * fix missing bugs

    * fix line

    * introduce debug_flag

    * add unit test

    ---------

    Co-authored-by: Jungkyo Jung <jungkyoj@nisar-adt-dev-3.jpl.nasa.gov>

commit ed636b4
Author: Hirad Ghaemi <92554775+rad-eng-59@users.noreply.github.com>
Date:   Mon Sep 14 19:56:01 2026 -0700

    Support for Second-band Channel Adjustments in INT-CAL HDF5 (#325)

    * Augment instrumnent intcal parser for new freq-B imbalances "amplitudeB"
    * Add an intcal file "REE_INSTRUMENT_TABLE_V3P0.h5" to test new feature
    * Update test suite of instrument parser to exercise new feature/product
    * Add description to README file for new intcal sample product
    * Support second-band fudge factors in pattern.py for EAP
    * Augment the test suite for instrument parser with new HDF5 file

commit 943007d
Author: Hirad Ghaemi <92554775+rad-eng-59@users.noreply.github.com>
Date:   Mon Sep 14 16:22:58 2026 -0700

    Switch to MVE Noise Est For Single Noise-only RangeLine in RSLC (NSDS-4938) (#347)

    * Use MVE noise est method in rslc if there is one noise rangeline
    * Add a note in noise section of schema for focus.yaml

commit 02509e9
Author: xhuang-jpl <118782850+xhuang-jpl@users.noreply.github.com>
Date:   Mon Sep 14 11:04:16 2026 -0700

    Add whirlwind unwrapping algorithm (#367)

    * sm r3.4

    * change the SM commit id for R4.0.2

    * add whirlwind

    * update the description

    * fix the whirlwind

    * update the unwrapping mask

    * combine the snaphu and whilrwind common functions

    * add the whirlwind dependencies to the environment.yml

    * fix the algorithm selection bug

    * update the whirlwind-insar version to be 0.10.0

    * update the python version to 3.11

    * add two additional options from the whirlwind 0.10.0

    ---------

    Co-authored-by: Xiaodong Huang <xhuang@nisar-adt-dev-3.jpl.nasa.gov>

commit cb7c3cb
Author: xhuang-jpl <118782850+xhuang-jpl@users.noreply.github.com>
Date:   Mon Sep 14 11:03:14 2026 -0700

    Fix InSAR Product Spec Issue #304 (#361)

    * sm r3.4

    * change the SM commit id for R4.0.2

    * fix the prod spec 304

    * raise the value error when the datatype is not Float32 or Float64

    ---------

    Co-authored-by: Xiaodong Huang <xhuang@nisar-adt-dev-3.jpl.nasa.gov>

commit 2c5a1fd
Author: xhuang-jpl <118782850+xhuang-jpl@users.noreply.github.com>
Date:   Mon Sep 14 11:02:43 2026 -0700

    Fix HDF5 Statistics Attributes Set to Zero for Multiple RUNW Datasetshe zero stats in the RUNW product  (#303)

    * sm r3.4

    * change the SM commit id for R4.0.2

    * fix the zero stats

    * add the dataset

    * fix the iono stats

    * copy the attributes from the source dataset

    * update the stats

    * fix typos

    ---------

    Co-authored-by: Xiaodong Huang <xhuang@nisar-adt-dev-3.jpl.nasa.gov>

commit deaadf1
Author: Brian Hawkins <1729052+bhawkins@users.noreply.github.com>
Date:   Fri Sep 11 16:29:56 2026 -0700

    RSLC missing data mask (#370)

    * WIP account for missing pulses in RSLC valid data mask.

    * Make mask parameters configurable.

    * Add docstrings for pulse mask/segment helper functions in Raw.py

    Document get_valid_pulse_mask, find_valid_pulse_intervals, split_segments,
    and join_segments to clarify their inputs/outputs for the missing-pulse
    valid data mask logic.

    Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

    * Add getValidPulses method to Raw reader

    * Add option to use RX mask in getSubSwathBboxes

    * Export some functions

    * Add a function to turn valid polygons into an image mask

    * Rasterize subswath polygons with GDAL instead of per-row Shapely intersection

    The row-by-row scanline intersection against every polygon was dominated by
    per-call Shapely/GEOS overhead. Build one in-memory OGR layer from the
    polygons and rasterize each output block with gdal.RasterizeLayer instead,
    which also correctly handles arbitrary (non-quadrilateral) polygon shapes.

    Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

    * Refactor sub-swath polygon computation and add save_valid_data_mask

    Split get_focused_sub_swaths into get_focused_sub_swath_polygons (bbox to
    polygon conversion + reskew) and the existing rasterize-to-indices step, so
    the polygon computation can be reused. Add save_valid_data_mask, which pairs
    that polygon computation with save_subswath_polygons_to_image to write a
    full-resolution valid-data mask directly instead of per-pulse indices.

    Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

    * Allow setting a specific bit in mask image and return total valid pixels.

    * Propagate bit and num_valid through save_valid_data_mask

    Bring save_valid_data_mask in line with the bit/num_valid support
    added to save_subswath_polygons_to_image.

    * Move helper function to a separate file.

    * Fix integer overflow (again!)

    * Split nisar get_focused_sub_swaths and add save_valid_data_mask

    Extract raw sub-swath bbox computation (per-observation lookup, chirp
    duration, seamless-observation merging) into get_raw_sub_swath_bboxes so
    it can be shared between get_focused_sub_swaths and a new
    save_valid_data_mask, mirroring the isce3-level refactor of
    valid_regions.py.

    * Assign valid-mask bit per polarization in nisar save_valid_data_mask

    Replace the bit argument with a lookup keyed on out_chan.pol so each
    polarization channel gets its own fixed bit, and add PolValidMask
    (public IntFlag) so callers can decode which channel(s) mark a pixel
    valid.

    * Use rx pulse mask and explicit polarization in save_valid_data_mask

    getSubSwathBboxes was relying on its default polarization (first
    available for the frequency), not necessarily the channel actually
    being processed. Pass polarization explicitly, and have
    save_valid_data_mask request use_rx_pulse_mask=True so the per-pol
    valid-data mask accounts for pulseHasValidSamples metadata specific
    to the receive channel.

    * Cast IntEnum to int to avoid numpy type promotion problem.

    * Add mask dataset creation to SLC writer

    * Add mask creation to top-level workflow.

    * Add valid pixel fraction attributes to mask.

    * numpy bool -> bool_

    * Kludge for old L0B data.

    * Combine validity mask with inputDataExceptionMask

    * Handle case where segment has only one pulse.

    * Avoid subtracting unsigned int

    * Fix typo

    * Compute min_segment_length based on synthetic aperture length

    * Compute min_segment_length in get_focused_sub_swaths from aperture length

    Mirrors the change already made to save_valid_data_mask: derive
    min_segment_length from min_segment_fraction and the synthetic aperture
    length instead of taking a fixed pulse count.

    * Log the calculated segment length

    * Discard epoch in _get_prf

    * Add docstring to _get_min_segment_length

    * Compute fraction of valid pulses and save as mask attribute.

    * Add docstring to get_valid_pulse_fraction

    * Add rawValidPulseFraction attribute to mask

    ---------

    Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>

commit 43b5cdf
Author: xhuang-jpl <118782850+xhuang-jpl@users.noreply.github.com>
Date:   Fri Sep 11 10:11:04 2026 -0700

    Fix InSAR Product Spec Issue #318 (#363)

    * sm r3.4

    * change the SM commit id for R4.0.2

    * fix the prod issue 318

    * make it string data typle

    ---------

    Co-authored-by: Xiaodong Huang <xhuang@nisar-adt-dev-3.jpl.nasa.gov>

commit fef7d70
Author: xhuang-jpl <118782850+xhuang-jpl@users.noreply.github.com>
Date:   Fri Sep 11 10:10:29 2026 -0700

    Fix InSAR Product Spec Issue #319 (#365)

    * sm r3.4

    * change the SM commit id for R4.0.2

    * fix prod spec 319

    ---------

    Co-authored-by: Xiaodong Huang <xhuang@nisar-adt-dev-3.jpl.nasa.gov>

commit a7b103d
Author: xhuang-jpl <118782850+xhuang-jpl@users.noreply.github.com>
Date:   Fri Sep 11 10:08:42 2026 -0700

    Fix InSAR Product Spec Issue #312 (#362)

    * sm r3.4

    * change the SM commit id for R4.0.2

    * fix 345

    ---------

    Co-authored-by: Xiaodong Huang <xhuang@nisar-adt-dev-3.jpl.nasa.gov>

commit 67bccb0
Author: Brian Hawkins <1729052+bhawkins@users.noreply.github.com>
Date:   Tue Sep 8 12:04:02 2026 -0700

    Improve definition of isMixedMode for RSLC (#373)

    * Improve definition of isMixedMode

    * Add unit test

commit 2e29e48
Author: Brian Hawkins <1729052+bhawkins@users.noreply.github.com>
Date:   Tue Sep 8 11:16:35 2026 -0700

    Improve error message related to missing telemetry (NSDS-5235) (#332)

    * Give a better error message when needed copol telemetry are missing.

    * Avoid an IndexError when handling missing copol telemetry.

commit 92d85c4
Author: Brian Hawkins <1729052+bhawkins@users.noreply.github.com>
Date:   Tue Sep 8 11:14:48 2026 -0700

    Fix BCAL normalization of first block (#286)

    * handle cases where first pulse isn't BCAL

    * Handle initial BCAL case before over-writing it.

commit cd233cd
Author: xhuang-jpl <118782850+xhuang-jpl@users.noreply.github.com>
Date:   Thu Sep 3 20:20:48 2026 -0700

    Fix InSAR Product Spec Issue #297 (#360)

    * sm r3.4

    * change the SM commit id for R4.0.2

    * fix the fill value 297

    * one more fix

    * update the dtype using np.dtype

    ---------

    Co-authored-by: Xiaodong Huang <xhuang@nisar-adt-dev-3.jpl.nasa.gov>

commit f42cea7
Author: Lijun Zhu <38738201+lijun99@users.noreply.github.com>
Date:   Tue Sep 1 16:57:46 2026 -0700

    Use optimized HDF5 reader for resample_slc_v2 (#372)

    resample_secondary_rslc_onto_reference() read the secondary RSLC via
    RSLC.getSlcDatasetAsNativeComplex(), which opens the file with h5py's
    default chunk cache instead of one sized for the dataset's own chunking.
    Open the secondary RSLC directly via HDF5OptimizedReader instead, matching
    the pattern already used in unwrap.py, crossmul.py, and other workflows.

    This cuts the resample step's SLC I/O read time by roughly 5x.

commit 23f9932
Author: Brian Hawkins <1729052+bhawkins@users.noreply.github.com>
Date:   Mon Aug 24 19:05:28 2026 -0500

    Solve CI failures (#366)

    * Skip latest version of pybind11

    * Fix out-of-bounds valarray access in TopoLayers::writeData

    Layers whose raster was never requested (e.g. the ground-to-satellite
    east/north components, which nisar.workflows.rdr2geo always disables)
    are left with size-0 valarrays by setBlockSize. writeData()
    unconditionally took &valarray[0] for every layer to build the variant
    list, which is out-of-bounds when the valarray is empty. This
    undefined behavior aborts under libstdc++ builds with bounds-checked
    valarray::operator[], crashing the Python interpreter during
    test.python.pkg.nisar.workflows.rdr2geo. Only take the address when
    the valarray is non-empty; the existing per-raster nullptr guard in
    the write loop already lines up with each valarray's resize condition.

    Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

    * Fix uninitialized aztime guess in geo2rdr GeoToRdr test

    The LUT2d-Doppler overload of geo2rdr uses the incoming aztime value
    as its initial azimuth time guess rather than always performing its
    own coarse search. The test left aztime uninitialized, so it relied
    on whatever garbage happened to be on the stack; that value must fall
    within the orbit's time span and the Doppler LUT2d's valid domain, or
    the very first iteration fails with an out-of-bounds lookup. This was
    silently working by chance until a toolchain change altered the stack
    layout, causing OutOfBoundsLookup and a spurious test failure.
    Initialize aztime to orbit.midTime(), which is guaranteed to satisfy
    both constraints.

    Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

    * Fix out-of-bounds vector access in FrameTest::SetUp

    est_xyz_eaz/eoa/aoe/tp were reserve()'d instead of resize()'d before
    being indexed, leaving size() at 0. Newer libstdc++ bounds-checking
    in operator[] catches this and aborts test.cxx.isce3.antenna.frame.

    * Put upper bound on pybind11 version

    * Fix missing pybind11/eigen.h include in DryTroposphereModel.cpp

    dryTropoDelayTSX() binds Eigen (Vec3) arguments but this file never included pybind11/eigen.h, so Vec3 resolved to the generic type_caster_base here instead of the Eigen/numpy specialization used elsewhere (e.g. slantRangeFromLookVec in pntintersect.cpp, which shares the exact same argument types). That is an ODR violation: two translation units instantiate the same pybind11 internal template with genuinely different content. It was apparently harmless under pybind11 2.x, but crashes import on pybind11 3.x with an internal type-signature-parsing error.

    Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

    * Revert cmake upper bound on pybind11 version and bump fetch to latest version.

    * Revert "Skip latest version of pybind11"

    This reverts commit b4559f4.

    * Rename lambda ptr -> safe_deref

    * Explicitly initialize slantRange

    ---------

    Co-authored-by: bhawkins <bhawkins@github.com>
    Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants