0.76.0

Changes

⚠️ Breaks

• Remove ArrayAccessor (#8603) @robert3005
• Fix bounded max partial state (#8515) @gatesn
• Use aggregate descriptors for zoned stats (#7938) @gatesn
• Remove duckdbfs (#8468) @myrrc
• Move execute-parent kernels into session registry (#8482) @gatesn
• break: add Validity::definitely_all_null() method (#8447) @joseph-isaacs
• Make Mask execution strict about nullable booleans (#8121) @joseph-isaacs
• Support reporting statistics in spark datasource (#8057) @robert3005

✨ Features

• Add swimlane view to vortex-web that visualizes physical layout of columns in files (#8634) @robert3005
• feat(vortex-bench): infra of SpatialBench on DuckDB, plain binary (#8598) @HarukiMoriarty
• Expose array buffer replacement (#8606) @gatesn
• bool -> Primitive casts (#8621) @myrrc
• feat: add metadata bridge for Python CUDA export (#8604) @0ax1
• Add list_length scalar function (#8495) @mhk197
• feat(file): make Footer::new public (#8602) @tobias-fire
• feat(array): push struct validity into children (#8589) @miniex
• Support session-time source configuration in DataFusion, and clear up precedence between config sources (#8575) @AdamGS
• feat(python): package CUDA as an optional extension (#8510) @0ax1
• Decimal casting (#8562) @gatesn
• Add JSON to Parquet Variant conversion with shredding (#8391) @AdamGS
• Add extension storage scalar function (#8540) @gatesn
• Numerical aggregate functions have an option to skip or include nans in calculation, skip by default (#8457) @robert3005
• feat(vortex-geo): ST_Distance via the geo crate + Polygon type (#8497) @HarukiMoriarty
• feat[gpu]: arrow device array stream support (#8483) @0ax1
• feat(vortex-datafusion): struct scalar conversion + extension-over-struct scan (#8453) @HarukiMoriarty
• feat: DeltaScheme adjustable compression ratio threshold (#8461) @joseph-isaacs
• Add Delta scheme to the integer compressor (unstable_encodings) (#8212) @joseph-isaacs
• Push down some expressions to Dict layout reader's cached values (#8341) @myrrc
• --patterns flag for random-access-bench (#8446) @myrrc
• ffi: create data source from memory buffer (#8426) @myrrc
• Constant comparison and byte_length OnPair kernels (#8371) @myrrc
• REGEX support for sqllogictests (#8392) @myrrc
• feat(vortex-geo): Arrow import/export for the native Point type (#8374) @HarukiMoriarty

🚀 Performance

• perf(datafusion): push down list_length expression (#8600) @mhk197
• perf: cache static identifiers to avoid per-call interner locks (#8614) @miniex
• perf(duckdb): push down list length expressions (#8544) @mhk197
• Spawn arrow conversion in jni bindings (#8595) @robert3005
• Compute zone layout stats concurrently when writing (#8594) @robert3005
• Chunked dict values pullup (#8577) @gatesn
• Tune varbinview compaction for Arrow export (#8585) @gatesn
• Report DuckDB max cardinality for exact scans (#8582) @gatesn
• perf(scan): intra-file decode parallelism — sub-split large chunk spans (#8400) @lukekim
• perf(alp): avoid per-combination allocations when searching exponents (#8565) @miniex
• Optimize interleave boolean gather (#8350) @joseph-isaacs
• RLEExporter for duckdb (#8539) @myrrc
• ReusableDict for RunEnd duckdb exporter (#8538) @myrrc
• perf: align buffer to Alignment::DEFAULT_ALIGNMENT = Alignment::new(256) (#8490) @joseph-isaacs
• DuckDB chunk exporter (#8520) @gatesn
• Faster take for runend array (#8228) @robert3005
• Inline Kleene boolean kernels (#8478) @gatesn
• Scalar functions pushdown to Duckdb (#8303) @myrrc
• perf: don't use fill_null for executing an array into a mask (#8466) @joseph-isaacs
• Optimize buffer ops (#8322) @AdamGS
• perf[gpu]: export arrow device validity on the gpu (#8440) @0ax1
• perf[buffer]: iteration for fallible operations with validity (#8120) @joseph-isaacs
• Faster str interning (#8459) @AdamGS
• perf[gpu]: generate FixedSizeList offsets on device (#8458) @0ax1
• Arithmetic Operations (#8398) @gatesn
• perf(buffer): read byte-aligned bit operands directly as words (#8436) @miniex
• perf: non-blocking Arrow Device array release (#8390) @0ax1

🐛 Bug Fixes

21 changes

• Convert extension type scalars to Value correctly (#8635) @myrrc
• fix(vortex-bench): map gs:// scheme to gcs storage label (#8630) @brancz
• fix: CUDA bitpacked sliced output allocation (#8622) @0ax1
• Fix wasm32 build by gating MultiFileSession on non-wasm targets (#8612) @robert3005
• fix: support 64K alignment (#8599) @a10y
• Guard calling is_nan on scalar value on dtype being a float (#8593) @robert3005
• vortex-datafusion: Pipe session through to converter (#8591) @brancz
• Save children dtypes/length in ZstdBuffersMetadata (#8572) @myrrc
• fix(array): don't panic on unsupported arrow types (#8564) @miniex
• fix(layout): don't panic collecting an empty stream (#8472) @miniex
• Correctly calculate FSST compressed output size (#8551) @robert3005
• refactor[fsst]: take ArrayRef in fsst_compress, decide offset width upfront (#7900) @mprammer
• Move optimizer kernels into a dedicated KernelSession (#8511) @gatesn
• Fix is_zero() for FixedSizeList and Struct scalar types (#8500) @connortsui20
• Handle null lanes in primitive take (#8505) @gatesn
• Preserve dict referenced values metadata (#8499) @gatesn
• fix: preserve operand width in DecimalValue checked arithmetic (#7380) @abnobdoss
• fix Fuzz constant OOM for CUDA (#8481) @a10y
• vortex-tui: query tab should work for non-*.vortex files (#8473) @a10y
• Ignore PyO3 rustsec (#8381) @connortsui20
• fix: vx_data_source_new_buffer sem-merge conflict (#8437) @joseph-isaacs

📖 Documentation

• Expanding conceptual docs (and some other minor docs) (#8552) @AdamGS
• docs: update doc links (#8375) @joseph-isaacs

🧰 Maintenance

49 changes

• Back VortexSession with ArcSwap for mutable but less errorprone session management (#8631) @robert3005
• chore: forbid the locking Id constructors with a clippy lint (#8617) @miniex
• Patches have correct dtype by construction instead of normalised during array construction (#8626) @robert3005
• chore: Python CUDA bridge: CI and buffer handoff ABI (#8618) @0ax1
• Lock file maintenance (#8616) @renovate[bot]
• Polish treemap view in vortex explorer (#8613) @robert3005
• Migrate WASM Arrow conversion to use VortexSession (#8619) @robert3005
• Lock file maintenance (#8615) @renovate[bot]
• Improve SLT test harness (#8567) @AdamGS
• Add benchmarks for decimal casting (#8569) @robert3005
• Use AllNonDistinct in assert_arrays_eq and implement it for variant types (#8546) @robert3005
• Refactor DictEncoder to not use ArrayAccessors (#7759) @robert3005
• Clear 3 orphan json files from root of repo (#8558) @AdamGS
• Remove LEGACY_SESSION from benchmarks and back compat tests (#8554) @robert3005
• Bump codspeed to 5.0 (#8553) @robert3005
• Remove usages of LEGACY_SESSION from tests (#8547) @robert3005
• assert_arrays_eq and assert_nth_scalar require ExecutionCtx (#8509) @robert3005
• Update Rust crate memmap2 to v0.9.11 [SECURITY] (#8545) @renovate[bot]
• Update lance benchmark dependencies (major) (#8525) @renovate[bot]
• Update actions/setup-java digest to ad2b381 (#8521) @renovate[bot]
• Update anthropics/claude-code-action digest to 2fee155 (#8522) @renovate[bot]
• Update taiki-e/install-action digest to 9e1e580 (#8523) @renovate[bot]
• Update actions/checkout action to v7 (#8524) @renovate[bot]
• Lock file maintenance JS lock file maintenance (#8536) @renovate[bot]
• Lock file maintenance Rust lock file maintenance (#8537) @renovate[bot]
• Make sure both pushdown features are always enabled for DF benchmarks (#8507) @AdamGS
• Misc upgrades to VortexSource (#8501) @AdamGS
• Generalize SIMD take operations to Copy (#8496) @connortsui20
• Migrate to new stats falsification rules (#8345) @gatesn
• codegen only used functions in duckdb C API (#8491) @myrrc
• Change ALIGNMENT_TO_HOST_COPY to Alignment::HOST_COPY (#8488) @AdamGS
• Touch up some of the interfaces and docs in vortex-datafusion (#8485) @AdamGS
• Add benchmarks for take on runend array (#8469) @robert3005
• Include vortex-compute in codspeed benchmarks (#8477) @robert3005
• Add Kleene boolean benchmarks (#8476) @gatesn
• Immutable session backed by HashMap (#8454) @AdamGS
• test[gpu]: expand cuDF CUDA e2e coverage (#8463) @0ax1
• Upgrade pyO3 to 0.29.0 (#8462) @robert3005
• Update dependency starlette to v1.3.1 [SECURITY] (#8435) @renovate[bot]
• fmt: normalize_comments = true (#8428) @joseph-isaacs
• Update taiki-e/install-action digest to 7a79fe8 (#8409) @renovate[bot]
• Update gradle/actions digest to 3f131e8 (#8408) @renovate[bot]
• Update anthropics/claude-code-action digest to d5726de (#8407) @renovate[bot]
• Update dependency duckdb to v1.4.2 [SECURITY] (#8431) @renovate[bot]
• Update dependency pip to v26.1.2 [SECURITY] (#8432) @renovate[bot]
• Align data buffers in fastlanes benchmarks (#8393) @AdamGS
• Update plugin com.palantir.java-format to v2.93.0 (#8416) @renovate[bot]
• More arithmetic benchmarks (#8402) @gatesn
• Add binary operator benchmarks (#8396) @gatesn

0.75.0

Changes

⚠️ Breaks

• aggregate fns to have grouped aggregate kernels for sum and count (#8314) @onursatici
• Remove len parameter from ScalarFnArray::try_new (#8378) @connortsui20
• Remove all TurboQuant stuff (#8377) @connortsui20
• Fix Validity::mask_eq semantics for mixed variants (#8334) @joseph-isaacs
• Rename Validity::no_nulls to definitely_no_nulls and add execute_no_nulls (#8333) @joseph-isaacs
• Pass execution context through list view rebuild (#8274) @connortsui20
• Initialize LazyBitBufferBuilder from Mask if available, when appending masks pass them by reference (#8221) @robert3005
• layout reader ctx (#8037) @onursatici

🚧 Deprecation

• Refactor validity checks to require explicit ExecutionCtx (#8273) @joseph-isaacs
• Deprecate vortex-array public APIs that use the hidden LEGACY_SESSION (#8269) @joseph-isaacs

✨ Features

• feat(vortex-geo): native Point extension type and GeoDistance scalar function (#8372) @HarukiMoriarty
• Add arrow import/export for vortex-json (#8339) @AdamGS
• WKB import/export VTables to Arrow (#8325) @a10y
• feat[gpu]: export as Arrow device array binary (#8320) @0ax1
• feat[gpu]: sliced validity in Arrow device export (#8318) @0ax1
• add vortex-geo crate and WKB extension type (#7722) @a10y
• Expose Merge expression in vortex-jni (#8319) @robert3005
• feat(vortex-array): add Interleave array encoding (#8277) @joseph-isaacs
• feat: add cuDF C FFI (#8236) @0ax1
• feat: export dictionary arrow device arrays (#8251) @0ax1
• Add zstd compression schemes for binary data (#8304) @AdamGS
• Upgrade DataFusion to 54 (#8044) @AdamGS
• byte_length() expression (#8298) @myrrc
• feat(vortex-row): row-oriented byte encoder (size + encode passes) (#8253) @joseph-isaacs
• Support pack expressions through JNI (#8266) @AdamGS
• feat: very wide column benchmark (#8252) @joseph-isaacs
• Support ParquetVariant through JNI (#8129) @AdamGS
• Introduce a new JSON extension type in vortex-json (#8215) @AdamGS
• feat[gpu]: arrow device array decimal export (#8155) @0ax1
• feat[gpu]: arrow device array list view export (#8219) @0ax1
• Add RoaringBitmap support to vortex-jni bindings (#8220) @robert3005
• Implement ZipKernel for ListViewArray (#8218) @robert3005
• move towards BitBuffer/{Mut}View over owned bit buffers (#8208) @joseph-isaacs

🚀 Performance

• perf: branchless primitive zip kernel (#8270) @joseph-isaacs
• perf[gpu]: coalesce streaming kernels (#8357) @0ax1
• perf[mask]: check AllTrue/AllFalse in eq (#8354) @joseph-isaacs
• BitBuffer binary operations support in place updates to lhs (#8162) @robert3005
• Optimize slice for dict and minor changes in other arrays (#8321) @AdamGS
• perf: remove unneeded scalar_at and valid_at and use arrays instead (#8307) @joseph-isaacs
• Allow caching LayoutReader in VortexFile (#8244) @myrrc
• perf: branchless boolean zip kernel (#8275) @joseph-isaacs
• Initial impl for sort pushdown in DataFusion FileSource implementation (#8235) @AdamGS
• Optimize dict array validation to use true_count() (#8263) @joseph-isaacs
• Add #[inline] hints to hot path functions (#8260) @joseph-isaacs
• perf(fastlanes): fuse bit-packed compare into a transposed mask + untranspose (#8239) @joseph-isaacs
• case_when with is_null -> fill_null (#8245) @onursatici
• perf: optimize varbinview construction (#8146) @joseph-isaacs
• ViewedLayoutChildren child layout cache (#8234) @myrrc
• Don't recalculate chunk offsets for ChunkedReader (#8231) @myrrc
• Share FSST symbol table state between executed variants of arrays. (#8222) @AdamGS

🐛 Bug Fixes

13 changes

• fix: keep Arrow device schemas aligned with exports (#8360) @0ax1
• aggregate_fn: Return NaN instead of null for mean of all-null input (#8365) @dimitarvdimitrov
• fix[file]: read the one-row pruning result in can_prune instead of re… (#8369) @tomsanbear
• Fix generation of stat pruning expression for unsupported dtypes (#8326) @robert3005
• fix[gpu]: decode extension arrays on the GPU (#8353) @0ax1
• fix issue with FileSystem path handling (#8248) @robert3005
• Wrap object stores in python with Compat layer (#8316) @robert3005
• Fix semantic conflict in vortex-array (#8312) @AdamGS
• fix: [profile.bench] codegen-units = 16 (#8257) @joseph-isaacs
• fix grouped sum index for listview array (#8255) @onursatici
• split registration to respect nested projection and filters (#8213) @onursatici
• Fix zoned min/max over fixed-size-list stats (#8243) @gatesn
• fix session deadlock (#8217) @onursatici

📖 Documentation

• Validate all rustdoc builds (#8386) @AdamGS

🧰 Maintenance

40 changes

• chore: clean up Arrow device export (#8359) @0ax1
• chore: polish cuDF C FFI (#8358) @0ax1
• Remove the unused website and clean other dependencies (#8362) @AdamGS
• Minor touchups for vortex-datafusion (#8356) @AdamGS
• Add docs for StatsRewriteRule (#8355) @AdamGS
• Disable wide-columns benchmark to stop benchmarks from timing out (#8331) @AdamGS
• Optimize fsst_compress_offsets_overflow_i32 by 2X locally (#8324) @AdamGS
• Add slicing benchmarks for vortex-buffer (#8323) @AdamGS
• test[gpu]: more e2e tests (#8317) @0ax1
• Publish Parquet-Variant encoding (#8313) @AdamGS
• Fix javadoc warnings and run lints in ci (#8309) @robert3005
• Update all patch updates (#8286) @renovate[bot]
• Update Rust crate tabled to 0.21.0 (#8184) @renovate[bot]
• Update logback monorepo to v1.5.34 (#8287) @renovate[bot]
• Update taiki-e/install-action digest to 0631aa6 (#8285) @renovate[bot]
• Update actions/checkout digest to df4cb1c (#8281) @renovate[bot]
• Update anthropics/claude-code-action digest to fbda2eb (#8282) @renovate[bot]
• Lock file maintenance Rust lock file maintenance (#8296) @renovate[bot]
• Update aws-actions/configure-aws-credentials digest to e7f100c (#8283) @renovate[bot]
• Update netty monorepo to v4.2.15.Final (#8288) @renovate[bot]
• Update storybook monorepo to v10.4.2 (#8290) @renovate[bot]
• Update react monorepo (#8289) @renovate[bot]
• Update tokio-prost monorepo to v0.14.4 (#8291) @renovate[bot]
• Update typescript-eslint monorepo to v8.60.1 (#8292) @renovate[bot]
• Update tanstack-virtual monorepo to v3.14.2 (#8293) @renovate[bot]
• Lock file maintenance JS lock file maintenance (#8295) @renovate[bot]
• Update codecov/codecov-action action to v7 (#8294) @renovate[bot]
• Update codecov/codecov-action digest to fb8b358 (#8284) @renovate[bot]
• Ignore RUSTSEC-2026-0173 (#8300) @AdamGS
• Rename ArrayEq/Hash Precision to EqMode (#8214) @AdamGS
• use octet_length() instead of len() in clickbench (#8267) @myrrc
• Mask bool iterator (#8258) @onursatici
• Update dependency starlette to v1 [SECURITY] (#8254) @renovate[bot]
• bench: sweep bit-packed compare across all int types and bit widths (#8238) @joseph-isaacs
• Make from_arrow not panic (#8242) @connortsui20
• clean: drop unused API lock-file (#8241) @0ax1
• CachedIds for functions (#8240) @myrrc
• vortex-io: Add explicit span on spawn_io (#8232) @brancz
• Add select_unpredictable hint to search_sorted (#8225) @AdamGS
• Add ArcSwapMap and use it throughout our session registries (#8072) @robert3005

0.74.0

Changes

⚠️ Breaks

• ffi: remove vx_file scan (#8152) @myrrc

✨ Features

• Expose row_idx in vortex-jni bindings (#8211) @robert3005
• Support for importing Arrow extension arrays into core vortex arrays and ParquetVariant support. (#8125) @AdamGS
• feat: add experimental onpair string encoding (#8144) @joseph-isaacs
• Teach zoned pruning to lower StatFn (#7937) @gatesn
• TurboQuant: Add distortion benchmark (#8070) @connortsui20
• Add UUID literal expression to vortex-jni (#8154) @robert3005
• Introduce basic binary compressor schemes and refactor some other bits (#8153) @AdamGS

🚀 Performance

• perf: trim zctl listview instead of full rebuild (#8164) @mhk197
• Use Vec instead of BTreeSet for splits (#8194) @myrrc
• Skip empty chunks when aggregating over chunked arrays (#8168) @robert3005
• perf: fast FixedSizeListArray canonicalization for chunked arrays (#8161) @joseph-isaacs
• Always use FileStatsLayoutReader if available for VortexFile reader (#8147) @connortsui20
• Use selection in repeated scans (#8137) @myrrc
• Reduce monomorphization by dispatching over unsigned integer widths (#8110) @joseph-isaacs

🐛 Bug Fixes

• fix(fuzz): VortexError in file_io (#8189) (#8201) @vortex-claude[bot]
• fix: onpair use crates verison (#8200) @joseph-isaacs
• Fix unsound Delta unsigned-widening cast (#8195) @joseph-isaacs
• fix[fastlanes]: remove unsound delta cast? (#8192) @joseph-isaacs
• Always try compressing extension array's storage directly (#8157) @AdamGS
• fix: code coverage ci task (#8142) @0ax1
• fix: speed up coverage tests and LLVM tools path (#8136) @0ax1
• Pin cargo-edit to version that still supports rust 1.91.0 (#8141) @robert3005

📖 Documentation

• docs: link to Vortex Explorer (#8132) @danking

🧰 Maintenance

22 changes

• Refactor: Split monolithic scheme files into modular submodules (#8207) @joseph-isaacs
• test: expand utf8view and binaryview cuda export coverage (#8159) @0ax1
• Update all patch updates (#8174) @renovate[bot]
• don't account array deallocation in random access bench (#8160) @myrrc
• Update aws-actions/configure-aws-credentials digest to 99214aa (#8170) @renovate[bot]
• Update docker/setup-qemu-action digest to 0611638 (#8171) @renovate[bot]
• Update EmbarkStudios/cargo-deny-action digest to bb137d7 (#8172) @renovate[bot]
• Update taiki-e/install-action digest to e49978b (#8173) @renovate[bot]
• Update Rust crate opentelemetry_sdk to v0.32.1 (#8178) @renovate[bot]
• Update dependency concurrently to v10 (#8186) @renovate[bot]
• Update Rust crate serial_test to v3.5.0 (#8183) @renovate[bot]
• Update plugin com.diffplug.spotless to v8.6.0 (#8182) @renovate[bot]
• Update dependency typescript-eslint to v8.60.0 (#8181) @renovate[bot]
• Update dependency lucide-react to v1.17.0 (#8180) @renovate[bot]
• Update crate-ci/typos action to v1.47.0 (#8179) @renovate[bot]
• Update dependency eslint to v10.4.1 (#8177) @renovate[bot]
• Update dependency ch.qos.logback:logback-classic to v1.5.33 (#8176) @renovate[bot]
• Update dependency @tanstack/react-virtual to v3.13.26 (#8175) @renovate[bot]
• test: expand cuDF e2e primitive coverage (#8149) @0ax1
• Fix file size baseline detection (#8151) @AdamGS
• test[gpu]: expand primitive Arrow Device export coverage (#8134) @0ax1
• test: nullable cuDF e2e tests (#8128) @0ax1

0.73.0

Changes

✨ Features

• feat: support nullable Arrow Device array export (#8104) @0ax1
• Add built-in stats rewrite rules (#7935) @gatesn
• Add FilterPushdown support to spark data source (#7785) @robert3005
• Explicit Precision::Absent variant instead of Option (#8042) @AdamGS
• Expression pushdown for duckdb (#7727) @myrrc
• Benchmarks Website V3: Admin and Auto-Deploy (#7849) @connortsui20
• Add pruning aggregate functions (#8025) @gatesn
• Allow writing Variant to files and test parquet-variant IO (#7945) @AdamGS

🚀 Performance

• TurboQuant: better centroid initialization (#8116) @connortsui20
• Use CachedId for ArrayExpr scalar function ID (#8124) @joseph-isaacs
• Add Sparse pushdown kernels for is_constant, sum, and compare (#8028) @joseph-isaacs
• perf: aggregate min/max (#8061) @joseph-isaacs
• fastlanes: streaming compare + between kernels for BitPacked (#8015) @joseph-isaacs
• perf: remove implicit ListViewArray rebuild during take and filter operations (#8048) @mhk197
• perf[array]: bool filter kernel optimisation (#7125) @joseph-isaacs
• perf[array]: cast to prim faster (#8062) @joseph-isaacs
• Add bit-packed widening cast pushdown (#8059) @joseph-isaacs
• Add TakeExecute implementation for FilterArray (#7393) @robert3005

🐛 Bug Fixes

10 changes

• Correctly handle Nan/Inf comparison in ALP between reduce (#8126) @robert3005
• fsst like to respect sql escape codes in the pattern (#8107) @onursatici
• Fix native library publish for non amd64 platforms (#8108) @robert3005
• wipe duckdb source in build.rs if not fully extracted before (#8106) @onursatici
• like stat falsification respects escape characters (#8103) @onursatici
• Fix dtype mismatch in Filter::take logic (#8073) @robert3005
• Handle overflow in decimal between implementation if the passed scalars fit in the array's logical but not the physical type (#8017) @vortex-claude[bot]
• When delegating to arrow comparison make sure that both left and right hand side have the same arrow type (#8018) @vortex-claude[bot]
• fix: build CUDA kernels as multi-arch fatbin with PTX fallback (#8047) @0ax1
• Fallback from fsst specialised like expression if there are escape characters in the like string (#8038) @robert3005

📖 Documentation

• Document existing TurboQuant types (#8053) @connortsui20

🧰 Maintenance

31 changes

• Remove public-api lockfiles (#8099) @AdamGS
• Remove pre df 53.0 polarsignals benchmark workaround (#8122) @robert3005
• Use plain functions for copy function in duckdb (#8109) @myrrc
• -isystem for Duckdb headers, build with -Werror (#7747) @myrrc
• Update actions/stale digest to eb5cf3a (#8077) @renovate[bot]
• Update anthropics/claude-code-action digest to 787c5a0 (#8078) @renovate[bot]
• Update codecov/codecov-action digest to e79a696 (#8079) @renovate[bot]
• Update docker/build-push-action digest to f9f3042 (#8080) @renovate[bot]
• Update docker/login-action digest to 650006c (#8081) @renovate[bot]
• Update docker/setup-buildx-action digest to d7f5e7f (#8082) @renovate[bot]
• Lock file maintenance (#8094) @renovate[bot]
• Lock file maintenance (#8095) @renovate[bot]
• Add CITATION.cff (#8112) @mprammer
• Update taiki-e/install-action digest to 8f531ea (#8084) @renovate[bot]
• PrimitiveArrayExt::narrow expose ExecutionCtx (#8096) @joseph-isaacs
• Update all patch updates (#8085) @renovate[bot]
• Use plain functions instead of vtab in duckdb (#8102) @myrrc
• test: improve arrow device array test coverage (#8101) @0ax1
• Simplify some of the buffer impl (#8098) @AdamGS
• Update EmbarkStudios/cargo-deny-action digest to a531616 (#8083) @renovate[bot]
• Update junit-framework monorepo to v6.1.0 (#8091) @renovate[bot]
• Update storybook monorepo to v10.4.1 (#8090) @renovate[bot]
• Update plugin com.palantir.java-format to v2.91.0 (#8092) @renovate[bot]
• Update dependency typescript-eslint to v8.59.4 (#8089) @renovate[bot]
• Update dependency io.netty:netty-bom to v4.2.14.Final (#8088) @renovate[bot]
• Update dependency @tanstack/react-virtual to v3.13.25 (#8086) @renovate[bot]
• Update dependency @types/react to v19.2.15 (#8087) @renovate[bot]
• Fix random-access benchmark console display (#8054) @robert3005
• Skip computation of expression return dtype unless there are no chunks to return (#8045) @robert3005
• chore: polish ArrowDeviceArray (#8023) @0ax1
• Revert "Remove type coercion" (#8041) @gatesn

0.72.0

Changes

✨ Features

• Thread scope dtype through stats rewrites (#8024) @gatesn
• Add Samply and benchmark skills (#8021) @gatesn

🚀 Performance

• Slice list/list_view elements in duckdb exporter (#8020) @myrrc

🐛 Bug Fixes

• Fix nullability of Constant BetweenReduce rule (#8029) @robert3005

🧰 Maintenance

4 changes

• Avoid re-building vortex-duckdb twice (#8035) @AdamGS
• Bump DuckDB to 1.5.3 (#8031) @AdamGS
• Remove type coercion (#8032) @gatesn
• chore[gpu]: error out in case of mixed cpu/gpu execution (#8016) @0ax1

0.71.0

Changes

⚠️ Breaks

• Switch python runtime to CurrentThreadRuntime (#7896) @robert3005

✨ Features

• pluggable registry for input/export arrow kernels (#7824) @a10y
• feat[fastlanes]: allow delta to support signed bases (#7923) @joseph-isaacs
• Updated Variant array and the new VariantGet expression (#7877) @AdamGS
• Add Arrow to Vortex conversion to C FFI (#7906) @robert3005
• Add statistic expression (#7854) @gatesn
• Add DType::Union variant carrying just Nullability (#7901) @connortsui20
• tracing logger for duckdb (#7892) @myrrc
• simplify projection expression for SELECT * in duckdb (#7885) @myrrc
• TurboQuant again! (#7829) @connortsui20
• feat: all_non_distinct aggregation make fuzzer faster (#7827) @joseph-isaacs
• Add a new AggregateFn for UncompressedSize (#7715) @AdamGS
• feat: iterative execution for SparseArray (#7711) @joseph-isaacs
• feat: check VortexReadAt::read_at results in the I/O driver (#7783) @danking
• feat[vortex-cuda]: GPU FSST decompression kernel (#7776) @asubiotto
• Make struct cast implementation pluggable (#7684) @robert3005
• [claude] feat(bench): emit v3 JSONL records and dual-write to bench server (#7780) @connortsui20

🚀 Performance

• perf[array]: add the SimplifyCache to optimize (#7948) @joseph-isaacs
• Mimic duckdb's post-filter cardinality estimates (#7895) @myrrc
• Use many connections on the benchmarks server (#7852) @connortsui20
• Faster Mask::from_slices (#7857) @gatesn
• Improve intersect_by_rank performance (#7744) @robert3005
• perf[array]: small vec for slots (#7825) @joseph-isaacs
• perf[array]: outline the array parts data from the DynArray (#7750) @joseph-isaacs
• Remove more scalar_at from sparse arrays (#7439) @gatesn

🐛 Bug Fixes

21 changes

• BufferMut::zeroed_aligned stores actually allocated length instead of passed length (#7979) @robert3005
• move getrandom backend override to config.toml (#7983) @onursatici
• fix[gpu]: retain device buffers for dyn dispatch kernel (#7980) @0ax1
• fix[gpu]: handle sliced BP arrays in CUDA (#7912) @0ax1
• duckdb: flatten runend arrays on export if requested (#7951) @myrrc
• Fix segfault when calling vx CLI from Python bindings (#7947) @joseph-isaacs
• register_splits to get both offset and relative row_range (#7913) @onursatici
• Fix IsSorted return dtype (#7914) @connortsui20
• fix[gpu]: CUDA sliced patch views with device patches (#7911) @0ax1
• Reorder agg kernel dispatch, and have Combined use inner accumulators (#7889) @gatesn
• Widen VarBinBuilder offets during FSST compress (#7853) @connortsui20
• fix[array]: get_item reduce + struct wrong nullability (#7887) @joseph-isaacs
• fix[bitpacked]: slice patches in execute method (not reduce). (#7839) @joseph-isaacs
• Stop ignoring tests (#7844) @robert3005
• Fix features of vortex-python (#7837) @robert3005
• fix[py]: remove mimalloc in library (#7826) @joseph-isaacs
• fix[sparse]: ensure cached patches are updated and use PatchesData (#7811) @joseph-isaacs
• fix(ffi): double-free in partition scan on error (#7817) @m7kss1
• fix: partition in filter should err (not panic). (#7816) @joseph-isaacs
• Fix logo on benchmarks website (#7779) @connortsui20
• fix: make ExtDType metadata deserialization total over byte input (#7782) @gatesn

🧰 Maintenance

74 changes

• Update Rust crate lance-encoding to v6 (#7974) @renovate[bot]
• Lock file maintenance (#7976) @renovate[bot]
• Update anthropics/claude-code-action digest to 51ea8ea (#7959) @renovate[bot]
• Update cloudflare/wrangler-action action to v4 (#7972) @renovate[bot]
• Lock file maintenance (#7975) @renovate[bot]
• Update EmbarkStudios/cargo-deny-action digest to 6c8f9fa (#7960) @renovate[bot]
• Update taiki-e/install-action digest to 7be9fd8 (#7961) @renovate[bot]
• Update all patch updates (#7962) @renovate[bot]
• Update dependency lucide-react to v1.16.0 (#7967) @renovate[bot]
• Update arrow-rs (#7965) @renovate[bot]
• Update storybook monorepo to v10.4.0 (#7970) @renovate[bot]
• Update Rust crate dashmap to v6.2.1 (#7969) @renovate[bot]
• Update plugin com.diffplug.spotless to v8.5.1 (#7968) @renovate[bot]
• Update dependency eslint to v10.4.0 (#7966) @renovate[bot]
• Update slf4j monorepo to v2.0.18 (#7964) @renovate[bot]
• Update dependency typescript-eslint to v8.59.3 (#7963) @renovate[bot]
• Clippy deny absolute_paths longer than 3 elements (#7950) @robert3005
• Add stats rewrite session API (#7930) @gatesn
• Add a nicer progress bar and file-based filter (#7942) @AdamGS
• Use async_fs file for java and python writes to avoid object_store buffering on local disks (#7920) @robert3005
• Add NullCount aggregate function (#7933) @gatesn
• Centralize aggregate stat bridge (#7931) @gatesn
• Remove chunked special case from stat execution (#7928) @gatesn
• Remove needless fmt_sql (#7409) @gatesn
• Add more benchmarks for take on filter array (#7876) @robert3005
• use nonnull in lifetime_wrapper (#7908) @myrrc
• Pin nightly toolchain version in public-api (#7909) @robert3005
• Update flatbuffers generated (#7899) @connortsui20
• Add Executor::spawn_io (#7894) @AdamGS
• Update dependency urllib3 to v2.7.0 [SECURITY] (#7888) @renovate[bot]
• Remove unused protobuf dependency in java bindings (#7886) @robert3005
• bench: dyn dispatch / standalone perf comparsion (#7883) @0ax1
• Update all patch updates (#7862) @renovate[bot]
• Lock file maintenance (#7863) @renovate[bot]
• Pin actions/checkout action to de0fac2 (#7858) @renovate[bot]
• Update anthropics/claude-code-action digest to 476e359 (#7859) @renovate[bot]
• Update Rust crate noodles-vcf to 0.88.0 (#7870) @renovate[bot]
• Update aws-actions/configure-aws-credentials digest to d979d5b (#7860) @renovate[bot]
• Update taiki-e/install-action digest to fa0dd4c - autoclosed (#7861) @renovate[bot]
• Lock file maintenance (#7864) @renovate[bot]
• Update dependency io.netty:netty-bom to v4.2.13.Final (#7865) @renovate[bot]
• Update Rust crate noodles-bgzf to 0.47.0 (#7869) @renovate[bot]
• Update opentelemetry-rust monorepo to 0.32.0 (#7867) @renovate[bot]
• Update tailwindcss monorepo to v4.3.0 (#7872) @renovate[bot]
• Update release-drafter/release-drafter action to v7.3.0 (#7868) @renovate[bot]
• Update react monorepo to v19.2.6 (#7866) @renovate[bot]
• Collect slots directly into smallvec instead of using vec and into (#7847) @robert3005
• bench[gpu]: CUDA device memory pool benchmarks (#7831) @0ax1
• Use smallvec for internal stats storage (#7823) @robert3005
• bench[gpu]: overlap of compute and copying (#7828) @0ax1
• bench: CUDA host-to-device copy modes (#7815) @0ax1
• use rstest for some test cleanups (#7813) @a10y
• Update dependency pip to v26.1 [SECURITY] (#7805) @renovate[bot]
• Run vortex-mask benchmarks with codspeed (#7804) @robert3005
• Add more benchmarks for intersect by rank (#7800) @robert3005
• bench: CUDA sync load-to-device benchmark (#7799) @0ax1
• Add Patches lookup benchmarks (#7795) @palaska
• Allow Claude to open PRs (#7798) @gatesn
• Fix weird signature of with_slots functions (#7758) @robert3005
• Update anthropics/claude-code-action digest to 2cc1ac1 (#7761) @renovate[bot]
• Update taiki-e/install-action digest to 711e1c3 (#7762) @renovate[bot]
• Update release-drafter/release-drafter action to v7.2.1 (#7763) @renovate[bot]
• Update storybook monorepo to v10.3.6 (#7765) @renovate[bot]
• Update crate-ci/typos action to v1.46.0 (#7766) @renovate[bot]
• Update dependency eslint to v10.3.0 (#7767) @renovate[bot]
• Update dependency globals to v17.6.0 (#7768) @renovate[bot]
• Update Gradle to v9.5.0 (#7770) @renovate[bot]
• Lock file maintenance (#7772) @renovate[bot]
• Lock file maintenance (#7771) @renovate[bot]
• Update dependency typescript-eslint to v8.59.2 (#7764) @renovate[bot]
• Update dependency lucide-react to v1.14.0 (#7769) @renovate[bot]
• Allow bot approvals (#7774) @gatesn
• Fix Claude workflow cancellation (#7775) @gatesn
• chore: codspeed display names (#7752) @0ax1

0.70.0

Changes

⚠️ Breaks

• Remove PartialOrd implementation for ScalarValue (#7742) @connortsui20
• New Java Scan API (#7527) @robert3005
• Make StatsCache and ArrayAndStats have interior mutability (#7583) @connortsui20
• Pass ExecutionCtx through the compressor (#7578) @connortsui20
• break: remove deprecated methods and plumb ExecutionCtx (#7512) @joseph-isaacs
• break: remove try_from struct array to record batch (#7488) @joseph-isaacs

🚧 Deprecation

• deprecate: into_arrow (#7577) @joseph-isaacs
• deprecate: execute_mask over to_mask (#7574) @joseph-isaacs
• depreacte non compute methods without a ctx (e.g. to_canonical) (#7473) @joseph-isaacs

✨ Features

• feat: add builder execution path with AppendChild step (#7677) @joseph-isaacs
• feat: iterative execution for DateTimePartsArray (#7698) @joseph-isaacs
• feat(ffi): add literal expression support (#7675) @m7kss1
• feat: teach Array try_concat(&[Array]) (#7632) @danking
• Late materialization support for duckdb (#7631) @myrrc
• feat: masked array iterative execution clean up (#7690) @joseph-isaacs
• feat(cuda): support f64 in alp dyn dispatch (#7666) @0ax1
• feat(cuda): prefer standalone kernels in auto mode (#7667) @0ax1
• feat: Mean aggregate (#7201) @blaginin
• feat: dict into parts (#7668) @joseph-isaacs
• Add ability to override function behaviour via registry in VortexSession (#7588) @robert3005
• feat(cuda): make GPU dispatch mode configurable (#7621) @0ax1
• feat: add a runner field to all runners (#7622) @joseph-isaacs
• feat(cuda): fuse narrower-than-output Dict codes and RunEnd ends (#7617) @0ax1
• feat(cuda): fuse narrower-than-output Dict codes and RunEnd ends (#7603) @0ax1
• Add spans in compressor for perfetto (#7607) @connortsui20
• data-parallel patched ALP standalone kernel (#7576) @a10y
• feat: GPU dyn dispatch patches support (#7563) @0ax1
• Add tracing support to the compressor (#7385) @connortsui20
• Vector similarity search scan benchmarks (#7499) @connortsui20
• GPU kernel for sorted patches with chunk_offsets (#7440) @a10y
• High level C Scan API (#7212) @myrrc
• Unify download management (#7490) @connortsui20
• Add new vector-search-bench benchmarking crate (#7458) @connortsui20

🚀 Performance

• Late materialization support for duckdb (#7721) @myrrc
• perf: zero copy validity export to duckdb (#7371) @joseph-isaacs
• Improve stats reporting to DF (#7628) @AdamGS
• Try and improve the perf of natural file splits in DF (#7609) @AdamGS
• Cache available parallelism (#7620) @AdamGS
• Faster BoolArray::min_max via true_count instead of set_slices (#7599) @robert3005
• perf: faster execution ctx and no opt (#7597) @joseph-isaacs
• Use layout file splits when DF re-partitions individual files (#7591) @AdamGS
• Implement smarter sampler (#7585) @connortsui20
• Remove ScalarFnConstantRule (#7575) @robert3005
• add unchecked array slot take and put (#7514) @joseph-isaacs
• Propagate min/max/string length statistics to duckdb (#7416) @myrrc
• Optimize L2Norm for ConstantArray (#7495) @connortsui20
• fix: avoid ListView take_reduce rebuild for dense selections (#7339) @dimitarvdimitrov
• perf: correctly try execute parent in the iterative child execute loop (#7386) @joseph-isaacs
• perf[gpu]: reduce register pressure in dyn dispatch (#7489) @0ax1

🐛 Bug Fixes

16 changes

• fix: reduce CUDA benchmark noise on codspeed (#7749) @0ax1
• Actually wire the pluggable expression convertor (#7730) @AdamGS
• Fix cosine similarity optimization bug (#7724) @connortsui20
• Prunning expressions can reference rowcount (#7589) @robert3005
• Fix dtype mismatch in FileStatsLayoutReader for stat scalars (#7639) @brainhart
• Validate zoned build (#7627) @connortsui20
• fix: nested block on, make materialize async (#7618) @0ax1
• fix deadlock on unknown policy request (#7606) @myrrc
• Extend DType::least_supertype to cover FixedSizeList, List, and tensor extensions (#7596) @palaska
• Remove ExtensionData (#7587) @connortsui20
• Implement validity for Between scalar function (#7519) @robert3005
• avoid including experimental encoding in footer (#7569) @a10y
• Fix reduction/execute cycle (#7522) @connortsui20
• fix potential UB in ByteBool encoding (#7518) @a10y
• Add extension constant pushdown rule and fix InnerProduct rule (#7507) @connortsui20
• fix: add validity no_nulls and fix usage (#7487) @joseph-isaacs

📖 Documentation

• Add epic and tracking issue templates (#7680) @connortsui20
• C API scan examples (#7564) @myrrc
• Improve docs for DataFusion integration (#7442) @AdamGS

🧰 Maintenance

85 changes

• Remove unused parameters from vtab functions (#7738) @myrrc
• Rename fixed shape tensor module and id (#7733) @connortsui20
• Validity casting doesn't hide usage of execution context (#7710) @robert3005
• ci[gpu]: run cuda micro-benchmarks with codspeed (#7696) @0ax1
• Fix java formatting (#7703) @robert3005
• Java CI is self contained and writes files through java instead of rust (#7702) @robert3005
• Update Rust crate daachorse to v2 (#7688) @renovate[bot]
• remove SessionVar blanket impl, replace with type impls (#7695) @a10y
• Update Rust crate similar to v3 (#7689) @renovate[bot]
• chore: clippy deny ref_option and ref_option_ref (#7692) @joseph-isaacs
• Update Rust crate cudarc to 0.19.0 (#7649) @renovate[bot]
• Add info email (#7682) @whelanBoyd
• Update benchmarks-website dependencies (#7676) @robert3005
• Update dependency ray to v2.55.0 [SECURITY] - abandoned (#7629) @renovate[bot]
• Update dependency concurrently to v9 (#7656) @renovate[bot]
• Upgrade all vortex-web dependencies (#7669) @robert3005
• Rename ScalarValue::List to ScalarValue::Tuple (#7672) @connortsui20
• Pin dependencies (#7647) @renovate[bot]
• Update all patch updates (#7648) @renovate[bot]
• Update dorny/paths-filter action to v4 (#7665) @renovate[bot]
• Update dependency node to v24 (#7661) @renovate[bot]
• Update actions/github-script action to v9 (#7652) @renovate[bot]
• Update dependency com.adobe.testing:s3mock-testcontainers to v5 (#7655) @renovate[bot]
• Update Rust crate zip to v8.6.0 (#7650) @renovate[bot]
• Update dependency typescript to v6 (#7662) @renovate[bot]
• zoned cleanup (#7634) @connortsui20
• DuckDB backend for benchmarks website (#7491) @AdamGS
• duckdb: don't copy vtab on function bind (#7616) @myrrc
• Clean up vortex-tensor even more (#7610) @connortsui20
• add another oneshot tsan suppression (#7608) @myrrc
• Modify renovate config for better experience (#7605) @AdamGS
• remove unused methods from duckdb vtab (#7602) @myrrc
• duckdb to_string: don't copy (#7582) @myrrc
• Lock file maintenance (#7526) @renovate[bot]
• Bump msrv and rust-toolchain to 1.91 (and lance for benchmarks) (#7595) @AdamGS
• Try and make the bot-review-action not hang and/or just fail (#7593) @AdamGS
• Skip nightly lance benchmark (#7594) @AdamGS
• Rename duplicated ScalarFn types (#7586) @connortsui20
• Allow Claude to write (#7294) @gatesn
• Move tensor types under types/ submodule (#7573) @connortsui20
• duckdb: define virtual columns on c++ side (#7571) @myrrc
• Update Rust crate sha2 to 0.11 (#7553) @renovate[bot]
• Update Rust crate reqwest to 0.13.0 (#7552) @renovate[bot]
• Clean up vortex-tensor (#7525) @connortsui20
• Update Rust crate cudarc to 0.19.0 (#7545) @renovate[bot]
• Update Rust crate uuid to v1.23.1 (#7557) @renovate[bot]
• Update arrow to v19 (major) (#7559) @renovate[bot]
• Update codecov/codecov-action action to v6 (#7561) @renovate[bot]
• Update Rust crate zip to v8.5.1 (#7558) @renovate[bot]
• Update Rust crate tokio to v1.52.1 (#7556) @renovate[bot]
• Update Rust crate hashbrown to 0.17.0 (#7547) @renovate[bot]
• Update Rust crate test-with to 0.16 (#7555) @renovate[bot]
• Update Rust crate taffy to 0.10.0 (#7554) @renovate[bot]
• Update Rust crate noodles-vcf to 0.87.0 (#7550) @renovate[bot]
• Update plugin com.google.protobuf to v0.10.0 (#7546) @renovate[bot]
• Update dependency org.testcontainers:junit-jupiter to v1.21.4 (#7539) @renovate[bot]
• Update arrow-rs to v58.1.0 (#7531) @renovate[bot]
• Update dependency com.google.guava:guava to v33.6.0-jre (#7535) @renovate[bot]
• Update release-drafter/release-drafter action to v7.2.0 (#7544) @renovate[bot]
• Update dependency typescript-eslint to v8.58.2 (#7542) @renovate[bot]
• Update dependency typescript to ~5.9.0 (#7541) @renovate[bot]
• Update dependency lucide-react to ^0.577.0 (#7538) @renovate[bot]
• Update all patch updates (#7528) @[renovate[bot]](https://github.com/apps/renovat...

0.69.0

What's Changed

• ALP decoding plugin by @a10y in #7370
• fix(cuda): handle validity in GPU kernels by @0ax1 in #7372
• Fix AGENTS.md sccache by @connortsui20 in #7377
• Replace &PrimitiveArray with ArrayView in a bunch of places by @connortsui20 in #7378
• Remove MaskMut and unused functions from BitBufferMut by @robert3005 in #7379
• AllInvalid arrays have non empty distinct info by @robert3005 in #7369
• duckdb: Reset validity instead of filling by @myrrc in #7365
• use expect over allow in clippy by @joseph-isaacs in #7373
• Remove portable_simd take implementation and remove unused cargo flags by @robert3005 in #7388
• Decompose TurboQuant into 2 ScalarFnArrays and DictArray by @connortsui20 in #7374
• add vector similarity benchmark by @connortsui20 in #7391
• perf: Arc::ptr_eq short-circuit to DType eq comparisons by @0ax1 in #7398
• Update dependency io.netty:netty-bom to v4.2.12.Final by @renovate[bot] in #7403
• Update react monorepo to v19.2.5 by @renovate[bot] in #7404
• Update rust-wasm-bindgen monorepo by @renovate[bot] in #7406
• bump rand to 0.10.1 by @connortsui20 in #7405
• Update all patch updates by @renovate[bot] in #7402
• chore[vortex-array]: add integer casting benchmark by @asubiotto in #7400
• Ignore dead_code warnings on scalar bit transpose code by @robert3005 in #7408
• support multiple globs in DuckDB and MultiFileDataSource by @a10y in #7390
• Optimize maths expressions with Constant children by @connortsui20 in #7394
• TurboQuant InnerProduct optimizations by @connortsui20 in #7396
• duckdb: Exit earlier on Validity::AllFalse by @myrrc in #7411
• perf: Arc::ptr_eq short-circuit for List & FixedSizeList DType eq by @0ax1 in #7410
• feat: implement IsNotNull expression in vortex expression library by @xiaoxuandev in #6969
• More robust types in the compressor by @connortsui20 in #7415
• fix[ci]: bench no fail-fast random access and compress bench by @joseph-isaacs in #7418
• Remove getrandom feature flag overrides by @robert3005 in #7421
• perf[vortex-array]: use from_trusted_len_iter in primitive casts by @asubiotto in #7401
• Update dependency pytest to v9 [SECURITY] by @renovate[bot] in #7419
• chore[vortex-layout]: allow specifying a set of stats on CompressingStrategy by @asubiotto in #7422
• fix[vortex-array]: avoid unnecessary checks in check_listview_constant by @asubiotto in #7423
• Lower days before stale to 14 from 30 by @robert3005 in #7425
• fix check for ArrayPlugin encoding ID on deserialize by @a10y in #7427
• chore: benchmark with execution context input by @joseph-isaacs in #7429
• fix ALP patch plugin by @a10y in #7430
• add unstable encodings and patches to benchmarks by @joseph-isaacs in #7426
• Add benchmarks for take on a FilterArray by @robert3005 in #7420
• Optimize inner product, update boilerplate, remove bad benchmarks by @connortsui20 in #7428
• ScalarFns as Arrays by @gatesn in #7361
• Fix lockfiles by @connortsui20 in #7433
• perf: intern identifiers (array, layout, etc). by @joseph-isaacs in #7412
• binstall for vortex-tui by @a10y in #7434
• Remove metadata from DynArray by @connortsui20 in #7435
• Remove ListViewBuilder scalar_at by @gatesn in #7438
• Support serializion tensor scalar fns by @connortsui20 in #7437
• chore[gpu]: don't inline bitunpack lane impls by @0ax1 in #7441
• add BtrBlocksCompressorBuilder::empty() by @connortsui20 in #7443
• Update locked rustls-webpki to address sec advisory by @AdamGS in #7444
• chore[gpu]: trim Criterion CUDA benchmarks by @0ax1 in #7445
• Add spark value writers for Date/Timestamp/TimestampNTZ and Struct types by @robert3005 in #7424
• Gate scalar fn array plugin for tensor exprs by @connortsui20 in #7447
• chore(ci): workflow cleanup and fixes by @0ax1 in #7448
• Vector datasets catalog and downloader by @connortsui20 in #7446
• fix(ci): restore vortex-compact file size tracking for PR clickbench by @0ax1 in #7452
• expose execution context for validity::to_bool and remove validity mask from array vtable by @joseph-isaacs in #7432
• chore: dedup formats in sql engine runner by @0ax1 in #7459
• chore: bump duckdb to 1.5.2 by @0ax1 in #7461
• chore: unify ci timeouts to 4 tiers by @0ax1 in #7460
• Correct rust version channel by @robert3005 in #7463
• fix: wipe duckdb archive as part of cargo clean by @0ax1 in #7465
• Unify benchmark file download logic by @robert3005 in #7462
• Fix web deployment github action by @robert3005 in #7467
• Unify all runtime env var access to functions wrapping LazyLock by @AdamGS in #7468
• Deprecate scalar_at for execute_scalar by @gatesn in #7457
• Demo TurboQuant basic search with serialization by @connortsui20 in #7451
• vortex-array: Attempt casting inner array to target for extensions by @brancz in #7469
• Fix scalar_at in example by @connortsui20 in #7471
• Split up the monster ci.yml workflow by @AdamGS in #7472

New Contributors

• @xiaoxuandev made their first contribution in #6969

Full Changelog: 0.68.0...0.69.0

0.68.0

Changes

⚠️ Breaks

• Move Array::serialize -> ArrayPlugin::Serialize (#7348) @gatesn
• Unify id creation for ScalarFns and rename is_null to vortex.is_null (#7325) @robert3005
• Rework Scheme estimation in compressor (#7230) @connortsui20
• block serialization for the existing aggregations (#7322) @blaginin
• break[array]: remove old vtable! macro (#7317) @joseph-isaacs
• Hoist Slots (#7287) @gatesn
• Fix Array::validity (#7285) @gatesn
• Array VTables Follow Up (#7276) @gatesn
• Clean up btrblocks compressor interface (#7274) @connortsui20
• Invert thevortex-tensor dependency (#7271) @connortsui20
• Array VTables (#7215) @gatesn
• break: apply should use apply(self: Arc<Self>, ...) over apply(&self, ...) (#7259) @joseph-isaacs
• ffi: expose array validity, allow creating primitive arrays (#7148) @myrrc
• remove deprecated StructStrategy (#7242) @a10y
• Remove deprecated compute traits (#7231) @gatesn
• Pluggable Compressor (#7018) @connortsui20
• array slots (#6870) @joseph-isaacs

✨ Features

• Add IsNull expression to vortex-jni (#7358) @robert3005
• feat: add an arrays slots macro (#7360) @joseph-isaacs
• Allow loading "foreign" plugins for UI/TUI/Serde (#7347) @gatesn
• Pull out L2Denorm from TurboQuant (#7349) @connortsui20
• Buffer Allocator Abstraction (#7337) @gatesn
• feat: iterative execution for most arrays in vortex-array (#7335) @joseph-isaacs
• Dynamic WHT rounds in TurboQuant (#7330) @connortsui20
• feat[gpu]: support mixed-width types in dynamic dispatch (#7331) @0ax1
• feat: fsst inline varbin child (#7316) @joseph-isaacs
• L2 Denorm expression (#7329) @connortsui20
• Normalize Execution (#7278) @gatesn
• Allow inserting non-Default values to VortexSession (#7321) @gatesn
• feat: First Last (#7313) @blaginin
• benchmarks website dark mode (#6886) @myrrc
• Tensor Matchers (#7300) @connortsui20
• feat: Count aggegate (#7267) @blaginin
• TurboQuant encoding for Vectors (#7269) @connortsui20
• Cleaner WriteStrategyBuilder interface (#7275) @connortsui20
• Fix tensor expressions and add InnerProduct expression (#7277) @connortsui20
• ffi: StructArray creation (#7260) @myrrc
• feat: iterative execute alp/alp_rd/bitpacking (#7233) @joseph-isaacs
• Crosshairs for filemap (#7247) @gatesn
• Support partitionBy in VortexSparkDataSource (#7218) @robert3005
• Add compressor for constant nonnullable and all valid bool arrays (#7221) @robert3005
• PatchedArray: basics and wiring (#7002) @a10y
• Approximate expressions for tensor types (#7226) @connortsui20
• Deploy Explorer (#7211) @gatesn
• Expose expressions for C API (#7190) @myrrc
• feat(cuda): single-pass dispatch plan builder (#7197) @0ax1
• New Parquet-Variant encoding (#7130) @AdamGS

🚀 Performance

• Add inner product and cosine similarity optimizations (#7364) @connortsui20
• feat: iterative execution patched array (#7345) @joseph-isaacs
• Fix StructArray / StructData after migration (#7305) @gatesn
• alp_rd: speedup (#7064) @joseph-isaacs
• perf: speed up TryFrom for Primitive (#7223) @joseph-isaacs
• feat: Introduce writeBatchFfi in Java JNI to avoid IPC serialization overhead (#7205) @JingsongLi

🐛 Bug Fixes

17 changes

• fix: maturin never logs (#7367) @danking
• Fix patch_chunk index OOB when slicing ALP arrays mid-chunk (#7354) @abnobdoss
• AVX2 take handles indices that are equal to the index type max value (#7359) @robert3005
• Fix TSan data race with FFI tests (#7244) @myrrc
• Don't re-run benchmarks nightly (#7338) @AdamGS
• fix[encodings]: take in progress buffer into account on fsst builder … (#7318) @asubiotto
• RLE handles decompression of indices where invalid positions are clobbered (#7255) @robert3005
• fix[vortex-array]: fix offset_within_chunk underflow on patches array (#7312) @asubiotto
• fix: array correctly update their validity (#7307) @joseph-isaacs
• Fix TurboQuant metadata to be protobuf (#7301) @connortsui20
• Fix ALP mask patches dtype (#7293) @gatesn
• fix[ffi]: to and from ffi types should have a fully defined cast (#7261) @joseph-isaacs
• fix: fix typo in compressor scheme (#7241) @joseph-isaacs
• buffered strategy to not use eof for the final chunk (#7219) @onursatici
• fix[vortex-array]: fix overflow on FSL element take indices (#7214) @asubiotto
• fix[ci]: correct perms to upload back compat fixtures (#7207) @joseph-isaacs
• fix: restrict wasm-bindgen to browser wasm, fixing wasip1 builds (#7208) @0ax1

📖 Documentation

• docs: add back array children docs moved from slots (#7249) @joseph-isaacs

🧰 Maintenance

30 changes

• add env flag for Patched array (#7314) @a10y
• Add a dedicated CI build profile (#7357) @AdamGS
• Remove generic rle (#7350) @connortsui20
• fix lint on develop (#7346) @a10y
• Enable clippy::clone_on_ref_ptr lint and fix all violations (#7262) @joseph-isaacs
• Symlink CLAUDE.md to AGENTS.md to support other coding agents (#7333) @AdamGS
• TurboQuant cleanup part 2 (#7326) @connortsui20
• TurboQuant Cleanup part 1 (#7320) @connortsui20
• chore(deps): update dependency vite to v6.4.2 [security] (#7303) @renovate[bot]
• drop unused code (#7304) @a10y
• chore(deps): update taiki-e/cache-cargo-install-action digest to 66c9585 (#7291) @renovate[bot]
• chore(deps): update actions/github-script digest to 450193c (#7290) @renovate[bot]
• ArrayPlugin (#7302) @gatesn
• Bring back file-size comments (#7296) @a10y
• Fix clippy on develop (#7295) @gatesn
• Correctly gate zstd (#7283) @connortsui20
• Remove unused approvals workflow triggers (#7280) @gatesn
• chore(deps): update release-drafter/release-drafter action to v7 (#6943) @renovate[bot]
• Require two humans for bot authored PRs (#7232) @gatesn
• take VortexSession by ref (#7270) @connortsui20
• make struct_fields boxed, not arc'd (#7257) @myrrc
• Fill out a few small pieces in Variant (#7209) @AdamGS
• chore: have on demand validity and patches for array remove slot extraction (#7217) @joseph-isaacs
• fix[vortex-array]: update an overflow test (#7229) @asubiotto
• Move stuff around vortex-tensor (#7225) @connortsui20
• Assert_arrays_eq also executes the array and compares the results against scalar_at (#7222) @robert3005
• feat[array]: compositional array display logic (#7082) @joseph-isaacs
• chore(deps): update codspeedhq/action digest to d872884 (#7202) @renovate[bot]
• chore(deps): update taiki-e/cache-cargo-install-action digest to 511847d (#7203) @renovate[bot]
• Vortex Explorer UI (#7165) @gatesn

0.67.0

Changes

✨ Features

• feat: Java Bindings for FixedSizeList (#7188) @JingsongLi
• feat: make dispatch plan dynamically sized (#7186) @0ax1
• Add tsan for vortex-ffi, fix asan instrumentation (#7157) @myrrc
• Upgrade to DataFusion 53 (#6740) @AdamGS

🚀 Performance

• Cache python class imports (#7191) @AdamGS
• Specialize export of chunked list arrays (#7180) @AdamGS
• Filter indices without an iterator for Bool (#7176) @AdamGS

🐛 Bug Fixes

• Make sure all tokio runtime propegate observability in the same way (#7193) @AdamGS
• fix: vortex-cxx test race (#7189) @0ax1
• fix: support total_size=0 in byte_range_to_row_range (#7115) @aalexandrov

📖 Documentation

• feat: teach docs about object store (#7169) @danking

🧰 Maintenance

12 changes

• Python benchmarks script and re-organize dev dependencies (#7192) @AdamGS
• move BitPacked slice from kernel -> parent reduce (#7195) @a10y
• Unify benchmark dispatch into a workflow (#7178) @AdamGS
• Run TPC-H SF 100 nightly on Vortex (#7184) @AdamGS
• Array VTables Migration 3 (#7185) @gatesn
• scalar_at take execution_ctx (#7166) @a10y
• Array VTables Migration 2 (#7183) @gatesn
• chore[gpu]: split plan into unmaterialized / materialized (#7175) @0ax1
• Array VTables Migration 1b (#7182) @gatesn
• Array VTables Migration 1a (#7181) @gatesn
• Upgrade fsst-rs to 0.5.9 (#7177) @AdamGS
• make Mimalloc opt-in for ffi (#7179) @myrrc