We are allowing users to pass in padded_dataset types to CAGRA build.
In the IVF-PQ CAGRA build option, stride is used to interpret the number of dimensions of the dataset.
The stride includes extra zeros set by the padding.
This can cause the PQ encoding and heuristics to be different. For example the derived pq_len would be different since pq_len = dim (stride) / pq_dim. That would cause the codebooks to look different.
Moreover, it can change whether we apply rotation matrix or not. dim (stride) % pq_dim == 0.
We are allowing users to pass in padded_dataset types to CAGRA build.
In the IVF-PQ CAGRA build option, stride is used to interpret the number of dimensions of the dataset.
The stride includes extra zeros set by the padding.
This can cause the PQ encoding and heuristics to be different. For example the derived pq_len would be different since pq_len = dim (stride) / pq_dim. That would cause the codebooks to look different.
Moreover, it can change whether we apply rotation matrix or not.
dim (stride) % pq_dim == 0.