Repository navigation
Avoid forming misaligned typed pointers in (de)serialization - #888
Merged
Merged
Conversation
roaring64_bitmap_frozen_serialize accepts an unaligned output buffer (only roaring64_bitmap_frozen_view requires alignment), but it carved the output into uint64_t*/uint16_t*/rle16_t* cursors aligned relative to the buffer start, so with an unaligned base pointer those typed pointers were misaligned. Likewise roaring_bitmap_deserialize and roaring_bitmap_deserialize_safe formed a const uint32_t* at buf + 5 and did pointer arithmetic on it before reading through memcpy. The data was always copied with memcpy, so nothing faults at run time, but forming a misaligned pointer to a type is undefined behavior (C11 6.3.2.3p7) and clang's -fsanitize=alignment reports it (GCC's UBSan, used in CI, does not check memcpy arguments). Use byte cursors instead and advance them by byte counts; the serialized bytes are unchanged. With clang and -DROARING_SANITIZE_UNDEFINED=ON, roaring64_unit reported "store to misaligned address ... for type 'uint16_t *'" in container_frozen_serialize, and toplevel_unit aborted in test_serialize. Both are clean now.
Member
|
It always seems crazy that just creating an unaligned pointer is UB in C, something that always catches me when going from rust to C/C++. |
Member
Author
|
@Dr-Emann I don't think it actually matters. I am being pedantic here. I would not fix it if not for the sanitizer warnings. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Two places formed misaligned typed pointers that were only ever passed to
memcpy:roaring64_bitmap_frozen_serializeaccepts an unaligned output buffer (onlyroaring64_bitmap_frozen_viewrequires 64-byte alignment), but it carved the output intouint64_t */uint16_t */rle16_t *cursors aligned relative to the buffer start. With an unaligned base pointer (ascheck_frozen_serializationinroaring64_unitdeliberately uses), those typed pointers are misaligned.roaring_bitmap_deserializeandroaring_bitmap_deserialize_safeformedconst uint32_t *elemsatbuf + 5and computedelems + ibefore thememcpy. The comment already said the data may be unaligned; the pointer type just didn't match that intent.Nothing faults at run time since the copies go through
memcpy, but forming a misaligned pointer to a type is undefined behavior (C11 6.3.2.3p7). clang's-fsanitize=alignmentchecks typedmemcpyarguments and reports both; GCC's UBSan (whatubuntu-sani-ub-ciuses) does not, which is why CI is green.The fix keeps byte (
char *) cursors and advances them by byte counts.container_get_frozen_size()already returns the byte count copied, socontainer_frozen_serializenow advances by it directly. Serialized output is byte-for-byte unchanged.Test plan
-DROARING_SANITIZE=ON -DROARING_SANITIZE_UNDEFINED=ON: 27/27 tests pass, zeroruntime errorlines in the ctest log (previouslyroaring64_unitreported a misaligneduint16_t *store incontainer_frozen_serialize, andtoplevel_unitaborted intest_serializeatroaring.c:1687)