Add L() macro for local labels and enable .subsections_via_symbols on aarch64-darwin - #6090
Open
topolarity wants to merge 5 commits into
Open
topolarity wants to merge 5 commits into
topolarity wants to merge 5 commits into
Conversation
Nothing uses it yet. L(name) is .Lname, except on Mach-O, where the assembler's prefix for local labels is "L" and a ".L" label is an ordinary symbol. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
No change to any object except on Mach-O. The kernels name their labels
.Lxxx, which makes them local to the assembler on ELF and COFF, but not on
Mach-O, where that prefix is "L": there every label is a symbol. Since
EPILOGUE is .subsections_via_symbols on Darwin, the linker takes each such
symbol for the start of a block of its own. It does not see that one block
falls through into the next, and
* folds a block that is byte for byte the same as one of another kernel
into that one (on by default when linking at -O2), and
* with -dead_strip, removes a block that no branch refers to,
either of which leaves the code before the block running on into whatever
comes next. With Apple clang 21 (ld-1267), libopenblas.dylib built for
NEHALEM returns garbage from cblas_sasum(5, {3,-1,2,-4,1}, 1) (sasum_k has
lost its final "ret"), and openblas_utest crashes in its first test.
With L(name) the labels are local on Mach-O as well, and a kernel is one
block.
Generated by asm-labels/use_local_label_macro.py, which rewrites every
".Lname" token of kernel/x86_64/*.S and nothing else: 23906 labels in 249
files. For an ELF target, all 255 files preprocess to the same text as
before, and the 2619 kernel objects of a DYNAMIC_ARCH build are byte for
byte the same.
Generated-by: asm-labels/use_local_label_macro.py
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
No change to any object: on ELF and COFF L(name) is .Lname, as before, and
on Mach-O, where it is Lname, the labels become local to the assembler
without changing the code (the arm64 kernels do not use
.subsections_via_symbols, so the linker did not act on the label symbols).
Preparation for using that directive on Darwin, which needs the labels of
a kernel to be local: with it, the assembler cannot even branch
conditionally to a label that is a symbol ("conditional branch requires
assembler-local label").
Generated by asm-labels/use_local_label_macro.py: 6565 labels in 76 files.
The two SME kernels, which do not include common.h, are not touched. For an
ELF target the 476 kernel objects of a DYNAMIC_ARCH build are byte for byte
the same, and on Darwin their text is.
Generated-by: asm-labels/use_local_label_macro.py
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
topolarity
force-pushed
the
ct/asm-local-labels
branch
from
October 2, 2026 03:50
66d3364 to
292b50b
Compare
A few kernels name labels in ways that the previous two commits did not
reach:
* x86_64/dgemm_kernel_6x4_piledriver.S: ._L_<n>_<kind>
* arm64/nrm2.S, znrm2.S, zscal.S: no prefix at all
* arm64/sgemm_direct_sme1_*.S: .M_Loop, .K_Loop, ...
* arm64/dznrm2_thunderx2t99_fast.c: .Lname, in inline assembly
* x86_64/dgemm_kernel_4x8_skylakex.c: .label<n>, in inline assembly
All but the fourth are symbols for every assembler. Spell them L(name).
The SME kernels include common.h for that, and end in EPILOGUE like the
others.
The two C kernels, which no KERNEL file uses at present, get an L() of
their own, which makes the string. clang always uses
.subsections_via_symbols for C, so on Darwin the first did not assemble
("conditional branch requires assembler-local label") and the second had
its loops split into blocks.
The text of each object is unchanged, on ELF and on Darwin, except that of
dgemm_kernel_4x8_skylakex.c on Darwin, where the jumps to its labels are now
short; the label symbols are gone from it.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Apple's linker never removes anything from an object without this directive, so a static link with -dead_strip kept every assembly kernel it had loaded, used or not. With it, a kernel that nothing refers to is left out, as the C kernels already are. This is safe now that every label inside a kernel is L(name), local to the assembler: each kernel object defines one symbol, its entry point, and so is one block for the linker. EPILOGUE, until now empty, is added to the five kernels that did not end in it. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
topolarity
force-pushed
the
ct/asm-local-labels
branch
from
October 2, 2026 04:12
292b50b to
c7bc1bc
Compare
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
On macOS, linker de-duplication and pruning is done at the symbol level.
As seen in #6089, it is important to be careful about how these symbols are exposed even if not using
-Wl,-dead-strip.This change is complementary to #6088, since
-Wl,-dead-stripcannot prune this assembly code on macOS without.subsections_via_symbols.On
arm64:developdgemmdgesv+daxpy+dgemv+dnrm2cblas_d*On
x86-64:developdgemmdgesv+daxpy+dgemv+dnrm2cblas_d*Another massive diff, although hopefully easy to review. Commits:
L(...)macro.L->L(...)replacement forx86-64.L->L(...)replacement forarm64.subsections_via_symbolsonarm64-darwin