feat: on-demand native lib download and optional feature splitting

[msluszniak]

Description

Removes prebuilt native binaries from the npm tarball and downloads them at postinstall from GitHub Releases. Apps declare exactly what they need under react-native-executorch in package.json — three optional arrays, all merged into a single set:

{
  "react-native-executorch": {
    "backends": ["xnnpack", "coreml", "vulkan"],
    "libs":     ["opencv", "phonemizer"],
    "features": ["llm", "textToSpeech", "objectDetection"]
  }
}

• features is the friendly opt-in: list the use* hooks you'll use and the postinstall expands each one to its required backends + libs (per FEATURE_MAP).
• backends / libs are the precise opt-in.
• Omitting the block enables everything (largest install, lowest friction).
• Mixing the three arrays unions their effects.

The 18 features track the documented use* hooks one-for-one; each row's backend list is the union of what at least one model in that family ships today (sourced from src/constants/modelRegistry.ts).

Component	iOS	Android
xnnpack backend	XnnpackBackend.xcframework force-loaded	separately-loaded libxnnpack_executorch_backend.so
coreml backend	CoreMLBackend.xcframework force-loaded	n/a
vulkan backend	n/a	separately-loaded libvulkan_executorch_backend.so
opencv lib	via opencv-rne CocoaPod	static libopencv_*.a + KleidiCV HAL (arm64)
phonemizer lib	compiled from in-tree source (third-party/common/phonemis submodule)	same submodule via CMake add_subdirectory

Each Android backend .so links only <backend>_backend (--whole-archive) + <backend>_schema + executorch_core — no CPU kernel registries — so loading multiple side-by-side does not trigger duplicate kernel registration. Supporting executorch fork branch: software-mansion-labs/executorch@ms/separate-backends (EXECUTORCH_BUILD_XNNPACK_BACKEND_SHARED + EXECUTORCH_BUILD_VULKAN_BACKEND_SHARED switches, a custom_ops fix to stop the transitive XNNPACK link from leaking into libexecutorch_jni.so, and a flatcc -Werror workaround for Apple clang 21).

Demo apps in this PR declare their actual feature set:

• bare-rn → ["llm"]
• llm → ["llm", "multimodalLLM", "privacyFilter"]
• speech → ["llm", "speechToText", "textToSpeech", "vad"]
• text-embeddings → ["textEmbeddings", "imageEmbeddings"]
• computer-vision → ["classification", "imageEmbeddings", "instanceSegmentation", "ocr", "objectDetection", "poseEstimation", "semanticSegmentation", "styleTransfer", "textEmbeddings", "textToImage", "verticalOCR"]

Introduces a breaking change?

• Yes
• No

Removes the legacy extras field. Apps that set it get an install-time error pointing at the new shape.

Type of change

• New feature (change which adds functionality)

Tested on

• iOS (physical iPhone SE 3rd gen / Xcode 26.4.1)
• Android (Galaxy S26 Ultra)

Testing instructions

Test the download flow:

cd packages/react-native-executorch
rm -rf third-party/android/libs third-party/ios/ExecutorchLib.xcframework third-party/ios/libs
rm -rf ~/.cache/react-native-executorch/0.9.0
RNET_BASE_URL=https://github.com/software-mansion/react-native-executorch/releases/download/v0.9.0-libs-test \
  INIT_CWD=<app-root> \
  node scripts/download-libs.js

Test feature opt-in (drop opencv / coreml by listing only LLM-shaped features):

1. In the app's package.json set "features": ["llm"].
2. yarn install → rne-build-config.json should show enableOpencv:false, enableCoreml:false, enableXnnpack:true.
3. Clean prebuild + run the app. LLM inference works; loading a CoreML-only model now produces a Backend CoreMLBackend is not registered error rather than crashing.

Test phonemizer opt-out:

1. In the app's package.json set "features": ["textEmbeddings"] (no TTS).
2. yarn install (regenerates rne-build-config.json), then pod install / regenerate android/.
3. The phonemis submodule is not entered by either CMake or the podspec — TTS sources are excluded from compilation.

Test that legacy extras is rejected:

1. In any app's package.json set "extras": ["opencv"].
2. yarn install fails with an explicit migration error pointing at backends, libs, features.

Related issues

Builds toward pytorch/executorch#10457 (hot-pluggable Android backends).

Checklist

• I have performed a self-review of my code
• I have commented my code, particularly in hard-to-understand areas
• I have updated the documentation accordingly
• My changes generate no new warnings

[msluszniak]

TODO: separate xnnpack and coreml backends to separate libs, so they can be opt-out the same way as opencv etc.

[msluszniak]

Adding support for vulkan is in progress

[msluszniak]

Status — verified working state

PR is in a tested, working state with a clear opt-in/opt-out model for every backend except XNNPACK on Android (which stays baked into libexecutorch.so).

Backend	iOS	Android
XNNPACK	separable (XnnpackBackend.xcframework, force-loaded when xnnpack extra is on)	always on — baked into libexecutorch.so; xnnpack extra has no effect on Android (postinstall warns)
CoreML	separable (CoreMLBackend.xcframework, force-loaded when coreml extra is on)	n/a
Vulkan	n/a	separable (libvulkan_executorch_backend.so, separately loaded when vulkan extra is on)

Verified on a Galaxy S26 Ultra and an iPhone 17 Pro simulator (Xcode 26.4.1):

• Android XNNPACK YOLO inference: works.
• Android Vulkan opt-in (YOLO26N exported with Vulkan delegate): works — libvulkan_executorch_backend.so packaged into the APK and loaded as a linker dependency of libreact-native-executorch.so.
• Android Vulkan opt-out: .so absent from APK, runtime emits E ExecuTorch: Backend VulkanBackend is not registered. and the app stays alive (no crash).
• iOS XNNPACK opt-out: dylib link drops -force_load XnnpackBackend.xcframework, app dylib has 0 XNNPACK symbols / 11 CoreML symbols, XNNPACK model fails gracefully without crashing.

Supporting executorch fork branch: msluszniak/executorch@ms/separate-backends (EXECUTORCH_BUILD_VULKAN_BACKEND_SHARED switch + Apple-clang-21 flatcc workaround).

Next work

Make XNNPACK separable on Android too — apply the same QNN-style EXECUTORCH_BUILD_*_BACKEND_SHARED pattern from the Vulkan commit to xnnpack_backend, link the resulting libxnnpack_executorch_backend.so only against xnnpack_backend + executorch_core + the XNNPACK third-party libs, and audit the link line to make sure no kernel registration libs (optimized_native_cpu_ops_lib, custom_ops, quantized_ops_lib, register_prim_ops) leak in. Once that holds, the xnnpack extra becomes meaningful on both platforms and the post-install Android warning can go away. Tracking towards pytorch/executorch#10457.

[msluszniak]

Update — XNNPACK on Android is now separable too

Every backend is now opt-in symmetrically across both platforms:

Backend	iOS	Android
XNNPACK	XnnpackBackend.xcframework (force-loaded when xnnpack extra is on)	libxnnpack_executorch_backend.so (separately loaded when xnnpack extra is on)
CoreML	CoreMLBackend.xcframework (force-loaded when coreml extra is on)	n/a
Vulkan	n/a	libvulkan_executorch_backend.so (separately loaded when vulkan extra is on)

Two changes on the executorch fork (msluszniak/executorch@ms/separate-backends):

1. New EXECUTORCH_BUILD_XNNPACK_BACKEND_SHARED switch mirrors the Vulkan one. When ON, xnnpack_backend is no longer whole-archive-linked into libexecutorch_jni.so; instead libxnnpack_executorch_backend.so is built linking only xnnpack_backend (--whole-archive) + the XNNPACK third-party libs (XNNPACK, pthreadpool, cpuinfo, xnnpack-microkernels-prod, kleidiai) + executorch_core + log. No CPU-kernel-registration archives leak in.
2. extension/llm/custom_ops/CMakeLists.txt drops the (transitive) PUBLIC link to xnnpack_backend when the new switch is ON. custom_ops doesn't actually call into XNNPACK in its sources — that link was purely transitive — but combined with WHOLE_ARCHIVE custom_ops in extension/android/CMakeLists.txt it was dragging XNNCompiler / XNNExecutor into libexecutorch_jni.so and would have caused the duplicate-registration crash when both shared libs loaded.

Verified on the same Galaxy S26 Ultra:

• libxnnpack_executorch_backend.so (~2.5 MB stripped) — only _GLOBAL__sub_I_XNNPACKBackend.cpp. No RegisterCodegen*, op_sdpa, register_prim_ops, regex_lookahead static initializers leaked in. 0 aten::* strings.
• libexecutorch.so shrank from 13 MB → 11.6 MB (XNNPACK no longer baked).
• Extras [xnnpack, vulkan] enabled: both backends register, both YOLO XNNPACK and YOLO26N Vulkan run. No RegistrationAlreadyRegistered (0x16). Confirms the duplicate-kernel issue we were avoiding by keeping XNNPACK baked is fully gone.
• Extras [vulkan] only (XNNPACK off): APK contains no libxnnpack_executorch_backend.so. Loading any XNNPACK-quantized .pte produces E ExecuTorch: Backend XnnpackBackend is not registered. and the app stays alive — clean failure mode, identical to the iOS XNNPACK opt-out path. Vulkan still works in the same build.

The download-libs.js platform-asymmetry warning for the xnnpack extra is removed. xnnpack-android-* tarballs are produced by package-release-artifacts.sh. Docs updated.

This finishes the work tracked above. PR is ready for review.

[kirklandsign]

Thanks @msluszniak . Mind working on small stack of PRs so it's easier to review?

[msluszniak]

@kirklandsign this particular PR solely address React Native ExecuTorch repo, but what is probably instresting for you is a not-yet-published PR from https://github.com/msluszniak/executorch/tree/@ms/separate-backends branch to executorch. For this one, for sure I can proceed with a series of small PRs :)). I just want to make sure changes are aligned with RNE repo first. I just need to check if iOS works the same way as Android.

[msluszniak]

PR is ready for review. Right now, I'm testing all demo apps on both iOS and Android.

[barhanc]

Started a review but had a moment of reflection. I think we should take a step back and consider on what "axes" so to speak the splitting should operate. The backends axis is an obvious and natural choice. The second (and final) axis, however, should be imo the domain (e.g. computer vision, nlp, speech, etc.) so that we can easily specify something along the lines "for my app I want to do computer vision and need CoreML for iOS" which would translate to a union of artefacts (lib binaries, etc.) required for CV task and CoreML backend accordingly. Per-feature splitting will lead to too much fragmentation and bloat in my opinion and I don't really see any benefits of this more granular division, but perhaps there are some? The libs choice on the other hand is particularly peculiar as it reverses the semantics---instead of declaratively specifying "I want CV with CoreML and XNNPACK", it imperatively tells that for something we will need e.g. OpenCV---as such it seems unsuitable.

[msluszniak]

@barhanc indeed there is at least one motivation between choosing features over broader categories as CV, etc. The split has only sense we are able to actualy disable some part of libs from bundling into an app. Which low-level translates into particular backends and libs used. In NLP, there are things like VLM. It utilizes OpenCV, which won't be disabled if we decide on less granular approach. Same thing for phonemis and speech category.

This will eventually cause that broader categories will "propagate" libs by the fact that we take union, and the final advantage of opting-out is less powerful.

So the axis of splitting by category is more intuitive at first sight, bur when you try to implement this mechanism optimally, I started to see, that split by libs is more acurate for implementation purpose, and the more granular split is not perfectly aligned with it.

[IgorSwat]

One question - why phonemizer and not phonemis (which is the common name)?

Also, I do believe Phonemis is currently being fetched from main - I think I will release it with some tag and we should bind it to this tag.

[msluszniak]

Good catch — renamed in 2b45f7fde: the feature lib is now phonemis, the build-config field is enablePhonemis, and the compile macro is RNE_ENABLE_PHONEMIS. Pre-existing public API names (TextToSpeechPhonemizerConfig, the phonemis::phonemizer namespace from the library itself, phonemizer_*.pte model files on HF) are left untouched.

On the submodule: agreed. The .gitmodules entry currently tracks the main branch of IgorSwat/Phonemis. Once you cut a tag, I'll pin the submodule SHA to it and update CI. Ping me when ready (or feel free to push the pin directly).