Skip to content

Swift interface export: findings from reconstructing a private framework #106

Description

@Mx-Iris

Findings from using this library's output for a real reconstruction job: hand-writing a compilable .swiftinterface subset of Xcode's private SourceEditor.framework (13289 Swift symbols, 1327 types), linking against it, and shipping it. The dumps were generated by RuntimeViewer, i.e. by this library.

The dumps were good enough that the job succeeded — superclasses, PWT offsets and enum case order were all correct and all load-bearing. Everything below is about the gap between "readable dump" and "transcribable interface".

Ordered by how much time each cost.


1. final is never emitted, but it is derivable from data the library already reads

This is the expensive one, because getting it wrong breaks linking:

Undefined symbols: dispatch thunk of SourceEditor.SourceEditorDataSource.string.getter

A member reached through a dispatch thunk must be declared plainly; a member the framework exports only as a direct symbol must be declared final. The dump doesn't say which, so today the only way to know is to ask the binary with nm and grep for dispatch thunk of … — one member at a time, or via a bespoke script.

final never appears anywhere in the codebase (grep -rn '"final"\|\.final\b' Sources → no hits), so this is a missing feature rather than a bug.

It is derivable. Four cases, verified against SourceEditor:

what the dump shows dispatch correct declaration
// VTable offset: vtable plain
computed member, only // Address: static final
stored let (// Field offset:) static final
stored var (// Field offset:) accessors in vtable plain

Measured across the whole framework, cross-referencing the dumps against exported dispatch thunks:

stored let: 1156 total,    0 have a dispatch thunk   (no exceptions)
stored var: 1712 total,  372 have a dispatch thunk   (the rest are non-public, so unexported)

The missing piece is attributing vtable entries back to stored properties. --emit-vtable-offset-comments is documented as "Generate vtable offset comments for class methods and computed properties" — and that is exactly the gap. A non-final stored var has getter/setter entries in the vtable, but the dump prints the property from the field descriptor only, so it carries no vtable comment and looks statically dispatched.

Concretely, SourceEditorView.dataSource and SourceEditorView.gutter are stored vars printed with // Field offset: and no vtable comment, yet both export dispatch thunks and must be declared plainly. I initially derived the rule as "no vtable comment ⇒ final" and it was wrong for precisely this reason.

SwiftAttributeInference/MemberAttributeInferrer.swift already infers @objc, @nonobjc and dynamic from MethodDescriptorFlags and thunk nodes; final seems to belong right beside them.


2. Report whether the module was built with library evolution

This decides whether the inference in §1 is even valid: in a non-resilient module no dispatch thunks are emitted at all, so "no thunk ⇒ final" would mark every member final.

Measured:

binary Swift symbols Tj Tq MV
SourceEditor (Xcode, resilient) 13289 2216 2216 1515
SourceModelSupport (Xcode, resilient) 601 121 121 68
a framework built by SwiftPM 103 0 0 17
another, non-resilient 35258 0 317 5907

Only Tj (dispatch thunk) discriminates. Tq (method descriptor) and MV (property descriptor) do not — the last row has 317 Tq and 5907 MV with library evolution off. Also worth noting Tj == Tq exactly in both resilient binaries.

Suggestion: one line in the export header, next to the existing Mach-O metadata (install name, UUID, …), e.g. Library evolution: enabled (2216 dispatch thunks). It tells the reader how to interpret the whole file.


3. Mark members that have no exported symbol

SourceEditorGutter.updateLineNumberDisplay() is printed with // VTable offset: 66 and // Address: 0x1CD60C, looking entirely callable. It is private — zero matches in the exported symbol table. I wrote a stub declaration for it before finding that out.

A // not exported marker (or an access-level annotation where derivable) would make the distinction visible without a second tool.


4. lazy var prints the storage type rather than the accessor type

// dump
lazy var languageService: SourceEditor.SourceEditorLanguageService?

The exported getter returns non-optional SourceEditorLanguageService. The optionality belongs to the lazy storage, not to the property as seen by callers. The stub only worked once declared:

public final var languageService: SourceEditor.SourceEditorLanguageService { get }

Verified at runtime, not just at link time.


5. Protocol extension blocks are emitted multiple times

$ grep -c '^extension SourceEditor.SourceEditorSelectionObserver' SourceEditor.SourceEditorSelectionObserver.swiftinterface
3
$ grep -c '^extension SourceEditor.FoldableLanguageService' SourceEditor.FoldableLanguageService.swiftinterface
3

Three identical copies each, same members, same addresses.

Relatedly: several members of a class are printed with the same placeholder address (// Address: 0x6608 on SourceEditorView.elide, .selectionWillChange, .didScrollPositionToVisible). These are protocol-extension defaults rather than class members; labelling them as such would avoid reading them as the class's own API.


6. A "compilable output" mode: real module names plus imports

Every declaration copied out of a dump has to be hand-translated, because imported C/ObjC types are printed with a __C. prefix. Frequencies across SourceEditor:

402  __C.NSColor      →  AppKit.NSColor
224  __C.CGRect       →  CoreGraphics.CGRect
222  __C.CGPoint      →  CoreGraphics.CGPoint
185  __C._NSRange     →  Foundation.NSRange
185  __C.NSView       →  AppKit.NSView
167  __C.NSEvent      →  AppKit.NSEvent
157  __C.NSImage      →  AppKit.NSImage
149  __C.NSFont       →  AppKit.NSFont

There is already a --show-imported-c-types flag; a mode that resolves __C.X to its defining module and emits the matching import lines would remove most of the mechanical work in a reconstruction.

7. Bundling several types into one file

The per-type layout is right for browsing, but reconstruction always needs "these N types, in one file that compiles". Emitting a chosen set together, with the imports from §6, would close the loop.


8. Document what is not recoverable

Blank fields read as "the original didn't have this" rather than "this can't be recovered". §1 cost me time for exactly that reason — I read the absence of any dispatch annotation as "nothing special about this member".

The clearest example is implicitly unwrapped optionals. Since SE-0054 (Swift 4.2) T! is not a type — it is Optional<T> plus an attribute on the declaration — so it is absent from both manglings and reflection metadata. A minimal probe:

open class Sample {
    public var storedOptional: NSString?
    public var storedImplicitlyUnwrapped: NSString!
    open func returnsOptional() -> NSString? { nil }
    open func returnsImplicitlyUnwrapped() -> NSString! { nil }
}

built with -enable-library-evolution:

returnsOptional              →  …So8NSStringCSgyF
returnsImplicitlyUnwrapped   →  …So8NSStringCSgyF    identical

__swift5_reflstr: 2 entries          __swift5_typeref: 1 entry
    storedOptional                       So8NSStringCSg
    storedImplicitlyUnwrapped

Two field names, one type reference — the compiler deduplicates them, because to it they are the same type.

Unlike final, guessing wrong here is harmless: printing ? is ABI-identical and only changes whether a caller unwraps. So the ask is just a note in the export header, something like:

IUO (T!) is not represented in the binary and is always printed as T?. This does not affect ABI.

Other things worth listing as unrecoverable rather than left blank: parameter internal names (only labels survive), @available, @discardableResult, default argument values, and the internal/fileprivate distinction.


Happy to open separate issues for any of these, or to send a PR for §1 and §2 — those two are the ones that turn a dump into something you can transcribe without a second tool.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions