feat(http-client-csharp): support dynamic loading of ExternalTypes from NuGet

[JoshLove-msft]

Fixes #10665.

Adds a NuGet-backed fallback to TypeFactory.CreateExternalType so external types declared in TypeSpec can resolve to types from arbitrary NuGet packages (e.g. Azure.Core.Expressions.DataFactoryExpression), not just BCL framework types.

Resolution order in CreateExternalType

1. CreateFrameworkType (unchanged — source of truth for BCL types, no I/O)
2. ExternalTypeReferenceResolver.TryResolve (NEW NuGet fallback when externalProperties.Package is set)
3. unsupported-external-type diagnostic (now includes attempted package metadata)

Implementation highlights

• src/Utilities/NugetPackageResolver.cs extracts the FindPackageAssembly and ResolveLatestPackageVersion helpers introduced in fix(http-client-csharp): resolve PackageReference assemblies for cust… #10229 from GeneratedCodeWorkspace so they can be shared. They are also now MinVersion-aware (lower bound: highest cached/feed-resolved version >= MinVersion).
• src/Utilities/ExternalTypeReferenceResolver.cs walks the InputLibrary up front (before the Roslyn workspaces are constructed) and resolves every InputExternalTypeMetadata whose Package is set. Resolved assemblies are loaded via byte arrays (Assembly.Load + MetadataReference.CreateFromImage) so we never hold a file handle on cached NuGet dlls.
• CSharpGen.ExecuteAsync now awaits ResolveAllAsync after AddPackageReferencesFromProject and GeneratedCodeWorkspace.Initialize() is moved to after both ref-adding steps so the cached project picks up all metadata references.

Tests

• 12 new ExternalTypeReferenceResolverTests covering null/missing-package short-circuits, NuGet-cache load, MinVersion selection (highest >= MinVersion), dedup of metadata refs, unknown package, and the ResolveAllAsync pre-walk on a complex InputLibrary graph (model -> property -> union -> external).
• ModelProviderTests.ExternalTypePropertyResolvedFromNuGetCache integration test exercises the end-to-end path through TypeFactory.CreateExternalType.
• UnsupportedExternalTypeEmitsDiagnostic strengthened to assert the unresolvable property is dropped.

All 2,957 existing dotnet tests pass; 25 emitter vitest files pass; Generate.ps1 produces no spurious diff.

--generated by Copilot

[pkg-pr-new]

Open in StackBlitz

npm i https://pkg.pr.new/@typespec/http-client-csharp@10666

commit: 0e2c6a0

[github-actions]

No changes needing a change description found.

[Copilot]

Pull request overview

Adds a NuGet-backed fallback to TypeFactory.CreateExternalType so external types declared in TypeSpec can resolve from arbitrary NuGet packages via a new ExternalTypeReferenceResolver and a shared NugetPackageResolver helper extracted from GeneratedCodeWorkspace.

Changes:

• Introduce ExternalTypeReferenceResolver that pre-walks the input library, resolves external metadata from the NuGet cache/feed, loads the assembly via Assembly.Load(bytes), and registers the resulting Roslyn metadata reference once.
• Extract FindPackageAssembly / ResolveLatestPackageVersion into a shared NugetPackageResolver that is now MinVersion-aware and uses NuGetVersion ordering.
• Wire the new pre-walk into CSharpGen.ExecuteAsync (before GeneratedCodeWorkspace.Initialize) and add NuGet fallback + improved diagnostic in TypeFactory.CreateExternalType.

Reviewed changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated 5 comments.

Show a summary per file

File	Description
src/Utilities/ExternalTypeReferenceResolver.cs	New resolver with process-wide caches; loads assemblies into the default ALC and tracks added metadata references.
src/Utilities/NugetPackageResolver.cs	New shared helper with MinVersion-aware cache lookup and feed version resolution.
src/PostProcessing/GeneratedCodeWorkspace.cs	Removes the inlined FindPackageAssembly/ResolveLatestPackageVersion and delegates to NugetPackageResolver.
src/CSharpGen.cs	Calls ExternalTypeReferenceResolver.ResolveAllAsync and reorders GeneratedCodeWorkspace.Initialize after metadata refs are added.
src/TypeFactory.cs	Adds NuGet fallback between framework lookup and the unsupported-external-type diagnostic; improves message.
test/Utilities/ExternalTypeReferenceResolverTests.cs	Unit tests covering cache, MinVersion selection, dedup, unknown package, and pre-walk.
test/Providers/ModelProviders/ModelProviderTests.cs	Integration test for end-to-end NuGet resolution and assertion that unresolvable external props are dropped.

Comments suppressed due to low confidence (7)

packages/http-client-csharp/generator/Microsoft.TypeSpec.Generator/src/Utilities/ExternalTypeReferenceResolver.cs:213

• Assembly.Load(byte[]) loads the external NuGet assembly into the generator's default AssemblyLoadContext. For real-world packages such as Azure.Core.Expressions.DataFactoryExpression, the dll transitively depends on Azure.Core and other assemblies that are already loaded into the generator process (and may be at different versions). Loading into the default ALC can produce type-identity / version conflicts or FileLoadExceptions, and the loaded types leak for the lifetime of the process. The issue specifically suggested using MetadataLoadContext for reflection-only discovery of the Type (with MetadataReference.CreateFromImage still used for the Roslyn compilation side). Consider switching the reflection lookup to a MetadataLoadContext (or an isolated AssemblyLoadContext) so the generator process is not polluted with arbitrary NuGet binaries.

            Type? loadedType;
            try
            {
                // Load from the in-memory byte array so we never hold a file handle on the dll
                // (NuGet packages may be cleaned up or replaced; tests need to be able to delete).
                var assembly = Assembly.Load(assemblyBytes);
                loadedType = assembly.GetType(external.Identity, throwOnError: false);
            }
            catch (Exception ex)
            {
                generator?.Emitter?.Debug(
                    $"Failed to load assembly '{assemblyPath}' for external type '{external.Identity}': {ex.Message}");
                CacheResult(key, null);
                return null;
            }

packages/http-client-csharp/generator/Microsoft.TypeSpec.Generator/src/Utilities/ExternalTypeReferenceResolver.cs:171

• When external.MinVersion is provided, the resolver short-circuits the feed lookup and just uses MinVersion as the version to download (resolvedVersion = external.MinVersion). This contradicts the semantic the rest of the new code follows (and the issue's acceptance criteria): MinVersion is the lower bound, not the exact version to install. As a result, if the cache has no qualifying version, the user always gets exactly MinVersion even though a newer compatible version exists on the feed. Consider calling ResolveLatestPackageVersion(package, settings, minVersion) in both branches so the feed download honors the highest stable version >= MinVersion (consistent with FindPackageAssembly's behavior).

                try
                {
                    var resolvedVersion = !string.IsNullOrEmpty(external.MinVersion)
                        ? external.MinVersion!
                        : await NugetPackageResolver.ResolveLatestPackageVersion(external.Package!, nugetSettings);

                    if (!string.IsNullOrEmpty(resolvedVersion))
                    {
                        var downloader = new NugetPackageDownloader(external.Package!, resolvedVersion!, null, nugetSettings);
                        var downloadedPath = await downloader.DownloadAndInstallPackage();
                        var downloadedAssembly = Path.Combine(downloadedPath, $"{external.Package}.dll");
                        if (File.Exists(downloadedAssembly))
                        {
                            assemblyPath = downloadedAssembly;
                        }
                    }

packages/http-client-csharp/generator/Microsoft.TypeSpec.Generator/src/Utilities/ExternalTypeReferenceResolver.cs:108

• TryResolve is called synchronously from TypeFactory.CreateExternalType, but on a cache miss the Lazy factory does Task.Run(() => ResolveAsync(external)).GetAwaiter().GetResult(), which performs blocking NuGet I/O (settings loading, feed enumeration, downloads) from a synchronous call. Although Task.Run mitigates the classic SynchronizationContext deadlock, it still blocks a thread-pool thread for the duration of a potentially multi-second network round-trip per unresolved external type. Given the design intent is that ResolveAllAsync pre-populates the cache, consider making the on-demand fallback null-returning (i.e. only return cached results from TryResolve) and logging a diagnostic when the pre-walk was skipped, rather than silently doing sync-over-async I/O during type construction.

        public static Type? TryResolve(InputExternalTypeMetadata? external)
        {
            if (external == null
                || string.IsNullOrEmpty(external.Identity)
                || string.IsNullOrEmpty(external.Package))
            {
                return null;
            }

            var key = MakeKey(external);
            var lazy = _resolved.GetOrAdd(key, _ => new Lazy<Type?>(
                () => Task.Run(() => ResolveAsync(external)).GetAwaiter().GetResult(),
                LazyThreadSafetyMode.ExecutionAndPublication));
            return lazy.Value;
        }

packages/http-client-csharp/generator/Microsoft.TypeSpec.Generator/src/Utilities/ExternalTypeReferenceResolver.cs:289

• CollectExternalTypes walks ns.Models, ns.Enums, ns.Constants, and ns.Clients, but does not visit the input types reachable from operation parameter/response properties outside of the model graph (e.g. header / query / path parameter types are reached via method.Operation.Parameters, but InputClient.Parameters — client-level parameters — and operation response headers are not traversed). Anything not visited up-front will fall through to the sync-over-async path in TryResolve during generation. It is worth confirming whether external metadata can appear on those types and, if so, extending the walker (or reusing an existing InputLibraryVisitor).

        private static void CollectExternalTypes(InputLibrary library, IDictionary<string, InputExternalTypeMetadata> collected)
        {
            var visited = new HashSet<InputType>(ReferenceEqualityComparer.Instance);
            var ns = library.InputNamespace;
            if (ns == null)
            {
                return;
            }

            foreach (var model in ns.Models)
            {
                VisitType(model, visited, collected);
            }
            foreach (var enumType in ns.Enums)
            {
                VisitType(enumType, visited, collected);
            }
            foreach (var constant in ns.Constants)
            {
                VisitType(constant, visited, collected);
            }
            foreach (var client in ns.Clients)
            {
                foreach (var method in client.Methods)
                {
                    if (method.Operation == null)
                    {
                        continue;
                    }
                    foreach (var p in method.Operation.Parameters)
                    {
                        VisitType(p.Type, visited, collected);
                    }
                    foreach (var r in method.Operation.Responses)
                    {
                        if (r.BodyType != null)
                        {
                            VisitType(r.BodyType, visited, collected);
                        }
                    }
                }
            }
        }

packages/http-client-csharp/generator/Microsoft.TypeSpec.Generator/src/Utilities/ExternalTypeReferenceResolver.cs:306

• The key construction is duplicated here ($"{type.External.Package}|{type.External.Identity}|{type.External.MinVersion ?? string.Empty}") and in MakeKey on line 120-121. If the key format ever changes, the two will silently drift. Call MakeKey(type.External) instead.

                var key = $"{type.External.Package}|{type.External.Identity}|{type.External.MinVersion ?? string.Empty}";
                if (!collected.ContainsKey(key))
                {
                    collected[key] = type.External;
                }

packages/http-client-csharp/generator/Microsoft.TypeSpec.Generator/src/Utilities/NugetPackageResolver.cs:102

• A bare catch here swallows every exception (including OperationCanceledException, OutOfMemoryException, etc.) silently with no diagnostic. The previous version in GeneratedCodeWorkspace had the same pattern, but since this helper is now shared and is the only place tracking feed failures, consider at least logging via CodeModelGenerator.Instance?.Emitter?.Debug(...) so that authentication / network failures are diagnosable.

                catch
                {
                    // Skip sources that fail (auth, network, etc.)
                }

packages/http-client-csharp/generator/Microsoft.TypeSpec.Generator/test/Utilities/ExternalTypeReferenceResolverTests.cs:47

• Directory.Delete(_tempDirectory!, true) will throw if the directory does not exist (e.g. Setup failed before creating it) or if a file under it is still locked by Roslyn/Assembly load. Combined with the env-var restore that follows, a teardown exception will short-circuit the Environment.SetEnvironmentVariable restore on line 46 and leak NUGET_PACKAGES into subsequent tests in the same fixture run. Consider wrapping the delete in a try/catch or restoring the env var first.

        public void Cleanup()
        {
            ExternalTypeReferenceResolver.ResetForTests();
            Directory.Delete(_tempDirectory!, true);
            Environment.SetEnvironmentVariable("NUGET_PACKAGES", _originalNugetPackageDir, EnvironmentVariableTarget.Process);
        }