RFC-0008: MVP Skill Registry (Phase 1) by jwm4 · Pull Request #26 · mlflow/rfcs

jwm4 · 2026-07-14T16:38:13Z

Summary

Add a Skill Registry to MLflow: a governed, metadata-first registry for AI agent skills.

This is a restructured Phase 1 of the work originally proposed in PR #10, narrowed per Slack discussion with Databricks maintainers to focus on skills and skill bundles only.

Skills: SKILL.md directories with versioning, lifecycle, aliases, tags
Skill bundles: versioned collections grouping related skills
Pull semantics: harness-agnostic content fetch from Git, OCI, ZIP, MLflow artifact storage
Package manager integration: delegates harness-specific installation to APM, Lola, or other plugins
Trace integration: mlflow.skill_context() creates SKILL spans with registry coordinates

Phase 2 (RFC-0009, separate PR) will extend bundles with subagents, hooks, and MCP server references.

Changes from PR #10

Removed subagent and hook entity types (deferred to Phase 2)
Skill bundles contain only skills (no MCP server cross-references)
Replaced custom harness adapters with package manager plugin interface
Removed import command and lock file format
Aligned with RFC-0004 (MCP Registry) patterns throughout
All design decisions from the PR RFC-0008 and RFC-0009: Skill Registry and Harness Integration #10 review are preserved

Files

rfcs/0008-mvp-skill-registry/0008-mvp-skill-registry.md: Main RFC
rfcs/0008-mvp-skill-registry/implementation-details.md: DB schema, store interface, REST API, SDK, CLI, package manager plugin interface

Test plan

Review main RFC for completeness and consistency with RFC-0004
Review implementation details for store/SDK/REST completeness
Verify all PR RFC-0008 and RFC-0009: Skill Registry and Harness Integration #10 settled decisions are preserved
Confirm no subagent/hook/MCP references leaked into Phase 1

Posted by Bill Murdock with assistance from Claude Code.

Metadata-first registry for AI agent skills with versioning, lifecycle management, trace integration, and package manager plugin interface. Phase 1 covers skills and skill bundles (skills-only). Phase 2 (RFC-0009, separate PR) will extend bundles with subagents, hooks, and MCP server references. Replaces the closed PR mlflow#10 with a phased approach per Slack discussion with Databricks maintainers. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Accept changes from Codex review: SDK namespace to mlflow.genai.*, plugin import section, automatic harness instrumentation, install via package manager. Trim auto instrumentation to essentials. Remove register_skill idempotency and align with MCP Server Registry behavior (fail if version exists), citing register_mcp_server() in mlflow/mlflow#23696 as rationale. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Fill in rfc_pr link, fix "The two entity types:" grammar, replace all RFC-0009 references with "follow-up RFC", and change non-skill member lists to examples (e.g., subagents, MCP server references) since the set may change before that RFC is written. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

…ntics Resolve both open investigation items: harness argument is now required on install commands, reproducibility depends on the package manager plugin (APM has full lockfile, Lola has version constraints). Drop Phase 3 from adoption strategy. Add agentskills.io citation. Clarify that bundles can contain non-skill content which is pulled and installed but does not receive individual registry entries in Phase 1. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Add missing --harness claude-code to both install-bundle examples. Fix Phase 2 phrasing to say "add registry entries for" instead of "extend bundles to include" non-skill members. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Add mlflow-skills.lock resolution lock for reproducible cross-machine installation. Package manager plugin interface now returns PackageManagerInstallResult with harness-local skill names so trace manifests are accurate even when plugins rename skills. Monolithic bundle install passes bundle_path to the plugin so non-skill content is installed as a unit. Fix span annotation from "registry" to "workspace" and "skipped" to "unregistered" wording. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Add Open questions section per RFC template. Note that structured security scan metadata is valuable but should be addressed as a cross-registry capability shared across all registries, not as a skill-specific feature. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

HumairAK · 2026-07-22T21:00:53Z

+full lifecycle (versioning, aliases, tags, status). Note: MLflow already has an
+`mlflow skills` CLI group with `list` and `view` subcommands for
+inspecting bundled Assistant skills. The registry subcommands
+(`register`, `pull`, `install`, etc.) extend this existing group;


MLflow already has an
mlflow skills CLI group with list and view

I think the existing mlflow skills cli group's intent is a bit different and more focused towards skills used and/or bundled with MLflow rather than skills used by users for purposes external to MLflow (like for their agents). So grouping them together could be confusing, what do you think?

I agree, but not using skills for the skill registry seems inconsistent with the other registries. What I would really like is for the existing MLflow skills command to get renamed, but that's a big breaking change and probably not worth the trouble it would cause. So just having these two purposes share a group seems like the least bad option to me. @mprahl , what do you think?

I agree with Humair. We can go with something suboptimal like mlflow skills-registry. It'd be less confusing.

OK, I will go with mlflow skills-registry for now and see if anyone comes up with a better name in the review process.

HumairAK · 2026-07-22T21:25:27Z

+MLflow discovers and registers the plugin's skills as members of a
+monolithic bundle. It preserves the Git source on the bundle and warns
+about subagents, hooks, and MCP configurations that are not registered
+in Phase 1.
+


and warns about subagents, hooks, and MCP configurations that are not registered
in Phase 1.

I see these warnings are provided in various cases (like import). It feels unnecessary because it is expected that a skill api / option would register only skills.

I could argue it either way, but Yuki specifically asked for the warnings, and they seem somewhat useful to me.

HumairAK · 2026-07-22T21:52:53Z

+
+New feature, not a breaking change. Phased rollout:
+
+- **Phase 1 (this RFC):** Skill and SkillBundle entities, store,


Feel free to disagree here but I'm of the opinion we should not message this as Phase 1 / Phase 2, but instead:

RFC-0008: MVP Skill Registry (Phase 1) ---> RFC-0008: Skill Registry
Phase 2 (follow-up RFC) --> Move to "Future Improvements" or "Follow up" section

@mprahl wdyt?

HumairAK · 2026-07-23T02:31:19Z

+   fill in name, version, source type, and source URL, then submit.
+2. Create a skill bundle version that pins these members:
+   ```bash
+   mlflow skills create-bundle-version --name pr-workflow --version 1.0.0 \


mlflow skills create-bundle-version or update-bundle-version starts to feel crowded and scales poorly as more bundle actions appear. My preference would be to do something like:

mlflow skills register ... mlflow skills update-version ... mlflow skills set-alias ... ... mlflow skills bundles create-version ... mlflow skills bundles update-version ... mlflow skills bundles set-alias ... etc.

HumairAK · 2026-07-23T02:55:51Z

+
+1. Import a Claude Code plugin from a remotely accessible source:
+   ```bash
+   mlflow skills import \


I continue to be a bit torn on this part of the design. We are going through the effort of offloading harness-specific installation behavior to package managers, while still baking harness- or format-specific import semantics into MLflow for mlflow skills import. That asymmetry feels a bit counterintuitive to me. Can we better justify why import belongs in MLflow core, or consider an adapter boundary there as well? cc @mprahl

In this RFC the actual complexity of skills import is quite low since all we are registering are skills, so the implementation for pretty much all harnesses would just be list all the skills in the skill directory and then register each one. It becomes a bigger issue for the next RFC (coming soon) which will introduce some support for non-skill elements. So we might want to discuss this further for that one.

HumairAK · 2026-07-23T03:46:52Z

+   traces = mlflow.search_traces(
+       experiment_ids=[experiment_id],
+       filter_string=(
+           'span.type = "SKILL"'
+           ' AND span.attributes.mlflow.skill.name = "code-review"'
+           ' AND span.attributes.mlflow.skill.version = "1.0.0"'
+       ),
+   )


this experience feels like it could be improved with first class filters like:

traces = mlflow.search_skill_traces( experiment_ids=[experiment_id], name="code-review", version="1.0.0", )

HumairAK · 2026-07-23T03:56:35Z

+A follow-up RFC will add registry entries for non-skill bundle
+members (e.g., subagents, MCP server references), enabling full
+"plugin"-style bundles.
+The member table schema includes a `member_type` field for forward
+compatibility with this extension.


You might want to add justification why the name "SkillBundle" should be used here then, because I have a feeling the same question is going to come up again.

HumairAK · 2026-07-23T04:15:49Z

+import](implementation-details.md#plugin-import) for the SDK return
+type, CLI mapping, discovery rules, and conflict behavior.
+
+### Pull semantics


I see that if I pull then install, I re-download from source, maybe consider adding a previously materialized local path rather than forcing it to resolve and fetch again.

HumairAK · 2026-07-23T04:21:27Z

+
+#### Automatic harness instrumentation
+
+Phase 1 extends the Claude Code autologger to recognize skill


It would be nice if the RFC also points out the plan for the others in a future enhancement section, which, if any, should be prime candidates for auto log support.

HumairAK · 2026-07-23T04:32:20Z

+
+### Manifest writing and discovery
+
+Installation commands write or update the manifest after all requested


One thing that caught me by surprise was the decision to have MLflow locally materialize the skill or skull-bundle. I think this is a good way to ensure MLflow retains domain ownership over resolution semantics, instead of trying to squeeze registry behavior into another tool's package-resolution abstraction.

I think it would be good to explicitly mention the justification somewhere.

I'd imagine there are some tradeoffs here as well, for example if a package manager has features / capabilities around remote source semantics. If you agree, consider adding a small note about it.

mprahl · 2026-07-23T18:56:03Z

+        registry skill names to harness-local names."""
+        ...
+
+    def supported_harnesses(self) -> list[str]:


Is there a CLI option to list the supported harnesses based on all the package manager plugins installed?

mprahl · 2026-07-23T20:32:06Z

+A follow-up RFC will add registry entries for non-skill bundle
+members (e.g., subagents, MCP server references), enabling full
+"plugin"-style bundles.
+The member table schema includes a `member_type` field for forward


Can we drop member_type for now? I think we can backfill the value if the subsequent RFC adds it.

mprahl · 2026-07-23T20:36:50Z

+
+Subagents, hooks, MCP configurations, and other non-skill content remain
+in the source artifact but are not registered as entities or members in
+Phase 1. The import result reports a warning for each skipped category.


Nit: It may be better to just report unrecognized types as Claude plugin contents may evolve faster than the MLflow client knows how to recognize new categories.

mprahl · 2026-07-23T20:42:18Z

+```python
+with mlflow.skill_context(
+    name="code-review", version="1.0.0"
+) as span:


Do we need to return the span? I worry about manipulation by the user after the fact.

mprahl · 2026-07-23T20:43:37Z

+closes the observability loop by letting agent developers indicate
+which registered skill is active during each part of a trace.
+
+#### `mlflow.skill_context()` context manager


Do we want to add an equivalent of link_prompt_versions_to_trace that is after the fact but for skills?

mprahl · 2026-07-23T20:46:49Z

+displaying traces and shows a "not found in registry" indicator if
+the coordinates do not resolve.
+
+#### Workspace resolution


I'm somewhat conflicted as it seems nice to be able to have skills from multiple workspaces but at the same time, there aren't APIs or UIs that operate across workspaces today. I think I lean towards keeping it.

mprahl · 2026-07-23T20:48:05Z

+tracking, deprecation impact analysis, per-skill cost attribution, and
+regression detection.
+
+A follow-up RFC will add registry entries for non-skill bundle


Let's remove references or promises to follow-up RFCs in this RFC. I think a simple bullet in the out of scope section saying other entities are deferred is sufficient.

mprahl · 2026-07-23T20:50:56Z

+`mlflow-skills.lock`, that records the exact registry coordinates and
+installation inputs selected by an install command: entity type, name,
+resolved version, workspace, package manager, harness, and scope. Aliases
+are resolved before they are written. Passing `--lock-file` to either


Can we keep the --from-lock CLI arg consistent with npm install and yarn install behavior? Preferably reuse their argument names too. I think this would be the most intuitive.

mprahl · 2026-07-23T20:54:48Z

+    def install_skill(
+        self,
+        name: str,
+        local_path: str,


Does it make sense to have a path convention rather than making it configurable (like vendor in Go)? That allows it to be committed in the repo and for MLflow tooling to easily find it.

mprahl · 2026-07-23T20:57:06Z

+installed via `pip install` without modifying MLflow core.
+
+```python
+class PackageManagerPlugin:


This approach is fine, but it'd be nice that along with https://github.com/mlflow/rfcs/pull/26/changes#r3641331561, the package manager plugin was able to write a lockfile in the format of the package manager so that the vendored content + package manager native lock file can be committed in the repo. For teams collaborating on an agent, it allows them to use this without knowing anything about MLflow.

mprahl · 2026-07-23T20:59:02Z

+installed via `pip install` without modifying MLflow core.
+
+```python
+class PackageManagerPlugin:


I think we need a method here that checks if the requirements are satisfied to use this package manager (e.g. apm is installed and is at least version X). This would be a preflight check that the MLflow CLI would perform.

mprahl · 2026-07-23T20:59:57Z

+creation:
+
+```json
+{


Could we include the tracking server URL that was used to resolve this lock file? This would allow someone mlflow skills pull without setting the tracking server URL.

mprahl · 2026-07-23T21:04:34Z

+
+`mlflow skills pull` provides a harness-agnostic way to fetch
+registered content from its source. Harness-specific installation
+delegates to package managers (APM, Lola, or others via a plugin


Is there a CLI subcommand to show the available package managers?

mprahl · 2026-07-23T21:08:22Z

+   There is no standard way to fetch content from all of these with a
+   single command.
+
+### User journeys


Optional: It may be worth updating this section to user persona driven (e.g. developer browsing available skills, agent developer wants to compare agent versions that use different skills).

mprahl · 2026-07-23T21:09:20Z

+| `creation_timestamp` | `BigInteger` | millis since epoch |
+| `last_updated_timestamp` | `BigInteger` | millis since epoch |
+
+### `skill_versions`


Could we keep track of install count? I think this would be really great to display in the UI and help developers find skills that are popular at their company.

mprahl

Looks great! I left a few comments.

jwm4 and others added 7 commits July 14, 2026 12:37

Fix internal inconsistencies in main RFC

4f93db5

Add missing --harness claude-code to both install-bundle examples. Fix Phase 2 phrasing to say "add registry entries for" instead of "extend bundles to include" non-skill members. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

HumairAK reviewed Jul 23, 2026

View reviewed changes

mprahl reviewed Jul 23, 2026

View reviewed changes

jwm4 mentioned this pull request Jul 23, 2026

Add RFC-0009: Extended Skill Bundles #27

Draft

3 tasks

mprahl reviewed Jul 23, 2026

View reviewed changes

mprahl requested changes Jul 23, 2026

View reviewed changes


		New feature, not a breaking change. Phased rollout:

		- Phase 1 (this RFC): Skill and SkillBundle entities, store,


		#### Automatic harness instrumentation

		Phase 1 extends the Claude Code autologger to recognize skill


		### Manifest writing and discovery

		Installation commands write or update the manifest after all requested

Uh oh!

Conversation

jwm4 commented Jul 14, 2026

Summary

Changes from PR #10

Files

Test plan

Uh oh!

HumairAK Jul 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

HumairAK Jul 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mprahl Jul 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mprahl Jul 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mprahl left a comment

Choose a reason for hiding this comment

HumairAK Jul 22, 2026 •

edited

Loading

HumairAK Jul 23, 2026 •

edited

Loading

mprahl Jul 23, 2026 •

edited

Loading

mprahl Jul 23, 2026 •

edited

Loading