feat(extension): add extension registry store module by margaretjgu · Pull Request #299 · elastic/cli

margaretjgu · 2026-05-12T16:31:48Z

Summary

Closes #293. Part of #222.

Adds src/extension/store.ts with five exports: readExtensions, writeExtensions, findExtension, upsertExtension, removeExtension
Registry persisted as JSON at ~/.elastic/extensions.json; parent directory is created automatically
Missing registry file treated as empty (no error on first use)
_testSetRegistryPath seam lets tests redirect to a tmp dir without touching the real home directory
14 unit tests covering all public functions across empty, single, and multi-entry states

Test plan

npx tsc --noEmit passes clean
node --import tsx/esm --test test/extension/store.test.ts -- 14/14 pass

github-actions · 2026-05-12T16:35:30Z

✅MegaLinter analysis: Success

Descriptor	Linter	Files	Errors	Warnings	Elapsed time
✅ COPYPASTE	jscpd	yes	no	no	10.43s
✅ REPOSITORY	gitleaks	yes	no	no	58.15s
✅ REPOSITORY	git_diff	yes	no	no	0.45s
✅ REPOSITORY	secretlint	yes	no	no	28.97s
✅ REPOSITORY	trivy	yes	no	no	17.71s
✅ TYPESCRIPT	eslint	11	0	0	5.51s

Notices

📣 MegaLinter 9.5.0 is out! Discover the new features and security recommendations in the release announcement. (Skip this info by defining SECURITY_SUGGESTIONS: false)

See detailed reports in MegaLinter artifacts
Set VALIDATE_ALL_CODEBASE: true in mega-linter.yml to validate all sources, not only the diff

MegaLinter is graciously provided by OX Security
Show us your support by starring ⭐ the repository

…t bad names

…object

MattDevy · 2026-05-14T11:49:27Z

Alternative design: filesystem as registry (no extensions.json)

After looking at how gh extension handles this, I'd suggest reconsidering the central extensions.json approach in favour of a filesystem-based registry. The current design has a security gap: any extension (running as the same user) can overwrite extensions.json and redirect other extensions' entrypoints to arbitrary paths.

gh solves this by having no central registry at all. Extensions are stored as named directories, and the directory structure encodes everything the CLI needs:

~/.elastic/extensions/
  github/
    elastic/
      my-extension/
        package.json          ← entrypoint declared here by the author
        bin/
          elastic-my-extension

How the five operations become trivial:

Current	Filesystem approach
`readExtensions()`	`readdir ~/.elastic/extensions/**`
`writeExtensions()`	no-op (install creates the dir)
`findExtension(name)`	path constructed from source, `stat` to check existence
`upsertExtension(entry)`	install writes into the namespaced dir
`removeExtension(name)`	`rm -rf` the dir

Entrypoint discovery: require extension authors to declare it in package.json's bin field (already standard for npm packages). The CLI reads package.json from the install directory at spawn time. No stored entrypoint field that can be tampered with.

{
  "name": "@elastic/my-extension",
  "bin": {
    "elastic-my-extension": "./bin/elastic-my-extension"
  }
}

Security improvement: tampering with extension A now requires writing into extension A's own directory. A malicious extension can still do that (same user), but it can no longer silently poison all extensions by editing one shared file. Pairing with a hash stored in the per-extension package.json at install time (and verified before spawn) closes the remaining gap.

This is a larger architectural call than what this PR is scoped to, so raising it now before the store interface is built on top of -- easier to change the shape here than after the install/remove/dispatch layers are wired in.

MattDevy · 2026-05-14T11:31:53Z

+    it('creates the parent directory if missing', async () => {
+      const nested = join(tmpDir, 'subdir', 'extensions.json')
+      _testSetRegistryPath(nested)
+      await writeExtensions([ext1])
+      const raw = await readFile(nested, 'utf-8')
+      assert.deepEqual(JSON.parse(raw), [ext1])
+      _testSetRegistryPath(registryFile)
+    })


If this fails mid-test, the registry path won't be reset, which could leak into other tests, making them fail.

Consider using a try/finally block here to ensure that the state is always cleaned up

good catch, fixed with try/finally

MattDevy · 2026-05-14T11:34:17Z

+  const filtered = extensions.filter((e) => e.name !== name)
+  await writeExtensions(filtered)


Suggested change

const filtered = extensions.filter((e) => e.name !== name)

await writeExtensions(filtered)

const filtered = extensions.filter((e) => e.name !== name)

if (filtered.length === extensions.length) return

await writeExtensions(filtered)

We should only write the extensions if one was actually removed. Currently it writes regardless instead of no-oping

fixed -- early return added so we skip the write when nothing changed

MattDevy · 2026-05-14T11:36:58Z

+  if (idx === -1) {
+    extensions.push(entry)
+  } else {
+    extensions[idx] = entry
+  }
+  await writeExtensions(extensions)


Suggested change

if (idx === -1) {

extensions.push(entry)

} else {

extensions[idx] = entry

}

await writeExtensions(extensions)

const updated = idx === -1

? [...extensions, entry]

: extensions.map((e, i) => (i === idx ? entry : e))

await writeExtensions(updated)

Nit: prefer to treat arrays as immutable instead of mutating in-place

went ahead and made the change... you're right it's cleaner. the array is a fresh local read on every call so there's no correctness difference, but prefer immutable too for consistency

MattDevy · 2026-05-14T11:38:25Z

+
+  return {
+    name,
+    source: obj['source'] as string,


source field has no format validation. It accepts any non-empty string. If the expected formats are github:org/repo and npm:package, a regex validation here (like SAFE_NAME_RE for names) would catch invalid registry entries early and give callers better error messages.

good point, added a loose regex check (SAFE_SOURCE_RE) that validates the prefix is a known format -- kept it intentionally loose so future source types (e.g. git:) aren't rejected. still tight enough to catch arbitrary strings like you described

- add try/finally in parent-dir test so registry path is always reset - skip write in removeExtension when no entry matched (no-op as documented) - add loose source format validation in validateEntry so tampered registry entries with unrecognised source strings are rejected on read

…ension

margaretjgu · 2026-05-14T17:59:29Z

@MattDevy Really appreciate the detailed writeup, the gh comparison is useful context. The main reason we landed on a central json file over the directory structure is the source\ field... we need to know whether the original install was github:\ or npm:\ to decide how to upgrade (git pull vs npm update). Without it we'd need a per-extension metadata file in each install dir anyway, which ends up being a registry with extra steps.

On the security side i think both models have roughly the same exposure in the same-user threat category... in the filesystem approach a malicious extension can still overwrite its own `package.json` bin field and get re-spawned from a tampered entrypoint next time. The blast radius is narrower (one extension vs all) but the attack still works. The thing that would actually close the gap is storing an install-time hash and verifying it before spawn... that's worth a separate issue regardless of which design we go with.

The one place the filesystem model does win is concurrent writes... if two CLI processes ran simultaneously the directory approach avoids the read-modify-write race on extensions.json, though in practice the CLI is single-invocation so this probably doesn't matter.

I'll address this offline with you for further discussion about pros and cons.

MattDevy · 2026-05-15T15:47:33Z

Alternative design: filesystem as registry (no extensions.json)

After looking at how gh extension handles this, I'd suggest reconsidering the central extensions.json approach in favour of a filesystem-based registry. The current design has a security gap: any extension (running as the same user) can overwrite extensions.json and redirect other extensions' entrypoints to arbitrary paths.

gh solves this by having no central registry at all. Extensions are stored as named directories, and the directory structure encodes everything the CLI needs:
~/.elastic/extensions/
  github/
    elastic/
      my-extension/
        package.json          ← entrypoint declared here by the author
        bin/
          elastic-my-extension
How the five operations become trivial:

Current Filesystem approach
readExtensions() readdir ~/.elastic/extensions/**
writeExtensions() no-op (install creates the dir)
findExtension(name) path constructed from source, stat to check existence
upsertExtension(entry) install writes into the namespaced dir
removeExtension(name) rm -rf the dir
Entrypoint discovery: require extension authors to declare it in package.json's bin field (already standard for npm packages). The CLI reads package.json from the install directory at spawn time. No stored entrypoint field that can be tampered with.
{
  "name": "@elastic/my-extension",
  "bin": {
    "elastic-my-extension": "./bin/elastic-my-extension"
  }
}
Security improvement: tampering with extension A now requires writing into extension A's own directory. A malicious extension can still do that (same user), but it can no longer silently poison all extensions by editing one shared file. Pairing with a hash stored in the per-extension package.json at install time (and verified before spawn) closes the remaining gap.

This is a larger architectural call than what this PR is scoped to, so raising it now before the store interface is built on top of -- easier to change the shape here than after the install/remove/dispatch layers are wired in.

Discussed with @margaretjgu on Zoom, we shall keep as-is for now. We can always migrate in the future if essential

MattDevy

LGTM

JoshMock

LGTM. Great start. 👏

…remove)

…ensions

…nd handlers installExtension, uninstallExtension, upgradeExtension, and searchExtensions can throw on invalid input or missing extensions. The factory only converts { error: { code } } return values to clean user-facing messages -- unhandled throws produce a raw Node.js stack trace. Wrap all four handler functions in try-catch and return handlerError() so errors are displayed as 'Error: <message>' with a non-zero exit code rather than crashing with a full stack trace. Fixes #312

…lSettled

…, handle local upgrade

margaretjgu · 2026-05-19T19:51:38Z

Tested the extension create command end-to-end before merging. A few things found and fixed in the latest commit (0dae1ae).

What was tested

elastic extension create my-tool --json
elastic my-tool --json

Outputs { name, esUrl, kibanaUrl, args } from the scaffolded index.js.

elastic extension create demo --path ./demo/elastic-demo --json
elastic demo --json

Outputs full context JSON from the existing demo/elastic-demo/index.js without overwriting it.

elastic extension upgrade demo --json

Returns { "error": { "code": "upgrade_failed", "message": "Extension \"demo\" is a local extension. Edit the files in ... directly." } } -- clean error instead of a stack trace.

elastic extension remove demo --json

Fixes made

create --path on an existing directory was clobbering the existing index.js and package.json. Fixed: scaffold files are now only written if they don't already exist. When --path points to a live directory, the command just registers it and resolves the entrypoint from the existing package.json bin field.

upgrade on a local extension was falling through to parseSource, which threw an unhelpful "Invalid GitHub source" error. Fixed: upgradeExtension now checks for the local: prefix up front and returns a clear message.

Missing SPDX header added to demo/elastic-demo/index.js.

Security notes

name is validated against /^[a-z0-9-]+$/ before any filesystem operation
--path is resolved via path.resolve() -- no path traversal possible in the stored source field
The generated index.js is user-editable code, not a trust boundary -- same threat model as gh extension create (user owns the file, CLI spawns it as the same user)
upgradeExtension skips local extensions rather than trying to git pull or npm update on them

Comparison to gh extension create

gh scaffolds a git repo with a shell/Go entrypoint. We scaffold a Node.js script instead (fits the JS-native CLI). The key difference is --path, which lets you point at an existing directory (useful for the demo/ in-repo example). gh doesn't have an equivalent -- it always creates a new directory. No git init on our end either, which keeps it lightweight.

… discovery

MattDevy · 2026-05-20T11:15:50Z

+ *   ELASTIC_ES_USERNAME         Elasticsearch username (basic auth)
+ *   ELASTIC_ES_PASSWORD         Elasticsearch password (basic auth)
+ *   ELASTIC_KIBANA_URL          Kibana URL
+ *   ELASTIC_KIBANA_API_KEY      Kibana API key


Kibana only supports username and password and not API key AFAIK

Good catch to double-check. Kibana's REST API does support API key authentication via the standard Authorization: ApiKey <base64> header (same mechanism as ES, documented since v7.x). The ServiceBlockSchema is shared across all three services, so if a user configures Kibana with api_key in their CLI config the var gets set correctly. Added a clarifying note to the docstring.

MattDevy

LGTM, nice work!

Added a small nit on kibana auth that might need verifying

- move demo/elastic-demo to examples/extensions/basic per JoshMock nit (sets up an examples/ tree for future extensions, skills, scripts) - replace long &&-chain with EARLY_CONFIG_COMMANDS Set in cli.ts (JoshMock) - derive BUILT_IN_COMMANDS from program.commands so it never goes stale (JoshMock) - upsertExtension now returns boolean (overwritten); installExtension propagates it so handleInstall emits a warning when reinstalling a name that already exists (JoshMock) - clarify in context.ts that Kibana REST API does support ApiKey auth via Authorization header (MattDevy nit)

…ion too

… } return shape

margaretjgu added 2 commits May 12, 2026 12:31

feat(extension): add extension registry store module

a4eee36

test(extension): add unit tests for extension store

51f8bba

fix(extension): validate registry schema, restrict permissions, rejec…

033b998

…t bad names

margaretjgu mentioned this pull request May 12, 2026

feat(extension): add install and remove logic for github and npm sources #301

Closed

3 tasks

margaretjgu added 2 commits May 12, 2026 12:46

fix(extension): skip chmod on win32 to fix Windows CI permissions test

6cd53a6

fix(extension): use early return for Windows skip instead of options …

24c4ad8

…object

MattDevy reviewed May 14, 2026

View reviewed changes

margaretjgu and others added 3 commits May 14, 2026 13:52

Merge branch 'main' into feat/extension-store

cf75f16

refactor(extension): treat extensions array as immutable in upsertExt…

52fc441

…ension

Merge branch 'main' into feat/extension-store

c75590e

MattDevy approved these changes May 15, 2026

View reviewed changes

JoshMock approved these changes May 15, 2026

View reviewed changes

margaretjgu added 13 commits May 19, 2026 15:18

feat(extension): add installer module for github and npm sources

9296069

test(extension): add unit tests for installer module

ea78d5b

feat(extension): add elastic extension command group (list, install, …

f600d12

…remove)

feat(extension): wire extension command group and skip config loading

90baa3f

feat(extension): add context env-var builder for extensions

4ad7ac8

docs(extension): document credential trust model in context.ts

b51ca40

feat(extension): add extension runner module

9fb36b6

feat(extension): dispatch unknown top-level commands to installed ext…

bf1c86a

…ensions

feat(extension): add upgradeExtension and upgradeAllExtensions

78ee677

feat(extension): add searchExtensions via GitHub topic API

49023b3

feat(extension): wire upgrade and search commands into extension group

327fd54

test(extension): add tests for upgrade and search

3c9a29a

fix(extension): remove unused params from fetch stub to satisfy eslint

7e3b5f6

margaretjgu added 4 commits May 19, 2026 15:18

feat(extension): add create command to scaffold local extensions

3b3ab2a

demo: add elastic-demo extension showing JSON output

a33e377

fix(extension): destructure bin field in resolveNpmBin

72db59b

This was referenced May 19, 2026

feat(extension): add context env-var propagation for extensions #300

Closed

feat(extension): add elastic extension command group (list, install, remove) #302

Closed

feat(extension): dispatch unknown top-level commands to installed extensions #303

Closed

Merge branch 'main' into feat/extension-store

1bf52b7

margaretjgu mentioned this pull request May 19, 2026

feat(extension): add search and upgrade commands #304

Closed

5 tasks

margaretjgu added 3 commits May 19, 2026 15:36

fix(extension): fix lint errors and concurrent upgrade via Promise.al…

6eb49b6

…lSettled

fix(extension): add SPDX header to demo, skip scaffold if files exist…

0dae1ae

…, handle local upgrade

docs: add Extensions section to README

4fa33b2

fix(extension): support non-JS extensions via --path using entrypoint…

89dcdad

… discovery

MattDevy reviewed May 20, 2026

View reviewed changes

MattDevy approved these changes May 20, 2026

View reviewed changes

JoshMock reviewed May 20, 2026

View reviewed changes

Comment thread examples/extensions/basic/index.js

Comment thread src/cli.ts Outdated

Comment thread src/cli.ts Outdated

Comment thread src/extension/store.ts Outdated

margaretjgu added 4 commits May 20, 2026 13:35

Merge branch 'main' into feat/extension-store

0cef8c7

fix(extension): propagate overwrite warning through createLocalExtens…

2c131f2

…ion too

fix(test): update createLocalExtension tests for { entry, overwritten…

ca15896

… } return shape

JoshMock approved these changes May 20, 2026

View reviewed changes

margaretjgu merged commit f42593e into main May 20, 2026
19 checks passed

margaretjgu deleted the feat/extension-store branch May 20, 2026 19:50

elastic-vault-github-plugin-prod Bot mentioned this pull request May 20, 2026

chore: release main #342

Draft

		const filtered = extensions.filter((e) => e.name !== name)
		await writeExtensions(filtered)

Conversation

margaretjgu commented May 12, 2026

Summary

Test plan

Uh oh!

github-actions Bot commented May 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

✅MegaLinter analysis: Success

Notices

Uh oh!

MattDevy commented May 14, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

margaretjgu commented May 14, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

MattDevy commented May 15, 2026

Uh oh!

MattDevy left a comment

Choose a reason for hiding this comment

Uh oh!

JoshMock left a comment

Choose a reason for hiding this comment

Uh oh!

margaretjgu commented May 19, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

margaretjgu May 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

MattDevy left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

github-actions Bot commented May 12, 2026 •

edited

Loading

margaretjgu commented May 14, 2026 •

edited

Loading

margaretjgu May 20, 2026 •

edited

Loading