Skip to content

Commit d9b3ff7

Browse files
authored
Merge pull request #1093 from jmpsec/node-flags-overrides
Per-node osquery flags overrides
2 parents 9b38bb3 + 2de914c commit d9b3ff7

31 files changed

Lines changed: 2912 additions & 166 deletions

‎ARCHITECTURE.md‎

Lines changed: 6 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -64,7 +64,7 @@ Main runtime components:
6464
- `pkg/environments`: environment model, enrollment/config path metadata, package/script metadata.
6565
- `pkg/health`: deployment health — the `service_status` heartbeat table, component status derivation (operational/degraded/down/stale/unknown), Go runtime snapshots, and a 24h-refreshed upgrade-version cache. Gated by `--health-enabled` (default off).
6666
- `pkg/nodes`: node registry, lookup, archive, metadata updates.
67-
- `pkg/nodeconfig`: per-node osquery configuration overrides (validation, deep merge, Redis-backed cache).
67+
- `pkg/nodeconfig`: per-node osquery configuration and flags overrides (validation, merge rules, Redis-backed caches) and the osctrld status record.
6868
- `pkg/queries`: distributed query definitions, targets, node-query state, saved queries.
6969
- `pkg/carves`: file carving metadata and storage backends (`db`, `local`, `s3`).
7070
- `pkg/logging`: TLS log pipeline, sink implementations (`db`, `file`, `stdout`, `graylog`, `splunk`, `logstash`, `kinesis`, `s3`, `kafka`, `elastic`), and DB/S3 readers.
@@ -321,7 +321,9 @@ Concrete example for config:
321321
6. It queues a `lastSeenUpdate` in the batch writer.
322322
7. It returns `env.Configuration` as the osquery config payload, with the node's per-node override (`pkg/nodeconfig`) deep-merged on top when one exists and has not expired.
323323

324-
Per-node configuration overrides are admin-only (environment admin, enforced server-side and audited) and layered on the environment configuration: objects merge key by key, other values replace, and environment changes keep reaching the node. Options that keep the host connected to osctrl (e.g. config/logger/distributed/enroll TLS settings, host identifier) and `null` values are refused. The override applies only while the node is in the environment it was saved for, and an invalid stored override falls back to the environment configuration. An override may carry an expiry (none by default); an expired one stops applying and stays listed until removed. Changes take effect on the host's next config fetch; osquery does not reset options that the environment doesn't set, so those keep their overridden value until osquery restarts. Flags overrides are not available yet because osctrld cannot identify the node.
324+
Per-node configuration overrides are admin-only (environment admin, enforced server-side and audited) and layered on the environment configuration: objects merge key by key, other values replace, and environment changes keep reaching the node. Options that keep the host connected to osctrl (e.g. config/logger/distributed/enroll TLS settings, host identifier) and `null` values are refused. The override applies only while the node is in the environment it was saved for, and an invalid stored override falls back to the environment configuration. An override may carry an expiry (none by default); an expired one stops applying and stays listed until removed. Changes take effect on the host's next config fetch; osquery does not reset options that the environment doesn't set, so those keep their overridden value until osquery restarts.
325+
326+
Per-node flags overrides (`GET|PUT|DELETE /api/v1/nodes/{env}/node/{uuid}/flags`, same admin/audit/expiry rules) follow the same model for the osquery flagfile. The text is one flag per line (`--name` or `--name=value`); connection flags (including their `no<name>` forms) are refused, while flags also set as configuration options are listed as conflicts (a warning, not refused). The override merges into the environment flags by name (replacing a flag in place, appending new ones). Only `osctrld` reads flags, and only in service mode, so the override reaches a host when osctrld's sync request carries that host's osquery UUID (`uuid`, from `system_info.uuid`) and the UUID matches the node: `osctrl-tls` then merges the active override into the osctrld `flags` and `verify` responses and records the fetch in `node_osctrld_status`. osctrld reports the outcome of its last flags change (`apply`); because osquery exits on a flag it rejects, osctrld checks that `osqueryd` stays running after a restart, otherwise restores the environment flags, restarts, reports `rejected` and does not re-apply that content while it is served. Flags apply on the next sync (default 60 minutes) with an osquery restart and fully revert on removal or expiry. Deleting a node (API or CLI) removes its overrides and osctrld status.
325327

326328
### Steady-state database load
327329

@@ -380,6 +382,8 @@ Feature-owned tables are migrated only when their manager is initialized. In par
380382
- `pkg/nodes.OsqueryNode` -> `osquery_nodes`
381383
- `pkg/nodes.ArchiveOsqueryNode` -> `archive_osquery_nodes`
382384
- `pkg/nodeconfig.Override` -> `node_config_overrides`
385+
- `pkg/nodeconfig.FlagsOverride` -> `node_flags_overrides`
386+
- `pkg/nodeconfig.OsctrldStatus` -> `node_osctrld_status`
383387
- Users and auth:
384388
- `pkg/users.AdminUser` -> `admin_users`
385389
- `pkg/users.UserPermission` -> `user_permissions`

‎README.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -44,7 +44,7 @@ Whether you’re running a small deployment or managing large fleets, **osctrl**
4444

4545
- **Modern operator UI**: React SPA powered by `osctrl-api`, with views for nodes, environments, queries, saved queries, carves, tags, users, enrollment, audit log, service configuration, log sinks, auth providers, alerting, node activity, and optional posture data.
4646
- **Node console**: Read-only console for a specific node using hidden accelerated distributed queries. It supports shell-like commands such as `pwd`, `cd`, `ls`, `stat`, `ps`, `sql`, `osquery`, `.tables`, and `get` for permission-checked file carves.
47-
- **Per-node configuration overrides**: Environment admins can layer a JSON override on one node's osquery configuration from the node page, with optional expiry. Connection settings are protected, every change is audited, and options the environment doesn't set keep their overridden value on the host until osquery restarts.
47+
- **Per-node configuration and flags overrides**: Environment admins can layer a JSON override on one node's osquery configuration, or a flags override on its flagfile, from the node page, with optional expiry. Connection settings are protected, every change is audited, and options the environment doesn't set keep their overridden value on the host until osquery restarts. Flags overrides reach hosts managed by osctrld's service mode on its next sync with an osquery restart; if osquery rejects them, osctrld rolls back and reports it. Removing or expiring a flags override reverts the host on osctrld's next sync.
4848
- **File explorer**: Accelerated per-node directory listing and stat requests backed by osquery distributed queries.
4949
- **Accelerated distributed queries**: Optional osquery accelerated query reads, defaulting to a 5 second interval when enabled. Console acceleration is scoped to the target node and fresh active console sessions.
5050
- **osquery schema awareness**: Ships osquery table metadata through 5.23.1 and exposes authenticated table metadata to the UI/API for query authoring and console `.tables`.

‎cmd/api/handlers/handlers.go‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -94,6 +94,7 @@ type HandlersApi struct {
9494
PostureEnabled bool
9595
Vulns *vulns.Reader // nil when --vuln-enabled is off
9696
NodeConfig *nodeconfig.Manager
97+
FlagsOverrides *nodeconfig.FlagsManager
9798
// MFA owns the second-factor tables. Nil disables every MFA route.
9899
MFA *mfa.Manager
99100
// WebAuthn is nil when no RP ID or origin could be resolved, which
@@ -315,6 +316,13 @@ func WithNodeConfig(m *nodeconfig.Manager) HandlersOption {
315316
}
316317
}
317318

319+
// WithFlagsOverrides wires the per-node osquery flags override manager.
320+
func WithFlagsOverrides(m *nodeconfig.FlagsManager) HandlersOption {
321+
return func(h *HandlersApi) {
322+
h.FlagsOverrides = m
323+
}
324+
}
325+
318326
// WithVulns wires the vulnerability read model (nil when disabled).
319327
func WithVulns(r *vulns.Reader) HandlersOption {
320328
return func(h *HandlersApi) {

‎cmd/api/handlers/node_flags.go‎

Lines changed: 245 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,245 @@
1+
package handlers
2+
3+
import (
4+
"context"
5+
"encoding/json"
6+
"errors"
7+
"net/http"
8+
"time"
9+
10+
"github.com/jmpsec/osctrl/pkg/environments"
11+
"github.com/jmpsec/osctrl/pkg/nodeconfig"
12+
"github.com/jmpsec/osctrl/pkg/nodes"
13+
"github.com/jmpsec/osctrl/pkg/types"
14+
"github.com/jmpsec/osctrl/pkg/utils"
15+
"github.com/rs/zerolog/log"
16+
)
17+
18+
// currentFlagsOverride returns the node's flags override in env, or nil. A row
19+
// left over from another environment is not applied by osctrl-tls, so it is
20+
// treated as absent here too.
21+
func (h *HandlersApi) currentFlagsOverride(env environments.TLSEnvironment, node nodes.OsqueryNode) (*nodeconfig.FlagsOverride, error) {
22+
o, err := h.FlagsOverrides.Get(node.UUID)
23+
if err != nil || o == nil || o.EnvironmentID != env.ID {
24+
return nil, err
25+
}
26+
return o, nil
27+
}
28+
29+
// nodeFlagsState builds the response. Applied mirrors what osctrl-tls serves
30+
// osctrld: the environment flags unless an unexpired override merges cleanly.
31+
func (h *HandlersApi) nodeFlagsState(env environments.TLSEnvironment, node nodes.OsqueryNode, o *nodeconfig.FlagsOverride) (types.NodeFlagsResponse, error) {
32+
envFlags, err := h.Envs.NodeFlags(env, "", "", h.OsqueryValues)
33+
if err != nil {
34+
return types.NodeFlagsResponse{}, err
35+
}
36+
cfg, err := h.currentOverride(env, node)
37+
if err != nil {
38+
return types.NodeFlagsResponse{}, err
39+
}
40+
st, err := h.FlagsOverrides.Status(node.UUID)
41+
if err != nil {
42+
return types.NodeFlagsResponse{}, err
43+
}
44+
resp := types.NodeFlagsResponse{Environment: envFlags, Applied: envFlags}
45+
if o != nil {
46+
updatedAt := o.UpdatedAt
47+
resp.Override = o.Flags
48+
resp.ExpiresAt = o.ExpiresAt
49+
resp.Expired = o.Expired(time.Now())
50+
resp.UpdatedBy = o.UpdatedBy
51+
resp.UpdatedAt = &updatedAt
52+
if !resp.Expired {
53+
if merged, err := nodeconfig.MergeFlags(envFlags, o.Flags); err == nil {
54+
resp.Applied = merged
55+
} else {
56+
log.Err(err).Msgf("node %s: stored flags override does not apply", o.NodeUUID)
57+
}
58+
}
59+
}
60+
nodeCfg := ""
61+
if cfg != nil {
62+
nodeCfg = cfg.Config
63+
}
64+
resp.Conflicts = nodeconfig.FlagOptionConflicts(resp.Override, env.Configuration, nodeCfg)
65+
// A status from another environment describes flags this one never served.
66+
if st != nil && st.EnvironmentID == env.ID {
67+
resp.Osctrld = osctrldStatusView(st, o)
68+
}
69+
return resp, nil
70+
}
71+
72+
// osctrldStatusView reports the node's osctrld status. RejectedCurrent compares
73+
// the host-reported apply time with the override's last update; it is clamped
74+
// to the server's fetch time so a skewed or forged future time cannot pin a
75+
// rejection. An expired override is no longer served, so it is never current.
76+
// SHA-256s cannot be compared: osctrld hashes the flags with the
77+
// host's real secret and certificate paths.
78+
func osctrldStatusView(st *nodeconfig.OsctrldStatus, o *nodeconfig.FlagsOverride) *types.OsctrldStatusView {
79+
v := &types.OsctrldStatusView{
80+
LastFetchedAt: st.LastFetchedAt,
81+
ApplyResult: st.ApplyResult,
82+
ApplyError: st.ApplyError,
83+
ApplyAt: st.ApplyAt,
84+
}
85+
if o != nil && !o.Expired(time.Now()) && st.ApplyResult == "rejected" && st.ApplyAt != nil {
86+
at := *st.ApplyAt
87+
if st.LastFetchedAt.Before(at) {
88+
at = st.LastFetchedAt
89+
}
90+
v.RejectedCurrent = at.After(o.UpdatedAt)
91+
}
92+
return v
93+
}
94+
95+
// serveNodeFlags writes the node's flags state for o.
96+
func (h *HandlersApi) serveNodeFlags(w http.ResponseWriter, env environments.TLSEnvironment, node nodes.OsqueryNode, o *nodeconfig.FlagsOverride) {
97+
resp, err := h.nodeFlagsState(env, node, o)
98+
if err != nil {
99+
apiErrorResponse(w, "error getting node flags", http.StatusInternalServerError, err)
100+
return
101+
}
102+
utils.HTTPResponse(w, utils.JSONApplicationUTF8, http.StatusOK, resp)
103+
}
104+
105+
// NodeFlagsGetHandler - GET Handler for a node's flags override
106+
// @Summary Get node flags override
107+
// @Description Returns the node's osquery flags override, the environment flags, the flags the node's osctrld is served, flags that are also set as configuration options, and the last osctrld fetch. The osctrld apply_result, apply_error and apply_at fields are reported by the host and are untrusted.
108+
// @Tags nodes
109+
// @Produce json
110+
// @Param env path string true "Environment name or UUID"
111+
// @Param uuid path string true "Node UUID"
112+
// @Success 200 {object} types.NodeFlagsResponse
113+
// @Failure 400 {object} types.ApiErrorResponse "Bad request"
114+
// @Failure 401 {object} types.ApiErrorResponse "Unauthorized"
115+
// @Failure 403 {object} types.ApiErrorResponse "Forbidden"
116+
// @Failure 404 {object} types.ApiErrorResponse "Not found"
117+
// @Failure 409 {object} types.ApiErrorResponse "Conflict"
118+
// @Failure 429 {object} types.ApiErrorResponse "Too many requests"
119+
// @Failure 500 {object} types.ApiErrorResponse "Internal server error"
120+
// @Failure 503 {object} types.ApiErrorResponse "Service unavailable"
121+
// @Security ApiKeyAuth
122+
// @Router /api/v1/nodes/{env}/node/{uuid}/flags [get]
123+
func (h *HandlersApi) NodeFlagsGetHandler(w http.ResponseWriter, r *http.Request) {
124+
env, node, _, ok := h.nodeConfigTarget(w, r)
125+
if !ok {
126+
return
127+
}
128+
o, err := h.currentFlagsOverride(env, node)
129+
if err != nil {
130+
apiErrorResponse(w, "error getting node flags override", http.StatusInternalServerError, err)
131+
return
132+
}
133+
h.serveNodeFlags(w, env, node, o)
134+
}
135+
136+
// NodeFlagsPutHandler - PUT Handler to set a node's flags override
137+
// @Summary Set node flags override
138+
// @Description Sets the node's osquery flags override, one flag per line, merged over the environment flags osctrld fetches. Flags that control the connection to osctrl and repeated flags are refused. The osctrld apply_result, apply_error and apply_at fields are reported by the host and are untrusted.
139+
// @Tags nodes
140+
// @Accept json
141+
// @Produce json
142+
// @Param env path string true "Environment name or UUID"
143+
// @Param uuid path string true "Node UUID"
144+
// @Param request body types.NodeFlagsRequest true "Request body"
145+
// @Success 200 {object} types.NodeFlagsResponse
146+
// @Failure 400 {object} types.ApiErrorResponse "Bad request"
147+
// @Failure 401 {object} types.ApiErrorResponse "Unauthorized"
148+
// @Failure 403 {object} types.ApiErrorResponse "Forbidden"
149+
// @Failure 404 {object} types.ApiErrorResponse "Not found"
150+
// @Failure 409 {object} types.ApiErrorResponse "Conflict"
151+
// @Failure 413 {object} types.ApiErrorResponse "Request body too large"
152+
// @Failure 429 {object} types.ApiErrorResponse "Too many requests"
153+
// @Failure 500 {object} types.ApiErrorResponse "Internal server error"
154+
// @Failure 503 {object} types.ApiErrorResponse "Service unavailable"
155+
// @Security ApiKeyAuth
156+
// @Router /api/v1/nodes/{env}/node/{uuid}/flags [put]
157+
func (h *HandlersApi) NodeFlagsPutHandler(w http.ResponseWriter, r *http.Request) {
158+
env, node, user, ok := h.nodeConfigTarget(w, r)
159+
if !ok {
160+
return
161+
}
162+
var req types.NodeFlagsRequest
163+
if err := json.NewDecoder(http.MaxBytesReader(w, r.Body, nodeConfigMaxBody)).Decode(&req); err != nil {
164+
var tooBig *http.MaxBytesError
165+
if errors.As(err, &tooBig) {
166+
apiErrorResponse(w, "request body too large", http.StatusRequestEntityTooLarge, err)
167+
} else {
168+
apiErrorResponse(w, "error parsing PUT body", http.StatusBadRequest, err)
169+
}
170+
return
171+
}
172+
if req.ExpiresAt != nil && !req.ExpiresAt.After(time.Now()) {
173+
apiErrorResponse(w, "expires_at must be in the future", http.StatusBadRequest, nil)
174+
return
175+
}
176+
if err := nodeconfig.ValidateFlagsOverride(req.Override); err != nil {
177+
apiErrorResponse(w, "invalid override: "+err.Error(), http.StatusBadRequest, err)
178+
return
179+
}
180+
before, err := h.currentFlagsOverride(env, node)
181+
if err != nil {
182+
apiErrorResponse(w, "error getting node flags override", http.StatusInternalServerError, err)
183+
return
184+
}
185+
stored, err := h.FlagsOverrides.Set(context.WithoutCancel(r.Context()), nodeconfig.FlagsOverride{
186+
NodeUUID: node.UUID,
187+
EnvironmentID: env.ID,
188+
Flags: req.Override,
189+
ExpiresAt: req.ExpiresAt,
190+
UpdatedBy: user,
191+
})
192+
if err != nil {
193+
apiErrorResponse(w, "error saving node flags override", http.StatusInternalServerError, err)
194+
return
195+
}
196+
expires := "never"
197+
if req.ExpiresAt != nil {
198+
expires = req.ExpiresAt.UTC().Format(time.RFC3339)
199+
}
200+
h.AuditLog.NodeAction(user, "set flags override on node "+node.UUID+" (expires "+expires+"): before="+overrideFlags(before)+" after="+req.Override, utils.GetIP(r), env.ID)
201+
h.serveNodeFlags(w, env, node, stored)
202+
}
203+
204+
// NodeFlagsDeleteHandler - DELETE Handler to remove a node's flags override
205+
// @Summary Remove node flags override
206+
// @Description Removes the node's osquery flags override so its osctrld is served the environment flags. The osctrld apply_result, apply_error and apply_at fields are reported by the host and are untrusted.
207+
// @Tags nodes
208+
// @Produce json
209+
// @Param env path string true "Environment name or UUID"
210+
// @Param uuid path string true "Node UUID"
211+
// @Success 200 {object} types.NodeFlagsResponse
212+
// @Failure 400 {object} types.ApiErrorResponse "Bad request"
213+
// @Failure 401 {object} types.ApiErrorResponse "Unauthorized"
214+
// @Failure 403 {object} types.ApiErrorResponse "Forbidden"
215+
// @Failure 404 {object} types.ApiErrorResponse "Not found"
216+
// @Failure 409 {object} types.ApiErrorResponse "Conflict"
217+
// @Failure 429 {object} types.ApiErrorResponse "Too many requests"
218+
// @Failure 500 {object} types.ApiErrorResponse "Internal server error"
219+
// @Failure 503 {object} types.ApiErrorResponse "Service unavailable"
220+
// @Security ApiKeyAuth
221+
// @Router /api/v1/nodes/{env}/node/{uuid}/flags [delete]
222+
func (h *HandlersApi) NodeFlagsDeleteHandler(w http.ResponseWriter, r *http.Request) {
223+
env, node, user, ok := h.nodeConfigTarget(w, r)
224+
if !ok {
225+
return
226+
}
227+
before, err := h.currentFlagsOverride(env, node)
228+
if err != nil {
229+
apiErrorResponse(w, "error getting node flags override", http.StatusInternalServerError, err)
230+
return
231+
}
232+
if err := h.FlagsOverrides.Delete(context.WithoutCancel(r.Context()), node.UUID); err != nil {
233+
apiErrorResponse(w, "error removing node flags override", http.StatusInternalServerError, err)
234+
return
235+
}
236+
h.AuditLog.NodeAction(user, "removed flags override on node "+node.UUID+": before="+overrideFlags(before), utils.GetIP(r), env.ID)
237+
h.serveNodeFlags(w, env, node, nil)
238+
}
239+
240+
func overrideFlags(o *nodeconfig.FlagsOverride) string {
241+
if o == nil {
242+
return "none"
243+
}
244+
return o.Flags
245+
}

0 commit comments

Comments
 (0)