Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -183,8 +183,8 @@ jobs:
- name: Verify protobufs up to date
run: |
make proto-gen
if [ -n "$(git status --porcelain -- pkg/server/proto)" ]; then
git status --short -- pkg/server/proto
if [ -n "$(git status --porcelain -- gen)" ]; then
git status --short -- gen
exit 1
fi

Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/codex-review.yml
Original file line number Diff line number Diff line change
Expand Up @@ -85,7 +85,7 @@ jobs:

STEP 1 — Enumerate. List every file changed by this PR, grouped by directory.
Note (and skip review of) any generated files: anything under config/crd/,
config/rbac/role.yaml, pkg/server/proto/, any file named zz_generated*. These
config/rbac/role.yaml, gen/, any file named zz_generated*. These
are regenerated, not hand-edited; findings on them are noise.

STEP 2 — For each non-generated changed file, walk all four checks below. Even
Expand Down Expand Up @@ -148,7 +148,7 @@ jobs:
4. Quality. New/changed behavior has tests (control-flow paths exercised, ideally
over the wire); errors are wrapped not swallowed; code lands in the right
package per the layout in CONTRIBUTING.md; generated code (config/crd,
zz_generated*, pkg/server/proto) is regenerated, not hand-edited.
zz_generated*, gen/) is regenerated, not hand-edited.

## Format constraints

Expand Down
12 changes: 6 additions & 6 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -223,21 +223,21 @@ See the README's "Repository layout" for the full map. In short:
| Controller / reconciler logic | `internal/controller/` |
| Controller ↔ server HTTP wire type | `internal/controlplaneapi/` |
| Pod-binding annotation / metadata contract | `internal/enginebinding/` |
| gRPC handlers, server wiring | `pkg/server/` |
| Cache-state index logic | `pkg/index/` |
| Mutable-slot rendering (the wedge) | `pkg/render/` |
| gRPC handlers, server wiring | `internal/server/` |
| Cache-state index logic | `internal/index/` |
| Planned reusable rendering API (reserved; not implemented) | `pkg/render/` |
| Stable adapter extension contract | `pkg/adapters/{backend,runtime}/` |
| Shipping adapter implementation / registration | `internal/adapters/builtin/` |
| Engine KV-event ingest implementation | `pkg/adapters/engine/` (pending the documented `internal/subscriber/` move) |
| Engine egress client (pre-tokenized request → engine; harness / benchmark, no binary owner) | `pkg/adapters/engineclient/` |
| Engine KV-event ingest implementation | `internal/subscriber/` |
| Engine egress client (pre-tokenized request → engine; harness / benchmark, no binary owner) | `pkg/engineclient/` |
| The gRPC contract | `proto/` → then `make proto-gen` |

Each package's `doc.go` (or package comment) states which binary owns it or why
it is a supported external Go API. Follow
[`docs/design/repository-boundaries.md`](docs/design/repository-boundaries.md)
for dependency direction and the staged internal-package migration.

**Generated code** — `config/crd/`, `config/rbac/role.yaml`, `api/**/zz_generated*.go`, `pkg/server/proto/` — is committed but never hand-edited. Regenerate and commit it with the source change (`make pre-pr` verifies there's no drift).
**Generated code** — `config/crd/`, `config/rbac/role.yaml`, `api/**/zz_generated*.go`, `gen/` — is committed but never hand-edited. Regenerate and commit it with the source change (`make pre-pr` verifies there's no drift).

**gRPC contract:** when you change `proto/`, update [`docs/design/grpc-contract.md`](docs/design/grpc-contract.md) in the same commit so the design doc stays accurate. The pre-commit hook blocks a commit that touches a `.proto` without touching that doc (override with `--no-verify` only if the change truly doesn't affect the contract).

Expand Down
8 changes: 4 additions & 4 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,7 @@ SBOM_TAG := $(subst /,_,$(TAG))
MINIMAL_IMAGE_DOCKERFILE ?= dockerfiles/Dockerfile
MINIMAL_RUNTIME_BASE ?= gcr.io/distroless/static-debian13:nonroot@sha256:f7f8f729987ad0fdf6b05eeeae94b26e6a0f613bdf46feea7fc40f7bd72953e6

version_pkg = $(MODULE)/pkg/version
version_pkg = $(MODULE)/internal/version
LD_FLAGS += -X '$(version_pkg).GitVersion=$(TAG)'
LD_FLAGS += -X '$(version_pkg).GitCommit=$(shell git rev-parse HEAD 2>/dev/null || echo unknown)'

Expand Down Expand Up @@ -313,7 +313,7 @@ vulncheck: $(LOCALBIN) ## Scan dependencies + reachable code for known Go vulner
COVER_MIN ?= 90
COVER_PROFILE ?= cover.out
COVER_PROFILE_LOGIC ?= cover.logic.out
COVER_EXCLUDE := pkg/server/proto/|zz_generated|/cmd/|/hack/|pkg/testing/
COVER_EXCLUDE := gen/|zz_generated|/cmd/|/hack/|internal/testutil/

.PHONY: cover
cover: ## Run tests with coverage and print the per-function report (logic packages, cross-package counted).
Expand Down Expand Up @@ -539,7 +539,7 @@ install-hooks: ## Install git hooks (vendor-neutral naming guard) via core.hooks
.PHONY: verify-naming
verify-naming: ## Fail if core-identity files reference OCI/Oracle (see CONTRIBUTING.md).
@bad=$$(grep -rniEI '\boci\b|oci\.com|oraclecloud|\boracle\b' \
api proto pkg/server/proto config/crd config/rbac config/default config/manager config/observability config/samples config/server config/webhook config/certmanager config/overlays docs/observability internal PROJECT go.mod 2>/dev/null || true); \
api proto gen pkg config/crd config/rbac config/default config/manager config/observability config/samples config/server config/webhook config/certmanager config/overlays docs/observability internal PROJECT go.mod 2>/dev/null || true); \
if [ -n "$$bad" ]; then \
echo "✗ OCI/Oracle reference in core-identity files (banned per CONTRIBUTING.md):"; \
echo "$$bad" | sed 's/^/ /'; \
Expand Down Expand Up @@ -624,7 +624,7 @@ ci: verify-naming verify-no-internal-refs verify-dco test-dco reuse-lint verify-
.PHONY: pre-pr
pre-pr: ci ## Pre-PR gate: CI gate + generated-code drift check + sample admission check + review checklist.
@$(MAKE) --no-print-directory manifests generate proto-gen >/dev/null
@gen='config/crd config/rbac/role.yaml config/webhook/manifests.yaml api/v1alpha1/zz_generated.deepcopy.go pkg/server/proto'; \
@gen='config/crd config/rbac/role.yaml config/webhook/manifests.yaml api/v1alpha1/zz_generated.deepcopy.go gen'; \
if ! git diff --quiet -- $$gen; then \
echo "✗ generated-code drift — regenerate and commit these files:"; \
git --no-pager diff --name-only -- $$gen; \
Expand Down
19 changes: 13 additions & 6 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -91,17 +91,24 @@ the operator CLI and the CRDs.

**`inferencecache-server`** (`cmd/server`) — gRPC policy server + cache-state index + metrics
- `cmd/server/` — gRPC + HTTP server entrypoint
- `pkg/server/` — gRPC service (`LookupRoute`, `RenderTemplate`, …), health, metrics
- `internal/server/` — gRPC service (`LookupRoute`, `RenderTemplate`, …), health, metrics
- `proto/` (+ generated stubs) — the gRPC contract
- `pkg/index/` — cache-state aggregator (`CacheIndex`)
- `pkg/render/` — mutable-slot prompt rendering engine (the wedge); importable library
- `pkg/adapters/engine/` — engine KV-event hook (feeds the index)
- `internal/index/` — cache-state aggregator (`CacheIndex`)
- `pkg/render/` — reserved path for a planned reusable renderer; no stable API yet

**`kvevent-subscriber`** (`cmd/kvevent-subscriber`) — engine-side KV-event ingestion
- `cmd/kvevent-subscriber/` — composition and lifecycle entrypoint
- `internal/subscriber/` — engine KV-event hook (feeds the index)

**Engine client library** — narrow pre-tokenized OpenAI-compatible completion client
- `pkg/engineclient/` — public `EngineClient` contract and `/v1/completions` implementation
- `internal/canary/` — repository-owned prefix-cache probe and live canary

**`inferencecache`** (`cmd/inferencecache`) — operator CLI; `doctor` runs a read-only pre-flight diagnostic
- `cmd/inferencecache/` — cobra entrypoint
- `pkg/cli/doctor/` — diagnostic checks + output formatters (see `docs/cli/doctor.md`)
- `internal/cli/doctor/` — diagnostic checks + output formatters (see `docs/cli/doctor.md`)

**Shared** — `pkg/version/`, `hack/`, `dockerfiles/`, `.githooks/`
**Shared** — `internal/version/`, `hack/`, `dockerfiles/`, `.githooks/`

Private cross-binary HTTP DTOs live in `internal/controlplaneapi/`; pod-binding
metadata shared by admission and reconcilers lives in `internal/enginebinding/`.
Expand Down
6 changes: 3 additions & 3 deletions REUSE.toml
Original file line number Diff line number Diff line change
Expand Up @@ -15,8 +15,8 @@ path = [
"**/*.table",
"**/*.xml",
"LICENSE",
"pkg/adapters/engine/testdata/*.txt",
"pkg/cli/doctor/output/testdata/*.txt",
"internal/cli/doctor/output/testdata/*.txt",
"internal/subscriber/testdata/*.txt",
]
precedence = "override"
SPDX-FileCopyrightText = "2026 The inference-cache Authors"
Expand All @@ -29,7 +29,7 @@ path = [
"config/crd/bases/*.yaml",
"config/rbac/role.yaml",
"config/webhook/manifests.yaml",
"pkg/server/proto/**/*.pb.go",
"gen/**/*.pb.go",
]
precedence = "override"
SPDX-FileCopyrightText = "2026 The inference-cache Authors"
Expand Down
2 changes: 1 addition & 1 deletion api/v1alpha1/cachepolicy_types.go
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ import (
)

// CachePolicyEvictionAlgorithm identifies an index entry-eviction algorithm.
// Each value has a corresponding implementation in pkg/index; the enum grows
// Each value has a corresponding implementation in internal/index; the enum grows
// as new algorithms land. The choice is per-namespace: the controller flattens
// it (lower-cased) into ResolvedPolicy.Eviction. The index reads it when the
// entry cap is exceeded (to order victims) and, for LFU, on the lookup path (to
Expand Down
2 changes: 1 addition & 1 deletion api/v1alpha1/remaining_crds_types_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ func TestRemainingCRDSchemas(t *testing.T) {
requireRequired(t, policySchema, "spec")
policySpec := mustPath[map[string]any](t, policySchema, "properties", "spec")
// Eviction selects the index cap-based eviction algorithm. Both values are
// implemented in pkg/index (LRU-by-lastSeen and LFU-by-access-count) and the
// implemented in internal/index (LRU-by-lastSeen and LFU-by-access-count) and the
// controller propagates the choice (lower-cased) into ResolvedPolicy.
evictionSchema := mustProperty(t, policySpec, "eviction")
requireEnum(t, evictionSchema, []string{"LRU", "LFU"})
Expand Down
10 changes: 10 additions & 0 deletions buf.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -32,3 +32,13 @@ lint:
# PREFIX_EVICTED, REPLICA_UPDATED, ALL_CLEARED) instead of a TYPE_ prefix;
# the zero value is TYPE_UNSPECIFIED (ENUM_ZERO_VALUE_SUFFIX still enforced).
- ENUM_VALUE_PREFIX
breaking:
use:
- FILE
ignore_only:
# The generated Go API intentionally moved out of the server implementation
# before the project was formally deployed. All other FILE compatibility
# rules remain enabled, while the repository boundary test pins this option
# to its new public gen/ path so a later import-path change still fails CI.
FILE_SAME_GO_PACKAGE:
- proto/inferencecache/v1alpha1/inferencecache.proto
13 changes: 6 additions & 7 deletions cmd/controller/main.go
Original file line number Diff line number Diff line change
Expand Up @@ -24,10 +24,9 @@ import (
cachev1alpha1 "github.com/cachebox-project/inference-cache/api/v1alpha1"
builtinadapters "github.com/cachebox-project/inference-cache/internal/adapters/builtin"
"github.com/cachebox-project/inference-cache/internal/controller"
"github.com/cachebox-project/inference-cache/internal/version"
podwebhook "github.com/cachebox-project/inference-cache/internal/webhook/pod"
cachewebhookv1alpha1 "github.com/cachebox-project/inference-cache/internal/webhook/v1alpha1"
adapterruntime "github.com/cachebox-project/inference-cache/pkg/adapters/runtime"
"github.com/cachebox-project/inference-cache/pkg/version"
)

const leaderLockName = "inference-cache-controller-leader-lock"
Expand Down Expand Up @@ -70,7 +69,7 @@ func defaultOptions() options {
cacheIndexRefreshEvery: controller.DefaultRefreshInterval,
policyPushEvery: controller.DefaultPolicyPushInterval,
subscriberImage: "",
policyServerGRPCAddress: adapterruntime.DefaultPolicyServerGRPCAddress,
policyServerGRPCAddress: "inference-cache-server.inference-cache-system.svc.cluster.local:9090",
zapOpts: zap.Options{
TimeEncoder: zapcore.RFC3339TimeEncoder,
},
Expand Down Expand Up @@ -133,10 +132,10 @@ func main() {
// address are operator-supplied: pinning the image to a digest in
// production and pointing the sidecar at the right Service DNS are
// deployment concerns, not CR-level knobs.
adapterRegistries := builtinadapters.New(
adapterruntime.WithSubscriberImage(opts.subscriberImage),
adapterruntime.WithPolicyServerGRPCAddress(opts.policyServerGRPCAddress),
)
adapterRegistries := builtinadapters.New(builtinadapters.Options{
SubscriberImage: opts.subscriberImage,
PolicyServerGRPCAddress: opts.policyServerGRPCAddress,
})
adapterRegistry := adapterRegistries.Runtime

// /probe wrapper for the CacheBackend reconciler's functional-probe gate.
Expand Down
4 changes: 2 additions & 2 deletions cmd/inferencecache/doctor.go
Original file line number Diff line number Diff line change
Expand Up @@ -27,8 +27,8 @@ import (
"sigs.k8s.io/controller-runtime/pkg/client"

cachev1alpha1 "github.com/cachebox-project/inference-cache/api/v1alpha1"
"github.com/cachebox-project/inference-cache/pkg/cli/doctor/checks"
"github.com/cachebox-project/inference-cache/pkg/cli/doctor/output"
"github.com/cachebox-project/inference-cache/internal/cli/doctor/checks"
"github.com/cachebox-project/inference-cache/internal/cli/doctor/output"
)

// Server-discovery defaults. The Service name and system namespace match the
Expand Down
4 changes: 2 additions & 2 deletions cmd/inferencecache/main.go
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@
// The binary deliberately keeps its glue thin — flag parsing, Kubernetes/gRPC
// client construction, and server-endpoint discovery live here, while the
// diagnostic logic and output formatting live in the unit-tested
// github.com/cachebox-project/inference-cache/pkg/cli/doctor packages.
// internal/cli/doctor packages.
package main

import (
Expand All @@ -19,7 +19,7 @@ import (

"github.com/spf13/cobra"

"github.com/cachebox-project/inference-cache/pkg/version"
"github.com/cachebox-project/inference-cache/internal/version"
)

func main() {
Expand Down
28 changes: 14 additions & 14 deletions cmd/kvevent-fake-engine/e2e_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -7,9 +7,9 @@ package main
// GPU-free, per-PR end-to-end gate for the content-fingerprint routing path:
//
// fake engine (this package, real ZMQ PUB socket)
// → kvevent-subscriber pipeline (engine.Subscriber → engine.Reporter —
// → kvevent-subscriber pipeline (subscriber.Subscriber → subscriber.Reporter —
// the same components cmd/kvevent-subscriber wires)
// → inference-cache server (pkg/server, real gRPC over loopback TCP)
// → inference-cache server (internal/server, real gRPC over loopback TCP)
// → LookupRoute
//
// This is the regression lock for the all-NO_HINT bug: the engine's own KV
Expand Down Expand Up @@ -42,10 +42,10 @@ import (
"google.golang.org/grpc"
"google.golang.org/grpc/credentials/insecure"

"github.com/cachebox-project/inference-cache/pkg/adapters/engine"
icpb "github.com/cachebox-project/inference-cache/gen/inferencecache/v1alpha1"
"github.com/cachebox-project/inference-cache/internal/server"
"github.com/cachebox-project/inference-cache/internal/subscriber"
"github.com/cachebox-project/inference-cache/pkg/fingerprint"
"github.com/cachebox-project/inference-cache/pkg/server"
icpb "github.com/cachebox-project/inference-cache/pkg/server/proto/inferencecache/v1alpha1"
)

const (
Expand Down Expand Up @@ -155,20 +155,20 @@ func startSubscriberPipeline(t *testing.T, grpcAddr, endpoint, tenant string, lo
}
client := icpb.NewInferenceCacheClient(conn)

cfg := engine.Config{
cfg := subscriber.Config{
ReplicaID: e2eReplica,
ModelID: e2eModel,
TenantID: tenant,
HashScheme: e2eScheme,
}
reporter := engine.NewReporter(client, cfg,
engine.WithWindow(10*time.Millisecond),
engine.WithLogger(logger))
sub := engine.NewSubscriber(endpoint, e2eTopic,
engine.WithSubscriberLogger(logger),
engine.WithSubscriberBackoff(50*time.Millisecond))

out := make(chan *engine.EventBatch, 256)
reporter := subscriber.NewReporter(client, cfg,
subscriber.WithWindow(10*time.Millisecond),
subscriber.WithLogger(logger))
sub := subscriber.NewSubscriber(endpoint, e2eTopic,
subscriber.WithSubscriberLogger(logger),
subscriber.WithSubscriberBackoff(50*time.Millisecond))

out := make(chan *subscriber.EventBatch, 256)
subCtx, cancelSub := context.WithCancel(context.Background())

// Run only exits via context cancellation (it reconnects forever, fail-soft)
Expand Down
10 changes: 5 additions & 5 deletions cmd/kvevent-fake-engine/main_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -8,23 +8,23 @@ import (
"bytes"
"testing"

"github.com/cachebox-project/inference-cache/pkg/adapters/engine"
"github.com/cachebox-project/inference-cache/internal/subscriber"
"github.com/cachebox-project/inference-cache/pkg/fingerprint"
)

// decodeStored decodes one payload through the REAL subscriber decoder and
// returns its BlockStored events. If the synthetic encoding drifts from what
// the subscriber decodes, a smoke would assert against a key the subscriber
// never produced (false green) — so every shape change must round-trip here.
func decodeStored(t *testing.T, payload []byte) []engine.BlockStored {
func decodeStored(t *testing.T, payload []byte) []subscriber.BlockStored {
t.Helper()
batch, err := engine.DecodeEventBatch(payload)
batch, err := subscriber.DecodeEventBatch(payload)
if err != nil {
t.Fatalf("DecodeEventBatch: %v", err)
}
out := make([]engine.BlockStored, 0, len(batch.Events))
out := make([]subscriber.BlockStored, 0, len(batch.Events))
for i, ev := range batch.Events {
bs, ok := ev.(engine.BlockStored)
bs, ok := ev.(subscriber.BlockStored)
if !ok {
t.Fatalf("event %d = %T, want BlockStored", i, ev)
}
Expand Down
Loading
Loading