Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
53 changes: 53 additions & 0 deletions container/20260123.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,53 @@
# 2026-01-23
## :family: Participants (GitHub handles preferred)
- @garloff
- @jschoone
- @schmidax

## 🧹 Housekeeping

### Build new Cluster Stacks
- no new Kubernetes version yet

## :notebook: Agenda

### Cluster Stacks PRs

sponsor: @jschoone

- [Add registry_mirror feature #197](https://github.com/SovereignCloudStack/cluster-stacks/pull/197)
- Since we stick to the nodes created by default image_builder configuration we can add them with containerd only
- Probably only one patch needed
- How to test [Create cluster-stack for rke2 #246](https://github.com/SovereignCloudStack/cluster-stacks/pull/246)?
- Use latest rancher (with enabled turtles) for testing
- Nice dashboard
- Move from calico to cilium outstanding
- Replacement for nginx: envoy gateway is a good candidate
- Has more functionality than cilium built-in pieces (e.g. backendTLSPolicy)
- [Apply Add-Ons again after cluster upgrade #208](https://github.com/SovereignCloudStack/cluster-stacks/pull/208)
- Not sure if really needed but would probably also not hurt
- Double application of the same set of (upgraded) add-ons
- Better approach might be to split add-ons into the ones that you want to upgrade before and the ones after
- AI jschoone: Check if there are even new hooks which are useful here
- Review
- [Remove zuul related files #249](https://github.com/SovereignCloudStack/cluster-stacks/pull/249)
- Will be removed and restored if needed
- [Remove outdated providers and Cluster Stacks #250](https://github.com/SovereignCloudStack/cluster-stacks/pull/250)
- hosted-control-plane may be a good alternative to kamaji
- [Improve generate_versions.py #251](https://github.com/SovereignCloudStack/cluster-stacks/pull/251)
- [New build scripts #248](https://github.com/SovereignCloudStack/cluster-stacks/pull/248)

### Trouble reported by Vinicius (matrix)

sponsor: @garloff

- Too new capi gets deployed by latest clusterctl?
- Understood and documented?
- capo-0.13.x contains v1beta2 pieces which leads to issues (ipam-v1beta2), not installed by clusterctl < 1.11
- clusterctl 1.12 works (because things are live-converted)
- Solution: pin capo to 0.12.6 or (better) use clusterctl >= 1.12
- Hope that capo 0.15 moves fully to v1beta2

### Wild questions
- Why does CoreDNS depend on OCCM?
- Seems to slow down cluster creation significantly (as everything waits for name resolution)
41 changes: 41 additions & 0 deletions container/20260206.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
# 2026-02-06
## :family: Participants (GitHub handles preferred)

- @jschoone
- @fkr
- @garloff
- @janiskemper
- @guettli

## 🧹 Housekeeping

### Build new Cluster Stacks
- no new Kubernetes version yet

## :notebook: Agenda
### Adopt CNI-aware conformance for networking (include Cilium KPR) (SCTP)

- sponsor: @fkr

- Issue: https://github.com/SovereignCloudStack/standards/issues/982

#### Netpol NetworkPolicy
- One of them fixed upstream (thanks syself!)
- Remaining cilium shortcomings (one without KPR mode, more with)

#### SCTP
- probably not in use by most users
- Decision: Downgrade to RECOMMENDED for now

#### Failures: ExternalIP
- CAPH has no InternalIP which is required in the tests
- will be disabled (if possible) until [Issue](https://github.com/kubernetes/kubernetes/issues/136626) finds its way into the Kubernetes sources
- May be hard to switch off in testing, idea is to better record the fail, but only consider it a warning -- parse jUnit.xml and evaluate ourselves

### Conformance Tests in general

- sponsor: @fkr

- Sonobuoy is a "wrapper" for the Kubernetes tests and currently the recommended way to execute them


39 changes: 39 additions & 0 deletions container/20260220.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
# 2026-04-17

## :family: Participants (GitHub handles preferred)
Participants (GitHub handles preferred)
- @jschoone
- @garloff
- @schneidarno

## 🧹 Housekeeping

## :notebook: Agenda
### Cluster Stacks Rancher Plugin (@schmidax)
- Repo needed (AI: @schmidax)
- Demo :star2:

### Release current set of cluster stacks (@garloff)
- CAPI-1.14 expected in August, won't support v1beta1
- v1.35: Convert to v1beta2 first before release
- v1.34: Already released with v1beta1, stick to it
- Conversion of clusters should work, but clusterclasses are probably not converted
- Check if CSO will work with v1beta2 ClusterClasss (AI: @jschoone)
- AI @garloff: Create 1.33 and 1.34 test releases and test, @jschoone to review (and approve if good)

### Deal with breaking changes (@jschoone)
- [PR #255](https://github.com/SovereignCloudStack/cluster-stacks/pull/255), [PR #259](https://github.com/SovereignCloudStack/cluster-stacks/pull/259)
- ClusterStack name change prevented by CSO (Webhook)
- Migration from scs->scs2 not smooth and scs3 won't be either (discussion in #255)
- Move everything back to scs? (#259)
- Create version subdirectories and deal with changes there
- Variables ...
- overrides specifically for control plane or workers (machineDeployments)
- has the advantage to avoid divergence between scs and hcp/kosmotron/kamaji
- need to ensure we have SSD flavors by default for control plane in scs cluster classes, so they work

### Using registry.scs.community for node images? (@garloff)
- Uhurutec offered to build and push images
- registry (harbor) runs on gxscs2 (Plusserver test env)
- Need to decide -> 2026-05-xx

39 changes: 39 additions & 0 deletions container/20260417.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
# 2026-04-17

## :family: Participants (GitHub handles preferred)
Participants (GitHub handles preferred)
- @jschoone
- @garloff
- @schneidarno

## 🧹 Housekeeping

## :notebook: Agenda
### Cluster Stacks Rancher Plugin (@schmidax)
- Repo needed (AI: @schmidax)
- Demo :star2:

### Release current set of cluster stacks (@garloff)
- CAPI-1.14 expected in August, won't support v1beta1
- v1.35: Convert to v1beta2 first before release
- v1.34: Already released with v1beta1, stick to it
- Conversion of clusters should work, but clusterclasses are probably not converted
- Check if CSO will work with v1beta2 ClusterClasss (AI: @jschoone)
- AI @garloff: Create 1.33 and 1.34 test releases and test, @jschoone to review (and approve if good)

### Deal with breaking changes (@jschoone)
- [PR #255](https://github.com/SovereignCloudStack/cluster-stacks/pull/255), [PR #259](https://github.com/SovereignCloudStack/cluster-stacks/pull/259)
- ClusterStack name change prevented by CSO (Webhook)
- Migration from scs->scs2 not smooth and scs3 won't be either (discussion in #255)
- Move everything back to scs? (#259)
- Create version subdirectories and deal with changes there
- Variables ...
- overrides specifically for control plane or workers (machineDeployments)
- has the advantage to avoid divergence between scs and hcp/kosmotron/kamaji
- need to ensure we have SSD flavors by default for control plane in scs cluster classes, so they work

### Using registry.scs.community for node images? (@garloff)
- Uhurutec offered to build and push images
- registry (harbor) runs on gxscs2 (Plusserver test env)
- Need to decide -> 2026-05-xx

44 changes: 44 additions & 0 deletions container/20260529.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
# 2026-05-15/2026-05-29
## :family: Participants
- @garloff
- @jschoone

## 🧹 Housekeeping
- So lonely :notes:

## :notebook: Agenda backlog
### Putting capi images into SCS registry

### csctl/cso: Accept dashes
* [PR #221](https://github.com/SovereignCloudStack/csctl/pull/221)
### csctl: Hash cluster class
* [PR #223](https://github.com/SovereignCloudStack/csctl/pull/223)
### csctl: Modernize build infra versions
* [PR #225] (https://github.com/SovereignCloudStack/csctl/pull/225)
### Support multi-AZ volume structure
* Some clouds have 1:1 volume and compute AZs, and cross-attaching is impossible or slow
* Pin volumes to VMs option `diskPinAZ`,https://docs.scs.community/standards/scs-0210-v2-k8s-version-policy [PR#263](https://github.com/SovereignCloudStack/cluster-stacks/pull/263)
### K8s-updates
* Process description: https://input.scs.community/sP49NVfCShScjmC1iceSqA
* How to structure parallel maintenance of several k8s trees
* Discussion: [PR#266](https://github.com/SovereignCloudStack/cluster-stacks/pull/266): Multiple branches? Multiple directories? csctl postprocessing?
* 1.34.8: [PR#268](https://github.com/SovereignCloudStack/cluster-stacks/pull/268)
* 1.33.11 [PR#267](https://github.com/SovereignCloudStack/cluster-stacks/pull/267)/ 1.33.12 [issue#326](https://github.com/osism/k8s-capi-images/issues/326)
* 1.35.5 [PR#269](https://github.com/SovereignCloudStack/cluster-stacks/pull/269)
### Need for scs3 Cluster-Stacks?
* hosted control plane
* kube-proxy-less & Gateway API through cilium?
### Cilium Gateway API and kube-proxy-less mode
* Can this be done non-disruptive? Opt-in? Opt-out?
### How to replace node image (without k8s patch)
* Motivation: [LPE issues](https://docs.scs.community/blog/kernel_local_root_exploits)
### Standardization SCS-0210 (k8s version policy)
* We don't explicitly require that kernels and other security-relevant pieces are up-to-date, this was just best practice (and a collateral benefit of how the capi images are built in our ref.impl.)
* Should we make this an explicit requirement?
### OVN loadbalancer and security-groups
* octavia-ovn allows for direct TCP/UDP connections from real user to backend (node-port)
* Great for efficiency
* Great to see real IP address in backend (for applying policies ....)
* But: Security Groups need manual tweaking to allow traffic from 0.0.0.0/0 to the exposed node ports.
* Is this a known issue? Shouldn't Google's "Direct Return" have the same advantages and disadvantages? Best practice for handling the security groups automatically?

86 changes: 86 additions & 0 deletions container/20260612.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,86 @@
# 2026-06-12
## :family:
- @garloff
- @jschoone
- @arnoschmid

## 🧹 Housekeeping

## :notebook: Agenda
### Putting capi images into SCS registry
- Uhurutec offered to volunteer with capi image building
- Currently, OSISM builds them
- Hosting location at registry.scs.community
- Also looks more official
- Double check with PS about SCS2 future (@garloff)
- Double check viability and mechanics of hosting *VM* images in harbor (@garloff)
- Otherwise fall back to swift/s3 on SCS2

### scs-0210: k8s version recency
- We require latest k8s patches mainly to ensure customers can secure their clusters
- We don't explicitly require kernels (and other distro packages) to be up2date
- Shouldn't we?
- Yes, we should
- Better: New standard
- Testing is hard (some work has been done with trivy)
- @garloff
- Question on -0210: Does support also require creating new clusters with up to 12 month old k8s versions?
- Important for provider switching
- Allows users to stay on a k8s minor version for up to one year, seems relevant for larger or less agile organisations (@mbuechse)

### csctl work
* csctl & cso: Accept dashes: [PR #221](https://github.com/SovereignCloudStack/csctl/pull/221) - not needed, let's rule out names with dashes, rather have a a patch the prevents "-" in names
* Hash cluster class: [PR #223](https://github.com/SovereignCloudStack/csctl/pull/223)
* Modernize csctl build infra versions: [PR #225] (https://github.com/SovereignCloudStack/csctl/pull/225)
* Do we need csctl?
* Lots of complexity for relatively easy task
* Just do it with oras (and some scripts)
* Will cause work due to CI changes in k8s world ...
* Dependency hell
* @jschoone working on alternative

### scs2 cluster classes: Multi-AZ volume support
Support multi-AZ volume structure
* Some clouds have 1:1 volume and compute AZs, and cross-attaching is impossible or slow
* Pin volumes to VMs option `diskPinAZ`,https://docs.scs.community/standards/scs-0210-v2-k8s-version-policy [PR#263](https://github.com/SovereignCloudStack/cluster-stacks/pull/263)
* Testing: nSC requires this
* (@garloff to ensure access to nSC for @jschoone)

### scs2: Push out updates
* How to structure parallel maintenance of several k8s trees
* Discussion: [PR#266](https://github.com/SovereignCloudStack/cluster-stacks/pull/266): Multiple branches? Multiple directories? csctl postprocessing?
* Subdirectories was how we did it originally
* Branch maintenance in other k8s projects use a bot for cherry-picking cross-branch changes
* Use new minor versions for breaking changes
* @jschoone to extract new subdirectory structure from existing WIP branch
* Expect 1.35.6, 1.34.9 and 1.33.13 in the next days
* @garloff
* Automation via github job/zuul of e2e testing would be great

### Cilium Gateway & Kube-proxy-less
* Is the latter a requirement for proper Gateway support?
* No: Node-port should work as well (needs enablement)
* New cluster-class variable as opt-in? (@garloff)
* Gateway CRDs still need to be downloaded ... (huge)
* kube-proxy-replacement is better (performance, features)
* Need to switch off kube-proxy (kubeadm skip ...)
* How?

### scs3: Roadmap?
* Hosted control plane support (@cwrau ...)
* Most capi-native solution
* Kube-Proxy-less & Gateway?

---

### OVN loadbalancer & security groups
* octavia-ovn allows for direct TCP/UDP connections from real user to backend (node-port)
* Great for efficiency
* Great to see real IP address in backend (for applying policies ....)
* But: Security Groups need manual tweaking to allow traffic from 0.0.0.0/0 to the LB-exposed node ports.
* Is this a known issue? Shouldn't Google's "Direct Return" have the same advantages and disadvantages? Best practice for handling the security groups automatically?

### CSO issues
* Crash [Issue #337](https://github.com/SovereignCloudStack/cluster-stack-operator/issues/337)
* CI [issues #340](https://github.com/SovereignCloudStack/cluster-stack-operator/issues/340) need work

28 changes: 28 additions & 0 deletions container/20260710.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
# 2027-07-10
## :family:
- @garloff
- @schmidax
- @jklare

## 🧹 Housekeeping
* @garloff to propose recency standard beyond k8s version: open
* @mbuechse to clarify that clusters of all supported versions must be creatable: ?
## :notebook: Agenda

### Reorganizing Cluster Stack release tooling
* Branches vs. directories [#270](https://github.com/SovereignCloudStack/cluster-stacks/pull/270)
* Enhancing or sunsetting csctl?
* @schmidax does it with [ansible playbooks](https://github.com/SovereignCloudStack/cluster-stacks-rancher) -> @jschoone
* Merging/Aligning classical capi with hcp?
* Do this for 1.36?

### Publish latest k8s patches
* Blocked by lack of images (OSISM [#326](https://github.com/osism/k8s-capi-images/issues/326) [#328](https://github.com/osism/k8s-capi-images/issues/328) [#332](https://github.com/osism/k8s-capi-images/issues/332))
* Images currently under test
* Images should be ready but need validation, see matrix chat
* UhuruTec/OSISM are both providing help here
* More important would be automated testing
* Build new patch release automatically and do CI tests (using @schmidax's playbooks?)
* Look at registry.scs/community to publish (@garloff)
* Use restructure work [#270](https://github.com/SovereignCloudStack/cluster-stacks/pull/270)