Re-using release structures for gitops cluster management

THIS IS WIP AND MAKES A LOT ASSUMPTIONS CURRENTLY

Problem Statement

We want to …

enable customers to have more agency in upgrading individual components.
give a lower barrier of entry for customers to deeply interact with our product.
allow customers to use known gitops tooling for cluster management.

Going gitops all the way

A next step in gitops adoption would be the addition of having the cluster custom resources in gitops as well. So the shared git repository would contain not only release information but also the clusters themselves.

gauss: # installation
  peu01: #clusterID
    base: # managed fully by GS
      cluster.yaml
      ...
      cert-exporter.yaml # default app
    overlay: # overwrites by customer
      cluster.yaml
      ...
      kong.yaml # non default app
aws: # releases as mentioned before
  v21.0.0:
  ...

Interaction with this repository could then be improved by adding functionality to our CLI kubectl-gs.

Cluster creation would become a pull request which commits code generated by kubectl-gs.

Upgrading a cluster is a bit more complex: Upgrades become a overwrite of configuration from the release to the base folder. We can show this with a MachinePool example:

# gauss/peu01/machinepool.yaml
apiVersion: cluster.x-k8s.io/v1alpha3
kind: MachinePool
metadata:
  name: peu01-mp-0
  namespace: my-org
spec:
  clusterName: peu01
  replicas: 2
  template:
    spec:
      bootstrap:
        configRef:
          apiVersion: bootstrap.cluster.x-k8s.io/v1alpha3
          kind: KubeadmConfig
          name: peu01-mp-0
      clusterName: peu01
      infrastructureRef:
        apiVersion: infrastructure.cluster.x-k8s.io/v1alpha3
        kind: AWSMachinePool
        name: peu01-mp-0
      version: v1.16.8

The upgrade to version v22.0.0 would now be done by overwriting from aws/v21.0.0/base/machinepool.yaml + aws/v21.0.0/overlay/machinepool.yaml. For this example we assume the overlay is empty and the base is as follows:

apiVersion: v1
kind: ConfigMap
metadata:
  name: machinepool-21-0-0
  namespace: giantswarm
data:
  spec:
    replicas: 3
    template:
      spec:
        version: v1.21.4

Then the resulting pull request for gauss/peu01/machinepool.yaml would change:

# gauss/peu01/machinepool.yaml
apiVersion: cluster.x-k8s.io/v1alpha3
kind: MachinePool
metadata:
  name: peu01-mp-0
  namespace: my-org
spec:
  clusterName: peu01
  replicas: 3
  template:
    spec:
      bootstrap:
        configRef:
          apiVersion: bootstrap.cluster.x-k8s.io/v1alpha3
          kind: KubeadmConfig
          name: peu01-mp-0
      clusterName: peu01
      infrastructureRef:
        apiVersion: infrastructure.cluster.x-k8s.io/v1alpha3
        kind: AWSMachinePool
        name: peu01-mp-0
      version: v1.21.4

Last modified November 21, 2023: Update rendered RFCs (#176) (7f6b6e4)