How to route app alerts to a team

Routing app alerts via annotations in Chart.yaml or app CRs.

App CR Annotation

  • If an App CR is annotated with this annotation takes precedence.
kubectl -n giantswarm annotate app app-exporter-unique

# port forwarding to the app-exporter service.
# kubectl -n giantswarm port-forward svc/app-exporter-unique 8000:8000
# Forwarding from -> 8000
# Forwarding from [::1]:8000 -> 8000
curl -s http://localhost:8000/metrics  | grep app-exporter
app_operator_app_info{app="app-exporter",catalog="control-plane-catalog",name="app-exporter-unique",namespace="giantswarm",status="deployed",team="halo",version="0.4.0"} 1
  • Otherwise we check for team or owner annotations in the Helm Chart.yaml.
  • If the annotation is missing for existing releases the team can be mapped via the app-exporter configmap.

Team Annotation

  • When a component is owned by a single team you just need to set the annotation in Chart.yaml.
# Chart.yaml
apiVersion: "v1"
name: "app-exporter"
annotations: "batman"
  • The annotation is added to the AppCatalogEntry CRs by app-operator-unique.
kubectl get appcatalogentry -A -l,latest=true -o yaml | yq  '.items[].metadata.annotations'
"": "batman"
  • It is used by [app-exporter] to set the team label in app info metrics.
curl -s http://localhost:8000/metrics  | grep app-exporter
app_operator_app_info{app="app-exporter",catalog="control-plane-catalog",name="app-exporter-unique",namespace="giantswarm",status="deployed",team="batman",version="0.4.0"} 1

Team label in resources

Team chart annotation should be propagated to a label for generated resources, because some alerts rely on the specific resource’s labels.

The recommended solution is to define a list of common labels in a _helpers template, containing this: {{ index .Chart.Annotations "" | default "batman" | quote }}

See grafana app for reference.

Owners Annotation

  • When a component is owned by multiple teams you can set the annotation in Chart.yaml.
  • Ownership can be assigned by catalog e.g. control-plane-catalog and / or provider e.g. aws.
apiVersion: v1
appVersion: v1.20.0
description: A Helm chart for the cluster autoscaler.
name: cluster-autoscaler-app
version: [[ .Version ]]
annotations: |
    - catalog: control-plane-catalog
      team: biscuit
    - catalog: control-plane-test-catalog
      provider: aws
      team: biscuit
    - catalog: default
      provider: aws
      team: firecracker
    - catalog: default
      provider: azure
      team: celestial