The printable version is no longer supported and may have rendering errors. Please update your browser bookmarks and please use the default browser print function instead.

Toolforge Kubernetes 1.25 upgrade

Parent task: phab:T316107

prep

[x] run prepare upgrade cookbook
[x] downtime project via https://prometheus-alerts.wikimedia.org
[x] update topic on -cloud

control nodes

tools-k8s-control-7

[x] run upgrade node cookbook
[x] check that services start healthy

tools-k8s-control-8

[x] run upgrade node cookbook
[x] check that services start healthy

tools-k8s-control-9

[x] run upgrade node cookbook
[x] check that services start healthy

worker nodes

run upgrade node cookbook for each. it's ok to do a couple in parallel

[x] tools-k8s-worker-nfs-1
[x] tools-k8s-worker-nfs-2
[x] tools-k8s-worker-nfs-3

[x] tools-k8s-worker-nfs-4
[x] tools-k8s-worker-nfs-5
[x] tools-k8s-worker-nfs-6
[x] tools-k8s-worker-nfs-7
[x] tools-k8s-worker-nfs-8
[x] tools-k8s-worker-nfs-9
[x] tools-k8s-worker-nfs-10
[x] tools-k8s-worker-nfs-11
[x] tools-k8s-worker-nfs-12
[x] tools-k8s-worker-nfs-13
[x] tools-k8s-worker-nfs-14
[x] tools-k8s-worker-nfs-15
[x] tools-k8s-worker-nfs-16
[x] tools-k8s-worker-nfs-17
[x] tools-k8s-worker-nfs-18
[x] tools-k8s-worker-nfs-19
[x] tools-k8s-worker-nfs-20
[x] tools-k8s-worker-nfs-21
[x] tools-k8s-worker-nfs-22
[x] tools-k8s-worker-nfs-23
[x] tools-k8s-worker-nfs-24
[x] tools-k8s-worker-nfs-25
[x] tools-k8s-worker-nfs-26
[x] tools-k8s-worker-nfs-27
[x] tools-k8s-worker-nfs-28
[x] tools-k8s-worker-nfs-29
[x] tools-k8s-worker-nfs-30
[x] tools-k8s-worker-nfs-31
[x] tools-k8s-worker-nfs-32
[x] tools-k8s-worker-nfs-33
[x] tools-k8s-worker-nfs-34
[x] tools-k8s-worker-nfs-35
[x] tools-k8s-worker-nfs-36
[x] tools-k8s-worker-nfs-37
[x] tools-k8s-worker-nfs-38
[x] tools-k8s-worker-nfs-39
[x] tools-k8s-worker-nfs-40
[x] tools-k8s-worker-nfs-41
[x] tools-k8s-worker-nfs-42
[x] tools-k8s-worker-nfs-43
[x] tools-k8s-worker-nfs-44
[x] tools-k8s-worker-nfs-45
[x] tools-k8s-worker-nfs-46
[x] tools-k8s-worker-nfs-47
[x] tools-k8s-worker-nfs-48
[x] tools-k8s-worker-nfs-49
[x] tools-k8s-worker-nfs-50
[x] tools-k8s-worker-nfs-51
[x] tools-k8s-worker-nfs-52
[x] tools-k8s-worker-nfs-53
[x] tools-k8s-worker-nfs-54
[x] tools-k8s-worker-nfs-55
[x] tools-k8s-worker-nfs-56

[x] tools-k8s-worker-102
[x] tools-k8s-worker-103
[x] tools-k8s-worker-104
[x] tools-k8s-worker-105
[x] tools-k8s-worker-106
[x] tools-k8s-worker-107

ingress nodes

[x] kubectl -n ingress-nginx-gen2 scale deployment ingress-nginx-gen2-controller --replicas=2
run upgrade node cookbook for each:

[x] tools-k8s-ingress-7
[x] tools-k8s-ingress-8
[x] tools-k8s-ingress-9

[x] revert afterwards: kubectl -n ingress-nginx-gen2 scale deployment ingress-nginx-gen2-controller --replicas=3

cleanup

[x] remove downtime
[x] revert topic change