WIP: StatefulSet config for gitea-server

A deployment cannot actually attach the same persistent storage across
nodes. We have to switch to a StatefulSet for any and all programs that
need shared persistent storage.

I tried applying this config, but it didn't attach the same volume/disk
as before, so I had to revert to the Deployment for now.

.gitignore

@@ -1 +1 @@
-/kubernetes/custom/config/
+/kubernetes/config/

.gitmodules

@@ -1,3 +0,0 @@
-[submodule "vendor/ark"]
-	path = vendor/ark
-	url = git@github.com:heptio/ark.git

README.md

@@ -1,9 +1,9 @@
 # gitea.kosmos.org
 
 This repository contains configuration files and other assets, that are used to
-deploy and operate this Gitea instance.
+deploy and operate this Gitea instance. Feel free to [open
+issues](https://gitea.kosmos.org/kosmos/gitea.kosmos.org/issues) for questions,
+suggestions, bugs, to-do items, and whatever else you want to discuss or
+resolve.
 
-Feel free to [open issues] for questions, suggestions, bugs, to-do items, and
-whatever else you want to discuss or resolve.
-
-[open issues]: https://gitea.kosmos.org/kosmos/gitea.kosmos.org/issues
+See `doc/` folder for some technical info.

custom/options/label/Kosmos

@@ -1,4 +1,5 @@
 #db231d bug ; Something is not working
+#ead746 docs ; Documentation
 #76db1d enhancement ; Improving existing functionality
 #1d76db feature ; New functionality
 #db1d76 idea ; Something to consider

doc/backup-and-restore.md

@@ -1,36 +1,28 @@
 # Backups
 
-We're using [Ark][1] for backing up Kubernetes config and GKE resources. It is
-available as a Git submodule in the `vendor/` folder (incl. the `ark`
-executable).
+We're using [Velero][1] (formerly Ark) for backing up Kubernetes config and GKE
+resources. It is available as a compiled binary for your platform [on GitHub][2]
 
-In order to initialize and update submodules in your local repo, run once:
-
-    git submodule update --init
-
-Then, to fetch/update the modules, run:
-
-    git submodule update
-
-The Ark service is running on the Sidamo cluster and was set up using the
-[official docs' GCP instructions and config files][4]. There's a daily backup
+The Velero service is running on the Sidamo cluster and was set up using the
+[official docs' GCP instructions][3]. There's a daily backup
 schedule in effect for Gitea (using the label `app=gitea`).
 
-Please refer to Ark's [ Getting Started ][5] doc for all backup and restore
+Please refer to Velero's [ Getting Started ][4] doc for all backup and restore
 commands.
 
 ## Backup location
 
 Cluster configuration (including all live resources) is backed up to [a Google
-Cloud Storage container][3].
+Cloud Storage container][5].
 
 ## Persistent volumes
 
-Persistent volumes are just GCE disks. Thus, with the current config, Ark
-creates volume snapshots as native [GCE disk snapshots][2].
+Persistent volumes are just GCE disks. Thus, with the current config, Velero
+creates volume snapshots as native [GCE disk snapshots][6].
 
-[1]: https://heptio.github.io/ark/v0.10.0
-[2]: https://console.cloud.google.com/compute/snapshots?organizationId=772167872692&project=fluted-magpie-218106&tab=snapshots&snapshotssize=50
-[3]: https://console.cloud.google.com/storage/browser/sidamo-backups?project=fluted-magpie-218106&organizationId=772167872692
-[4]: https://heptio.github.io/ark/v0.10.0/gcp-config
-[5]: https://heptio.github.io/ark/v0.10.0/get-started
+[1]: https://velero.io/docs/v1.0.0
+[2]: https://github.com/heptio/velero/releases/tag/v1.0.0
+[3]: https://velero.io/docs/v1.0.0/gcp-config/
+[4]: https://velero.io/docs/v1.0.0/about/
+[5]: https://console.cloud.google.com/storage/browser/sidamo-backups-new?project=fluted-magpie-218106&organizationId=772167872692
+[6]: https://console.cloud.google.com/compute/snapshots?organizationId=772167872692&project=fluted-magpie-218106&tab=snapshots&snapshotssize=50

doc/customizations-image.md

@@ -0,0 +1,20 @@
+## Customizations image
+
+### Build
+
+To create a new Docker image containing our Gitea customizations (label sets,
+styles, page content, etc.):
+
+Edit `packer/custom.json` to increment the tag, then run this script (needs
+[Packer](https://www.packer.io/) in your path)
+
+    ./script/build_customizations_image
+
+### Deploy
+
+Edit `kubernetes/gitea-server.yaml` to use the new tag
+(`image: eu.gcr.io/fluted-magpie-218106/gitea_custom:$VERSION`) and apply the
+change:
+
+    cd kubernetes
+    kubectl apply -f gitea-server.yaml

kubernetes/gitea-db.yaml

@@ -32,13 +32,19 @@ spec:
           value: gitea
         image: mariadb:10.3.10
         name: gitea-db
-        resources: {}
         ports:
           - containerPort: 3306
             name: mysql
         volumeMounts:
         - mountPath: /var/lib/mysql
           name: gitea-db-data
+        resources:
+          requests:
+            cpu: 250m
+            memory: 150Mi
+          limits:
+            cpu: 500m
+            memory: 300Mi
       restartPolicy: Always
       volumes:
       - name: gitea-db-data

kubernetes/gitea-server-statefulset.yaml

@@ -0,0 +1,103 @@
+apiVersion: apps/v1
+kind: StatefulSet
+metadata:
+  name: gitea-server
+  labels:
+    app: gitea
+spec:
+  serviceName: gitea-server
+  updateStrategy:
+    type: RollingUpdate
+  replicas: 1
+  selector:
+    matchLabels:
+      app: gitea
+  template:
+    metadata:
+      labels:
+        name: gitea-server
+        app: gitea
+    spec:
+      initContainers:
+        - name: init-config
+          # This is a busybox image with our gitea customizations saved to
+          # /custom, built using ./script/build_customizations_image from the
+          # root of the repo
+          image: eu.gcr.io/fluted-magpie-218106/gitea_custom:0.1.2
+          command: [
+            'sh', '-c',
+            'mkdir -p /data/gitea/conf && mkdir -p /data/gitea/https && cp /root/conf/app.ini /data/gitea/conf/app.ini && chown 1000:1000 /data/gitea/conf/app.ini && chmod 660 /data/gitea/conf/app.ini && cp /root/conf/*.pem /data/gitea/https && chmod 600 /data/gitea/https/*.pem && cp -R /custom/* /data/gitea && chown -R 1000:1000 /data/gitea'
+          ]
+          volumeMounts:
+          - mountPath: /data
+            name: gitea-server-data
+          - mountPath: /root/conf
+            name: config
+      containers:
+      - name: gitea-server
+        image: gitea/gitea:1.10.1
+        ports:
+        - containerPort: 3000
+        - containerPort: 3001
+        - containerPort: 22
+        volumeMounts:
+        - mountPath: /data
+          name: gitea-server-data
+        resources:
+          requests:
+            cpu: 150m
+            memory: 256Mi
+          limits:
+            cpu: 250m
+            memory: 512Mi
+      restartPolicy: Always
+      volumes:
+      - name: config
+        secret:
+          secretName: gitea-config
+          items:
+            - key: app.ini
+              path: app.ini
+              mode: 256
+            - key: cert.pem
+              path: cert.pem
+              mode: 256
+            - key: key.pem
+              path: key.pem
+              mode: 256
+  volumeClaimTemplates:
+    - metadata:
+        name: gitea-server-data
+        labels:
+          app: gitea
+      spec:
+        accessModes:
+        - ReadWriteOnce
+        resources:
+          requests:
+            storage: 20Gi
+        storageClassName: gcePersistentDisk
+---
+apiVersion: v1
+kind: Service
+metadata:
+  name: gitea-server
+  labels:
+    name: gitea-server
+    app: gitea
+spec:
+  type: LoadBalancer
+  # preserves the client source IP
+  externalTrafficPolicy: Local
+  ports:
+    - name: "ssh"
+      port: 22
+      targetPort: 22
+    - name: "http"
+      port: 80
+      targetPort: 3001
+    - name: "https"
+      port: 443
+      targetPort: 3000
+  selector:
+    name: gitea-server

kubernetes/gitea-server.yaml

@@ -1,4 +1,4 @@
-apiVersion: extensions/v1beta1
+apiVersion: apps/v1
 kind: Deployment
 metadata:
   name: gitea-server
@@ -6,6 +6,9 @@ metadata:
     app: gitea
 spec:
   replicas: 1
+  selector:
+    matchLabels:
+      app: gitea
   template:
     metadata:
       labels:
@@ -14,24 +17,22 @@ spec:
     spec:
       initContainers:
         - name: init-config
-          image: busybox
+          # This is a busybox image with our gitea customizations saved to
+          # /custom, built using ./script/build_customizations_image from the
+          # root of the repo
+          image: eu.gcr.io/fluted-magpie-218106/gitea_custom:0.1.2
           command: [
             'sh', '-c',
-            'mkdir -p /data/gitea/conf && mkdir -p /data/gitea/https && mkdir -p /data/gitea/options/label && cp /root/conf/app.ini /data/gitea/conf/app.ini && chown 1000:1000 /data/gitea/conf/app.ini && chmod 660 /data/gitea/conf/app.ini && cp /root/conf/*.pem /data/gitea/https && chmod 600 /data/gitea/https/*.pem && cp /root/options/label/* /data/gitea/options/label/ && chown -R 1000:1000 /data/gitea'
+            'mkdir -p /data/gitea/conf && mkdir -p /data/gitea/https && cp /root/conf/app.ini /data/gitea/conf/app.ini && chown 1000:1000 /data/gitea/conf/app.ini && chmod 660 /data/gitea/conf/app.ini && cp /root/conf/*.pem /data/gitea/https && chmod 600 /data/gitea/https/*.pem && cp -R /custom/* /data/gitea && chown -R 1000:1000 /data/gitea'
           ]
           volumeMounts:
           - mountPath: /data
             name: gitea-server-data
           - mountPath: /root/conf
             name: config
-          # The labels have been created as a ConfigMap from local files using this command:
-          #
-          # kubectl create configmap gitea-options-label --from-file=custom/options/label/
-          - mountPath: /root/options/label
-            name: label
       containers:
       - name: gitea-server
-        image: gitea/gitea:1.7.1
+        image: gitea/gitea:1.10.1
         ports:
         - containerPort: 3000
         - containerPort: 3001
@@ -39,6 +40,13 @@ spec:
         volumeMounts:
         - mountPath: /data
           name: gitea-server-data
+        resources:
+          requests:
+            cpu: 150m
+            memory: 256Mi
+          limits:
+            cpu: 250m
+            memory: 512Mi
       restartPolicy: Always
       volumes:
       - name: gitea-server-data
@@ -57,9 +65,6 @@ spec:
             - key: key.pem
               path: key.pem
               mode: 256
-      - name: label
-        configMap:
-          name: gitea-options-label
 ---
 apiVersion: v1
 kind: PersistentVolumeClaim
@@ -72,7 +77,7 @@ spec:
   - ReadWriteOnce
   resources:
     requests:
-      storage: 1Gi
+      storage: 20Gi
 ---
 apiVersion: v1
 kind: Service

packer/custom.json

@@ -0,0 +1,29 @@
+{
+  "builders": [{
+    "type": "docker",
+    "image": "busybox",
+    "run_command": ["-d", "-i", "-t", "{{.Image}}", "/bin/sh"],
+    "commit": true
+  }],
+  "provisioners": [
+    {
+      "inline": ["mkdir /custom"],
+      "type": "shell"
+    },
+    {
+      "type": "file",
+      "source": "../custom/",
+      "destination": "/custom"
+    }
+  ],
+  "post-processors": [
+    [
+      {
+        "type": "docker-tag",
+        "repository": "eu.gcr.io/fluted-magpie-218106/gitea_custom",
+        "tag": "0.1.2"
+      },
+      "docker-push"
+    ]
+  ]
+}

script/build_customizations_image

@@ -0,0 +1,7 @@
+#!/usr/bin/env bash
+# fail fast
+set -e
+
+cd packer/
+packer build custom.json
+cd -

script/get_secrets

@@ -7,7 +7,7 @@ secret = `kubectl get secret gitea-config -o yaml`
 yaml = YAML.load(secret)
 
 yaml['data'].each do |key, data|
-  filename = File.join('kubernetes', 'custom', 'config', key)
+  filename = File.join('kubernetes', 'config', key)
   File.open(filename, "w+") do |f|
     puts "Writing #{filename}"
     f.write Base64.decode64(data)

script/replace_secrets

@@ -2,8 +2,8 @@
 
 # Delete the gitea-config secrets
 kubectl delete secret gitea-config
-# Replace it from the local files in kubernetes/custom/config/* (acquired by running
+# Replace it from the local files in kubernetes/config/* (acquired by running
 # ./script/get_secrets)
-kubectl create secret generic gitea-config --from-file=cert.pem=kubernetes/custom/config/cert.pem --from-file=key.pem=kubernetes/custom/config/key.pem --from-file=app.ini=kubernetes/custom/config/app.ini
+kubectl create secret generic gitea-config --from-file=cert.pem=kubernetes/config/cert.pem --from-file=key.pem=kubernetes/config/key.pem --from-file=app.ini=kubernetes/config/app.ini
 # Force the pod to restart by patching the deployment resource
 kubectl patch deployment gitea-server -p "{\"spec\":{\"template\":{\"metadata\":{\"annotations\":{\"date\":\"`date +'%s'`\"}}}}}"