Turns out everything already works perfectly. A simple ark backup create [name] will create a complete backup of all our resources and upload the files to the storage bucket. It will also create volume snapshots, which are a native feature of GCP, and which you can find here.
I started setting up Ark, as it looks like the best open-source solution to me. Went through https://heptio.github.io/ark/v0.10.0/gcp-config and have an ark deployment running on the sidamo cluster now.
What happened to removing the obsolete env vars? I just found them in master.
Update: as discussed on IRC, just changing the secrets isn't actually enough to re-deploy and apply them. Need to change the deployment config for that.
Very cool! I used the scripts to get the configs, edited them a bit (disabled OpenID and creating orgs for example), then pushed them back and applied the server. All worked flawlessly.
What initial setup would that be? Wouldn't the ini be there from the very start now?