@greg and I have finished the migration of our PostgreSQL cluster from the host systems of Andromeda, Centaurus, and Draco, to new VMs on Centaurus and Draco.
Instead of the previous encfs setup, we now use standard full-disk encryption (LUKS/LVM) for the VMs to encrypt data at rest. So the decryption password has to be provided during the booting of the VM now.
The new setup also uses our Zerotier private network exclusively for database connections, and thus no public connections or TLS anymore.
In order to make both this migration seamless, as well make the switching of the primary node to a hot standby node much easier and quicker in the future as well, we have changed all database clients to use a custom /etc/hosts entry (pg.kosmos.local).
Then we created a script to run all the steps for switching the primary at once, both triggering the promotion of a standby node to primary, as well as stopping the old one, and updating the hosts entries without full Chef runs. This way, all clients will immediately re-connect to the new primary after promotion, which worked exactly as intended when we did it for this migration.
It was created with an encfs volume inside a VM, we want full disk encryption
instead. I have deleted the VM from centaurus as well as its disk and
have also unauthorized its id from the zerotier controller