Add garage S3 config for Mastodon

Add nginx proxy hosts for Garage Web access
The respective bucket needs to be configured with a domain alias. When a new alias is added to the `s3_web_domains` config, a new nginx site can then be deployed to the `nginx_proxy` hosts.
2022-11-30 12:15:27 +01:00 · 2022-11-30 12:13:39 +01:00
46 changed files with 111 additions and 510 deletions
--- a/README.md
+++ b/README.md
@@ -1,16 +1,3 @@
-This repository contains all infrastructure automation code that we use to set
-up and configure servers, virtual machines, and applications for Kosmos hosted
-services.
-
-Chef cookbooks are written in Ruby, and based on [Chef Infra
-resources](https://docs.chef.io/resources/). Some cookbooks contain integration
-test suites based on [Test Kitchen](https://docs.chef.io/workstation/kitchen/).
-
-Note: Manual configuration of servers and applications is highly discouraged,
-and can be overwritten or lost without notice!
-
-## Setup
-
 ### Install Chef Workstation

 * macOS, Windows, RHEL, Ubuntu: https://docs.chef.io/workstation/install_workstation/
@@ -19,28 +6,24 @@ and can be overwritten or lost without notice!
 #### rbenv

 If you use rbenv to manage Ruby versions on your system, install the
-[rbenv-chef-workstation](https://github.com/docwhat/rbenv-chef-workstation)
+(rbenv-chef-workstation)[https://github.com/docwhat/rbenv-chef-workstation]
 plugin.

 ### Install gem dependencies

-Clone this repository, `cd` into it, and run:
-
    bundle install

-## Common tasks
+### Bootstrap a new server

-### Bootstrap a new host server
-
-    knife zero bootstrap root@server-name.kosmos.org --run-list "role[base],role[kvm_host]" --secret-file .chef/encrypted_data_bag_secret
+    knife zero bootstrap root@dev.kosmos.org --run-list "recipe[kosmos-base],..." -j '{"example_cookbook":{"memory_max":"256M"}}' --secret-file .chef/encrypted_data_bag_secret

 ### Bootstrap a new VM

-    knife zero bootstrap ubuntu@zerotier-ip-address -x ubuntu --sudo --run-list "role[base],role[kvm_guest]" --secret-file .chef/encrypted_data_bag_secret
+    knife zero bootstrap ubuntu@zerotier-ip-address -x ubuntu --sudo --run-list "recipe[kosmos-base]" --secret-file .chef/encrypted_data_bag_secret

-### Run Chef Zero on a host server
+### Run Chef Zero

-    knife zero converge -p2222 name:server-name.kosmos.org
+    knife zero converge name:dev.kosmos.org

 ### Run Chef Zero on a VM

@@ -50,7 +33,7 @@ Clone this repository, `cd` into it, and run:

    knife zero converge name:dev.kosmos.org --client-version 15.3.14

-## Managing cookbooks
+### Managing cookbooks

 Cookbooks are managed via Berkshelf. Run `berks --help` for command help.

@@ -62,7 +45,7 @@ Vendor installed cookbooks to the `cookbooks/` dir:

    berks vendor cookbooks/ --delete

-## "Expired" TLS certificates
+### "Expired" TLS certificates

 If you encounter expired TLS certificates during a Chef run (e.g. for remote
 files), the issue is likely that the certificate has been issued by Let's
--- a/clients/ldap-4.json
+++ b/clients/ldap-4.json
@@ -1,4 +0,0 @@
-{
-  "name": "ldap-4",
-  "public_key": "-----BEGIN PUBLIC KEY-----\nMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAmzFyZh5/J2BsKRunghis\nwUGbv4j/ynAF7QY+CYoOwDBcbLHk6odn1JyUqCgfhCIX0mh8F/fDKyU9Aw6+HHZ/\nX0DTt/enLTaWc2vxRfyJLRXP7/ymHOr4u6HYEINMdVJp4yQ9XLcWpuRHfA+fHrZ7\n9fI8sCMSEawvVpEKytYdVnm3VCjfIVrfCAkY0lP0mNG908edX2ZuJ4GS1UwADUZX\nLZuMhbGX9JqIQYWCyiMDakD7P7PlEDf/JVkvkao4HQatkqJGmGDhvfIPodIo8JC0\n6FsYxWtvrLJBArYjnVBKRuxIlBqq/7Yx0gj09kGf84aSXvkMDgio7AO4xSp9GJTJ\n4wIDAQAB\n-----END PUBLIC KEY-----\n"
-}
--- a/data_bags/credentials/lndhub-go.json
+++ b/data_bags/credentials/lndhub-go.json
@@ -1,24 +0,0 @@
-{
-  "id": "lndhub-go",
-  "jwt_secret": {
-    "encrypted_data": "cFost8pLsoJ/8Gp5m/TgN8xjMkvk0oZuEZ3XfxDIaYjOVYi3fEX8\n",
-    "iv": "47gV4v/D+10B6xqu\n",
-    "auth_tag": "MKEyVFfJ3f5pxWRSyMH4Rw==\n",
-    "version": 3,
-    "cipher": "aes-256-gcm"
-  },
-  "postgresql_password": {
-    "encrypted_data": "YSMEIWdZn08lyrZeJNAUZ5xwKhWHESa1A5MojKJ/5iiE\n",
-    "iv": "0mlURPOohnKbG+i8\n",
-    "auth_tag": "bqIOqFEEIxA99wlvpTqxFA==\n",
-    "version": 3,
-    "cipher": "aes-256-gcm"
-  },
-  "admin_token": {
-    "encrypted_data": "Jv2vQySZT9qn87g24IOYK1dpfSbZoUE/8VtZhzljQGIL\n",
-    "iv": "kjtrzmjTFKQq+nTV\n",
-    "auth_tag": "3YbOzU/ndVARbHTU1hoa9g==\n",
-    "version": 3,
-    "cipher": "aes-256-gcm"
-  }
-}
--- a/nodes/akkounts-1.json
+++ b/nodes/akkounts-1.json
@@ -12,9 +12,7 @@
    "hostname": "akkounts-1",
    "ipaddress": "192.168.122.160",
    "roles": [
-      "base",
      "kvm_guest",
-      "ldap_client",
      "akkounts",
      "postgresql_client"
    ],
@@ -22,7 +20,6 @@
      "kosmos-base",
      "kosmos-base::default",
      "kosmos_kvm::guest",
-      "kosmos-dirsrv::hostsfile",
      "kosmos_postgresql::hostsfile",
      "kosmos-akkounts",
      "kosmos-akkounts::default",
@@ -49,6 +46,7 @@
      "redis::default",
      "backup::default",
      "logrotate::default",
+      "kosmos-dirsrv::hostsfile",
      "nodejs::npm",
      "nodejs::install",
      "kosmos-nginx::default",
@@ -80,9 +78,8 @@
    }
  },
  "run_list": [
-    "role[base]",
+    "recipe[kosmos-base]",
    "role[kvm_guest]",
-    "role[ldap_client]",
    "role[akkounts]"
  ]
 }
--- a/nodes/bitcoin-2.json
+++ b/nodes/bitcoin-2.json
@@ -12,14 +12,9 @@
    "hostname": "bitcoin-2",
    "ipaddress": "192.168.122.148",
    "roles": [
-      "base",
      "kvm_guest",
-      "bitcoind",
-      "cln",
-      "lnd",
-      "lndhub",
-      "postgresql_client",
-      "btcpay"
+      "btcpay",
+      "postgresql_client"
    ],
    "recipes": [
      "kosmos-base",
@@ -27,16 +22,14 @@
      "kosmos_kvm::guest",
      "tor-full",
      "tor-full::default",
-      "kosmos-bitcoin::bitcoind",
+      "kosmos-bitcoin::source",
      "kosmos-bitcoin::c-lightning",
      "kosmos-bitcoin::lnd",
      "kosmos-bitcoin::lnd-scb-s3",
      "kosmos-bitcoin::boltz",
      "kosmos-bitcoin::rtl",
-      "kosmos-bitcoin::peerswap-lnd",
-      "kosmos_postgresql::hostsfile",
      "kosmos-bitcoin::lndhub",
-      "kosmos-bitcoin::lndhub-go",
+      "kosmos_postgresql::hostsfile",
      "kosmos-bitcoin::dotnet",
      "kosmos-bitcoin::nbxplorer",
      "kosmos-bitcoin::btcpay",
@@ -77,6 +70,7 @@
      "redisio::disable_os_default",
      "redisio::configure",
      "redisio::enable",
+      "kosmos-base::letsencrypt",
      "kosmos-nginx::default",
      "nginx::default",
      "nginx::package",
@@ -86,8 +80,7 @@
      "nginx::commons_dir",
      "nginx::commons_script",
      "nginx::commons_conf",
-      "kosmos-nginx::firewall",
-      "kosmos-base::letsencrypt"
+      "kosmos-nginx::firewall"
    ],
    "platform": "ubuntu",
    "platform_version": "20.04",
@@ -104,13 +97,16 @@
    }
  },
  "run_list": [
-    "role[base]",
+    "recipe[kosmos-base]",
    "role[kvm_guest]",
    "recipe[tor-full]",
-    "role[bitcoind]",
-    "role[cln]",
-    "role[lnd]",
-    "role[lndhub]",
+    "recipe[kosmos-bitcoin::source]",
+    "recipe[kosmos-bitcoin::c-lightning]",
+    "recipe[kosmos-bitcoin::lnd]",
+    "recipe[kosmos-bitcoin::lnd-scb-s3]",
+    "recipe[kosmos-bitcoin::boltz]",
+    "recipe[kosmos-bitcoin::rtl]",
+    "recipe[kosmos-bitcoin::lndhub]",
    "role[btcpay]"
  ]
 }
--- a/nodes/ejabberd-4.json
+++ b/nodes/ejabberd-4.json
@@ -59,9 +59,8 @@
    }
  },
  "run_list": [
-    "role[base]",
+    "recipe[kosmos-base]",
    "role[kvm_guest]",
-    "role[ldap_client]",
    "role[ejabberd]"
  ]
-}
+}
--- a/nodes/ejabberd-8.json
+++ b/nodes/ejabberd-8.json
@@ -57,9 +57,8 @@
    }
  },
  "run_list": [
-    "role[base]",
+    "recipe[kosmos-base]",
    "role[kvm_guest]",
-    "role[ldap_client]",
    "role[ejabberd]"
  ]
 }
--- a/nodes/fornax.kosmos.org.json
+++ b/nodes/fornax.kosmos.org.json
@@ -31,21 +31,20 @@
      "kosmos_assets::nginx_site",
      "kosmos_discourse::nginx",
      "kosmos_drone::nginx",
-      "kosmos_garage",
-      "kosmos_garage::default",
-      "kosmos_garage::firewall_rpc",
-      "kosmos_garage::nginx_web",
      "kosmos_gitea::nginx",
      "kosmos_website",
      "kosmos_website::default",
      "kosmos-akkounts::nginx_api",
-      "kosmos-bitcoin::nginx_lndhub",
      "kosmos-ejabberd::nginx",
      "kosmos-hubot::nginx_botka_irc-libera-chat",
      "kosmos-hubot::nginx_hal8000_xmpp",
      "kosmos-ipfs::nginx_public_gateway",
      "kosmos-mastodon::nginx",
      "remotestorage_discourse::nginx",
+      "kosmos_garage",
+      "kosmos_garage::default",
+      "kosmos_garage::firewall_rpc",
+      "kosmos_garage::nginx_web",
      "kosmos_zerotier::controller",
      "kosmos_zerotier::firewall",
      "kosmos_zerotier::zncui",
@@ -74,11 +73,11 @@
      "nginx::commons_conf",
      "kosmos-nginx::firewall",
      "discourse::nginx",
-      "firewall::default",
-      "chef-sugar::default",
      "git::default",
      "git::package",
      "kosmos-base::letsencrypt",
+      "firewall::default",
+      "chef-sugar::default",
      "fail2ban::default"
    ],
    "platform": "ubuntu",
--- a/nodes/gitea-2.json
+++ b/nodes/gitea-2.json
@@ -64,7 +64,6 @@
  "run_list": [
    "role[base]",
    "role[kvm_guest]",
-    "role[ldap_client]",
    "role[garage_gateway]",
    "role[gitea]"
  ]
--- a/nodes/ldap-3.kosmos.org.json
+++ b/nodes/ldap-3.kosmos.org.json
@@ -59,6 +59,6 @@
  "run_list": [
    "recipe[kosmos-base]",
    "role[kvm_guest]",
-    "role[dirsrv_supplier]"
+    "role[dirsrv_primary]"
  ]
-}
+}
--- a/nodes/ldap-4.kosmos.org.json
+++ b/nodes/ldap-4.kosmos.org.json
@@ -1,57 +0,0 @@
-{
-  "name": "ldap-4.kosmos.org",
-  "normal": {
-    "knife_zero": {
-      "host": "10.1.1.106"
-    }
-  },
-  "automatic": {
-    "fqdn": "ldap-4.kosmos.org",
-    "os": "linux",
-    "os_version": "5.4.0-1079-kvm",
-    "hostname": "ldap-4",
-    "ipaddress": "192.168.122.73",
-    "roles": [
-      "base",
-      "kvm_guest"
-    ],
-    "recipes": [
-      "kosmos-base",
-      "kosmos-base::default",
-      "kosmos_kvm::guest",
-      "apt::default",
-      "timezone_iii::default",
-      "timezone_iii::debian",
-      "ntp::default",
-      "ntp::apparmor",
-      "kosmos-base::systemd_emails",
-      "apt::unattended-upgrades",
-      "kosmos-base::firewall",
-      "kosmos-postfix::default",
-      "postfix::default",
-      "postfix::_common",
-      "postfix::_attributes",
-      "postfix::sasl_auth",
-      "hostname::default"
-    ],
-    "platform": "ubuntu",
-    "platform_version": "20.04",
-    "cloud": null,
-    "chef_packages": {
-      "chef": {
-        "version": "17.10.3",
-        "chef_root": "/opt/chef/embedded/lib/ruby/gems/3.0.0/gems/chef-17.10.3/lib",
-        "chef_effortless": null
-      },
-      "ohai": {
-        "version": "17.9.0",
-        "ohai_root": "/opt/chef/embedded/lib/ruby/gems/3.0.0/gems/ohai-17.9.0/lib/ohai"
-      }
-    }
-  },
-  "run_list": [
-    "role[base]",
-    "role[kvm_guest]",
-    "role[dirsrv_supplier]"
-  ]
-}
--- a/nodes/postgres-2.json
+++ b/nodes/postgres-2.json
@@ -21,10 +21,8 @@
      "kosmos_kvm::guest",
      "kosmos_postgresql::primary",
      "kosmos_postgresql::firewall",
-      "kosmos-bitcoin::lndhub-go_pg_db",
-      "kosmos_drone::pg_db",
      "kosmos_gitea::pg_db",
-      "kosmos-mastodon::pg_db",
+      "kosmos_drone::pg_db",
      "apt::default",
      "timezone_iii::default",
      "timezone_iii::debian",
--- a/nodes/wiki-1.json
+++ b/nodes/wiki-1.json
@@ -74,9 +74,8 @@
    }
  },
  "run_list": [
-    "role[base]",
+    "recipe[kosmos-base]",
    "role[kvm_guest]",
-    "role[ldap_client]",
    "recipe[kosmos-mediawiki]"
  ]
-}
+}
--- a/roles/bitcoind.rb
+++ b/roles/bitcoind.rb
@@ -1,5 +0,0 @@
-name "bitcoind"
-
-run_list %w(
-  kosmos-bitcoin::bitcoind
-)
--- a/roles/cln.rb
+++ b/roles/cln.rb
@@ -1,5 +0,0 @@
-name "cln"
-
-run_list %w(
-  kosmos-bitcoin::c-lightning
-)
--- a/roles/dirsrv_supplier.rb
+++ b/roles/dirsrv_supplier.rb
@@ -1,4 +1,4 @@
-name "dirsrv_supplier"
+name "dirsrv_primary"

 run_list %w(
  recipe[kosmos-dirsrv]
--- a/roles/hubot.rb
+++ b/roles/hubot.rb
@@ -7,6 +7,6 @@ default_run_list = %w(

 env_run_lists(
  '_default' => default_run_list,
-  'development' => default_run_list,
+  'development' => [],
  'production' => default_run_list
 )
--- a/roles/ldap_client.rb
+++ b/roles/ldap_client.rb
@@ -1,5 +0,0 @@
-name "ldap_client"
-
-run_list %w(
-  kosmos-dirsrv::hostsfile
-)
--- a/roles/lnd.rb
+++ b/roles/lnd.rb
@@ -1,9 +0,0 @@
-name "lnd"
-
-run_list %w(
-  kosmos-bitcoin::lnd
-  kosmos-bitcoin::lnd-scb-s3
-  kosmos-bitcoin::boltz
-  kosmos-bitcoin::rtl
-  kosmos-bitcoin::peerswap-lnd
-)
--- a/roles/lndhub.rb
+++ b/roles/lndhub.rb
@@ -1,7 +0,0 @@
-name "lndhub"
-
-run_list %w(
-  role[postgresql_client]
-  kosmos-bitcoin::lndhub
-  kosmos-bitcoin::lndhub-go
-)
--- a/roles/nginx_proxy.rb
+++ b/roles/nginx_proxy.rb
@@ -18,19 +18,18 @@ default_run_list = %w(
  kosmos_assets::nginx_site
  kosmos_discourse::nginx
  kosmos_drone::nginx
-  kosmos_garage::default
-  kosmos_garage::firewall_rpc
-  kosmos_garage::nginx_web
  kosmos_gitea::nginx
  kosmos_website::default
  kosmos-akkounts::nginx_api
-  kosmos-bitcoin::nginx_lndhub
  kosmos-ejabberd::nginx
  kosmos-hubot::nginx_botka_irc-libera-chat
  kosmos-hubot::nginx_hal8000_xmpp
  kosmos-ipfs::nginx_public_gateway
  kosmos-mastodon::nginx
  remotestorage_discourse::nginx
+  kosmos_garage::default
+  kosmos_garage::firewall_rpc
+  kosmos_garage::nginx_web
 )

 env_run_lists(
--- a/roles/postgresql_primary.rb
+++ b/roles/postgresql_primary.rb
@@ -3,8 +3,7 @@ name "postgresql_primary"
 run_list %w(
  kosmos_postgresql::primary
  kosmos_postgresql::firewall
-  kosmos-bitcoin::lndhub-go_pg_db
-  kosmos_drone::pg_db
  kosmos_gitea::pg_db
+  kosmos_drone::pg_db
  kosmos-mastodon::pg_db
 )
--- a/site-cookbooks/kosmos-akkounts/attributes/default.rb
+++ b/site-cookbooks/kosmos-akkounts/attributes/default.rb
@@ -1,5 +1,5 @@
 node.default['akkounts']['repo'] = 'https://gitea.kosmos.org/kosmos/akkounts.git'
-node.default['akkounts']['revision'] = 'feature/73-lndhub-go'
+node.default['akkounts']['revision'] = 'master'
 node.default['akkounts']['port'] = 3000
 node.default['akkounts']['domain'] = 'accounts.kosmos.org'

--- a/site-cookbooks/kosmos-akkounts/recipes/default.rb
+++ b/site-cookbooks/kosmos-akkounts/recipes/default.rb
@@ -22,6 +22,7 @@ package "libpq-dev"

 include_recipe 'kosmos-nodejs'
 include_recipe "kosmos-redis"
+include_recipe "kosmos-dirsrv::hostsfile"

 npm_package "yarn" do
  version "1.22.4"
--- a/site-cookbooks/kosmos-bitcoin/attributes/default.rb
+++ b/site-cookbooks/kosmos-bitcoin/attributes/default.rb
@@ -79,26 +79,6 @@ node.default['lndhub']['revision'] = 'master'
 node.default['lndhub']['port'] = '3023'
 node.default['lndhub']['domain'] = 'lndhub.kosmos.org'

-node.default['lndhub-go']['repo'] = 'https://github.com/getAlby/lndhub.go.git'
-node.default['lndhub-go']['revision'] = '0.12.0'
-node.default['lndhub-go']['source_dir'] = '/opt/lndhub-go'
-node.default['lndhub-go']['port'] = 3026
-node.default['lndhub-go']['domain'] = 'lndhub.kosmos.org'
-node.default['lndhub-go']['postgres']['database'] = 'lndhub'
-node.default['lndhub-go']['postgres']['user'] = 'lndhub'
-node.default['lndhub-go']['postgres']['port'] = 5432
-node.default['lndhub-go']['default_rate_limit'] = 20
-node.default['lndhub-go']['strict_rate_limit'] =  1
-node.default['lndhub-go']['burst_rate_limit'] =  10
-node.default['lndhub-go']['branding'] = {
-  'title' => 'LndHub - Kosmos Lightning',
-  'desc' => 'Kosmos accounts for the Lightning Network',
-  'url' => 'https://lndhub.kosmos.org',
-  'logo' => 'https://assets.kosmos.org/img/icon-lndhub-400px.png',
-  'favicon' => 'https://kosmos.org/favicon.ico',
-  'footer' => 'about=https://kosmos.org'
-}
-
 node.default['dotnet']['ms_packages_src_url'] = "https://packages.microsoft.com/config/ubuntu/20.04/packages-microsoft-prod.deb"
 node.default['dotnet']['ms_packages_src_checksum'] = "4df5811c41fdded83eb9e2da9336a8dfa5594a79dc8a80133bd815f4f85b9991"

@@ -118,7 +98,3 @@ node.default["btcpay"]["domain"] = 'btcpay.kosmos.org'
 node.default['btcpay']['postgres']['port'] = 5432
 node.default['btcpay']['postgres']['database'] = 'btcpayserver'
 node.default['btcpay']['postgres']['user'] = 'satoshi'
-
-node.default['peerswap']['repo'] = 'https://github.com/ElementsProject/peerswap.git'
-node.default['peerswap']['revision'] = 'master'
-node.default['peerswap-lnd']['source_dir'] = '/opt/peerswap'
--- a/site-cookbooks/kosmos-bitcoin/metadata.rb
+++ b/site-cookbooks/kosmos-bitcoin/metadata.rb
@@ -7,15 +7,25 @@ long_description 'Installs/configures bitcoin-related software'
 version '0.1.0'
 chef_version '>= 14.0'

-depends 'application_javascript'
+# The `issues_url` points to the location where issues for this cookbook are
+# tracked.  A `View Issues` link will be displayed on this cookbook's page when
+# uploaded to a Supermarket.
+#
+# issues_url 'https://github.com/<insert_org_here>/kosmos-bitcoin/issues'
+
+# The `source_url` points to the development repository for this cookbook.  A
+# `View Source` link will be displayed on this cookbook's page when uploaded to
+# a Supermarket.
+#
+# source_url 'https://github.com/<insert_org_here>/kosmos-bitcoin'
+
 depends 'ark'
 depends 'backup'
-depends 'firewall'
 depends 'git'
 depends 'golang'
 depends 'kosmos-nginx'
 depends 'kosmos-nodejs'
-depends 'kosmos_postgresql'
-depends 'postgresql'
-depends 'redisio'
+depends 'firewall'
+depends 'application_javascript'
 depends 'tor-full'
+depends 'redisio'
--- a/site-cookbooks/kosmos-bitcoin/recipes/golang.rb
+++ b/site-cookbooks/kosmos-bitcoin/recipes/golang.rb
@@ -1,6 +1,6 @@
 #
 # Cookbook:: kosmos-bitcoin
-# Recipe:: golang
+# Recipe:: boltz
 #
 # Internal recipe for managing the Go installation in one place
 #
--- a/site-cookbooks/kosmos-bitcoin/recipes/lndhub-go.rb
+++ b/site-cookbooks/kosmos-bitcoin/recipes/lndhub-go.rb
@@ -1,107 +0,0 @@
-#
-# Cookbook:: kosmos-bitcoin
-# Recipe:: lndhub-go
-#
-
-include_recipe 'git'
-include_recipe 'kosmos-bitcoin::golang'
-include_recipe 'kosmos-bitcoin::user'
-
-bitcoin_user  = node['bitcoin']['username']
-bitcoin_group = node['bitcoin']['usergroup']
-lnd_dir       = node['lnd']['lnd_dir']
-lncli_bin     = '/opt/go/bin/lncli'
-source_dir    = node['lndhub-go']['source_dir']
-macaroon_path = "#{lnd_dir}/data/lndhub.macaroon"
-credentials   = data_bag_item('credentials', 'lndhub-go')
-postgres_host = "pg.kosmos.local"
-postgres_user = node['lndhub-go']['postgres']['user']
-postgres_db   = node['lndhub-go']['postgres']['database']
-postgres_port = node['lndhub-go']['postgres']['port']
-
-git source_dir do
-  repository node['lndhub-go']['repo']
-  revision node['lndhub-go']['revision']
-  action :sync
-  notifies :run, 'bash[compile_lndhub-go]', :immediately
-end
-
-bash 'compile_lndhub-go' do
-  cwd source_dir
-  code 'make'
-  action :nothing
-  notifies :restart, 'service[lndhub-go]', :delayed
-end
-
-bash 'bake_lndhub_macaroon' do
-  user bitcoin_user
-  cwd lnd_dir
-  code "#{lncli_bin} bakemacaroon --save_to=./data/lndhub.macaroon info:read invoices:read invoices:write offchain:read offchain:write"
-  not_if { File.exist?(macaroon_path) }
-end
-
-template "#{source_dir}/.env" do
-  source 'lndhub-go.env.erb'
-  owner bitcoin_user
-  group bitcoin_group
-  mode 0600
-  sensitive true
-  variables config: {
-    database_uri: "postgresql://#{postgres_user}:#{credentials['postgresql_password']}@#{postgres_host}:#{postgres_port}/#{postgres_db}?sslmode=disable",
-    jwt_secret: credentials['jwt_secret'],
-    lnd_address: 'localhost:10009', # gRPC address,
-    lnd_macaroon_file: macaroon_path,
-    lnd_cert_file:  "#{lnd_dir}/tls.cert",
-    custom_name: node['lndhub-go']['domain'],
-    port: node['lndhub-go']['port'],
-    admin_token: credentials['admin_token'],
-    default_rate_limit: node['lndhub-go']['default_rate_limit'],
-    strict_rate_limit: node['lndhub-go']['strict_rate_limit'],
-    burst_rate_limit: node['lndhub-go']['burst_rate_limit'],
-    branding: node['lndhub-go']['branding']
-  }
-  notifies :restart, 'service[lndhub-go]', :delayed
-end
-
-systemd_unit 'lndhub-go.service' do
-  content({
-    Unit: {
-      Description: 'LndHub compatible API written in Go',
-      Documentation: ['https://github.com/getAlby/lndhub.go/blob/main/README.md'],
-      Requires: 'lnd.service',
-      After: 'lnd.service'
-    },
-    Service: {
-      User: bitcoin_user,
-      Group: bitcoin_group,
-      Type: 'simple',
-      WorkingDirectory: source_dir,
-      ExecStart: "#{source_dir}/lndhub",
-      Restart: 'always',
-      RestartSec: '10',
-      TimeoutSec: '60',
-      PrivateTmp: true,
-      ProtectSystem: 'full',
-      NoNewPrivileges: true,
-      PrivateDevices: true,
-      MemoryDenyWriteExecute: true
-    },
-    Install: {
-      WantedBy: 'multi-user.target'
-    }
-  })
-  verify false
-  triggers_reload true
-  action [:create, :enable, :start]
-end
-
-service 'lndhub-go' do
-  action :nothing
-end
-
-firewall_rule 'lndhub-go' do
-  port     node['lndhub-go']['port']
-  source   '10.1.1.0/24'
-  protocol :tcp
-  command  :allow
-end
--- a/site-cookbooks/kosmos-bitcoin/recipes/lndhub-go_pg_db.rb
+++ b/site-cookbooks/kosmos-bitcoin/recipes/lndhub-go_pg_db.rb
@@ -1,19 +0,0 @@
-#
-# Cookbook Name:: kosmos-bitcoin
-# Recipe:: lndhub-go_pg_db
-#
-
-credentials = data_bag_item('credentials', 'lndhub-go')
-
-postgres_user = node['lndhub-go']['postgres']['user']
-postgres_db   = node['lndhub-go']['postgres']['database']
-
-postgresql_user postgres_user do
-  action :create
-  password credentials['postgresql_password']
-end
-
-postgresql_database postgres_db do
-  owner postgres_user
-  action :create
-end
--- a/site-cookbooks/kosmos-bitcoin/recipes/lndhub.rb
+++ b/site-cookbooks/kosmos-bitcoin/recipes/lndhub.rb
@@ -90,7 +90,27 @@ firewall_rule 'lndhub_private' do
  command  :allow
 end

-return if node.chef_environment == "development"
+unless node.chef_environment == "development"
+  include_recipe "kosmos-base::letsencrypt"
+  include_recipe "kosmos-nginx"

-node.override["backup"]["archives"]["lndhub"] = ["/var/lib/redis/dump-6379.rdb"]
-include_recipe "backup"
+  nginx_certbot_site node[app_name]['domain']
+
+  template "#{node['nginx']['dir']}/sites-available/#{node[app_name]['domain']}" do
+    source 'nginx_conf_lndhub.erb'
+    owner node["nginx"]["user"]
+    mode 0640
+    variables port: node[app_name]['port'],
+              server_name:  node[app_name]['domain'],
+              ssl_cert:     "/etc/letsencrypt/live/#{node[app_name]['domain']}/fullchain.pem",
+              ssl_key:      "/etc/letsencrypt/live/#{node[app_name]['domain']}/privkey.pem"
+    notifies :reload, 'service[nginx]', :delayed
+  end
+
+  nginx_site node[app_name]['domain'] do
+    action :enable
+  end
+
+  node.override["backup"]["archives"]["lndhub"] = ["/var/lib/redis/dump-6379.rdb"]
+  include_recipe "backup"
+end
--- a/site-cookbooks/kosmos-bitcoin/recipes/nginx_lndhub.rb
+++ b/site-cookbooks/kosmos-bitcoin/recipes/nginx_lndhub.rb
@@ -1,29 +0,0 @@
-#
-# Cookbook:: kosmos-bitcoin
-# Recipe:: nginx_lndhub
-#
-
-include_recipe "kosmos-base::letsencrypt"
-include_recipe "kosmos-nginx"
-
-domain = node['lndhub-go']['domain']
-
-nginx_certbot_site domain
-
-upstream_host = search(:node, "role:lndhub").first["knife_zero"]["host"]
-
-template "#{node['nginx']['dir']}/sites-available/#{domain}" do
-  source 'nginx_conf_lndhub.erb'
-  owner node["nginx"]["user"]
-  mode 0640
-  variables port: node['lndhub-go']['port'],
-            server_name: domain,
-            ssl_cert: "/etc/letsencrypt/live/#{domain}/fullchain.pem",
-            ssl_key:  "/etc/letsencrypt/live/#{domain}/privkey.pem",
-            upstream_host: upstream_host
-  notifies :reload, 'service[nginx]', :delayed
-end
-
-nginx_site domain do
-  action :enable
-end
--- a/site-cookbooks/kosmos-bitcoin/recipes/peerswap-lnd.rb
+++ b/site-cookbooks/kosmos-bitcoin/recipes/peerswap-lnd.rb
@@ -1,86 +0,0 @@
-#
-# Cookbook:: kosmos-bitcoin
-# Recipe:: peerswap-lnd
-#
-
-include_recipe 'git'
-include_recipe 'kosmos-bitcoin::golang'
-include_recipe 'kosmos-bitcoin::user'
-
-bitcoin_user  = node['bitcoin']['username']
-bitcoin_group = node['bitcoin']['usergroup']
-lnd_dir       = node['lnd']['lnd_dir']
-macaroon_path = "#{lnd_dir}/data/chain/bitcoin/#{node['bitcoin']['network']}/admin.macaroon"
-source_dir    = node['peerswap-lnd']['source_dir']
-config_dir    = "/home/#{bitcoin_user}/.peerswap"
-
-directory config_dir do
-  owner bitcoin_user
-  group bitcoin_group
-  mode '0700'
-  action :create
-end
-
-git source_dir do
-  repository node['peerswap']['repo']
-  revision node['peerswap']['revision']
-  action :sync
-  notifies :run, 'bash[compile_peerswap]', :immediately
-end
-
-bash 'compile_peerswap' do
-  cwd source_dir
-  environment 'GOPATH' => '/opt/go'
-  code 'make lnd-release'
-  action :run
-  notifies :restart, 'service[peerswap]', :delayed
-end
-
-template "#{config_dir}/peerswap.conf" do
-  source 'peerswap-lnd.conf.erb'
-  owner bitcoin_user
-  group bitcoin_group
-  mode 0600
-  sensitive true
-  variables config: {
-    tlscertpath:  "#{lnd_dir}/tls.cert",
-    macaroonpath: macaroon_path
-  }
-  notifies :restart, 'service[peerswap]', :delayed
-end
-
-systemd_unit 'peerswap.service' do
-  content({
-    Unit: {
-      Description: 'PeerSwap Lightning channel balancing',
-      Documentation: ['https://github.com/ElementsProject/peerswap'],
-      Requires: 'lnd.service',
-      After: 'lnd.service'
-    },
-    Service: {
-      User: bitcoin_user,
-      Group: bitcoin_group,
-      Type: 'simple',
-      WorkingDirectory: source_dir,
-      ExecStart: "/opt/go/bin/peerswapd",
-      Restart: 'always',
-      RestartSec: '10',
-      TimeoutSec: '60',
-      PrivateTmp: true,
-      ProtectSystem: 'full',
-      NoNewPrivileges: true,
-      PrivateDevices: true,
-      MemoryDenyWriteExecute: true
-    },
-    Install: {
-      WantedBy: 'multi-user.target'
-    }
-  })
-  verify false
-  triggers_reload true
-  action [:create, :enable, :start]
-end
-
-service 'peerswap' do
-  action :nothing
-end
--- a/site-cookbooks/kosmos-bitcoin/recipes/bitcoind.rb
+++ b/site-cookbooks/kosmos-bitcoin/recipes/bitcoind.rb
@@ -1,6 +1,6 @@
 #
 # Cookbook:: kosmos-bitcoin
-# Recipe:: bitcoind
+# Recipe:: source
 #

 build_essential
--- a/site-cookbooks/kosmos-bitcoin/templates/lndhub-go.env.erb
+++ b/site-cookbooks/kosmos-bitcoin/templates/lndhub-go.env.erb
@@ -1,9 +0,0 @@
-<% @config.each do |key, value| %>
-<% if value.is_a?(Hash) %>
-<% value.each do |k, v| %>
-<%= "#{key.upcase}_#{k.upcase}" %>=<%= v.to_s %>
-<% end %>
-<% else %>
-<%= key.upcase %>=<%= value.to_s %>
-<% end %>
-<% end %>
--- a/site-cookbooks/kosmos-bitcoin/templates/nginx_conf_lndhub.erb
+++ b/site-cookbooks/kosmos-bitcoin/templates/nginx_conf_lndhub.erb
@@ -2,9 +2,10 @@
 # Generated by Chef
 #
 upstream _lndhub {
-  server <%= @upstream_host || "localhost" %>:<%= @port %>;
+  server localhost:<%= @port %>;
 }

+<% if File.exist?(@ssl_cert) && File.exist?(@ssl_key) -%>
 server {
  listen 443 ssl http2;
  server_name <%= @server_name %>;
@@ -15,13 +16,10 @@ server {
  error_log <%= node[:nginx][:log_dir] %>/<%= @server_name %>.error.log warn;

  location / {
-    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
-    proxy_set_header X-Forwarded-Proto https;
-    proxy_set_header Host $http_host;
-    proxy_redirect off;
    proxy_pass http://_lndhub;
-  }
+   }

  ssl_certificate <%= @ssl_cert %>;
  ssl_certificate_key <%= @ssl_key %>;
 }
+<% end -%>
--- a/site-cookbooks/kosmos-bitcoin/templates/peerswap-lnd.conf.erb
+++ b/site-cookbooks/kosmos-bitcoin/templates/peerswap-lnd.conf.erb
@@ -1,3 +0,0 @@
-<% @config.each do |k, v| %>
-<%= "lnd.#{k}=#{v}" %>
-<% end %>
--- a/site-cookbooks/kosmos-dirsrv/recipes/default.rb
+++ b/site-cookbooks/kosmos-dirsrv/recipes/default.rb
@@ -3,15 +3,12 @@
 # Recipe:: default
 #

-credentials = data_bag_item("credentials", "dirsrv")
-local_hostname = "#{node["hostname"]}.kosmos.local"
+include_recipe "kosmos-dirsrv::hostsfile"

-hostsfile_entry "127.0.0.1" do
-  hostname local_hostname
-end
+credentials = data_bag_item("credentials", "dirsrv")

 dirsrv_instance "master" do
-  hostname local_hostname
+  hostname "ldap.kosmos.local"
  admin_password credentials['admin_password']
  suffix "dc=kosmos,dc=org"
 end
--- a/site-cookbooks/kosmos-dirsrv/recipes/hostsfile.rb
+++ b/site-cookbooks/kosmos-dirsrv/recipes/hostsfile.rb
@@ -3,12 +3,12 @@
 # Recipe:: hostsfile
 #

-dirsrv_supplier = search(:node, "role:dirsrv_supplier AND chef_environment:#{node.chef_environment}").first
+dirsrv_primary = search(:node, "role:dirsrv_primary AND chef_environment:#{node.chef_environment}").first

-unless dirsrv_supplier.nil?
-  supplier_ip = dirsrv_supplier['knife_zero']['host']
+unless dirsrv_primary.nil?
+  primary_ip = dirsrv_primary['knife_zero']['host']

-  hostsfile_entry supplier_ip do
+  hostsfile_entry primary_ip do
    hostname  "ldap.kosmos.local"
    unique    true
  end
--- a/site-cookbooks/kosmos-ejabberd/recipes/default.rb
+++ b/site-cookbooks/kosmos-ejabberd/recipes/default.rb
@@ -3,6 +3,8 @@
 # Recipe:: default
 #

+include_recipe "kosmos-dirsrv::hostsfile"
+
 ejabberd_credentials = data_bag_item("credentials", "ejabberd")

 ejabberd_version = node["kosmos-ejabberd"]["version"]
--- a/site-cookbooks/kosmos-mastodon/recipes/default.rb
+++ b/site-cookbooks/kosmos-mastodon/recipes/default.rb
@@ -70,7 +70,7 @@ npm_package "yarn" do
  version "1.22.4"
 end

-ruby_version = "3.0.4"
+ruby_version = "3.0.3"

 execute "systemctl daemon-reload" do
  command "systemctl daemon-reload"
@@ -192,6 +192,7 @@ application mastodon_path do
  end

  execute 'rake db:migrate' do
+    # environment "RAILS_ENV" => "production", "HOME" => mastodon_path#, "SKIP_POST_DEPLOYMENT_MIGRATIONS" => "true"
    environment "RAILS_ENV" => "production", "HOME" => mastodon_path, "SKIP_POST_DEPLOYMENT_MIGRATIONS" => "true"
    user mastodon_user
    group mastodon_user
--- a/site-cookbooks/kosmos-mastodon/templates/default/mastodon-sidekiq.systemd.service.erb
+++ b/site-cookbooks/kosmos-mastodon/templates/default/mastodon-sidekiq.systemd.service.erb
@@ -10,7 +10,7 @@ WorkingDirectory=<%= @app_dir %>
 Environment="RAILS_ENV=production"
 Environment="DB_POOL=50"
 Environment="LD_PRELOAD=/usr/lib/x86_64-linux-gnu/libjemalloc.so.2"
-ExecStart=<%= @bundle_path %> exec sidekiq -c <%= @sidekiq_threads %> -q default -q mailers -q pull -q push -q ingress
+ExecStart=<%= @bundle_path %> exec sidekiq -c <%= @sidekiq_threads %> -q default -q mailers -q pull -q push
 TimeoutSec=15
 Restart=always

--- a/site-cookbooks/kosmos-mediawiki/recipes/default.rb
+++ b/site-cookbooks/kosmos-mediawiki/recipes/default.rb
@@ -27,6 +27,7 @@
 include_recipe 'apt'
 include_recipe 'ark'
 include_recipe 'composer'
+include_recipe 'kosmos-dirsrv::hostsfile'

 server_name = 'wiki.kosmos.org'

--- a/site-cookbooks/kosmos_gitea/attributes/default.rb
+++ b/site-cookbooks/kosmos_gitea/attributes/default.rb
@@ -1,7 +1,7 @@
-gitea_version = "1.18.0"
+gitea_version = "1.17.3"
 node.default["gitea"]["version"] = gitea_version
 node.default["gitea"]["binary_url"] = "https://dl.gitea.io/gitea/#{gitea_version}/gitea-#{gitea_version}-linux-amd64"
-node.default["gitea"]["binary_checksum"] = "b45b715d519a97086208c6b42528d291dd1c4dfdf40321dc940030e1cf3de6e6"
+node.default["gitea"]["binary_checksum"] = "38c4e1228cd051b785c556bcadc378280d76c285b70e8761cd3f5051aed61b5e"
 node.default["gitea"]["working_directory"] = "/var/lib/gitea"
 node.default["gitea"]["port"] = 3000
 node.default["gitea"]["postgresql_host"] = "localhost:5432"
--- a/site-cookbooks/kosmos_gitea/recipes/default.rb
+++ b/site-cookbooks/kosmos_gitea/recipes/default.rb
@@ -3,6 +3,8 @@
 # Recipe:: default
 #

+include_recipe "kosmos-dirsrv::hostsfile"
+
 working_directory         = node["gitea"]["working_directory"]
 git_home_directory        = "/home/git"
 repository_root_directory = "#{git_home_directory}/gitea-repositories"
@@ -10,8 +12,6 @@ config_directory          = "/etc/gitea"
 gitea_binary_path         = "/usr/local/bin/gitea"
 gitea_data_bag_item       = data_bag_item("credentials", "gitea")
 smtp_credentials          = data_bag_item("credentials", "smtp")
-smtp_addr                 = smtp_credentials["relayhost"].split(":")[0]
-smtp_port                 = smtp_credentials["relayhost"].split(":")[1]
 jwt_secret                = gitea_data_bag_item["jwt_secret"]
 internal_token            = gitea_data_bag_item["internal_token"]
 secret_key                = gitea_data_bag_item["secret_key"]
@@ -86,8 +86,7 @@ config_variables = {
  secret_key: secret_key,
  postgresql_host: node["gitea"]["postgresql_host"],
  postgresql_password: gitea_data_bag_item["postgresql_password"],
-  smtp_addr: smtp_addr,
-  smtp_port: smtp_port,
+  smtp_host: smtp_credentials["relayhost"],
  smtp_user: smtp_credentials["user_name"],
  smtp_password: smtp_credentials["password"],
  config: node["gitea"]["config"],
--- a/site-cookbooks/kosmos_gitea/templates/default/app.ini.erb
+++ b/site-cookbooks/kosmos_gitea/templates/default/app.ini.erb
@@ -39,12 +39,10 @@ COOKIE_SECURE = true

 [mailer]
 ENABLED = true
-PROTOCOL = smtp+startls
-SMTP_ADDR = <%= @smtp_addr %>
-SMTP_PORT = <%= @smtp_port %>
+HOST = <%= @smtp_host %>
+FROM = gitea@kosmos.org
 USER = <%= @smtp_user %>
 PASSWD = <%= @smtp_password %>
-FROM = gitea@kosmos.org

 [security]
 INTERNAL_TOKEN = <%= @internal_token %>
--- a/site-cookbooks/kosmos_postgresql/libraries/helpers.rb
+++ b/site-cookbooks/kosmos_postgresql/libraries/helpers.rb
@@ -1,7 +1,7 @@
 class Chef
  class Recipe
    def postgresql_primary
-      postgresql_primary = search(:node, "role:postgresql_primary").first
+      postgresql_primary = search(:node, "role:postgresql_primary AND chef_environment:#{node.chef_environment}").first

      unless postgresql_primary.nil?
        primary_ip = ip_for(postgresql_primary)
Author	SHA1	Message	Date
Râu Cao	9fcf92be05	Add garage S3 config for Mastodon	2022-11-30 12:15:27 +01:00
Râu Cao	8c3bd2e939	Add nginx proxy hosts for Garage Web access The respective bucket needs to be configured with a domain alias. When a new alias is added to the `s3_web_domains` config, a new nginx site can then be deployed to the `nginx_proxy` hosts.	2022-11-30 12:13:39 +01:00