Add garage S3 config for Mastodon

Update README
Merge pull request 'Upgrade Gitea to 1.18.0' (#460 ) from chore/upgrade_gitea into master
2023-01-14 15:03:38 +08:00 · 2023-01-05 17:14:46 +08:00 · 2022-12-30 16:08:18 +00:00 · 2022-12-30 23:05:40 +07:00 · 2022-12-27 06:29:09 +00:00 · 2022-12-27 06:23:09 +00:00
46 changed files with 510 additions and 111 deletions
--- a/README.md
+++ b/README.md
@ -1,3 +1,16 @@
+This repository contains all infrastructure automation code that we use to set
+up and configure servers, virtual machines, and applications for Kosmos hosted
+services.
+
+Chef cookbooks are written in Ruby, and based on [Chef Infra
+resources](https://docs.chef.io/resources/). Some cookbooks contain integration
+test suites based on [Test Kitchen](https://docs.chef.io/workstation/kitchen/).
+
+Note: Manual configuration of servers and applications is highly discouraged,
+and can be overwritten or lost without notice!
+
+## Setup
+
 ### Install Chef Workstation

 * macOS, Windows, RHEL, Ubuntu: https://docs.chef.io/workstation/install_workstation/
@ -6,24 +19,28 @@
 #### rbenv

 If you use rbenv to manage Ruby versions on your system, install the
-(rbenv-chef-workstation)[https://github.com/docwhat/rbenv-chef-workstation]
+[rbenv-chef-workstation](https://github.com/docwhat/rbenv-chef-workstation)
 plugin.

 ### Install gem dependencies

+Clone this repository, `cd` into it, and run:
+
    bundle install

-### Bootstrap a new server
+## Common tasks

-    knife zero bootstrap root@dev.kosmos.org --run-list "recipe[kosmos-base],..." -j '{"example_cookbook":{"memory_max":"256M"}}' --secret-file .chef/encrypted_data_bag_secret
+### Bootstrap a new host server
+
+    knife zero bootstrap root@server-name.kosmos.org --run-list "role[base],role[kvm_host]" --secret-file .chef/encrypted_data_bag_secret

 ### Bootstrap a new VM

-    knife zero bootstrap ubuntu@zerotier-ip-address -x ubuntu --sudo --run-list "recipe[kosmos-base]" --secret-file .chef/encrypted_data_bag_secret
+    knife zero bootstrap ubuntu@zerotier-ip-address -x ubuntu --sudo --run-list "role[base],role[kvm_guest]" --secret-file .chef/encrypted_data_bag_secret

-### Run Chef Zero
+### Run Chef Zero on a host server

-    knife zero converge name:dev.kosmos.org
+    knife zero converge -p2222 name:server-name.kosmos.org

 ### Run Chef Zero on a VM

@ -33,7 +50,7 @@ plugin.

    knife zero converge name:dev.kosmos.org --client-version 15.3.14

-### Managing cookbooks
+## Managing cookbooks

 Cookbooks are managed via Berkshelf. Run `berks --help` for command help.

@ -45,7 +62,7 @@ Vendor installed cookbooks to the `cookbooks/` dir:

    berks vendor cookbooks/ --delete

-### "Expired" TLS certificates
+## "Expired" TLS certificates

 If you encounter expired TLS certificates during a Chef run (e.g. for remote
 files), the issue is likely that the certificate has been issued by Let's
--- a/clients/ldap-4.json
+++ b/clients/ldap-4.json
@ -0,0 +1,4 @@
+{
+  "name": "ldap-4",
+  "public_key": "-----BEGIN PUBLIC KEY-----\nMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAmzFyZh5/J2BsKRunghis\nwUGbv4j/ynAF7QY+CYoOwDBcbLHk6odn1JyUqCgfhCIX0mh8F/fDKyU9Aw6+HHZ/\nX0DTt/enLTaWc2vxRfyJLRXP7/ymHOr4u6HYEINMdVJp4yQ9XLcWpuRHfA+fHrZ7\n9fI8sCMSEawvVpEKytYdVnm3VCjfIVrfCAkY0lP0mNG908edX2ZuJ4GS1UwADUZX\nLZuMhbGX9JqIQYWCyiMDakD7P7PlEDf/JVkvkao4HQatkqJGmGDhvfIPodIo8JC0\n6FsYxWtvrLJBArYjnVBKRuxIlBqq/7Yx0gj09kGf84aSXvkMDgio7AO4xSp9GJTJ\n4wIDAQAB\n-----END PUBLIC KEY-----\n"
+}
--- a/data_bags/credentials/lndhub-go.json
+++ b/data_bags/credentials/lndhub-go.json
@ -0,0 +1,24 @@
+{
+  "id": "lndhub-go",
+  "jwt_secret": {
+    "encrypted_data": "cFost8pLsoJ/8Gp5m/TgN8xjMkvk0oZuEZ3XfxDIaYjOVYi3fEX8\n",
+    "iv": "47gV4v/D+10B6xqu\n",
+    "auth_tag": "MKEyVFfJ3f5pxWRSyMH4Rw==\n",
+    "version": 3,
+    "cipher": "aes-256-gcm"
+  },
+  "postgresql_password": {
+    "encrypted_data": "YSMEIWdZn08lyrZeJNAUZ5xwKhWHESa1A5MojKJ/5iiE\n",
+    "iv": "0mlURPOohnKbG+i8\n",
+    "auth_tag": "bqIOqFEEIxA99wlvpTqxFA==\n",
+    "version": 3,
+    "cipher": "aes-256-gcm"
+  },
+  "admin_token": {
+    "encrypted_data": "Jv2vQySZT9qn87g24IOYK1dpfSbZoUE/8VtZhzljQGIL\n",
+    "iv": "kjtrzmjTFKQq+nTV\n",
+    "auth_tag": "3YbOzU/ndVARbHTU1hoa9g==\n",
+    "version": 3,
+    "cipher": "aes-256-gcm"
+  }
+}
--- a/nodes/akkounts-1.json
+++ b/nodes/akkounts-1.json
@ -12,7 +12,9 @@
    "hostname": "akkounts-1",
    "ipaddress": "192.168.122.160",
    "roles": [
+      "base",
      "kvm_guest",
+      "ldap_client",
      "akkounts",
      "postgresql_client"
    ],
@ -20,6 +22,7 @@
      "kosmos-base",
      "kosmos-base::default",
      "kosmos_kvm::guest",
+      "kosmos-dirsrv::hostsfile",
      "kosmos_postgresql::hostsfile",
      "kosmos-akkounts",
      "kosmos-akkounts::default",
@ -46,7 +49,6 @@
      "redis::default",
      "backup::default",
      "logrotate::default",
-      "kosmos-dirsrv::hostsfile",
      "nodejs::npm",
      "nodejs::install",
      "kosmos-nginx::default",
@ -78,8 +80,9 @@
    }
  },
  "run_list": [
-    "recipe[kosmos-base]",
+    "role[base]",
    "role[kvm_guest]",
+    "role[ldap_client]",
    "role[akkounts]"
  ]
 }
--- a/nodes/bitcoin-2.json
+++ b/nodes/bitcoin-2.json
@ -12,9 +12,14 @@
    "hostname": "bitcoin-2",
    "ipaddress": "192.168.122.148",
    "roles": [
+      "base",
      "kvm_guest",
-      "btcpay",
-      "postgresql_client"
+      "bitcoind",
+      "cln",
+      "lnd",
+      "lndhub",
+      "postgresql_client",
+      "btcpay"
    ],
    "recipes": [
      "kosmos-base",
@ -22,14 +27,16 @@
      "kosmos_kvm::guest",
      "tor-full",
      "tor-full::default",
-      "kosmos-bitcoin::source",
+      "kosmos-bitcoin::bitcoind",
      "kosmos-bitcoin::c-lightning",
      "kosmos-bitcoin::lnd",
      "kosmos-bitcoin::lnd-scb-s3",
      "kosmos-bitcoin::boltz",
      "kosmos-bitcoin::rtl",
-      "kosmos-bitcoin::lndhub",
+      "kosmos-bitcoin::peerswap-lnd",
      "kosmos_postgresql::hostsfile",
+      "kosmos-bitcoin::lndhub",
+      "kosmos-bitcoin::lndhub-go",
      "kosmos-bitcoin::dotnet",
      "kosmos-bitcoin::nbxplorer",
      "kosmos-bitcoin::btcpay",
@ -70,7 +77,6 @@
      "redisio::disable_os_default",
      "redisio::configure",
      "redisio::enable",
-      "kosmos-base::letsencrypt",
      "kosmos-nginx::default",
      "nginx::default",
      "nginx::package",
@ -80,7 +86,8 @@
      "nginx::commons_dir",
      "nginx::commons_script",
      "nginx::commons_conf",
-      "kosmos-nginx::firewall"
+      "kosmos-nginx::firewall",
+      "kosmos-base::letsencrypt"
    ],
    "platform": "ubuntu",
    "platform_version": "20.04",
@ -97,16 +104,13 @@
    }
  },
  "run_list": [
-    "recipe[kosmos-base]",
+    "role[base]",
    "role[kvm_guest]",
    "recipe[tor-full]",
-    "recipe[kosmos-bitcoin::source]",
-    "recipe[kosmos-bitcoin::c-lightning]",
-    "recipe[kosmos-bitcoin::lnd]",
-    "recipe[kosmos-bitcoin::lnd-scb-s3]",
-    "recipe[kosmos-bitcoin::boltz]",
-    "recipe[kosmos-bitcoin::rtl]",
-    "recipe[kosmos-bitcoin::lndhub]",
+    "role[bitcoind]",
+    "role[cln]",
+    "role[lnd]",
+    "role[lndhub]",
    "role[btcpay]"
  ]
 }
--- a/nodes/ejabberd-4.json
+++ b/nodes/ejabberd-4.json
@ -59,8 +59,9 @@
    }
  },
  "run_list": [
-    "recipe[kosmos-base]",
+    "role[base]",
    "role[kvm_guest]",
+    "role[ldap_client]",
    "role[ejabberd]"
  ]
-}
+}
--- a/nodes/ejabberd-8.json
+++ b/nodes/ejabberd-8.json
@ -57,8 +57,9 @@
    }
  },
  "run_list": [
-    "recipe[kosmos-base]",
+    "role[base]",
    "role[kvm_guest]",
+    "role[ldap_client]",
    "role[ejabberd]"
  ]
 }
--- a/nodes/fornax.kosmos.org.json
+++ b/nodes/fornax.kosmos.org.json
@ -31,20 +31,21 @@
      "kosmos_assets::nginx_site",
      "kosmos_discourse::nginx",
      "kosmos_drone::nginx",
+      "kosmos_garage",
+      "kosmos_garage::default",
+      "kosmos_garage::firewall_rpc",
+      "kosmos_garage::nginx_web",
      "kosmos_gitea::nginx",
      "kosmos_website",
      "kosmos_website::default",
      "kosmos-akkounts::nginx_api",
+      "kosmos-bitcoin::nginx_lndhub",
      "kosmos-ejabberd::nginx",
      "kosmos-hubot::nginx_botka_irc-libera-chat",
      "kosmos-hubot::nginx_hal8000_xmpp",
      "kosmos-ipfs::nginx_public_gateway",
      "kosmos-mastodon::nginx",
      "remotestorage_discourse::nginx",
-      "kosmos_garage",
-      "kosmos_garage::default",
-      "kosmos_garage::firewall_rpc",
-      "kosmos_garage::nginx_web",
      "kosmos_zerotier::controller",
      "kosmos_zerotier::firewall",
      "kosmos_zerotier::zncui",
@ -73,11 +74,11 @@
      "nginx::commons_conf",
      "kosmos-nginx::firewall",
      "discourse::nginx",
+      "firewall::default",
+      "chef-sugar::default",
      "git::default",
      "git::package",
      "kosmos-base::letsencrypt",
-      "firewall::default",
-      "chef-sugar::default",
      "fail2ban::default"
    ],
    "platform": "ubuntu",
--- a/nodes/gitea-2.json
+++ b/nodes/gitea-2.json
@ -64,6 +64,7 @@
  "run_list": [
    "role[base]",
    "role[kvm_guest]",
+    "role[ldap_client]",
    "role[garage_gateway]",
    "role[gitea]"
  ]
--- a/nodes/ldap-3.kosmos.org.json
+++ b/nodes/ldap-3.kosmos.org.json
@ -59,6 +59,6 @@
  "run_list": [
    "recipe[kosmos-base]",
    "role[kvm_guest]",
-    "role[dirsrv_primary]"
+    "role[dirsrv_supplier]"
  ]
-}
+}
--- a/nodes/ldap-4.kosmos.org.json
+++ b/nodes/ldap-4.kosmos.org.json
@ -0,0 +1,57 @@
+{
+  "name": "ldap-4.kosmos.org",
+  "normal": {
+    "knife_zero": {
+      "host": "10.1.1.106"
+    }
+  },
+  "automatic": {
+    "fqdn": "ldap-4.kosmos.org",
+    "os": "linux",
+    "os_version": "5.4.0-1079-kvm",
+    "hostname": "ldap-4",
+    "ipaddress": "192.168.122.73",
+    "roles": [
+      "base",
+      "kvm_guest"
+    ],
+    "recipes": [
+      "kosmos-base",
+      "kosmos-base::default",
+      "kosmos_kvm::guest",
+      "apt::default",
+      "timezone_iii::default",
+      "timezone_iii::debian",
+      "ntp::default",
+      "ntp::apparmor",
+      "kosmos-base::systemd_emails",
+      "apt::unattended-upgrades",
+      "kosmos-base::firewall",
+      "kosmos-postfix::default",
+      "postfix::default",
+      "postfix::_common",
+      "postfix::_attributes",
+      "postfix::sasl_auth",
+      "hostname::default"
+    ],
+    "platform": "ubuntu",
+    "platform_version": "20.04",
+    "cloud": null,
+    "chef_packages": {
+      "chef": {
+        "version": "17.10.3",
+        "chef_root": "/opt/chef/embedded/lib/ruby/gems/3.0.0/gems/chef-17.10.3/lib",
+        "chef_effortless": null
+      },
+      "ohai": {
+        "version": "17.9.0",
+        "ohai_root": "/opt/chef/embedded/lib/ruby/gems/3.0.0/gems/ohai-17.9.0/lib/ohai"
+      }
+    }
+  },
+  "run_list": [
+    "role[base]",
+    "role[kvm_guest]",
+    "role[dirsrv_supplier]"
+  ]
+}
--- a/nodes/postgres-2.json
+++ b/nodes/postgres-2.json
@ -21,8 +21,10 @@
      "kosmos_kvm::guest",
      "kosmos_postgresql::primary",
      "kosmos_postgresql::firewall",
-      "kosmos_gitea::pg_db",
+      "kosmos-bitcoin::lndhub-go_pg_db",
      "kosmos_drone::pg_db",
+      "kosmos_gitea::pg_db",
+      "kosmos-mastodon::pg_db",
      "apt::default",
      "timezone_iii::default",
      "timezone_iii::debian",
--- a/nodes/wiki-1.json
+++ b/nodes/wiki-1.json
@ -74,8 +74,9 @@
    }
  },
  "run_list": [
-    "recipe[kosmos-base]",
+    "role[base]",
    "role[kvm_guest]",
+    "role[ldap_client]",
    "recipe[kosmos-mediawiki]"
  ]
-}
+}
--- a/roles/bitcoind.rb
+++ b/roles/bitcoind.rb
@ -0,0 +1,5 @@
+name "bitcoind"
+
+run_list %w(
+  kosmos-bitcoin::bitcoind
+)
--- a/roles/cln.rb
+++ b/roles/cln.rb
@ -0,0 +1,5 @@
+name "cln"
+
+run_list %w(
+  kosmos-bitcoin::c-lightning
+)
--- a/roles/dirsrv_supplier.rb
+++ b/roles/dirsrv_supplier.rb
@ -1,4 +1,4 @@
-name "dirsrv_primary"
+name "dirsrv_supplier"

 run_list %w(
  recipe[kosmos-dirsrv]
--- a/roles/hubot.rb
+++ b/roles/hubot.rb
@ -7,6 +7,6 @@ default_run_list = %w(

 env_run_lists(
  '_default' => default_run_list,
-  'development' => [],
+  'development' => default_run_list,
  'production' => default_run_list
 )
--- a/roles/ldap_client.rb
+++ b/roles/ldap_client.rb
@ -0,0 +1,5 @@
+name "ldap_client"
+
+run_list %w(
+  kosmos-dirsrv::hostsfile
+)
--- a/roles/lnd.rb
+++ b/roles/lnd.rb
@ -0,0 +1,9 @@
+name "lnd"
+
+run_list %w(
+  kosmos-bitcoin::lnd
+  kosmos-bitcoin::lnd-scb-s3
+  kosmos-bitcoin::boltz
+  kosmos-bitcoin::rtl
+  kosmos-bitcoin::peerswap-lnd
+)
--- a/roles/lndhub.rb
+++ b/roles/lndhub.rb
@ -0,0 +1,7 @@
+name "lndhub"
+
+run_list %w(
+  role[postgresql_client]
+  kosmos-bitcoin::lndhub
+  kosmos-bitcoin::lndhub-go
+)
--- a/roles/nginx_proxy.rb
+++ b/roles/nginx_proxy.rb
@ -18,18 +18,19 @@ default_run_list = %w(
  kosmos_assets::nginx_site
  kosmos_discourse::nginx
  kosmos_drone::nginx
+  kosmos_garage::default
+  kosmos_garage::firewall_rpc
+  kosmos_garage::nginx_web
  kosmos_gitea::nginx
  kosmos_website::default
  kosmos-akkounts::nginx_api
+  kosmos-bitcoin::nginx_lndhub
  kosmos-ejabberd::nginx
  kosmos-hubot::nginx_botka_irc-libera-chat
  kosmos-hubot::nginx_hal8000_xmpp
  kosmos-ipfs::nginx_public_gateway
  kosmos-mastodon::nginx
  remotestorage_discourse::nginx
-  kosmos_garage::default
-  kosmos_garage::firewall_rpc
-  kosmos_garage::nginx_web
 )

 env_run_lists(
--- a/roles/postgresql_primary.rb
+++ b/roles/postgresql_primary.rb
@ -3,7 +3,8 @@ name "postgresql_primary"
 run_list %w(
  kosmos_postgresql::primary
  kosmos_postgresql::firewall
-  kosmos_gitea::pg_db
+  kosmos-bitcoin::lndhub-go_pg_db
  kosmos_drone::pg_db
+  kosmos_gitea::pg_db
  kosmos-mastodon::pg_db
 )
--- a/site-cookbooks/kosmos-akkounts/attributes/default.rb
+++ b/site-cookbooks/kosmos-akkounts/attributes/default.rb
@ -1,5 +1,5 @@
 node.default['akkounts']['repo'] = 'https://gitea.kosmos.org/kosmos/akkounts.git'
-node.default['akkounts']['revision'] = 'master'
+node.default['akkounts']['revision'] = 'feature/73-lndhub-go'
 node.default['akkounts']['port'] = 3000
 node.default['akkounts']['domain'] = 'accounts.kosmos.org'

--- a/site-cookbooks/kosmos-akkounts/recipes/default.rb
+++ b/site-cookbooks/kosmos-akkounts/recipes/default.rb
@ -22,7 +22,6 @@ package "libpq-dev"

 include_recipe 'kosmos-nodejs'
 include_recipe "kosmos-redis"
-include_recipe "kosmos-dirsrv::hostsfile"

 npm_package "yarn" do
  version "1.22.4"
--- a/site-cookbooks/kosmos-bitcoin/attributes/default.rb
+++ b/site-cookbooks/kosmos-bitcoin/attributes/default.rb
@ -79,6 +79,26 @@ node.default['lndhub']['revision'] = 'master'
 node.default['lndhub']['port'] = '3023'
 node.default['lndhub']['domain'] = 'lndhub.kosmos.org'

+node.default['lndhub-go']['repo'] = 'https://github.com/getAlby/lndhub.go.git'
+node.default['lndhub-go']['revision'] = '0.12.0'
+node.default['lndhub-go']['source_dir'] = '/opt/lndhub-go'
+node.default['lndhub-go']['port'] = 3026
+node.default['lndhub-go']['domain'] = 'lndhub.kosmos.org'
+node.default['lndhub-go']['postgres']['database'] = 'lndhub'
+node.default['lndhub-go']['postgres']['user'] = 'lndhub'
+node.default['lndhub-go']['postgres']['port'] = 5432
+node.default['lndhub-go']['default_rate_limit'] = 20
+node.default['lndhub-go']['strict_rate_limit'] =  1
+node.default['lndhub-go']['burst_rate_limit'] =  10
+node.default['lndhub-go']['branding'] = {
+  'title' => 'LndHub - Kosmos Lightning',
+  'desc' => 'Kosmos accounts for the Lightning Network',
+  'url' => 'https://lndhub.kosmos.org',
+  'logo' => 'https://assets.kosmos.org/img/icon-lndhub-400px.png',
+  'favicon' => 'https://kosmos.org/favicon.ico',
+  'footer' => 'about=https://kosmos.org'
+}
+
 node.default['dotnet']['ms_packages_src_url'] = "https://packages.microsoft.com/config/ubuntu/20.04/packages-microsoft-prod.deb"
 node.default['dotnet']['ms_packages_src_checksum'] = "4df5811c41fdded83eb9e2da9336a8dfa5594a79dc8a80133bd815f4f85b9991"

@ -98,3 +118,7 @@ node.default["btcpay"]["domain"] = 'btcpay.kosmos.org'
 node.default['btcpay']['postgres']['port'] = 5432
 node.default['btcpay']['postgres']['database'] = 'btcpayserver'
 node.default['btcpay']['postgres']['user'] = 'satoshi'
+
+node.default['peerswap']['repo'] = 'https://github.com/ElementsProject/peerswap.git'
+node.default['peerswap']['revision'] = 'master'
+node.default['peerswap-lnd']['source_dir'] = '/opt/peerswap'
--- a/site-cookbooks/kosmos-bitcoin/metadata.rb
+++ b/site-cookbooks/kosmos-bitcoin/metadata.rb
@ -7,25 +7,15 @@ long_description 'Installs/configures bitcoin-related software'
 version '0.1.0'
 chef_version '>= 14.0'

-# The `issues_url` points to the location where issues for this cookbook are
-# tracked.  A `View Issues` link will be displayed on this cookbook's page when
-# uploaded to a Supermarket.
-#
-# issues_url 'https://github.com/<insert_org_here>/kosmos-bitcoin/issues'
-
-# The `source_url` points to the development repository for this cookbook.  A
-# `View Source` link will be displayed on this cookbook's page when uploaded to
-# a Supermarket.
-#
-# source_url 'https://github.com/<insert_org_here>/kosmos-bitcoin'
-
+depends 'application_javascript'
 depends 'ark'
 depends 'backup'
+depends 'firewall'
 depends 'git'
 depends 'golang'
 depends 'kosmos-nginx'
 depends 'kosmos-nodejs'
-depends 'firewall'
-depends 'application_javascript'
-depends 'tor-full'
+depends 'kosmos_postgresql'
+depends 'postgresql'
 depends 'redisio'
+depends 'tor-full'
--- a/site-cookbooks/kosmos-bitcoin/recipes/bitcoind.rb
+++ b/site-cookbooks/kosmos-bitcoin/recipes/bitcoind.rb
@ -1,6 +1,6 @@
 #
 # Cookbook:: kosmos-bitcoin
-# Recipe:: source
+# Recipe:: bitcoind
 #

 build_essential
--- a/site-cookbooks/kosmos-bitcoin/recipes/golang.rb
+++ b/site-cookbooks/kosmos-bitcoin/recipes/golang.rb
@ -1,6 +1,6 @@
 #
 # Cookbook:: kosmos-bitcoin
-# Recipe:: boltz
+# Recipe:: golang
 #
 # Internal recipe for managing the Go installation in one place
 #
--- a/site-cookbooks/kosmos-bitcoin/recipes/lndhub-go.rb
+++ b/site-cookbooks/kosmos-bitcoin/recipes/lndhub-go.rb
@ -0,0 +1,107 @@
+#
+# Cookbook:: kosmos-bitcoin
+# Recipe:: lndhub-go
+#
+
+include_recipe 'git'
+include_recipe 'kosmos-bitcoin::golang'
+include_recipe 'kosmos-bitcoin::user'
+
+bitcoin_user  = node['bitcoin']['username']
+bitcoin_group = node['bitcoin']['usergroup']
+lnd_dir       = node['lnd']['lnd_dir']
+lncli_bin     = '/opt/go/bin/lncli'
+source_dir    = node['lndhub-go']['source_dir']
+macaroon_path = "#{lnd_dir}/data/lndhub.macaroon"
+credentials   = data_bag_item('credentials', 'lndhub-go')
+postgres_host = "pg.kosmos.local"
+postgres_user = node['lndhub-go']['postgres']['user']
+postgres_db   = node['lndhub-go']['postgres']['database']
+postgres_port = node['lndhub-go']['postgres']['port']
+
+git source_dir do
+  repository node['lndhub-go']['repo']
+  revision node['lndhub-go']['revision']
+  action :sync
+  notifies :run, 'bash[compile_lndhub-go]', :immediately
+end
+
+bash 'compile_lndhub-go' do
+  cwd source_dir
+  code 'make'
+  action :nothing
+  notifies :restart, 'service[lndhub-go]', :delayed
+end
+
+bash 'bake_lndhub_macaroon' do
+  user bitcoin_user
+  cwd lnd_dir
+  code "#{lncli_bin} bakemacaroon --save_to=./data/lndhub.macaroon info:read invoices:read invoices:write offchain:read offchain:write"
+  not_if { File.exist?(macaroon_path) }
+end
+
+template "#{source_dir}/.env" do
+  source 'lndhub-go.env.erb'
+  owner bitcoin_user
+  group bitcoin_group
+  mode 0600
+  sensitive true
+  variables config: {
+    database_uri: "postgresql://#{postgres_user}:#{credentials['postgresql_password']}@#{postgres_host}:#{postgres_port}/#{postgres_db}?sslmode=disable",
+    jwt_secret: credentials['jwt_secret'],
+    lnd_address: 'localhost:10009', # gRPC address,
+    lnd_macaroon_file: macaroon_path,
+    lnd_cert_file:  "#{lnd_dir}/tls.cert",
+    custom_name: node['lndhub-go']['domain'],
+    port: node['lndhub-go']['port'],
+    admin_token: credentials['admin_token'],
+    default_rate_limit: node['lndhub-go']['default_rate_limit'],
+    strict_rate_limit: node['lndhub-go']['strict_rate_limit'],
+    burst_rate_limit: node['lndhub-go']['burst_rate_limit'],
+    branding: node['lndhub-go']['branding']
+  }
+  notifies :restart, 'service[lndhub-go]', :delayed
+end
+
+systemd_unit 'lndhub-go.service' do
+  content({
+    Unit: {
+      Description: 'LndHub compatible API written in Go',
+      Documentation: ['https://github.com/getAlby/lndhub.go/blob/main/README.md'],
+      Requires: 'lnd.service',
+      After: 'lnd.service'
+    },
+    Service: {
+      User: bitcoin_user,
+      Group: bitcoin_group,
+      Type: 'simple',
+      WorkingDirectory: source_dir,
+      ExecStart: "#{source_dir}/lndhub",
+      Restart: 'always',
+      RestartSec: '10',
+      TimeoutSec: '60',
+      PrivateTmp: true,
+      ProtectSystem: 'full',
+      NoNewPrivileges: true,
+      PrivateDevices: true,
+      MemoryDenyWriteExecute: true
+    },
+    Install: {
+      WantedBy: 'multi-user.target'
+    }
+  })
+  verify false
+  triggers_reload true
+  action [:create, :enable, :start]
+end
+
+service 'lndhub-go' do
+  action :nothing
+end
+
+firewall_rule 'lndhub-go' do
+  port     node['lndhub-go']['port']
+  source   '10.1.1.0/24'
+  protocol :tcp
+  command  :allow
+end
--- a/site-cookbooks/kosmos-bitcoin/recipes/lndhub-go_pg_db.rb
+++ b/site-cookbooks/kosmos-bitcoin/recipes/lndhub-go_pg_db.rb
@ -0,0 +1,19 @@
+#
+# Cookbook Name:: kosmos-bitcoin
+# Recipe:: lndhub-go_pg_db
+#
+
+credentials = data_bag_item('credentials', 'lndhub-go')
+
+postgres_user = node['lndhub-go']['postgres']['user']
+postgres_db   = node['lndhub-go']['postgres']['database']
+
+postgresql_user postgres_user do
+  action :create
+  password credentials['postgresql_password']
+end
+
+postgresql_database postgres_db do
+  owner postgres_user
+  action :create
+end
--- a/site-cookbooks/kosmos-bitcoin/recipes/lndhub.rb
+++ b/site-cookbooks/kosmos-bitcoin/recipes/lndhub.rb
@ -90,27 +90,7 @@ firewall_rule 'lndhub_private' do
  command  :allow
 end

-unless node.chef_environment == "development"
-  include_recipe "kosmos-base::letsencrypt"
-  include_recipe "kosmos-nginx"
+return if node.chef_environment == "development"

-  nginx_certbot_site node[app_name]['domain']
-
-  template "#{node['nginx']['dir']}/sites-available/#{node[app_name]['domain']}" do
-    source 'nginx_conf_lndhub.erb'
-    owner node["nginx"]["user"]
-    mode 0640
-    variables port: node[app_name]['port'],
-              server_name:  node[app_name]['domain'],
-              ssl_cert:     "/etc/letsencrypt/live/#{node[app_name]['domain']}/fullchain.pem",
-              ssl_key:      "/etc/letsencrypt/live/#{node[app_name]['domain']}/privkey.pem"
-    notifies :reload, 'service[nginx]', :delayed
-  end
-
-  nginx_site node[app_name]['domain'] do
-    action :enable
-  end
-
-  node.override["backup"]["archives"]["lndhub"] = ["/var/lib/redis/dump-6379.rdb"]
-  include_recipe "backup"
-end
+node.override["backup"]["archives"]["lndhub"] = ["/var/lib/redis/dump-6379.rdb"]
+include_recipe "backup"
--- a/site-cookbooks/kosmos-bitcoin/recipes/nginx_lndhub.rb
+++ b/site-cookbooks/kosmos-bitcoin/recipes/nginx_lndhub.rb
@ -0,0 +1,29 @@
+#
+# Cookbook:: kosmos-bitcoin
+# Recipe:: nginx_lndhub
+#
+
+include_recipe "kosmos-base::letsencrypt"
+include_recipe "kosmos-nginx"
+
+domain = node['lndhub-go']['domain']
+
+nginx_certbot_site domain
+
+upstream_host = search(:node, "role:lndhub").first["knife_zero"]["host"]
+
+template "#{node['nginx']['dir']}/sites-available/#{domain}" do
+  source 'nginx_conf_lndhub.erb'
+  owner node["nginx"]["user"]
+  mode 0640
+  variables port: node['lndhub-go']['port'],
+            server_name: domain,
+            ssl_cert: "/etc/letsencrypt/live/#{domain}/fullchain.pem",
+            ssl_key:  "/etc/letsencrypt/live/#{domain}/privkey.pem",
+            upstream_host: upstream_host
+  notifies :reload, 'service[nginx]', :delayed
+end
+
+nginx_site domain do
+  action :enable
+end
--- a/site-cookbooks/kosmos-bitcoin/recipes/peerswap-lnd.rb
+++ b/site-cookbooks/kosmos-bitcoin/recipes/peerswap-lnd.rb
@ -0,0 +1,86 @@
+#
+# Cookbook:: kosmos-bitcoin
+# Recipe:: peerswap-lnd
+#
+
+include_recipe 'git'
+include_recipe 'kosmos-bitcoin::golang'
+include_recipe 'kosmos-bitcoin::user'
+
+bitcoin_user  = node['bitcoin']['username']
+bitcoin_group = node['bitcoin']['usergroup']
+lnd_dir       = node['lnd']['lnd_dir']
+macaroon_path = "#{lnd_dir}/data/chain/bitcoin/#{node['bitcoin']['network']}/admin.macaroon"
+source_dir    = node['peerswap-lnd']['source_dir']
+config_dir    = "/home/#{bitcoin_user}/.peerswap"
+
+directory config_dir do
+  owner bitcoin_user
+  group bitcoin_group
+  mode '0700'
+  action :create
+end
+
+git source_dir do
+  repository node['peerswap']['repo']
+  revision node['peerswap']['revision']
+  action :sync
+  notifies :run, 'bash[compile_peerswap]', :immediately
+end
+
+bash 'compile_peerswap' do
+  cwd source_dir
+  environment 'GOPATH' => '/opt/go'
+  code 'make lnd-release'
+  action :run
+  notifies :restart, 'service[peerswap]', :delayed
+end
+
+template "#{config_dir}/peerswap.conf" do
+  source 'peerswap-lnd.conf.erb'
+  owner bitcoin_user
+  group bitcoin_group
+  mode 0600
+  sensitive true
+  variables config: {
+    tlscertpath:  "#{lnd_dir}/tls.cert",
+    macaroonpath: macaroon_path
+  }
+  notifies :restart, 'service[peerswap]', :delayed
+end
+
+systemd_unit 'peerswap.service' do
+  content({
+    Unit: {
+      Description: 'PeerSwap Lightning channel balancing',
+      Documentation: ['https://github.com/ElementsProject/peerswap'],
+      Requires: 'lnd.service',
+      After: 'lnd.service'
+    },
+    Service: {
+      User: bitcoin_user,
+      Group: bitcoin_group,
+      Type: 'simple',
+      WorkingDirectory: source_dir,
+      ExecStart: "/opt/go/bin/peerswapd",
+      Restart: 'always',
+      RestartSec: '10',
+      TimeoutSec: '60',
+      PrivateTmp: true,
+      ProtectSystem: 'full',
+      NoNewPrivileges: true,
+      PrivateDevices: true,
+      MemoryDenyWriteExecute: true
+    },
+    Install: {
+      WantedBy: 'multi-user.target'
+    }
+  })
+  verify false
+  triggers_reload true
+  action [:create, :enable, :start]
+end
+
+service 'peerswap' do
+  action :nothing
+end
--- a/site-cookbooks/kosmos-bitcoin/templates/lndhub-go.env.erb
+++ b/site-cookbooks/kosmos-bitcoin/templates/lndhub-go.env.erb
@ -0,0 +1,9 @@
+<% @config.each do |key, value| %>
+<% if value.is_a?(Hash) %>
+<% value.each do |k, v| %>
+<%= "#{key.upcase}_#{k.upcase}" %>=<%= v.to_s %>
+<% end %>
+<% else %>
+<%= key.upcase %>=<%= value.to_s %>
+<% end %>
+<% end %>
--- a/site-cookbooks/kosmos-bitcoin/templates/nginx_conf_lndhub.erb
+++ b/site-cookbooks/kosmos-bitcoin/templates/nginx_conf_lndhub.erb
@ -2,10 +2,9 @@
 # Generated by Chef
 #
 upstream _lndhub {
-  server localhost:<%= @port %>;
+  server <%= @upstream_host || "localhost" %>:<%= @port %>;
 }

-<% if File.exist?(@ssl_cert) && File.exist?(@ssl_key) -%>
 server {
  listen 443 ssl http2;
  server_name <%= @server_name %>;
@ -16,10 +15,13 @@ server {
  error_log <%= node[:nginx][:log_dir] %>/<%= @server_name %>.error.log warn;

  location / {
+    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
+    proxy_set_header X-Forwarded-Proto https;
+    proxy_set_header Host $http_host;
+    proxy_redirect off;
    proxy_pass http://_lndhub;
-   }
+  }

  ssl_certificate <%= @ssl_cert %>;
  ssl_certificate_key <%= @ssl_key %>;
 }
-<% end -%>
--- a/site-cookbooks/kosmos-bitcoin/templates/peerswap-lnd.conf.erb
+++ b/site-cookbooks/kosmos-bitcoin/templates/peerswap-lnd.conf.erb
@ -0,0 +1,3 @@
+<% @config.each do |k, v| %>
+<%= "lnd.#{k}=#{v}" %>
+<% end %>
--- a/site-cookbooks/kosmos-dirsrv/recipes/default.rb
+++ b/site-cookbooks/kosmos-dirsrv/recipes/default.rb
@ -3,12 +3,15 @@
 # Recipe:: default
 #

-include_recipe "kosmos-dirsrv::hostsfile"
-
 credentials = data_bag_item("credentials", "dirsrv")
+local_hostname = "#{node["hostname"]}.kosmos.local"
+
+hostsfile_entry "127.0.0.1" do
+  hostname local_hostname
+end

 dirsrv_instance "master" do
-  hostname "ldap.kosmos.local"
+  hostname local_hostname
  admin_password credentials['admin_password']
  suffix "dc=kosmos,dc=org"
 end
--- a/site-cookbooks/kosmos-dirsrv/recipes/hostsfile.rb
+++ b/site-cookbooks/kosmos-dirsrv/recipes/hostsfile.rb
@ -3,12 +3,12 @@
 # Recipe:: hostsfile
 #

-dirsrv_primary = search(:node, "role:dirsrv_primary AND chef_environment:#{node.chef_environment}").first
+dirsrv_supplier = search(:node, "role:dirsrv_supplier AND chef_environment:#{node.chef_environment}").first

-unless dirsrv_primary.nil?
-  primary_ip = dirsrv_primary['knife_zero']['host']
+unless dirsrv_supplier.nil?
+  supplier_ip = dirsrv_supplier['knife_zero']['host']

-  hostsfile_entry primary_ip do
+  hostsfile_entry supplier_ip do
    hostname  "ldap.kosmos.local"
    unique    true
  end
--- a/site-cookbooks/kosmos-ejabberd/recipes/default.rb
+++ b/site-cookbooks/kosmos-ejabberd/recipes/default.rb
@ -3,8 +3,6 @@
 # Recipe:: default
 #

-include_recipe "kosmos-dirsrv::hostsfile"
-
 ejabberd_credentials = data_bag_item("credentials", "ejabberd")

 ejabberd_version = node["kosmos-ejabberd"]["version"]
--- a/site-cookbooks/kosmos-mastodon/recipes/default.rb
+++ b/site-cookbooks/kosmos-mastodon/recipes/default.rb
@ -70,7 +70,7 @@ npm_package "yarn" do
  version "1.22.4"
 end

-ruby_version = "3.0.3"
+ruby_version = "3.0.4"

 execute "systemctl daemon-reload" do
  command "systemctl daemon-reload"
@ -192,7 +192,6 @@ application mastodon_path do
  end

  execute 'rake db:migrate' do
-    # environment "RAILS_ENV" => "production", "HOME" => mastodon_path#, "SKIP_POST_DEPLOYMENT_MIGRATIONS" => "true"
    environment "RAILS_ENV" => "production", "HOME" => mastodon_path, "SKIP_POST_DEPLOYMENT_MIGRATIONS" => "true"
    user mastodon_user
    group mastodon_user
--- a/site-cookbooks/kosmos-mastodon/templates/default/mastodon-sidekiq.systemd.service.erb
+++ b/site-cookbooks/kosmos-mastodon/templates/default/mastodon-sidekiq.systemd.service.erb
@ -10,7 +10,7 @@ WorkingDirectory=<%= @app_dir %>
 Environment="RAILS_ENV=production"
 Environment="DB_POOL=50"
 Environment="LD_PRELOAD=/usr/lib/x86_64-linux-gnu/libjemalloc.so.2"
-ExecStart=<%= @bundle_path %> exec sidekiq -c <%= @sidekiq_threads %> -q default -q mailers -q pull -q push
+ExecStart=<%= @bundle_path %> exec sidekiq -c <%= @sidekiq_threads %> -q default -q mailers -q pull -q push -q ingress
 TimeoutSec=15
 Restart=always

--- a/site-cookbooks/kosmos-mediawiki/recipes/default.rb
+++ b/site-cookbooks/kosmos-mediawiki/recipes/default.rb
@ -27,7 +27,6 @@
 include_recipe 'apt'
 include_recipe 'ark'
 include_recipe 'composer'
-include_recipe 'kosmos-dirsrv::hostsfile'

 server_name = 'wiki.kosmos.org'

--- a/site-cookbooks/kosmos_gitea/attributes/default.rb
+++ b/site-cookbooks/kosmos_gitea/attributes/default.rb
@ -1,7 +1,7 @@
-gitea_version = "1.17.3"
+gitea_version = "1.18.0"
 node.default["gitea"]["version"] = gitea_version
 node.default["gitea"]["binary_url"] = "https://dl.gitea.io/gitea/#{gitea_version}/gitea-#{gitea_version}-linux-amd64"
-node.default["gitea"]["binary_checksum"] = "38c4e1228cd051b785c556bcadc378280d76c285b70e8761cd3f5051aed61b5e"
+node.default["gitea"]["binary_checksum"] = "b45b715d519a97086208c6b42528d291dd1c4dfdf40321dc940030e1cf3de6e6"
 node.default["gitea"]["working_directory"] = "/var/lib/gitea"
 node.default["gitea"]["port"] = 3000
 node.default["gitea"]["postgresql_host"] = "localhost:5432"
--- a/site-cookbooks/kosmos_gitea/recipes/default.rb
+++ b/site-cookbooks/kosmos_gitea/recipes/default.rb
@ -3,8 +3,6 @@
 # Recipe:: default
 #

-include_recipe "kosmos-dirsrv::hostsfile"
-
 working_directory         = node["gitea"]["working_directory"]
 git_home_directory        = "/home/git"
 repository_root_directory = "#{git_home_directory}/gitea-repositories"
@ -12,6 +10,8 @@ config_directory          = "/etc/gitea"
 gitea_binary_path         = "/usr/local/bin/gitea"
 gitea_data_bag_item       = data_bag_item("credentials", "gitea")
 smtp_credentials          = data_bag_item("credentials", "smtp")
+smtp_addr                 = smtp_credentials["relayhost"].split(":")[0]
+smtp_port                 = smtp_credentials["relayhost"].split(":")[1]
 jwt_secret                = gitea_data_bag_item["jwt_secret"]
 internal_token            = gitea_data_bag_item["internal_token"]
 secret_key                = gitea_data_bag_item["secret_key"]
@ -86,7 +86,8 @@ config_variables = {
  secret_key: secret_key,
  postgresql_host: node["gitea"]["postgresql_host"],
  postgresql_password: gitea_data_bag_item["postgresql_password"],
-  smtp_host: smtp_credentials["relayhost"],
+  smtp_addr: smtp_addr,
+  smtp_port: smtp_port,
  smtp_user: smtp_credentials["user_name"],
  smtp_password: smtp_credentials["password"],
  config: node["gitea"]["config"],
--- a/site-cookbooks/kosmos_gitea/templates/default/app.ini.erb
+++ b/site-cookbooks/kosmos_gitea/templates/default/app.ini.erb
@ -39,10 +39,12 @@ COOKIE_SECURE = true

 [mailer]
 ENABLED = true
-HOST = <%= @smtp_host %>
-FROM = gitea@kosmos.org
+PROTOCOL = smtp+startls
+SMTP_ADDR = <%= @smtp_addr %>
+SMTP_PORT = <%= @smtp_port %>
 USER = <%= @smtp_user %>
 PASSWD = <%= @smtp_password %>
+FROM = gitea@kosmos.org

 [security]
 INTERNAL_TOKEN = <%= @internal_token %>
--- a/site-cookbooks/kosmos_postgresql/libraries/helpers.rb
+++ b/site-cookbooks/kosmos_postgresql/libraries/helpers.rb
@ -1,7 +1,7 @@
 class Chef
  class Recipe
    def postgresql_primary
-      postgresql_primary = search(:node, "role:postgresql_primary AND chef_environment:#{node.chef_environment}").first
+      postgresql_primary = search(:node, "role:postgresql_primary").first

      unless postgresql_primary.nil?
        primary_ip = ip_for(postgresql_primary)
Author	SHA1	Message	Date
Râu Cao	345ba14f0e	Add garage S3 config for Mastodon	2023-01-14 15:03:38 +08:00
Râu Cao	2eb0544ea9	Update README	2023-01-05 17:14:46 +08:00
raucao	2d50c7dfd6	Merge pull request 'Upgrade Gitea to 1.18.0' (#460 ) from chore/upgrade_gitea into master Reviewed-on: #460	2022-12-30 16:08:18 +00:00
Râu Cao	e54112418e	Upgrade Gitea 1.18.0 Requires some config changes for the mailer.	2022-12-30 23:05:40 +07:00
raucao	0c37504ac6	Merge pull request 'Set up lndhub.go' (#457 ) from feature/454-lndhub.go into master Reviewed-on: #457	2022-12-27 06:29:09 +00:00
raucao	d8e0124c46	Merge pull request 'Set up PeerSwap for LND' (#459 ) from feature/peerswap into feature/454-lndhub.go Reviewed-on: #459	2022-12-27 06:23:09 +00:00
Râu Cao	176dd64438	Remove peerswap policy file from recipe This will be auto-created anyway, and we don't want to overwrite changes added by the CLI.	2022-12-26 11:29:17 +07:00
Râu Cao	4f1b1aff30	Set up PeerSwap Allows to swap sats in and out of Lightning channels without a 3rd party (and their fees). Instead, swaps can be initiated directly with the channel peer. https://www.peerswap.dev/	2022-12-26 11:16:22 +07:00
Râu Cao	b3465e186f	Fix comment	2022-12-26 11:16:01 +07:00
Râu Cao	ea635a52e9	Formatting	2022-12-26 11:14:40 +07:00
Râu Cao	90e17b0abc	Rename bitcoind recipe Was still using a name from when the cookbook didn't set up anything else	2022-12-25 16:28:14 +07:00
Râu Cao	8c8e978ae9	Update node configs	2022-12-24 00:58:31 +07:00
Râu Cao	3d7b4df376	Add rate limit config for lndhub-go	2022-12-24 00:58:11 +07:00
Râu Cao	b738dc1e80	Add nginx proxy hosts for Garage Web access The respective bucket needs to be configured with a domain alias. When a new alias is added to the `s3_web_domains` config, a new nginx site can then be deployed to the `nginx_proxy` hosts.	2022-12-23 18:07:39 +07:00
Râu Cao	3641ea7a60	Deploy lndhub.go branch of akkounts	2022-12-23 18:02:42 +07:00
Râu Cao	a7e04f4e63	Exclude lndhub backups in dev	2022-12-23 14:17:43 +07:00
Râu Cao	e0c400c007	Use correct asset URL for lndhub logo	2022-12-22 20:03:58 +07:00
Râu Cao	7802ea25e6	Ignore chef environment when looking up primary We use mixed environments still, not everything is in "production" yet.	2022-12-22 19:45:45 +07:00
Râu Cao	fb1206d03f	Refactor bitcoin-related roles and node config	2022-12-22 19:35:41 +07:00
Râu Cao	379a503dd0	Move lndhub nginx site to proxy And configure for lndhub-go. Also configure branding for public lndhub dashboard	2022-12-22 19:35:30 +07:00
Râu Cao	7d11450c4e	Set up lndhub.go closes #454	2022-12-11 14:30:27 +01:00
greg	430f8b36b6	Merge pull request 'Set up LDAP/dirsrv replication' (#453 ) from feature/ldap_replication into master Reviewed-on: #453	2022-12-08 13:12:22 +00:00
greg	26a34a69d3	Merge branch 'master' into feature/ldap_replication	2022-12-08 13:08:00 +00:00
greg	98785a1a86	Merge pull request 'Move Mastodon nginx to proxy/LB' (#452 ) from feature/mastodon_nginx into master Reviewed-on: #452	2022-12-08 13:07:35 +00:00
greg	66c91b5a7f	Merge branch 'master' into feature/mastodon_nginx	2022-12-08 13:07:09 +00:00
Râu Cao	a460302728	Add missing sidekiq queue	2022-12-01 17:08:17 +01:00
greg	f8fd10c4c2	Merge pull request 'Upgrade Mastodon to 4.0' (#456 ) from chore/upgrade_mastodon into feature/mastodon_nginx Reviewed-on: #456	2022-12-01 14:33:20 +00:00
Râu Cao	2ecb4e2385	Upgrade Ruby to 3.0.4	2022-12-01 15:29:38 +01:00
Râu Cao	cdd3f026c4	Always use config for skipping post-deployment migrations	2022-12-01 15:05:24 +01:00
greg	83fea1026c	Merge pull request 'Install/configure Garage (S3-compatible object storage cluster)' (#450 ) from feature/garage into master Reviewed-on: #450	2022-11-28 17:38:41 +00:00
Râu Cao	c3e98688fd	Deploy second dirsrv supplier Manually configured (once) to replicate data from and to the first supplier on `ldap-3`.	2022-11-26 16:47:55 +01:00
Râu Cao	2958ba4b81	Use *.kosmos.local hostnames for LDAP nodes	2022-11-26 16:47:28 +01:00
Râu Cao	991458208d	Use a role for configuring LDAP hostname on clients This way it's also easy to converge all LDAP clients at once.	2022-11-26 16:45:45 +01:00
Râu Cao	8d4db7290e	Rename dirsrv_primary role The term used in 389 docs is "supplier" instead (ex "master")	2022-11-26 16:44:05 +01:00