search(:node, 'role:hubot') also matched retired nodes outside production
(e.g. nodejs-4 in _default), so the vhosts could proxy to a dead host and
return 502. Restrict the search to chef_environment:production and pick
deterministically.
The per-bot firewall_rule declared inside the kosmos_hubot_bot custom
resource was not collected by the ufw provider when it rebuilt the rules
file, so the bots' HTTP ports were never opened. Declare the rules in the
default recipe instead, matching the pattern used by the other cookbooks,
and drop the now-unused firewall property from the resource.
hubot-auth is CoffeeScript-only (needs porting) and nothing in the bots calls
robot.auth; remove it from hal8000's external scripts and drop HUBOT_AUTH_ADMIN
from both bots.
Replace the per-bot and per-vhost recipes with a single data-driven setup.
- resources/bot.rb (kosmos_hubot_bot): git checkout, npm install, generated
external-scripts.json, extra files, and one hardened systemd unit for every
hubot (Type=simple, Restart=always, RestartSec=2, TimeoutSec=10, PrivateTmp,
ProtectSystem=full, NoNewPrivileges, PrivateDevices)
- recipes/default.rb: shared setup + loop over node['kosmos-hubot']['bots'];
recipes/nginx.rb: loop over node['kosmos-hubot']['vhosts'];
recipes/hal8000_extras.rb: augments the hal8000 bot (searched IPFS/RSK hosts,
kredits .env, wallet) and declares no resources itself
- bot config and vhosts now live in attributes; secrets come from the encrypted
credentials/hubot bag via secret_env
- Node 22.23.4; kosmos_redis replaces redisio; REDIS_URL uses ?<prefix>; the
adapter is passed as -a <module>; botka gains hubot-incoming-webhook
- roles/hubot, nginx_proxy, openresty_proxy and the chef-nodes pointer updated
Move kredits-github onto Node 22.23.4 and behind the openresty reverse proxy
on the proxy hosts, replacing its own nginx + the custom haproxy entry.
- pin Node 22.23.4; run the app with HOST=0.0.0.0 / NODE_ENV=production and
open port 3000 to 10.1.1.0/24 so the proxy can reach it over ZeroTier
- create /opt/kredits-github owned by the deploy user before cloning
- add kredits-github::nginx_proxy (tls_cert_for + openresty_site, upstream
from role:kredits_github) and register it in the openresty_proxy run list
- drop the app-node nginx recipe/template and its role entry
Pin v0.15.15 and set blockExplorerUrl (config-only since 0.15.14). Discard
local checkout changes before the pinned checkout, and use
`npm ci --omit=dev --legacy-peer-deps` per upstream guidance.
Bump Node to patched releases (mastodon/sockethub 24.22.1, RTL 22.23.4).
Force the pinned major's latest patch in kosmos-nodejs: nodejs_install only
selects the repo major and a bare `package nodejs` never upgrades an already
installed package.
For hosts stuck on Node 16 (no patched 16.x), run the services with
NODE_OPTIONS=--use-openssl-ca so they use the patched system trust store:
ipfs-pinner, kredits-github, hubot botka/hal8000, sockethub.
cacert.org is intermittently unreachable, which fails bootstraps at the
remote_file step. Ship both roots in the cookbook and install them with
cookbook_file; contents match what was already installed fleet-wide.
Resolve the client config dir at runtime (Chef::Config[:config_file] /
:client_d_dir) instead of hardcoding /etc/chef, and create client.d when
missing so fresh nodes work.
Reload only the attributes file we just wrote, not the whole client.rb:
reloading client.rb reset chef_server_url and dropped knife-zero's -S (TCP)
override, so later server calls fell back to socketless chefzero and failed
with No socketless Cinc Zero server.
The migration chain starts mastodon-search-deploy.service immediately, but the systemd_unit resource was declared later in the recipe, so the unit did not exist yet and systemctl failed with 'Unit not found'.
The stamp files' lazy git call ran as root, which git rejects on the mastodon-owned repository (dubious ownership), silently writing an empty stamp. Use execute resources with the mastodon user instead so git works and the revision is recorded.
The mastodon user's home was /opt/mastodon, i.e. the clone destination, so writing the SSH deploy key into ~/.ssh made the directory non-empty and Chef's git resource silently skipped cloning (it only clones into an empty directory). Use a dedicated /home/mastodon home for the user and keep the clone destination separate. Also clear a non-git (partial) clone destination before cloning.
Bundler 4 (shipped with Ruby 4) removed --deployment. Use 'bundle config set --local deployment true' (frozen + vendor/bundle) and the 'without' setting instead, in the Mastodon, akkounts and liquor-cabinet cookbooks.
The repository is private, so use git@gitea.kosmos.org with a read-only deploy key stored in credentials/mastodon (repo_deploy_key). The pinned gitea host key is an attribute.
Split the deployment into kosmos-mastodon::build (checkout + bundle/yarn/assets, no database) and kosmos-mastodon::deploy (migrations + services), dispatched by the new 'build' attribute. Both phases are idempotent per checked-out revision via stamps, and the migration/restart/search-index chain is now triggered by the deployed revision instead of the git resource, so it still runs when the build was pre-staged.
The shared backup cookbook hardcodes postgresql-client-12, which does not exist on Ubuntu 24.04. Drop the Mastodon backup recipe and its role inclusion; the PostgreSQL standby is the safety net. The backup cookbook itself is still used by other services.
Ubuntu 24.04's system Python is externally managed (PEP 668) and refuses pip installs. Also bump LibreTranslate from 1.3.8 to 1.9.6, since the former pulls ctranslate2 2.24.0 which has no Python 3.12 wheels.
The java cookbook's openjdk recipe unconditionally adds the dead openjdk-r PPA on Ubuntu and cannot be told not to, which breaks on Ubuntu 24.04. Elasticsearch 7.x ships a bundled JDK and no JAVA_HOME is configured, so no system Java is needed.
Mastodon 4.4+ can create/upgrade the Elasticsearch indices and mappings without importing data. Do that synchronously during the deployment so search does not fail on a missing index, then populate the (potentially very long) import via a systemd unit started with --no-block so the maintenance window is not extended.
Add a kosmos-mastodon::dependencies recipe with everything needed to run Mastodon (Node, Ruby, libvips, Elasticsearch, packages) but without the app deployment. The default recipe includes it and only runs the deployment when node['kosmos-mastodon']['deploy'] is true, so a VM can be prepared ahead of a maintenance window.
Stop using the local redisio instance and connect to the external Redis cluster (redis_server role, db 2, password from the credentials data bag) instead. Remove the redisio service dependencies from the systemd units and drop the redisio cookbook dependency.
Bump the production branch to 4.7 and adjust for the changes between 4.3 and 4.7:
- Node 24.21.0 and Ruby 4.0.7 (via ruby-build v20260924)
- Redis 7.4.11 (Mastodon 4.5 requires >= 7.0)
- Replace ImageMagick with libvips (required since 4.6) and libidn11 with libidn
- Split database migrations into pre-/post-deployment phases and rebuild
the Elasticsearch accounts index mappings (required since 4.4)
- Remove the OTP_SECRET environment variable (removed in 4.4)
- Disable email subscriptions (new optional feature in 4.6) and force the
default locale (DEFAULT_LOCALE no longer overrides it since 4.4)
- Add the new fasp Sidekiq queue
- Enable corepack for yarn instead of the removed no-arg 'corepack prepare'
raucao
greg