Add chef-managed member bridge (strfry router, LDAP-only)
Add kosmos_strfry::member_bridge: a strfry router stream group that subscribes to the local submission relay (ws://localhost:7778) and writes member-authored events into the production database. dir = "down" keeps it strictly one-way, and pluginDown is the new LDAP-only policy (ldap-only-policy.ts) from akkounts extras/strfry, so membership is decided per event by LDAP with no member list. The two promotion paths (approved_sync, member_bridge) are enabled as role overrides in role[strfry_submission], not in the production environment config. Also add a render regression test for the router config and update the runbook. Refs raucao/marco#97.
This commit is contained in:
9 files changed
+277
-20
No files matched your search
@@ -16,13 +16,16 @@ are prepared but not converged.
|
||||
| Firewall for `:7778` | same node | `kosmos_strfry_submission::firewall` |
|
||||
| `wss://submit.nostr.kosmos.org` vhost + cert | openresty proxy nodes | `kosmos_strfry_submission::nginx` (via `role[openresty_proxy]`) |
|
||||
| Approved-event sync (staging `:7779` → production DB) | production relay node | `kosmos_strfry::approved_sync` (via `role[strfry]`) |
|
||||
| Node 24 LTS for both write policies | relay nodes | role-scoped `kosmos_nodejs.version` override |
|
||||
| Member bridge (`:7778` → production DB, LDAP-only) | production relay node | `kosmos_strfry::member_bridge` (via `role[strfry]`) |
|
||||
| Node 24 LTS for the Node write policies | relay nodes | role-scoped `kosmos_nodejs.version` override |
|
||||
|
||||
Two write policies from `kosmos/akkounts` `extras/strfry/` are fetched at
|
||||
converge time:
|
||||
Write policies from `kosmos/akkounts` `extras/strfry/` are fetched at converge
|
||||
time:
|
||||
|
||||
* `submission-policy.mjs` — open write gates for the submission relay.
|
||||
* `sync-policy.mjs` — fail-closed policy for `strfry sync` (see below).
|
||||
* `submission-policy.mjs` — open write gates for the submission relay (Node).
|
||||
* `sync-policy.mjs` — fail-closed policy for `strfry sync` (Node; see below).
|
||||
* `ldap-only-policy.ts` — LDAP membership gate for the member bridge (Deno; see
|
||||
below).
|
||||
|
||||
## Design notes / invariants
|
||||
|
||||
@@ -56,13 +59,32 @@ converge time:
|
||||
route it through nginx). The member bridge uses loopback and is unaffected.
|
||||
* **No `--filter` on the sync.** The worker only publishes approved events to
|
||||
the staging relay; the policy is the gate.
|
||||
* **The member bridge is a live `strfry router`, not a service.** A stream group
|
||||
with `dir = "down"` subscribes to `ws://localhost:7778` and writes
|
||||
member-authored events into the production LMDB; its `pluginDown` is
|
||||
`ldap-only-policy.ts`, so membership is decided per event by LDAP and no member
|
||||
list is maintained. The group's `filter` bounds the REQ (and therefore the LDAP
|
||||
lookups) to the submission kinds. `dir = "down"` makes it strictly one-way: the
|
||||
router never watches or forwards the local DB. `pluginDown`'s path contains no
|
||||
spaces, so strfry watches its mtime and hot-reloads it without a restart. It
|
||||
writes to the same LMDB as the relay (the same multi-process access pattern as
|
||||
the approved sync).
|
||||
* **Both promotion paths are enabled from `role[strfry_submission]`.** The
|
||||
`strfry.approved_sync.enabled` and `strfry.member_bridge.enabled` switches are
|
||||
set as role overrides, not in the production environment, so they travel with
|
||||
this feature's roles. (The recipes themselves live in `role[strfry]`; the relay
|
||||
node runs both roles.)
|
||||
|
||||
## Attributes
|
||||
|
||||
| Attribute | Default | Notes |
|
||||
| --- | --- | --- |
|
||||
| `strfry.staging_relay_url` | `ws://10.1.1.183:7779` | nuc ZeroTier IP + staging port |
|
||||
| `strfry.approved_sync.enabled` | `false` | **no-op until enabled** |
|
||||
| `strfry.approved_sync.enabled` | `false` | **no-op until enabled** (`true` in `role[strfry_submission]`) |
|
||||
| `strfry.member_bridge.enabled` | `false` | **no-op until enabled** (`true` in `role[strfry_submission]`) |
|
||||
| `strfry.member_bridge.subscribe_relay` | `ws://localhost:7778` | local submission relay |
|
||||
| `strfry.member_bridge.plugin_cmd` | `/opt/strfry/ldap-only-policy.ts` | LDAP-only `pluginDown` |
|
||||
| `strfry.member_bridge.kinds` | `[0, 3, 5, 360, 30360]` | router REQ kind filter |
|
||||
| `strfry_submission.domain` | `submit.nostr.kosmos.org` | |
|
||||
| `strfry_submission.port` | `7778` | |
|
||||
| `strfry_submission.config.db_params.mapsize` | `4 GiB` | disk cap for the disposable DB |
|
||||
@@ -79,17 +101,27 @@ server nor a converge — only the `chef` gem and stdlib ERB:
|
||||
ruby site-cookbooks/kosmos_strfry/test/strfry_sync_conf_test.rb
|
||||
```
|
||||
|
||||
`site-cookbooks/kosmos_strfry/test/member_bridge_conf_test.rb` renders
|
||||
`member_bridge.conf.erb` with the attribute shapes `member_bridge.rb` passes and
|
||||
fails if the stream group is not one-way (`dir != "down"`) or not gated by the
|
||||
LDAP-only policy (missing/wrong `pluginDown`, or a plugin path containing a
|
||||
space, which strfry would not hot-reload). Only stdlib ERB is needed:
|
||||
|
||||
```
|
||||
ruby site-cookbooks/kosmos_strfry/test/member_bridge_conf_test.rb
|
||||
```
|
||||
|
||||
## Deployment order
|
||||
|
||||
0. **HARD PRECONDITION — akkounts `live` branch.** Before *any* converge, the
|
||||
`live` branch of `kosmos/akkounts` must contain **both**
|
||||
`extras/strfry/submission-policy.mjs` and `extras/strfry/sync-policy.mjs`.
|
||||
Chef fetches them from
|
||||
`live` branch of `kosmos/akkounts` must contain
|
||||
`extras/strfry/submission-policy.mjs`, `extras/strfry/sync-policy.mjs` **and**
|
||||
`extras/strfry/ldap-only-policy.ts`. Chef fetches them from
|
||||
`https://gitea.kosmos.org/kosmos/akkounts/raw/branch/live/extras/strfry/…` at
|
||||
converge time; if either is missing, the `remote_file` resource fails and the
|
||||
converge time; if any is missing, the `remote_file` resource fails and the
|
||||
converge aborts (with a partially configured host). Verify with:
|
||||
`curl -fsSI https://gitea.kosmos.org/kosmos/akkounts/raw/branch/live/extras/strfry/submission-policy.mjs`
|
||||
and the same for `sync-policy.mjs`. These files are produced by WS1, not by
|
||||
and the same for the other two files. These files are produced by WS1, not by
|
||||
this repo.
|
||||
1. **DNS.** `submit.nostr.kosmos.org` is a **CNAME to `lb.kosmos.org`** at Gandi
|
||||
(already configured). Chef does not manage DNS; the `gandi_dns` Let's Encrypt
|
||||
@@ -101,15 +133,21 @@ ruby site-cookbooks/kosmos_strfry/test/strfry_sync_conf_test.rb
|
||||
proxy nodes should already be registered as Chef nodes.
|
||||
3. **Proxy nodes.** Converge `role[openresty_proxy]`; this adds the
|
||||
`submit.nostr.kosmos.org` site and requests the certificate.
|
||||
4. **Enable the sync.** Only after the worker + staging relay (WS2) are live:
|
||||
set `strfry.approved_sync.enabled = true` (environment or role override) and
|
||||
converge `role[strfry]`. This deploys `sync-policy.mjs`, the sync config, and
|
||||
the `strfry-sync.timer`.
|
||||
4. **Enable the promotion paths.** Only after the worker + staging relay (WS2)
|
||||
are live: `role[strfry_submission]` sets `strfry.approved_sync.enabled` and
|
||||
`strfry.member_bridge.enabled` to `true` (role overrides, not the
|
||||
environment). Converge `role[strfry]`. This deploys `sync-policy.mjs`, the
|
||||
sync config, `strfry-sync.timer`, `ldap-only-policy.ts`, the member-bridge
|
||||
config and `strfry-member-bridge.service`.
|
||||
5. **Verify.**
|
||||
* `systemctl status strfry-submission` / `strfry-sync.timer`.
|
||||
* `systemctl status strfry-submission` / `strfry-sync.timer` /
|
||||
`strfry-member-bridge`.
|
||||
* `curl -s https://submit.nostr.kosmos.org | jq -r .name` →
|
||||
`Kosmos submission relay`.
|
||||
* `journalctl -u strfry-sync` shows accepted events after an approval.
|
||||
* `journalctl -u strfry-member-bridge` shows `Connected to ws://localhost:7778`
|
||||
and rejects non-member events (`pluginDown blocked event … Only members can
|
||||
publish notes on this relay`).
|
||||
* The production relay's NIP-11 and `_strfry` upstream are unchanged.
|
||||
|
||||
## Pre-go-live verification
|
||||
@@ -129,6 +167,8 @@ ruby site-cookbooks/kosmos_strfry/test/strfry_sync_conf_test.rb
|
||||
* **Sync:** set `strfry.approved_sync.enabled = false` and converge, then
|
||||
`systemctl disable --now strfry-sync.timer`. (The recipe becomes a no-op, but
|
||||
it does not stop an already-running timer.)
|
||||
* **Member bridge:** set `strfry.member_bridge.enabled = false` and converge,
|
||||
then `systemctl disable --now strfry-member-bridge`.
|
||||
* **Submission relay / proxy:** remove `kosmos_strfry_submission::nginx` from
|
||||
`role[openresty_proxy]` and `role[strfry_submission]` from the node, converge,
|
||||
then `systemctl disable --now strfry-submission`. The DB at
|
||||
@@ -137,9 +177,11 @@ ruby site-cookbooks/kosmos_strfry/test/strfry_sync_conf_test.rb
|
||||
|
||||
## Node.js caveat (check before converging)
|
||||
|
||||
Both new policies are Node 24 ESM scripts, so `role[strfry_submission]` and
|
||||
`role[strfry]` add a **role-scoped** `kosmos_nodejs.version` override to
|
||||
`24.22.2`. The global default (`14.21.3`) is deliberately **not** raised.
|
||||
The Node write policies (`submission-policy.mjs`, `sync-policy.mjs`) are Node 24
|
||||
ESM scripts, so `role[strfry_submission]` and `role[strfry]` add a
|
||||
**role-scoped** `kosmos_nodejs.version` override to `24.22.2`. The global
|
||||
default (`14.21.3`) is deliberately **not** raised. (`ldap-only-policy.ts` is a
|
||||
Deno script and needs no Node.)
|
||||
|
||||
If either role is applied to a host that already runs other Node/JS services,
|
||||
verify they do not depend on Node 14 *before* converging:
|
||||
@@ -155,5 +197,6 @@ verify they do not depend on Node 14 *before* converging:
|
||||
## Out of scope
|
||||
|
||||
* Deploying/converging, DNS changes, certificate issuance, ZeroTier changes.
|
||||
* The nuc staging relay and the moderator/member-bridge (WS2).
|
||||
* The nuc staging relay and the moderator worker (WS2; not Kosmos infra). The
|
||||
member bridge *is* covered here (`kosmos_strfry::member_bridge`).
|
||||
* The write policies themselves (WS1, `kosmos/akkounts`).
|
||||
@@ -17,4 +17,5 @@ run_list %w(
|
||||
kosmos_strfry::firewall
|
||||
kosmos_strfry::substr
|
||||
kosmos_strfry::approved_sync
|
||||
kosmos_strfry::member_bridge
|
||||
)
|
||||
@@ -7,6 +7,20 @@ override_attributes(
|
||||
# untouched for every other node.
|
||||
"kosmos_nodejs" => {
|
||||
"version" => "24.22.2"
|
||||
},
|
||||
|
||||
# Enable the two promotion paths that feed the production relay from the
|
||||
# submission pipeline. They are toggled here (role-scoped) rather than in the
|
||||
# production environment config, so the switches travel with this feature's
|
||||
# roles. The recipes themselves live in role[strfry] (the relay node runs both
|
||||
# roles).
|
||||
"strfry" => {
|
||||
"approved_sync" => {
|
||||
"enabled" => true
|
||||
},
|
||||
"member_bridge" => {
|
||||
"enabled" => true
|
||||
}
|
||||
}
|
||||
)
|
||||
|
||||
|
||||
@@ -14,3 +14,13 @@ rejects every synced event. No Chef server or converge is required:
|
||||
```
|
||||
ruby site-cookbooks/kosmos_strfry/test/strfry_sync_conf_test.rb
|
||||
```
|
||||
|
||||
`test/member_bridge_conf_test.rb` is a standalone regression check for
|
||||
`templates/member_bridge.conf.erb` (the `kosmos_strfry::member_bridge` router
|
||||
config). It asserts that the stream group is one-way (`dir = "down"`) and gated
|
||||
by the LDAP-only policy (`pluginDown`), with a space-free plugin path so strfry
|
||||
hot-reloads it. No Chef server or converge is required:
|
||||
|
||||
```
|
||||
ruby site-cookbooks/kosmos_strfry/test/member_bridge_conf_test.rb
|
||||
```
|
||||
@@ -10,6 +10,18 @@ node.default["strfry"]["approved_sync"]["enabled"] = false
|
||||
node.default["strfry"]["approved_sync"]["config_file"] = "/etc/strfry-sync.conf"
|
||||
node.default["strfry"]["approved_sync"]["policy_url"] = "https://gitea.kosmos.org/kosmos/akkounts/raw/branch/live/extras/strfry/sync-policy.mjs"
|
||||
|
||||
#
|
||||
# member bridge (WS3): live one-way promotion of member-authored events from the
|
||||
# local submission relay into the production database, gated by the LDAP-only
|
||||
# write policy. See kosmos_strfry/recipes/member_bridge.rb.
|
||||
#
|
||||
node.default["strfry"]["member_bridge"]["enabled"] = false
|
||||
node.default["strfry"]["member_bridge"]["config_file"] = "/etc/strfry-member-bridge.conf"
|
||||
node.default["strfry"]["member_bridge"]["subscribe_relay"] = "ws://localhost:7778"
|
||||
node.default["strfry"]["member_bridge"]["plugin_cmd"] = "#{node["strfry"]["extras_dir"]}/ldap-only-policy.ts"
|
||||
node.default["strfry"]["member_bridge"]["kinds"] = [0, 3, 5, 360, 30360]
|
||||
node.default["strfry"]["member_bridge"]["connection_timeout"] = 20
|
||||
|
||||
# node.default["substr"]["repo"] = "https://gitea.kosmos.org/kosmos/substr.git"
|
||||
# node.default["substr"]["revision"] = "master"
|
||||
node.default["substr"]["version"] = "nightly"
|
||||
|
||||
@@ -0,0 +1,70 @@
|
||||
#
|
||||
# Cookbook Name:: kosmos_strfry
|
||||
# Recipe:: member_bridge
|
||||
#
|
||||
# Live, one-way promotion of member-authored events from the local submission
|
||||
# relay (ws://localhost:7778) into the production database used by
|
||||
# nostr.kosmos.org.
|
||||
#
|
||||
# It runs `strfry router` with a stream group whose `dir` is "down" (so the
|
||||
# router never watches or forwards the local database) and whose `pluginDown` is
|
||||
# the LDAP-only write policy (/opt/strfry/ldap-only-policy.ts). Membership is
|
||||
# therefore decided per event by LDAP; no member list is maintained, and
|
||||
# non-member events are rejected before they reach the database.
|
||||
#
|
||||
# This recipe is a no-op until node["strfry"]["member_bridge"]["enabled"] is set
|
||||
# to true (in an environment or a node's role overrides).
|
||||
#
|
||||
|
||||
unless node["strfry"]["member_bridge"]["enabled"]
|
||||
Chef::Log.info("kosmos_strfry::member_bridge is disabled; skipping member bridge.")
|
||||
return
|
||||
end
|
||||
|
||||
config_file = node["strfry"]["member_bridge"]["config_file"]
|
||||
|
||||
template config_file do
|
||||
source "member_bridge.conf.erb"
|
||||
owner node["strfry"]["user"]
|
||||
group node["strfry"]["group"]
|
||||
mode "0644"
|
||||
variables subscribe_relay: node["strfry"]["member_bridge"]["subscribe_relay"],
|
||||
plugin_cmd: node["strfry"]["member_bridge"]["plugin_cmd"],
|
||||
kinds: node["strfry"]["member_bridge"]["kinds"],
|
||||
connection_timeout: node["strfry"]["member_bridge"]["connection_timeout"]
|
||||
end
|
||||
|
||||
#
|
||||
# `strfry router` reads /etc/strfry.conf for the database and the router config
|
||||
# for the stream groups. It writes accepted events straight into the production
|
||||
# LMDB (the same multi-process access pattern already used by the approved sync).
|
||||
#
|
||||
systemd_unit "strfry-member-bridge.service" do
|
||||
content({
|
||||
Unit: {
|
||||
Description: "strfry member bridge (submission relay -> production, LDAP-only)",
|
||||
Documentation: ["https://github.com/hoytech/strfry"],
|
||||
After: "strfry.service",
|
||||
Wants: "network-online.target"
|
||||
},
|
||||
Service: {
|
||||
Type: "simple",
|
||||
User: node["strfry"]["user"],
|
||||
Group: node["strfry"]["group"],
|
||||
ExecStart: "/usr/local/bin/strfry router #{config_file}",
|
||||
Restart: "on-failure",
|
||||
RestartSec: "5",
|
||||
Nice: 10,
|
||||
# The LDAP-only policy is a Deno script; Deno caches its modules under the
|
||||
# strfry user's home, so ProtectHome must stay off (as for strfry.service).
|
||||
ProtectHome: "no",
|
||||
NoNewPrivileges: "yes",
|
||||
ProtectSystem: "full"
|
||||
},
|
||||
Install: {
|
||||
WantedBy: "multi-user.target"
|
||||
}
|
||||
})
|
||||
triggers_reload true
|
||||
action [:create, :enable, :start]
|
||||
end
|
||||
@@ -81,3 +81,14 @@ remote_file "#{extras_dir}/strfry-sync.ts" do
|
||||
group node["strfry"]["group"]
|
||||
mode "0644"
|
||||
end
|
||||
|
||||
# LDAP-only policy for the member bridge (kosmos_strfry::member_bridge). It
|
||||
# reuses ldap-policy.ts above. No `notifies` is needed: strfry watches the
|
||||
# plugin file's mtime (the pluginDown path contains no spaces) and restarts it
|
||||
# on change.
|
||||
remote_file "#{extras_dir}/ldap-only-policy.ts" do
|
||||
source "#{base_url}/ldap-only-policy.ts"
|
||||
owner node["strfry"]["user"]
|
||||
group node["strfry"]["group"]
|
||||
mode "0755"
|
||||
end
|
||||
@@ -0,0 +1,20 @@
|
||||
# Managed by Chef (kosmos_strfry::member_bridge). Do not edit by hand.
|
||||
#
|
||||
# Live member bridge: a `strfry router` stream group subscribes to the local
|
||||
# submission relay and writes member-authored events into the production relay's
|
||||
# database. Membership is decided per event by the LDAP-only write policy
|
||||
# (pluginDown); no member list is maintained here.
|
||||
#
|
||||
# `dir = "down"` keeps this strictly one-way: the router never watches or
|
||||
# forwards the local database.
|
||||
|
||||
connectionTimeout = <%= @connection_timeout %>
|
||||
|
||||
streams {
|
||||
members {
|
||||
dir = "down"
|
||||
filter = { "kinds": [<%= @kinds.join(', ') %>] }
|
||||
urls = [ "<%= @subscribe_relay %>" ]
|
||||
pluginDown = "<%= @plugin_cmd %>"
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,76 @@
|
||||
#!/usr/bin/env ruby
|
||||
# frozen_string_literal: true
|
||||
#
|
||||
# Regression check for kosmos_strfry/templates/member_bridge.conf.erb.
|
||||
#
|
||||
# The member bridge is a `strfry router` stream group. Two properties must hold,
|
||||
# or the bridge is either not one-way or not a member gate:
|
||||
# * `dir` must be "down" (the router must never watch/forward the local DB), and
|
||||
# * `pluginDown` must be the LDAP-only policy, with a path that contains no
|
||||
# spaces (strfry only hot-reloads space-free plugin paths).
|
||||
#
|
||||
# Renders the real template (no Chef server needed) with the attribute shapes
|
||||
# kosmos_strfry::member_bridge passes.
|
||||
#
|
||||
# Run standalone: ruby site-cookbooks/kosmos_strfry/test/member_bridge_conf_test.rb
|
||||
|
||||
require 'erb'
|
||||
|
||||
TEMPLATE = File.expand_path('../templates/member_bridge.conf.erb', __dir__)
|
||||
|
||||
DEFAULTS = {
|
||||
subscribe_relay: 'ws://localhost:7778',
|
||||
plugin_cmd: '/opt/strfry/ldap-only-policy.ts',
|
||||
kinds: [0, 3, 5, 360, 30360],
|
||||
connection_timeout: 20
|
||||
}.freeze
|
||||
|
||||
def render(vars)
|
||||
context = Object.new
|
||||
vars.each { |name, value| context.instance_variable_set("@#{name}", value) }
|
||||
|
||||
# Chef renders templates with the `variables` hash exposed as instance
|
||||
# variables (@subscribe_relay, ...); mirror that with a bare binding.
|
||||
ERB.new(File.read(TEMPLATE), trim_mode: '-').result(context.instance_eval { binding })
|
||||
end
|
||||
|
||||
def check(description, vars)
|
||||
rendered = render(vars)
|
||||
|
||||
unless rendered.match?(/^\s*dir\s*=\s*"down"\s*$/)
|
||||
raise "#{description}: dir must be \"down\""
|
||||
end
|
||||
if rendered.match?(/^\s*dir\s*=\s*"(up|both)"\s*$/)
|
||||
raise "#{description}: dir must not be up/both (the bridge must be one-way)"
|
||||
end
|
||||
|
||||
unless rendered.include?(%(urls = [ "#{vars[:subscribe_relay]}" ]))
|
||||
raise "#{description}: urls does not contain the subscribe relay"
|
||||
end
|
||||
|
||||
unless rendered.include?(%(pluginDown = "#{vars[:plugin_cmd]}"))
|
||||
raise "#{description}: pluginDown does not point at the LDAP-only policy"
|
||||
end
|
||||
|
||||
if vars[:plugin_cmd].include?(' ')
|
||||
raise "#{description}: plugin path must contain no spaces (strfry only hot-reloads space-free paths)"
|
||||
end
|
||||
|
||||
unless rendered.include?(%(filter = { "kinds": [#{vars[:kinds].join(', ')}] }))
|
||||
raise "#{description}: kinds filter missing or unexpected"
|
||||
end
|
||||
|
||||
unless rendered.include?("connectionTimeout = #{vars[:connection_timeout]}")
|
||||
raise "#{description}: connectionTimeout missing"
|
||||
end
|
||||
|
||||
puts "ok - #{description}"
|
||||
end
|
||||
|
||||
begin
|
||||
check('default attributes', DEFAULTS)
|
||||
puts "\nPASS: member_bridge.conf.erb is one-way and gated by the LDAP-only policy."
|
||||
rescue StandardError => e
|
||||
warn "FAIL: #{e.message}"
|
||||
exit 1
|
||||
end
|
||||
Reference in new issue
Block a user