Add strfry policies for the public submission relay and approved sync
CI / Test (pull_request) Successful in 49s
CI / Test (pull_request) Successful in 49s
Adds the Node write policies used by the new public submission relay and by the approved-content sync on the members relay, for raucao/marco#97. submission-policy.mjs keeps the submission relay open to all authors while applying the usual gates: a kind allowlist, content and tag bounds, per-IP and per-author token buckets, a short-window content dedupe, and a hellthread limit. It does not do membership checks or media analysis. sync-policy.mjs is fail-closed: it accepts only events that arrived through strfry sync from the exact staging relay URL, with an allowed kind. Everything else, including a wrong source type or an unset STAGING_RELAY_URL, is rejected and logged with observed-vs-expected source information. Includes node:test coverage for both policies and a README documenting the environment variables and how strfry invokes them. The existing Deno policies are untouched. Note: chef fetches these files from the live branch, so master must be merged into live before the submission relay or approved sync is converged.
This commit is contained in:
5 files changed
+899
No files matched your search
@@ -0,0 +1,138 @@
|
||||
# strfry policies
|
||||
|
||||
Write-policy plugins for [strfry][strfry] relays, written as plain Node ESM
|
||||
(`.mjs`, Node 22 LTS). They are fetched by Chef from this directory into
|
||||
`/opt/strfry` on the relay hosts.
|
||||
|
||||
The legacy Deno policies (`strfry-policy.ts`, `ldap-policy.ts`,
|
||||
`strfry-sync.ts`) are still deployed for `nostr.kosmos.org` and are unrelated to
|
||||
the scripts below. See `docs/dev/nostr.md` for those.
|
||||
|
||||
## How strfry invokes a policy
|
||||
|
||||
strfry spawns the configured `relay.writePolicy.plugin` command and talks to it
|
||||
over stdin/stdout. Each line on **stdin** is a JSON object:
|
||||
|
||||
```json
|
||||
{"type":"new","event":{...},"receivedAt":1700000000,"sourceType":"IP4","sourceInfo":"203.0.113.7"}
|
||||
```
|
||||
|
||||
`sourceType` is one of `IP4`, `IP6`, `Import`, `Stream`, `Sync`, `Stored`.
|
||||
`sourceInfo` is an IP address (for `IP4`/`IP6`) or a relay URL (for
|
||||
`Sync`/`Stream`).
|
||||
|
||||
For every input line the plugin must write one line of JSON to **stdout**:
|
||||
|
||||
```json
|
||||
{"id":"<event id>","action":"accept|reject","msg":"..."}
|
||||
```
|
||||
|
||||
`msg` is only meaningful for `reject` and is returned to the client. Diagnostics
|
||||
go to **stderr** and end up in the strfry log.
|
||||
|
||||
## `submission-policy.mjs`
|
||||
|
||||
Write policy for the **public submission relay** (`submit.nostr.kosmos.org`).
|
||||
Open to all authors — there is **no LDAP/membership check**. It applies only the
|
||||
"usual gates":
|
||||
|
||||
1. `type` must be `new`.
|
||||
2. Event must be well-formed (`pubkey`, integer `kind`, `tags` array).
|
||||
3. `kind` must be in the allowlist (deny by default).
|
||||
4. `content` must be within the byte bound.
|
||||
5. Tag count must be within bounds.
|
||||
6. `p`-tag count must be within the hellthread bound.
|
||||
7. Per-IP token bucket (keyed on `sourceInfo`).
|
||||
8. Per-author token bucket (keyed on `event.pubkey`).
|
||||
9. Short-window content dedupe (sha256 of `content`, skipped for content shorter
|
||||
than the configured minimum so empty kind-3/kind-5 content is not falsely
|
||||
deduped).
|
||||
|
||||
All state is **in-memory** and resets whenever strfry reloads the plugin (e.g.
|
||||
on file mtime change or config change).
|
||||
|
||||
### Environment variables
|
||||
|
||||
| Variable | Default | Description |
|
||||
| --- | --- | --- |
|
||||
| `SUBMISSION_KINDS` | `0,3,5,360,30360` | Comma-separated kind allowlist. Empty value denies every kind. |
|
||||
| `SUBMISSION_MAX_CONTENT_LENGTH` | `16384` | Max `content` size in UTF-8 bytes. |
|
||||
| `SUBMISSION_MAX_TAGS` | `200` | Max number of tags per event. |
|
||||
| `SUBMISSION_MAX_P_TAGS` | `10` | Max number of `p` tags (hellthread cap). |
|
||||
| `SUBMISSION_IP_BURST` | `120` | Per-IP bucket capacity (burst). |
|
||||
| `SUBMISSION_IP_REFILL_PER_SEC` | `5` | Per-IP sustained refill rate. |
|
||||
| `SUBMISSION_AUTHOR_BURST` | `10` | Per-author bucket capacity (burst). |
|
||||
| `SUBMISSION_AUTHOR_REFILL_PER_SEC` | `0.2` | Per-author sustained refill rate. |
|
||||
| `SUBMISSION_DEDUPE_WINDOW_MS` | `60000` | Dedupe window. |
|
||||
| `SUBMISSION_DEDUPE_MIN_LENGTH` | `50` | Minimum content length that is deduped. |
|
||||
|
||||
### Chef / strfry configuration
|
||||
|
||||
```ini
|
||||
relay {
|
||||
writePolicy {
|
||||
plugin = "node /opt/strfry/submission-policy.mjs"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## `sync-policy.mjs`
|
||||
|
||||
Write policy for the **approved-content sync** config on strfry-1
|
||||
(`/etc/strfry-sync.conf`, same LMDB database as the production relay). It is
|
||||
**fail-closed**: an event is accepted only when *all* of the following hold:
|
||||
|
||||
- `type === "new"`;
|
||||
- `sourceType === "Sync"`;
|
||||
- `sourceInfo === STAGING_RELAY_URL` (byte-exact);
|
||||
- `event.kind` is in the allowlist.
|
||||
|
||||
Everything else is rejected, and the log line records the **observed vs
|
||||
expected** `sourceType` and `sourceInfo`. If `STAGING_RELAY_URL` is unset, the
|
||||
policy rejects every event.
|
||||
|
||||
### Environment variables
|
||||
|
||||
| Variable | Default | Description |
|
||||
| --- | --- | --- |
|
||||
| `STAGING_RELAY_URL` | _(unset)_ | Expected `sourceInfo` for synced events, e.g. `ws://10.1.1.183:7779`. Unset/empty rejects everything. Whitespace around the configured value is trimmed; `sourceInfo` is compared as-is. |
|
||||
| `SYNC_KINDS` | `0,360,30360,5` | Comma-separated kind allowlist. |
|
||||
|
||||
### Chef / strfry configuration
|
||||
|
||||
`/etc/strfry-sync.conf`:
|
||||
|
||||
```ini
|
||||
relay {
|
||||
writePolicy {
|
||||
plugin = "node /opt/strfry/sync-policy.mjs"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
The matching sync is driven by `strfry --config /etc/strfry-sync.conf sync
|
||||
ws://<nuc-zt-ip>:7779 --dir down`, which sets `sourceType=Sync` and
|
||||
`sourceInfo` to the relay URL passed on the command line. `STAGING_RELAY_URL`
|
||||
must be that exact URL.
|
||||
|
||||
## Tests
|
||||
|
||||
Unit tests use the built-in `node:test` runner (no dependencies):
|
||||
|
||||
```sh
|
||||
node --test extras/strfry/test/*.test.mjs
|
||||
```
|
||||
|
||||
## Smoke test
|
||||
|
||||
```sh
|
||||
printf '%s\n' \
|
||||
'{"type":"new","sourceType":"IP4","sourceInfo":"1.2.3.4","event":{"id":"a","pubkey":"b","kind":360,"tags":[],"content":"hi"}}' \
|
||||
| ./submission-policy.mjs
|
||||
|
||||
printf '%s\n' \
|
||||
'{"type":"new","sourceType":"Sync","sourceInfo":"ws://10.1.1.183:7779","event":{"id":"a","pubkey":"b","kind":360,"tags":[],"content":"hi"}}' \
|
||||
| STAGING_RELAY_URL='ws://10.1.1.183:7779' ./sync-policy.mjs
|
||||
```
|
||||
|
||||
[strfry]: https://github.com/hoytech/strfry
|
||||
Executable
+241
@@ -0,0 +1,241 @@
|
||||
#!/usr/bin/env node
|
||||
// strfry write policy for the public Kosmos submission relay.
|
||||
//
|
||||
// This relay is open to all authors (no LDAP membership check). It applies only
|
||||
// the "usual gates": a kind allowlist, content/tag bounds, per-IP and
|
||||
// per-author token-bucket rate limits, a short-window content dedupe, and a
|
||||
// hellthread (max p-tag) cap.
|
||||
//
|
||||
// Reads strfry JSONL input messages from stdin and writes one
|
||||
// {"id","action","msg"} JSON object per line to stdout.
|
||||
//
|
||||
// All state is in-memory and resets whenever strfry reloads the plugin.
|
||||
//
|
||||
// See README.md for the env vars and how strfry invokes this.
|
||||
|
||||
import readline from 'node:readline';
|
||||
import { createHash } from 'node:crypto';
|
||||
import { pathToFileURL } from 'node:url';
|
||||
|
||||
const DEFAULTS = {
|
||||
kinds: '0,3,5,360,30360',
|
||||
maxContentLength: 16384,
|
||||
maxTags: 200,
|
||||
maxPTags: 10,
|
||||
ipBurst: 120,
|
||||
ipRefillPerSec: 5,
|
||||
authorBurst: 10,
|
||||
authorRefillPerSec: 0.2,
|
||||
dedupeWindowMs: 60000,
|
||||
dedupeMinLength: 50,
|
||||
};
|
||||
|
||||
const NAME = 'submission-policy';
|
||||
|
||||
function parseKinds(raw, fallback) {
|
||||
const value = raw === undefined || raw === null ? fallback : raw;
|
||||
const kinds = new Set();
|
||||
|
||||
for (const part of String(value).split(',')) {
|
||||
const trimmed = part.trim();
|
||||
if (trimmed === '') continue;
|
||||
|
||||
const n = Number(trimmed);
|
||||
if (Number.isInteger(n)) kinds.add(n);
|
||||
}
|
||||
|
||||
return kinds;
|
||||
}
|
||||
|
||||
function parseNumber(env, key, fallback) {
|
||||
const raw = env[key];
|
||||
if (raw === undefined || raw === null || String(raw).trim() === '') return fallback;
|
||||
|
||||
const n = Number(raw);
|
||||
return Number.isFinite(n) && n >= 0 ? n : fallback;
|
||||
}
|
||||
|
||||
export function createState(env = process.env) {
|
||||
return {
|
||||
config: {
|
||||
kinds: parseKinds(env.SUBMISSION_KINDS, DEFAULTS.kinds),
|
||||
maxContentLength: parseNumber(env, 'SUBMISSION_MAX_CONTENT_LENGTH', DEFAULTS.maxContentLength),
|
||||
maxTags: parseNumber(env, 'SUBMISSION_MAX_TAGS', DEFAULTS.maxTags),
|
||||
maxPTags: parseNumber(env, 'SUBMISSION_MAX_P_TAGS', DEFAULTS.maxPTags),
|
||||
ipBurst: parseNumber(env, 'SUBMISSION_IP_BURST', DEFAULTS.ipBurst),
|
||||
ipRefillPerSec: parseNumber(env, 'SUBMISSION_IP_REFILL_PER_SEC', DEFAULTS.ipRefillPerSec),
|
||||
authorBurst: parseNumber(env, 'SUBMISSION_AUTHOR_BURST', DEFAULTS.authorBurst),
|
||||
authorRefillPerSec: parseNumber(env, 'SUBMISSION_AUTHOR_REFILL_PER_SEC', DEFAULTS.authorRefillPerSec),
|
||||
dedupeWindowMs: parseNumber(env, 'SUBMISSION_DEDUPE_WINDOW_MS', DEFAULTS.dedupeWindowMs),
|
||||
dedupeMinLength: parseNumber(env, 'SUBMISSION_DEDUPE_MIN_LENGTH', DEFAULTS.dedupeMinLength),
|
||||
},
|
||||
ipBuckets: new Map(),
|
||||
authorBuckets: new Map(),
|
||||
dedupe: new Map(),
|
||||
};
|
||||
}
|
||||
|
||||
// Lazily refills a token bucket and consumes one token. Returns true when a
|
||||
// token was available. `now` is in milliseconds.
|
||||
function takeToken(buckets, key, capacity, refillPerSec, now) {
|
||||
let bucket = buckets.get(key);
|
||||
|
||||
if (bucket === undefined) {
|
||||
bucket = { tokens: capacity, updated: now };
|
||||
buckets.set(key, bucket);
|
||||
} else {
|
||||
const elapsedSec = Math.max(0, now - bucket.updated) / 1000;
|
||||
bucket.tokens = Math.min(capacity, bucket.tokens + elapsedSec * refillPerSec);
|
||||
bucket.updated = now;
|
||||
}
|
||||
|
||||
if (bucket.tokens >= 1) {
|
||||
bucket.tokens -= 1;
|
||||
return true;
|
||||
}
|
||||
|
||||
return false;
|
||||
}
|
||||
|
||||
// Drops buckets that have fully refilled to keep the maps bounded.
|
||||
function pruneBuckets(buckets, capacity) {
|
||||
if (buckets.size < 10000) return;
|
||||
|
||||
for (const [key, bucket] of buckets) {
|
||||
if (bucket.tokens >= capacity) buckets.delete(key);
|
||||
}
|
||||
}
|
||||
|
||||
function isDuplicate(dedupe, key, now, windowMs) {
|
||||
if (dedupe.size > 1000) {
|
||||
for (const [k, seenAt] of dedupe) {
|
||||
if (now - seenAt >= windowMs) dedupe.delete(k);
|
||||
}
|
||||
}
|
||||
|
||||
const seenAt = dedupe.get(key);
|
||||
return seenAt !== undefined && now - seenAt < windowMs;
|
||||
}
|
||||
|
||||
function reject(id, msg) {
|
||||
return { id, action: 'reject', msg };
|
||||
}
|
||||
|
||||
export function decide(msg, state, now = Date.now()) {
|
||||
const id = msg && msg.event && typeof msg.event.id === 'string' ? msg.event.id : '';
|
||||
|
||||
if (msg === null || typeof msg !== 'object') {
|
||||
return reject(id, 'malformed input');
|
||||
}
|
||||
|
||||
if (msg.type !== 'new') {
|
||||
return reject(id, `unexpected request type: ${String(msg.type)}`);
|
||||
}
|
||||
|
||||
const event = msg.event;
|
||||
if (event === null || typeof event !== 'object') {
|
||||
return reject(id, 'malformed event');
|
||||
}
|
||||
|
||||
const { kind, tags, content, pubkey } = event;
|
||||
|
||||
if (typeof pubkey !== 'string' || pubkey === '') {
|
||||
return reject(id, 'malformed event: missing pubkey');
|
||||
}
|
||||
if (!Number.isInteger(kind)) {
|
||||
return reject(id, 'malformed event: invalid kind');
|
||||
}
|
||||
if (!Array.isArray(tags)) {
|
||||
return reject(id, 'malformed event: invalid tags');
|
||||
}
|
||||
|
||||
const text = typeof content === 'string' ? content : '';
|
||||
const config = state.config;
|
||||
|
||||
if (!config.kinds.has(kind)) {
|
||||
return reject(id, `kind ${kind} not allowed`);
|
||||
}
|
||||
|
||||
if (Buffer.byteLength(text, 'utf8') > config.maxContentLength) {
|
||||
return reject(id, 'content too long');
|
||||
}
|
||||
|
||||
if (tags.length > config.maxTags) {
|
||||
return reject(id, 'too many tags');
|
||||
}
|
||||
|
||||
let pTags = 0;
|
||||
for (const tag of tags) {
|
||||
if (Array.isArray(tag) && tag[0] === 'p') pTags += 1;
|
||||
}
|
||||
if (pTags > config.maxPTags) {
|
||||
return reject(id, 'too many p tags (hellthread)');
|
||||
}
|
||||
|
||||
const source = typeof msg.sourceInfo === 'string' ? msg.sourceInfo : '';
|
||||
|
||||
pruneBuckets(state.ipBuckets, config.ipBurst);
|
||||
if (!takeToken(state.ipBuckets, source, config.ipBurst, config.ipRefillPerSec, now)) {
|
||||
return reject(id, 'rate limited');
|
||||
}
|
||||
|
||||
pruneBuckets(state.authorBuckets, config.authorBurst);
|
||||
if (!takeToken(state.authorBuckets, pubkey, config.authorBurst, config.authorRefillPerSec, now)) {
|
||||
return reject(id, 'rate limited');
|
||||
}
|
||||
|
||||
if (text.length >= config.dedupeMinLength) {
|
||||
const hash = createHash('sha256').update(text).digest('hex');
|
||||
if (isDuplicate(state.dedupe, hash, now, config.dedupeWindowMs)) {
|
||||
return reject(id, 'duplicate content');
|
||||
}
|
||||
state.dedupe.set(hash, now);
|
||||
}
|
||||
|
||||
return { id, action: 'accept', msg: '' };
|
||||
}
|
||||
|
||||
function writeResult(output, result) {
|
||||
output.write(`${JSON.stringify(result)}\n`);
|
||||
}
|
||||
|
||||
export async function run({ input = process.stdin, output = process.stdout, error = process.stderr, env = process.env, state = undefined } = {}) {
|
||||
const policyState = state ?? createState(env);
|
||||
const rl = readline.createInterface({ input, crlfDelay: Infinity });
|
||||
|
||||
for await (const line of rl) {
|
||||
if (line.trim() === '') continue;
|
||||
|
||||
let msg;
|
||||
try {
|
||||
msg = JSON.parse(line);
|
||||
} catch (e) {
|
||||
error.write(`[${NAME}] malformed JSON: ${e.message}\n`);
|
||||
writeResult(output, reject('', 'malformed JSON'));
|
||||
continue;
|
||||
}
|
||||
|
||||
let result;
|
||||
try {
|
||||
result = decide(msg, policyState);
|
||||
} catch (e) {
|
||||
error.write(`[${NAME}] unexpected error: ${e.stack || e.message}\n`);
|
||||
result = reject(msg && msg.event && typeof msg.event.id === 'string' ? msg.event.id : '', 'internal error');
|
||||
}
|
||||
|
||||
if (result.action !== 'accept') {
|
||||
error.write(`[${NAME}] reject id=${result.id} reason=${result.msg}\n`);
|
||||
}
|
||||
|
||||
writeResult(output, result);
|
||||
}
|
||||
}
|
||||
|
||||
const isMain = process.argv[1] !== undefined && import.meta.url === pathToFileURL(process.argv[1]).href;
|
||||
|
||||
if (isMain) {
|
||||
run().catch((e) => {
|
||||
process.stderr.write(`[${NAME}] fatal: ${e.stack || e.message}\n`);
|
||||
process.exit(1);
|
||||
});
|
||||
}
|
||||
Executable
+145
@@ -0,0 +1,145 @@
|
||||
#!/usr/bin/env node
|
||||
// Fail-closed strfry write policy for the strfry-1 approved-content sync.
|
||||
//
|
||||
// This policy only accepts events that arrived through `strfry sync` from the
|
||||
// dedicated staging relay. Everything else is rejected, including events with
|
||||
// the wrong source type or the wrong source URL, so that a misconfigured or
|
||||
// compromised sync source can never write into the production relay.
|
||||
//
|
||||
// Reads strfry JSONL input messages from stdin and writes one
|
||||
// {"id","action","msg"} JSON object per line to stdout.
|
||||
//
|
||||
// See README.md for the env vars and how strfry invokes this.
|
||||
|
||||
import readline from 'node:readline';
|
||||
import { pathToFileURL } from 'node:url';
|
||||
|
||||
const DEFAULTS = {
|
||||
kinds: '0,360,30360,5',
|
||||
};
|
||||
|
||||
const NAME = 'sync-policy';
|
||||
|
||||
function parseKinds(raw, fallback) {
|
||||
const value = raw === undefined || raw === null ? fallback : raw;
|
||||
const kinds = new Set();
|
||||
|
||||
for (const part of String(value).split(',')) {
|
||||
const trimmed = part.trim();
|
||||
if (trimmed === '') continue;
|
||||
|
||||
const n = Number(trimmed);
|
||||
if (Number.isInteger(n)) kinds.add(n);
|
||||
}
|
||||
|
||||
return kinds;
|
||||
}
|
||||
|
||||
export function createState(env = process.env) {
|
||||
return {
|
||||
config: {
|
||||
kinds: parseKinds(env.SYNC_KINDS, DEFAULTS.kinds),
|
||||
stagingRelayUrl: typeof env.STAGING_RELAY_URL === 'string' ? env.STAGING_RELAY_URL.trim() : '',
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
function reject(id, msg) {
|
||||
return { id, action: 'reject', msg };
|
||||
}
|
||||
|
||||
function describeSource(msg) {
|
||||
return `sourceType=${JSON.stringify(msg && msg.sourceType)} sourceInfo=${JSON.stringify(msg && msg.sourceInfo)}`;
|
||||
}
|
||||
|
||||
export function decide(msg, state) {
|
||||
const id = msg && msg.event && typeof msg.event.id === 'string' ? msg.event.id : '';
|
||||
const expected = state.config.stagingRelayUrl;
|
||||
|
||||
if (msg === null || typeof msg !== 'object') {
|
||||
return reject(id, 'malformed input');
|
||||
}
|
||||
|
||||
if (msg.type !== 'new') {
|
||||
return reject(id, `unexpected request type: ${String(msg.type)} (${describeSource(msg)})`);
|
||||
}
|
||||
|
||||
const event = msg.event;
|
||||
if (event === null || typeof event !== 'object') {
|
||||
return reject(id, 'malformed event');
|
||||
}
|
||||
|
||||
if (typeof event.pubkey !== 'string' || event.pubkey === '' || !Number.isInteger(event.kind) || !Array.isArray(event.tags)) {
|
||||
return reject(id, 'malformed event');
|
||||
}
|
||||
|
||||
if (expected === '') {
|
||||
return reject(id, `source not allowed: STAGING_RELAY_URL is not configured (${describeSource(msg)})`);
|
||||
}
|
||||
|
||||
if (msg.sourceType !== 'Sync') {
|
||||
return reject(id, `source not allowed: expected sourceType=${JSON.stringify('Sync')} got ${JSON.stringify(msg.sourceType)} (${describeSource(msg)})`);
|
||||
}
|
||||
|
||||
if (msg.sourceInfo !== expected) {
|
||||
return reject(id, `source not allowed: expected sourceInfo=${JSON.stringify(expected)} got ${JSON.stringify(msg.sourceInfo)} (${describeSource(msg)})`);
|
||||
}
|
||||
|
||||
if (!state.config.kinds.has(event.kind)) {
|
||||
return reject(id, `kind ${event.kind} not allowed`);
|
||||
}
|
||||
|
||||
return { id, action: 'accept', msg: '' };
|
||||
}
|
||||
|
||||
function writeResult(output, result) {
|
||||
output.write(`${JSON.stringify(result)}\n`);
|
||||
}
|
||||
|
||||
export async function run({ input = process.stdin, output = process.stdout, error = process.stderr, env = process.env, state = undefined } = {}) {
|
||||
const policyState = state ?? createState(env);
|
||||
|
||||
if (policyState.config.stagingRelayUrl === '') {
|
||||
error.write(`[${NAME}] warning: STAGING_RELAY_URL is not set; rejecting all events\n`);
|
||||
} else {
|
||||
error.write(`[${NAME}] accepting sourceType=Sync sourceInfo=${JSON.stringify(policyState.config.stagingRelayUrl)} kinds=${[...policyState.config.kinds].join(',')}\n`);
|
||||
}
|
||||
|
||||
const rl = readline.createInterface({ input, crlfDelay: Infinity });
|
||||
|
||||
for await (const line of rl) {
|
||||
if (line.trim() === '') continue;
|
||||
|
||||
let msg;
|
||||
try {
|
||||
msg = JSON.parse(line);
|
||||
} catch (e) {
|
||||
error.write(`[${NAME}] malformed JSON: ${e.message}\n`);
|
||||
writeResult(output, reject('', 'malformed JSON'));
|
||||
continue;
|
||||
}
|
||||
|
||||
let result;
|
||||
try {
|
||||
result = decide(msg, policyState);
|
||||
} catch (e) {
|
||||
error.write(`[${NAME}] unexpected error: ${e.stack || e.message}\n`);
|
||||
result = reject(msg && msg.event && typeof msg.event.id === 'string' ? msg.event.id : '', 'internal error');
|
||||
}
|
||||
|
||||
if (result.action !== 'accept') {
|
||||
error.write(`[${NAME}] reject id=${result.id} reason=${result.msg}\n`);
|
||||
}
|
||||
|
||||
writeResult(output, result);
|
||||
}
|
||||
}
|
||||
|
||||
const isMain = process.argv[1] !== undefined && import.meta.url === pathToFileURL(process.argv[1]).href;
|
||||
|
||||
if (isMain) {
|
||||
run().catch((e) => {
|
||||
process.stderr.write(`[${NAME}] fatal: ${e.stack || e.message}\n`);
|
||||
process.exit(1);
|
||||
});
|
||||
}
|
||||
@@ -0,0 +1,206 @@
|
||||
import test from 'node:test';
|
||||
import assert from 'node:assert/strict';
|
||||
import { Readable, Writable } from 'node:stream';
|
||||
|
||||
import { createState, decide, run } from '../submission-policy.mjs';
|
||||
|
||||
function newMsg(overrides = {}) {
|
||||
const { event: eventOverrides, ...rest } = overrides;
|
||||
|
||||
return {
|
||||
type: 'new',
|
||||
sourceType: 'IP4',
|
||||
sourceInfo: '10.0.0.1',
|
||||
event: {
|
||||
id: 'evt1',
|
||||
pubkey: 'pk1',
|
||||
kind: 360,
|
||||
tags: [],
|
||||
content: 'hello world',
|
||||
...(eventOverrides || {}),
|
||||
},
|
||||
...rest,
|
||||
};
|
||||
}
|
||||
|
||||
async function runLines(lines, { env, state } = {}) {
|
||||
const input = Readable.from(lines.map((line) => `${line}\n`));
|
||||
let stdout = '';
|
||||
let stderr = '';
|
||||
|
||||
const output = new Writable({ write(chunk, _enc, cb) { stdout += chunk.toString(); cb(); } });
|
||||
const error = new Writable({ write(chunk, _enc, cb) { stderr += chunk.toString(); cb(); } });
|
||||
|
||||
await run({ input, output, error, env, state });
|
||||
|
||||
const results = stdout.split('\n').filter((line) => line !== '').map((line) => JSON.parse(line));
|
||||
return { results, stdout, stderr };
|
||||
}
|
||||
|
||||
test('accepts a valid kind 360 submission', () => {
|
||||
const result = decide(newMsg(), createState({}));
|
||||
assert.deepEqual(result, { id: 'evt1', action: 'accept', msg: '' });
|
||||
});
|
||||
|
||||
test('accepts every kind in the default allowlist', () => {
|
||||
const state = createState({});
|
||||
|
||||
for (const kind of [0, 3, 5, 360, 30360]) {
|
||||
const result = decide(newMsg({ event: { id: `evt-${kind}`, kind } }), state, 1000);
|
||||
assert.equal(result.action, 'accept', `kind ${kind} should be accepted`);
|
||||
}
|
||||
});
|
||||
|
||||
test('rejects kinds outside the allowlist', () => {
|
||||
const state = createState({});
|
||||
|
||||
for (const kind of [1, 4, 7, 24133, 9735]) {
|
||||
const result = decide(newMsg({ event: { kind } }), state);
|
||||
assert.equal(result.action, 'reject');
|
||||
assert.match(result.msg, /not allowed/);
|
||||
}
|
||||
});
|
||||
|
||||
test('honours a custom SUBMISSION_KINDS allowlist', () => {
|
||||
const state = createState({ SUBMISSION_KINDS: '360,30360' });
|
||||
|
||||
assert.equal(decide(newMsg({ event: { kind: 360 } }), state).action, 'accept');
|
||||
assert.equal(decide(newMsg({ event: { kind: 30360 } }), state).action, 'accept');
|
||||
assert.equal(decide(newMsg({ event: { kind: 1 } }), state).action, 'reject');
|
||||
});
|
||||
|
||||
test('an empty allowlist denies everything', () => {
|
||||
const state = createState({ SUBMISSION_KINDS: '' });
|
||||
const result = decide(newMsg(), state);
|
||||
assert.equal(result.action, 'reject');
|
||||
});
|
||||
|
||||
test('rejects non-new message types', () => {
|
||||
const result = decide(newMsg({ type: 'lookback' }), createState({}));
|
||||
assert.equal(result.action, 'reject');
|
||||
assert.match(result.msg, /unexpected request type/);
|
||||
});
|
||||
|
||||
test('rejects malformed events', () => {
|
||||
const state = createState({});
|
||||
|
||||
assert.equal(decide(newMsg({ event: { pubkey: '' } }), state).action, 'reject');
|
||||
assert.equal(decide(newMsg({ event: { pubkey: 123 } }), state).action, 'reject');
|
||||
assert.equal(decide(newMsg({ event: { kind: 'x' } }), state).action, 'reject');
|
||||
assert.equal(decide(newMsg({ event: { tags: 'nope' } }), state).action, 'reject');
|
||||
assert.equal(decide(null, state).action, 'reject');
|
||||
assert.equal(decide(42, state).action, 'reject');
|
||||
});
|
||||
|
||||
test('rejects over-long content', () => {
|
||||
const state = createState({ SUBMISSION_MAX_CONTENT_LENGTH: '10' });
|
||||
|
||||
assert.equal(decide(newMsg({ event: { content: 'a'.repeat(10) } }), state).action, 'accept');
|
||||
assert.equal(decide(newMsg({ event: { content: 'a'.repeat(11) } }), state).action, 'reject');
|
||||
});
|
||||
|
||||
test('rejects events with too many tags', () => {
|
||||
const state = createState({ SUBMISSION_MAX_TAGS: '3' });
|
||||
const tags = [['e', '1'], ['e', '2'], ['e', '3'], ['e', '4']];
|
||||
|
||||
const result = decide(newMsg({ event: { tags } }), state);
|
||||
assert.equal(result.action, 'reject');
|
||||
assert.match(result.msg, /too many tags/);
|
||||
});
|
||||
|
||||
test('rejects hellthreads with too many p tags', () => {
|
||||
const state = createState({ SUBMISSION_MAX_P_TAGS: '2' });
|
||||
const tags = [['p', 'a'], ['p', 'b'], ['p', 'c']];
|
||||
|
||||
const result = decide(newMsg({ event: { tags } }), state);
|
||||
assert.equal(result.action, 'reject');
|
||||
assert.match(result.msg, /hellthread/);
|
||||
});
|
||||
|
||||
test('enforces the per-IP token bucket and refills over time', () => {
|
||||
const state = createState({ SUBMISSION_IP_BURST: '2', SUBMISSION_IP_REFILL_PER_SEC: '1' });
|
||||
|
||||
assert.equal(decide(newMsg({ event: { id: 'a', pubkey: 'pk1', content: 'a' } }), state, 1000).action, 'accept');
|
||||
assert.equal(decide(newMsg({ event: { id: 'b', pubkey: 'pk2', content: 'b' } }), state, 1000).action, 'accept');
|
||||
|
||||
const limited = decide(newMsg({ event: { id: 'c', pubkey: 'pk3', content: 'c' } }), state, 1000);
|
||||
assert.equal(limited.action, 'reject');
|
||||
assert.match(limited.msg, /rate limited/);
|
||||
|
||||
assert.equal(decide(newMsg({ event: { id: 'd', pubkey: 'pk4', content: 'd' } }), state, 2000).action, 'accept');
|
||||
});
|
||||
|
||||
test('enforces the per-author token bucket and refills over time', () => {
|
||||
const state = createState({ SUBMISSION_AUTHOR_BURST: '2', SUBMISSION_AUTHOR_REFILL_PER_SEC: '1' });
|
||||
|
||||
assert.equal(decide(newMsg({ event: { id: 'a', content: 'a' } }), state, 1000).action, 'accept');
|
||||
assert.equal(decide(newMsg({ event: { id: 'b', content: 'b' } }), state, 1000).action, 'accept');
|
||||
|
||||
const limited = decide(newMsg({ event: { id: 'c', content: 'c' } }), state, 1000);
|
||||
assert.equal(limited.action, 'reject');
|
||||
assert.match(limited.msg, /rate limited/);
|
||||
|
||||
assert.equal(decide(newMsg({ event: { id: 'd', content: 'd' } }), state, 2000).action, 'accept');
|
||||
});
|
||||
|
||||
test('dedupes identical long content within the window', () => {
|
||||
const state = createState({});
|
||||
const content = 'x'.repeat(60);
|
||||
|
||||
assert.equal(decide(newMsg({ event: { id: 'a', content } }), state, 1000).action, 'accept');
|
||||
|
||||
const duplicate = decide(newMsg({ event: { id: 'b', content } }), state, 1000);
|
||||
assert.equal(duplicate.action, 'reject');
|
||||
assert.match(duplicate.msg, /duplicate content/);
|
||||
|
||||
assert.equal(decide(newMsg({ event: { id: 'c', content } }), state, 61001).action, 'accept');
|
||||
});
|
||||
|
||||
test('does not dedupe short content', () => {
|
||||
const state = createState({});
|
||||
|
||||
assert.equal(decide(newMsg({ event: { id: 'a', content: 'hi' } }), state, 1000).action, 'accept');
|
||||
assert.equal(decide(newMsg({ event: { id: 'b', content: 'hi' } }), state, 1000).action, 'accept');
|
||||
});
|
||||
|
||||
test('run() processes JSONL end-to-end', async () => {
|
||||
const { results } = await runLines([
|
||||
JSON.stringify(newMsg({ event: { id: 'a', kind: 360 } })),
|
||||
JSON.stringify(newMsg({ event: { id: 'b', kind: 1 } })),
|
||||
]);
|
||||
|
||||
assert.deepEqual(results, [
|
||||
{ id: 'a', action: 'accept', msg: '' },
|
||||
{ id: 'b', action: 'reject', msg: 'kind 1 not allowed' },
|
||||
]);
|
||||
});
|
||||
|
||||
test('run() rejects malformed JSON without crashing and keeps going', async () => {
|
||||
const { results, stderr } = await runLines([
|
||||
'{not json',
|
||||
JSON.stringify(newMsg({ event: { id: 'ok' } })),
|
||||
'also not json',
|
||||
]);
|
||||
|
||||
assert.equal(results.length, 3);
|
||||
assert.equal(results[0].action, 'reject');
|
||||
assert.equal(results[1].action, 'accept');
|
||||
assert.equal(results[2].action, 'reject');
|
||||
assert.match(stderr, /malformed JSON/);
|
||||
});
|
||||
|
||||
test('run() skips blank lines', async () => {
|
||||
const { results } = await runLines(['', ' ', JSON.stringify(newMsg({ event: { id: 'ok' } }))]);
|
||||
assert.equal(results.length, 1);
|
||||
assert.equal(results[0].id, 'ok');
|
||||
});
|
||||
|
||||
test('run() turns unexpected errors into rejections', async () => {
|
||||
const state = { config: { kinds: { has() { throw new Error('boom'); } } } };
|
||||
|
||||
const { results, stderr } = await runLines([JSON.stringify(newMsg({ event: { id: 'boom' } }))], { state });
|
||||
|
||||
assert.equal(results.length, 1);
|
||||
assert.deepEqual(results[0], { id: 'boom', action: 'reject', msg: 'internal error' });
|
||||
assert.match(stderr, /unexpected error/);
|
||||
});
|
||||
@@ -0,0 +1,169 @@
|
||||
import test from 'node:test';
|
||||
import assert from 'node:assert/strict';
|
||||
import { Readable, Writable } from 'node:stream';
|
||||
|
||||
import { createState, decide, run } from '../sync-policy.mjs';
|
||||
|
||||
const STAGING = 'ws://10.1.1.183:7779';
|
||||
|
||||
function syncMsg(overrides = {}) {
|
||||
const { event: eventOverrides, ...rest } = overrides;
|
||||
|
||||
return {
|
||||
type: 'new',
|
||||
sourceType: 'Sync',
|
||||
sourceInfo: STAGING,
|
||||
event: {
|
||||
id: 'evt1',
|
||||
pubkey: 'pk1',
|
||||
kind: 360,
|
||||
tags: [],
|
||||
content: 'hello world',
|
||||
...(eventOverrides || {}),
|
||||
},
|
||||
...rest,
|
||||
};
|
||||
}
|
||||
|
||||
async function runLines(lines, { env, state } = {}) {
|
||||
const input = Readable.from(lines.map((line) => `${line}\n`));
|
||||
let stdout = '';
|
||||
let stderr = '';
|
||||
|
||||
const output = new Writable({ write(chunk, _enc, cb) { stdout += chunk.toString(); cb(); } });
|
||||
const error = new Writable({ write(chunk, _enc, cb) { stderr += chunk.toString(); cb(); } });
|
||||
|
||||
await run({ input, output, error, env, state });
|
||||
|
||||
const results = stdout.split('\n').filter((line) => line !== '').map((line) => JSON.parse(line));
|
||||
return { results, stdout, stderr };
|
||||
}
|
||||
|
||||
test('accepts Sync events from the configured staging relay', () => {
|
||||
const state = createState({ STAGING_RELAY_URL: STAGING });
|
||||
assert.deepEqual(decide(syncMsg(), state), { id: 'evt1', action: 'accept', msg: '' });
|
||||
});
|
||||
|
||||
test('accepts every kind in the default allowlist', () => {
|
||||
const state = createState({ STAGING_RELAY_URL: STAGING });
|
||||
|
||||
for (const kind of [0, 360, 30360, 5]) {
|
||||
const result = decide(syncMsg({ event: { id: `evt-${kind}`, kind } }), state);
|
||||
assert.equal(result.action, 'accept', `kind ${kind} should be accepted`);
|
||||
}
|
||||
});
|
||||
|
||||
test('rejects kinds outside the allowlist', () => {
|
||||
const state = createState({ STAGING_RELAY_URL: STAGING });
|
||||
|
||||
for (const kind of [1, 3, 7, 24133]) {
|
||||
const result = decide(syncMsg({ event: { kind } }), state);
|
||||
assert.equal(result.action, 'reject');
|
||||
assert.match(result.msg, /not allowed/);
|
||||
}
|
||||
});
|
||||
|
||||
test('honours a custom SYNC_KINDS allowlist', () => {
|
||||
const state = createState({ STAGING_RELAY_URL: STAGING, SYNC_KINDS: '360' });
|
||||
|
||||
assert.equal(decide(syncMsg({ event: { kind: 360 } }), state).action, 'accept');
|
||||
assert.equal(decide(syncMsg({ event: { kind: 5 } }), state).action, 'reject');
|
||||
});
|
||||
|
||||
test('rejects events that did not arrive via Sync', () => {
|
||||
const state = createState({ STAGING_RELAY_URL: STAGING });
|
||||
|
||||
for (const sourceType of ['IP4', 'IP6', 'Import', 'Stream', 'Stored']) {
|
||||
const result = decide(syncMsg({ sourceType }), state);
|
||||
assert.equal(result.action, 'reject', `${sourceType} should be rejected`);
|
||||
assert.match(result.msg, /expected sourceType/);
|
||||
}
|
||||
});
|
||||
|
||||
test('rejects Sync events from the wrong source URL', () => {
|
||||
const state = createState({ STAGING_RELAY_URL: STAGING });
|
||||
const result = decide(syncMsg({ sourceInfo: 'ws://evil.example:7779' }), state);
|
||||
|
||||
assert.equal(result.action, 'reject');
|
||||
assert.match(result.msg, /expected sourceInfo/);
|
||||
});
|
||||
|
||||
test('compares sourceInfo byte-exactly', () => {
|
||||
const state = createState({ STAGING_RELAY_URL: STAGING });
|
||||
|
||||
assert.equal(decide(syncMsg({ sourceInfo: `${STAGING} ` }), state).action, 'reject');
|
||||
assert.equal(decide(syncMsg({ sourceInfo: `${STAGING}/` }), state).action, 'reject');
|
||||
assert.equal(decide(syncMsg({ sourceInfo: STAGING.toUpperCase() }), state).action, 'reject');
|
||||
});
|
||||
|
||||
test('trims whitespace around the configured STAGING_RELAY_URL', () => {
|
||||
const state = createState({ STAGING_RELAY_URL: ` ${STAGING}\n` });
|
||||
assert.equal(decide(syncMsg(), state).action, 'accept');
|
||||
});
|
||||
|
||||
test('fails closed when STAGING_RELAY_URL is not configured', () => {
|
||||
const state = createState({});
|
||||
const result = decide(syncMsg(), state);
|
||||
|
||||
assert.equal(result.action, 'reject');
|
||||
assert.match(result.msg, /not configured/);
|
||||
});
|
||||
|
||||
test('rejects non-new message types', () => {
|
||||
const state = createState({ STAGING_RELAY_URL: STAGING });
|
||||
const result = decide(syncMsg({ type: 'lookback' }), state);
|
||||
|
||||
assert.equal(result.action, 'reject');
|
||||
assert.match(result.msg, /unexpected request type/);
|
||||
});
|
||||
|
||||
test('rejects malformed events', () => {
|
||||
const state = createState({ STAGING_RELAY_URL: STAGING });
|
||||
|
||||
assert.equal(decide(syncMsg({ event: { pubkey: '' } }), state).action, 'reject');
|
||||
assert.equal(decide(syncMsg({ event: { kind: 'x' } }), state).action, 'reject');
|
||||
assert.equal(decide(syncMsg({ event: { tags: 'nope' } }), state).action, 'reject');
|
||||
assert.equal(decide(null, state).action, 'reject');
|
||||
});
|
||||
|
||||
test('run() accepts an expected event and rejects the rest', async () => {
|
||||
const { results, stderr } = await runLines(
|
||||
[
|
||||
JSON.stringify(syncMsg({ event: { id: 'a' } })),
|
||||
JSON.stringify(syncMsg({ event: { id: 'b' }, sourceType: 'IP4' })),
|
||||
JSON.stringify(syncMsg({ event: { id: 'c' }, sourceInfo: 'ws://evil.example:7779' })),
|
||||
],
|
||||
{ env: { STAGING_RELAY_URL: STAGING } },
|
||||
);
|
||||
|
||||
assert.deepEqual(results.map((r) => r.action), ['accept', 'reject', 'reject']);
|
||||
assert.match(stderr, /expected sourceType/);
|
||||
assert.match(stderr, /expected sourceInfo/);
|
||||
});
|
||||
|
||||
test('run() rejects malformed JSON without crashing and keeps going', async () => {
|
||||
const { results, stderr } = await runLines(
|
||||
['{not json', JSON.stringify(syncMsg({ event: { id: 'ok' } }))],
|
||||
{ env: { STAGING_RELAY_URL: STAGING } },
|
||||
);
|
||||
|
||||
assert.equal(results.length, 2);
|
||||
assert.equal(results[0].action, 'reject');
|
||||
assert.equal(results[1].action, 'accept');
|
||||
assert.match(stderr, /malformed JSON/);
|
||||
});
|
||||
|
||||
test('run() turns unexpected errors into rejections', async () => {
|
||||
const state = {
|
||||
config: {
|
||||
stagingRelayUrl: STAGING,
|
||||
kinds: { has() { throw new Error('boom'); }, [Symbol.iterator]: function* () {} },
|
||||
},
|
||||
};
|
||||
|
||||
const { results, stderr } = await runLines([JSON.stringify(syncMsg({ event: { id: 'boom' } }))], { state });
|
||||
|
||||
assert.equal(results.length, 1);
|
||||
assert.deepEqual(results[0], { id: 'boom', action: 'reject', msg: 'internal error' });
|
||||
assert.match(stderr, /unexpected error/);
|
||||
});
|
||||
Reference in new issue
Block a user