Commit Graph

2163 Commits

Author SHA1 Message Date
raucao 66481aa4c4 Merge pull request 'Upgrade Sentry to 26.6.0' (#649) from chore/upgrade_sentry into master
Reviewed-on: #649
Reviewed-by: Greg <greg@kosmos.org>
2026-07-13 15:08:14 +00:00
raucao 626662a7ff Merge pull request 'Add a recipe to install the zerotier ohai plugin' (#647) from feature/ohai_zerotier into master
Reviewed-on: #647
Reviewed-by: Râu Cao <raucao@kosmos.org>
2026-07-13 15:00:47 +00:00
greg f2e0a0929a Update garage-16 node file after Chef run 2026-07-13 16:59:19 +02:00
greg 12948be193 Do not manually save node data
It is not needed
2026-07-13 16:58:47 +02:00
greg df4f1236de Clean up white list attributes in client.rb
Both old and new styles. We define our allow list in the client.d file
2026-07-13 16:57:24 +02:00
raucao 875b23c746 Upgrade Sentry to 26.6.0 2026-07-13 16:00:05 +02:00
greg db69e437d5 Remove sanitize node script and pre-commit hook
This is no longer needed with the properly set allowed attributes
2026-07-11 20:08:58 +02:00
greg 663c4da0a0 Update the prometheus-1 node after running Chef
It now contains the attributes from the ohai plugin
2026-07-11 17:35:04 +02:00
greg 010e2d0d64 Add zerotier ohai plugin to the base role 2026-07-11 17:34:39 +02:00
greg 065eac06b1 Add a recipe to install the zerotier ohai plugin
The attributes are saved as "automatic" attributes and look like this:

  "zerotier": {
    "version": "1.16.2",
    "node_id": "NODEID",
    "networks": {
      "NETWORKID": {
        "network_name": "networkname",
        "mac": "ff:ff:ff:ff:ff:ff",
        "status": "OK",
        "type": "PRIVATE",
        "interface": "abcdefg",
        "addresses": [
          "10.1.1.255/24"
        ]
      }
    }
  }

Refs #263
2026-07-11 17:29:31 +02:00
greg 1759af98b8 Simplify and improve chef attribute handling
Set the attributes in a Ruby file under /etc/chef/client.d/ instead of
editing /etc/chef/client.rb
Now uses the new whitelist attribute:
https://docs.chef.io/client/19/cookbooks/attributes/attribute_persistence/

Local chef config also uses the same file. That means there is no longer
a need to use the sanitize script to remove override and default
attributes from nodes.
2026-07-11 17:28:03 +02:00
raucao 736d4effa4 Upgrade RS discourse, increase service timeouts 2026-07-10 11:30:11 +02:00
raucao 1ebe0e4074 Merge pull request 'Update akkounts cookbook, deploy new akkounts VM' (#645) from chore/akkounts into master
Reviewed-on: #645
2026-07-10 09:29:44 +00:00
raucao 9250b2c726 Deploy new akkounts node, remove old one 2026-07-10 11:27:01 +02:00
raucao 6807b8f5bc Update akkounts cookbook, deploy new akkounts VM
* Fix: add missing directory resource
* New nodejs cookbook usage
* Remove Redis entirely (not using local Redis anymore)
2026-07-10 11:24:42 +02:00
raucao 0338e56d4c Merge pull request 'Upgrade nodejs cookbook from 7.3 to 11.0' (#644) from chore/upgrade_nodejs_cookbook into master
Reviewed-on: #644
2026-07-10 09:23:02 +00:00
raucao bd193567ca Upgrade nodejs cookbook from 7.3 to 11.0 2026-07-10 11:19:43 +02:00
raucao 86f7b68726 Merge pull request 'Set up Prometheus and Alertmanager' (#632) from feature/prometheus_cookbook into master
Reviewed-on: #632
2026-07-09 13:33:38 +00:00
greg 51af9c76e0 Fix node attribute 2026-07-09 15:30:38 +02:00
greg f15d5cb958 Move the alertmanager port to an attribute 2026-07-09 15:23:46 +02:00
greg ae14a1f353 Move the node_exporter port to an attribute 2026-07-09 15:18:06 +02:00
raucao b700baac37 Configure alerts for disk/filesystem usage, memory, CPU 2026-07-09 14:04:27 +02:00
raucao 6f93820936 Update node configs 2026-07-07 16:58:36 +02:00
raucao 9fca5386f3 Increase padding for NodeDown rule 2026-07-07 16:57:22 +02:00
raucao b6ba5ba679 Update node configs 2026-07-07 15:20:48 +02:00
raucao 4976888358 Remove unnecessary include 2026-07-07 15:20:33 +02:00
raucao e4ad52d413 Update dirsrv node configs 2026-07-07 15:05:25 +02:00
raucao 94a3c41964 Show severity for new alerts, number of firing vs resolved 2026-07-07 14:26:21 +02:00
raucao 704d0bca0e Document upgrading Chef client 2026-07-06 18:50:07 +02:00
raucao 02dc95b7de Update node configs 2026-07-06 18:47:51 +02:00
raucao 2a14837f64 WIP Add prometheus rules, improve alerts 2026-07-06 18:01:22 +02:00
raucao 1ae5210813 Send critical alerts via email (in addition to XMPP) 2026-07-06 14:52:17 +02:00
raucao 20f67d7261 Change prometheus scrape/eval intervals 2026-07-06 14:52:10 +02:00
raucao 4ce77e1a6e Fix postfix relay sending (on newer Ubuntu versions)
Requires relay restrictions to be configured explicitly
2026-07-06 14:50:49 +02:00
raucao 21513e6d10 Critical icon only for critical severity 2026-07-06 13:53:52 +02:00
raucao 33987402e3 Fix port attribute 2026-07-06 13:16:10 +02:00
raucao 81aae3d1bb Add icons to visualize alert severity 2026-07-06 12:38:07 +02:00
raucao 65c1933a99 WIP Add default alertmanager route and receiver 2026-07-06 12:38:07 +02:00
greg 5e7a0c07b0 Move web port to an attribute 2026-07-05 18:16:18 +02:00
greg 8601e33764 Fix year in license file 2026-07-05 18:03:12 +02:00
raucao c736468259 Merge branch 'master' into feature/prometheus_cookbook 2026-07-05 17:39:08 +02:00
raucao 6370069e9d Add new vm_host attribute/value to all VM nodes 2026-07-05 17:32:43 +02:00
raucao 0e9bcca72d Add vm_host label to prometheus targets 2026-07-05 17:32:14 +02:00
raucao c762dfe8c1 Merge pull request 'Sanitize node configs via script, and do it automatically in a Git hook' (#635) from dev/sanitize_node_configs into feature/prometheus_cookbook
Reviewed-on: #635
2026-07-05 15:26:30 +00:00
raucao 53cf95c4e7 Remove decommissioned nodes 2026-07-05 17:22:48 +02:00
raucao 959d506cfb Remove newline from all node files
Was actually added by manually editing the files, but Chef writes them
without newline, so our sanitization script also does it that way
2026-07-05 17:22:39 +02:00
raucao 8c71261fc9 Add pre-commit hook for sanitizing node configs 2026-07-05 17:22:34 +02:00
raucao 6c3da8a711 Add util script for sanitizing nodes
Deletes the "override" and "default" properties, which may contain
sensitive data
2026-07-05 17:22:26 +02:00
raucao 53de8ea119 Merge branch 'master' into feature/prometheus_cookbook 2026-07-05 17:20:48 +02:00
raucao 639800a838 Remove decommissioned node 2026-07-05 17:19:16 +02:00