Add l6@kosmos.org alias for hackerhouse@kosmos.org

Points the submodule at the re-converged garage-11 node object (Cinc
Client 19.4.46). Normal attributes (knife_zero, vm_host) are unchanged.

.cinc/config.rb.example

@@ -8,11 +8,10 @@ data_bag_path             "data_bags"
 encrypted_data_bag_secret "#{current_dir}/encrypted_data_bag_secret"
 local_mode                true # Chef local mode, replacing Solo
 
-# Added to /etc/chef/client.rb on node bootstrap
+# Added to /etc/cinc/client.rb on node bootstrap
 # https://docs.chef.io/attribute_persistence/
 # Source of truth: site-cookbooks/kosmos-base/files/default/chef_attributes.rb
 client_d_dir(File.expand_path("../site-cookbooks/kosmos-base/files/default/chef_attributes.rb"))
 
-# Chef binaries on packages.chef.io are license-gated (HTTP 402); use the
+# Cinc Workstation's knife already installs Cinc Client from omnitruck.cinc.sh
-# community Cinc build for bootstraps and converged nodes.
+# by default, so no `knife[:alter_project]` is needed here.
-knife[:alter_project] = "cinc"

.gitignore

@@ -1,7 +1,4 @@
-/.chef/encrypted_data_bag_secret
+/.cinc/encrypted_data_bag_secret
-/.chef/chef_guid
+/.cinc/chef_guid
-/.chef/config.rb
+/.cinc/config.rb
-/.bundle/
+.kitchen
-/.vagrant/
-/.kitchen
-/nodes/vagrant-node.json

.ruby-version

@@ -1 +1 @@
-chef-workstation
+cinc-workstation

AGENTS.md

@@ -5,7 +5,7 @@ Welcome, AI Agent! This file contains essential context and rules for interactin
 ## 🏢 Project Overview
 This repository contains the infrastructure automation code used by Kosmos to provision and configure bare metal servers (KVM hosts) and Ubuntu virtual machines (KVM guests). 
 
-We use **Chef Infra**, managed locally via **Knife Zero** (agentless Chef), and **Berkshelf** for dependency management.
+We use **Chef Infra**, managed locally via **Knife Zero** (agentless Chef), and **Berkshelf** for dependency management. Nodes run the community **Cinc Client** distribution (the license-free Chef Infra Client) instead of Chef Infra Client, and everything is driven by **Cinc Workstation** (the community fork of Chef Workstation). On-node configuration lives in `/etc/cinc`, and the repo's knife config lives in `.cinc/`.
 
 ## 📂 Directory Structure & Rules
 
@@ -13,10 +13,11 @@ We use **Chef Infra**, managed locally via **Knife Zero** (agentless Chef), and
 * **`cookbooks/`**: 🔴 **DO NOT EDIT.** This directory contains third-party/community cookbooks that are vendored. These are managed by Berkshelf. Modifying them directly will result in lost changes.
 * **`roles/`**: 🟢 **EDITABLE.** Contains Chef roles written in Ruby (e.g., `base.rb`, `kvm_guest.rb`, `postgresql_primary.rb`). These define run-lists and role-specific default attributes for servers.
 * **`environments/`**: Contains Chef environment definitions (like `production.rb`).
-* **`data_bags/`**: Contains data bag configurations, often encrypted. Be cautious and do not expose secrets. (Note: Agents should not manage data bag secrets directly unless provided the `.chef/encrypted_data_bag_secret`).
+* **`data_bags/`**: Contains data bag configurations, often encrypted. Be cautious and do not expose secrets. (Note: Agents should not manage data bag secrets directly unless provided the `.cinc/encrypted_data_bag_secret`).
 * **`nodes/`**: Contains JSON state files for bootstrapped nodes. *Agents typically do not edit these directly unless cleaning up a deleted node.*
 * **`Berksfile`**: Defines community cookbook dependencies.
-* **`Vagrantfile` / `.kitchen/`**: Used for local virtualization and integration testing.
+* **`.cinc/`**: Local knife configuration (`config.rb`, `encrypted_data_bag_secret`). Untracked; `.cinc/config.rb.example` is the template.
+* **`kitchen.yml` / `.kitchen/`**: Test Kitchen integration suites. They run against Cinc Client via Cinc Workstation's `kitchen-dokken` plugin, so invoke them with `kitchen` (see `README.md`).
 
 ## 🛠️ Tooling & Workflows
 
@@ -28,6 +29,7 @@ We use **Chef Infra**, managed locally via **Knife Zero** (agentless Chef), and
 2. **Provisioning (Knife Zero)**
    - Bootstrapping and converging nodes is done using `knife zero`. 
    - *Example:* `knife zero converge name:server-name.kosmos.org`
+   - To migrate an existing (Chef) node to Cinc Client: `scripts/knife/migrate_node_to_cinc.sh <node>`, then converge as usual.
 
 3. **Code Style & Conventions**
    - Chef recipes, resources, and roles are written in **Ruby**. 

Gemfile

@@ -1,3 +0,0 @@
-source 'https://rubygems.org'
-
-gem 'knife-zero', '~> 2.6.0'

Gemfile.lock

@@ -1,440 +0,0 @@
-GEM
-  remote: https://rubygems.org/
-  specs:
-    abbrev (0.1.2)
-    addressable (2.9.0)
-      public_suffix (>= 2.0.2, < 8.0)
-    ast (2.4.3)
-    aws-eventstream (1.4.0)
-    aws-partitions (1.1263.0)
-    aws-sdk-core (3.252.0)
-      aws-eventstream (~> 1, >= 1.3.0)
-      aws-partitions (~> 1, >= 1.992.0)
-      aws-sigv4 (~> 1.9)
-      base64
-      bigdecimal
-      jmespath (~> 1, >= 1.6.1)
-      logger
-    aws-sdk-kms (1.129.0)
-      aws-sdk-core (~> 3, >= 3.248.0)
-      aws-sigv4 (~> 1.5)
-    aws-sdk-s3 (1.226.0)
-      aws-sdk-core (~> 3, >= 3.248.0)
-      aws-sdk-kms (~> 1)
-      aws-sigv4 (~> 1.5)
-    aws-sdk-secretsmanager (1.133.0)
-      aws-sdk-core (~> 3, >= 3.248.0)
-      aws-sigv4 (~> 1.5)
-    aws-sigv4 (1.12.1)
-      aws-eventstream (~> 1, >= 1.0.2)
-    base64 (0.3.0)
-    bcrypt_pbkdf (1.1.2)
-    bcrypt_pbkdf (1.1.2-arm64-darwin)
-    bcrypt_pbkdf (1.1.2-x86_64-darwin)
-    benchmark (0.5.0)
-    bigdecimal (4.1.2)
-    builder (3.3.0)
-    chef (19.3.15)
-      addressable
-      aws-sdk-s3 (~> 1.91)
-      aws-sdk-secretsmanager (~> 1.46)
-      bcrypt_pbkdf (~> 1.0)
-      chef-config (= 19.3.15)
-      chef-licensing (~> 1.3)
-      chef-utils (= 19.3.15)
-      chef-vault
-      chef-zero (~> 15.1.0)
-      corefoundation (~> 0.3.4)
-      csv (~> 3.3.5)
-      diff-lcs (~> 1.6.0)
-      ed25519 (~> 1.2)
-      erubis (~> 2.7)
-      ffi (>= 1.15.5, < 1.18.0)
-      ffi-libarchive (~> 1.0, >= 1.0.3)
-      ffi-yajl (>= 2.2, < 4.0)
-      iniparse (~> 1.4)
-      inspec-core (~> 7.0.107)
-      license-acceptance (>= 1.0.5, < 3)
-      mixlib-archive (>= 0.4, < 2.0)
-      mixlib-authentication (>= 2.1, < 4)
-      mixlib-cli (>= 2.1.1, < 3.0)
-      mixlib-log (>= 2.0.3, < 4.0)
-      mixlib-shellout (>= 3.3.8, < 3.5.0)
-      net-ftp
-      net-sftp (>= 2.1.2, < 5.0)
-      ohai (~> 19.0)
-      plist (~> 3.2)
-      proxifier2 (~> 1.1)
-      syslog
-      syslog-logger (~> 1.6)
-      train-core (~> 3.13, >= 3.13.4)
-      train-rest (>= 0.4.1)
-      train-winrm (>= 0.2.17)
-      unf_ext (~> 0.0.9.1)
-      uri (>= 1.0.4, < 1.2.0)
-      vault (>= 0.18.2, < 0.21.0)
-    chef-config (19.3.15)
-      addressable
-      chef-utils (= 19.3.15)
-      fuzzyurl
-      mixlib-config (>= 2.2.12, < 4.0)
-      mixlib-shellout (>= 2.0, < 4.0)
-      racc
-      tomlrb (>= 1.2, < 3.0)
-    chef-gyoku (1.5.0)
-      builder (>= 2.1.2)
-      rexml (~> 3.4)
-    chef-licensing (1.4.1)
-      chef-config (>= 15)
-      faraday (>= 1, < 3)
-      faraday-http-cache
-      mixlib-log (~> 3.0)
-      ostruct (~> 0.6.0)
-      pstore (~> 0.1.1)
-      tty-prompt (~> 0.23)
-      tty-spinner (~> 0.9.3)
-    chef-telemetry (1.1.1)
-      chef-config
-      concurrent-ruby (~> 1.0)
-    chef-utils (19.3.15)
-      concurrent-ruby
-    chef-vault (4.2.12)
-      syslog (~> 0.3)
-    chef-winrm (2.5.0)
-      builder (>= 2.1.2)
-      chef-gyoku (~> 1.5)
-      erubi (~> 1.8)
-      gssapi (~> 1.2)
-      httpclient (~> 2.2, >= 2.2.0.2)
-      logging (>= 1.6.1, < 3.0)
-      nori (~> 2.7)
-      rexml (>= 3.4.2, < 4.0)
-      rubyntlm (~> 0.6.0, >= 0.6.3)
-    chef-winrm-elevated (1.2.5)
-      chef-winrm (>= 2.3.11)
-      chef-winrm-fs (>= 1.3.7)
-      erubi (~> 1.8)
-    chef-winrm-fs (1.4.2)
-      benchmark (~> 0.5.0)
-      chef-winrm (~> 2.4)
-      csv (~> 3.3)
-      erubi (>= 1.7)
-      logging (>= 1.6.1, < 3.0)
-      rubyzip (~> 2.0)
-    chef-zero (15.1.11)
-      ffi-yajl (>= 2.2, < 4.0)
-      hashie (>= 2.0, < 6.0)
-      mixlib-log (>= 2.0, < 4.0)
-      rack (~> 3.2, >= 3.2.6)
-      rackup (~> 2.3, >= 2.3.1)
-      uuidtools (>= 2.1, < 4.0)
-      webrick
-    coderay (1.1.3)
-    concurrent-ruby (1.3.7)
-    connection_pool (2.5.5)
-    cookstyle (8.7.6)
-      rubocop (= 1.86.1)
-    corefoundation (0.3.19)
-      ffi (>= 1.15.0)
-    csv (3.3.5)
-    date (3.5.1)
-    diff-lcs (1.6.2)
-    domain_name (0.6.20240107)
-    ed25519 (1.4.0)
-    erubi (1.13.1)
-    erubis (2.7.0)
-    faraday (2.14.3)
-      faraday-net_http (>= 2.0, < 3.5)
-      json
-      logger
-    faraday-follow_redirects (0.5.0)
-      faraday (>= 1, < 3)
-    faraday-http-cache (2.5.1)
-      faraday (>= 0.8)
-    faraday-net_http (3.4.4)
-      net-http (~> 0.5)
-    ffi (1.17.4-arm64-darwin)
-    ffi (1.17.4-x86_64-darwin)
-    ffi (1.17.4-x86_64-linux-gnu)
-    ffi-libarchive (1.1.14)
-      ffi (~> 1.0)
-    ffi-yajl (2.7.11)
-      libyajl2 (>= 2.1)
-    fuzzyurl (0.9.0)
-    gssapi (1.3.1)
-      ffi (>= 1.0.1)
-    hashie (5.1.0)
-      logger
-    highline (3.1.2)
-      reline
-    http-accept (1.7.0)
-    http-cookie (1.1.6)
-      domain_name (~> 0.5)
-    httpclient (2.9.0)
-      mutex_m
-    iniparse (1.5.0)
-    inspec-core (7.0.107)
-      addressable (~> 2.4)
-      chef-licensing (>= 1.2.0)
-      chef-telemetry (~> 1.0, >= 1.0.8)
-      cookstyle
-      csv (~> 3.0)
-      faraday (>= 1, < 3)
-      faraday-follow_redirects (~> 0.3)
-      hashie (>= 3.4, < 6.0)
-      license-acceptance (>= 0.2.13, < 3.0)
-      method_source (>= 0.8, < 2.0)
-      mixlib-log (~> 3.0)
-      multipart-post (~> 2.0)
-      ostruct (>= 0.1, < 0.7)
-      parallel (~> 1.9)
-      parslet (>= 1.5, < 3.0)
-      pry (~> 0.13)
-      rspec (>= 3.9, <= 3.14)
-      rspec-its (>= 1.2, < 3.0)
-      rubyzip (>= 1.2.2, < 4.0)
-      semverse (~> 3.0)
-      sslshake (~> 1.2)
-      syslog (~> 0.1)
-      thor (>= 0.20, < 1.5.0)
-      tomlrb (>= 1.3, < 2.1)
-      train-core (~> 3.16, >= 3.16.1)
-      tty-prompt (~> 0.17)
-      tty-table (~> 0.10)
-    io-console (0.8.2)
-    ipaddress (0.8.3)
-    jmespath (1.6.2)
-    json (2.20.0)
-    knife (19.0.134)
-      abbrev
-      bcrypt_pbkdf (~> 1.1)
-      chef-licensing (~> 1.2)
-      chef-vault
-      ed25519 (>= 1.2, < 2.0)
-      erubis (~> 2.7)
-      ffi (>= 1.15, < 1.18.0)
-      ffi-yajl (>= 2.2, < 3.0)
-      highline (>= 1.6.9, < 4)
-      license-acceptance (>= 1.0.5, < 3)
-      mixlib-archive (>= 0.4, < 2.0)
-      mixlib-cli (>= 2.1.1, < 3.0)
-      net-ssh (>= 5.1, < 8)
-      net-ssh-multi (~> 1.2, >= 1.2.1)
-      pastel
-      proxifier2 (~> 1.1)
-      train-core (~> 3.13, >= 3.13.4)
-      train-winrm (>= 0.2.17)
-      tty-prompt (~> 0.21)
-      tty-screen (~> 0.6)
-      tty-table (~> 0.11)
-    knife-zero (2.6.0)
-      chef (>= 16.6)
-      knife (>= 17.0)
-    language_server-protocol (3.17.0.5)
-    libyajl2 (2.1.0)
-    license-acceptance (2.1.13)
-      pastel (~> 0.7)
-      tomlrb (>= 1.2, < 3.0)
-      tty-box (~> 0.6)
-      tty-prompt (~> 0.20)
-    lint_roller (1.1.0)
-    little-plugger (1.1.4)
-    logger (1.7.0)
-    logging (2.4.0)
-      little-plugger (~> 1.1)
-      multi_json (~> 1.14)
-    method_source (1.1.0)
-    mime-types (3.7.0)
-      logger
-      mime-types-data (~> 3.2025, >= 3.2025.0507)
-    mime-types-data (3.2026.0701)
-    mixlib-archive (1.3.3)
-      mixlib-log
-    mixlib-authentication (3.0.10)
-    mixlib-cli (2.1.8)
-    mixlib-config (3.0.27)
-      tomlrb
-    mixlib-log (3.2.3)
-      ffi (>= 1.15.5)
-    mixlib-shellout (3.4.10)
-      chef-utils
-    multi_json (1.19.1)
-    multipart-post (2.4.1)
-    mutex_m (0.3.0)
-    net-ftp (0.3.9)
-      net-protocol
-      time
-    net-http (0.9.1)
-      uri (>= 0.11.1)
-    net-http-persistent (4.0.8)
-      connection_pool (>= 2.2.4, < 4)
-    net-protocol (0.2.2)
-      timeout
-    net-scp (4.1.0)
-      net-ssh (>= 2.6.5, < 8.0.0)
-    net-sftp (4.0.0)
-      net-ssh (>= 5.0.0, < 8.0.0)
-    net-ssh (7.3.3)
-    net-ssh-gateway (2.0.0)
-      net-ssh (>= 4.0.0)
-    net-ssh-multi (1.2.1)
-      net-ssh (>= 2.6.5)
-      net-ssh-gateway (>= 1.2.0)
-    netrc (0.11.0)
-    nori (2.7.1)
-      bigdecimal
-    ohai (19.1.40)
-      base64
-      chef-config (>= 14.12, < 20)
-      chef-utils (>= 16.0, < 20)
-      ffi (>= 1.15.5)
-      ffi-yajl (>= 2.2, < 3.0)
-      ipaddress
-      mixlib-cli (>= 1.7.0)
-      mixlib-config (>= 2.0, < 4.0)
-      mixlib-log (>= 2.0.1, < 4.0)
-      mixlib-shellout (>= 3.3.6, < 3.5.0)
-      plist (~> 3.1)
-      train-core
-      wmi-lite (~> 1.0)
-    ostruct (0.6.3)
-    parallel (1.28.0)
-    parser (3.3.11.1)
-      ast (~> 2.4.1)
-      racc
-    parslet (2.0.0)
-    pastel (0.8.0)
-      tty-color (~> 0.5)
-    plist (3.7.2)
-    prism (1.9.0)
-    proxifier2 (1.1.0)
-    pry (0.16.0)
-      coderay (~> 1.1)
-      method_source (~> 1.0)
-      reline (>= 0.6.0)
-    pstore (0.1.4)
-    public_suffix (6.0.2)
-    racc (1.8.1)
-    rack (3.2.6)
-    rackup (2.3.1)
-      rack (>= 3)
-    rainbow (3.1.1)
-    regexp_parser (2.12.0)
-    reline (0.6.3)
-      io-console (~> 0.5)
-    rest-client (2.1.0)
-      http-accept (>= 1.7.0, < 2.0)
-      http-cookie (>= 1.0.2, < 2.0)
-      mime-types (>= 1.16, < 4.0)
-      netrc (~> 0.8)
-    rexml (3.4.4)
-    rspec (3.13.2)
-      rspec-core (~> 3.13.0)
-      rspec-expectations (~> 3.13.0)
-      rspec-mocks (~> 3.13.0)
-    rspec-core (3.13.6)
-      rspec-support (~> 3.13.0)
-    rspec-expectations (3.13.5)
-      diff-lcs (>= 1.2.0, < 2.0)
-      rspec-support (~> 3.13.0)
-    rspec-its (2.0.0)
-      rspec-core (>= 3.13.0)
-      rspec-expectations (>= 3.13.0)
-    rspec-mocks (3.13.8)
-      diff-lcs (>= 1.2.0, < 2.0)
-      rspec-support (~> 3.13.0)
-    rspec-support (3.13.7)
-    rubocop (1.86.1)
-      json (~> 2.3)
-      language_server-protocol (~> 3.17.0.2)
-      lint_roller (~> 1.1.0)
-      parallel (>= 1.10)
-      parser (>= 3.3.0.2)
-      rainbow (>= 2.2.2, < 4.0)
-      regexp_parser (>= 2.9.3, < 3.0)
-      rubocop-ast (>= 1.49.0, < 2.0)
-      ruby-progressbar (~> 1.7)
-      unicode-display_width (>= 2.4.0, < 4.0)
-    rubocop-ast (1.49.1)
-      parser (>= 3.3.7.2)
-      prism (~> 1.7)
-    ruby-progressbar (1.13.0)
-    rubyntlm (0.6.5)
-      base64
-    rubyzip (2.4.1)
-    semverse (3.0.2)
-    socksify (1.8.1)
-    sslshake (1.3.1)
-    strings (0.2.1)
-      strings-ansi (~> 0.2)
-      unicode-display_width (>= 1.5, < 3.0)
-      unicode_utils (~> 1.4)
-    strings-ansi (0.2.0)
-    syslog (0.4.0)
-      logger
-    syslog-logger (1.6.8)
-    thor (1.4.0)
-    time (0.4.2)
-      date
-    timeout (0.6.1)
-    tomlrb (2.0.4)
-    train-core (3.16.5)
-      addressable (~> 2.5)
-      ffi (>= 1.16.0, < 1.18)
-      json (>= 2.19.2, < 3.0)
-      mixlib-shellout (>= 2.0, < 4.0)
-      net-scp (>= 1.2, < 5.0)
-      net-ssh (>= 2.9, < 8.0)
-    train-rest (0.5.0)
-      aws-sigv4 (~> 1.5)
-      rest-client (~> 2.1)
-      train-core (~> 3.0)
-    train-winrm (0.4.3)
-      chef-winrm (>= 2.4.4, < 3.0)
-      chef-winrm-elevated (>= 1.2.5, < 2.0)
-      chef-winrm-fs (>= 1.4.1, < 2.0)
-      socksify (~> 1.8)
-    tty-box (0.7.0)
-      pastel (~> 0.8)
-      strings (~> 0.2.0)
-      tty-cursor (~> 0.7)
-    tty-color (0.6.0)
-    tty-cursor (0.7.1)
-    tty-prompt (0.23.1)
-      pastel (~> 0.8)
-      tty-reader (~> 0.8)
-    tty-reader (0.9.0)
-      tty-cursor (~> 0.7)
-      tty-screen (~> 0.8)
-      wisper (~> 2.0)
-    tty-screen (0.8.2)
-    tty-spinner (0.9.3)
-      tty-cursor (~> 0.7)
-    tty-table (0.12.0)
-      pastel (~> 0.8)
-      strings (~> 0.2.0)
-      tty-screen (~> 0.8)
-    unf_ext (0.0.9.1)
-    unicode-display_width (2.6.0)
-    unicode_utils (1.4.0)
-    uri (1.1.1)
-    uuidtools (3.0.0)
-    vault (0.20.1)
-      aws-sigv4
-      base64
-      net-http-persistent (~> 4.0, >= 4.0.2)
-    webrick (1.9.2)
-    wisper (2.0.1)
-    wmi-lite (1.0.7)
-
-PLATFORMS
-  arm64-darwin-22
-  x86_64-darwin-18
-  x86_64-darwin-19
-  x86_64-linux
-
-DEPENDENCIES
-  knife-zero (~> 2.6.0)
-
-BUNDLED WITH
-   2.2.15

README.md

@@ -2,45 +2,61 @@ This repository contains all infrastructure automation code that we use to set
 up and configure servers, virtual machines, and applications for Kosmos hosted
 services.
 
-Chef cookbooks are written in Ruby, and based on [Chef Infra
+Cookbooks are written in Ruby, and based on [Chef Infra
 resources](https://docs.chef.io/resources/). Some cookbooks contain integration
 test suites based on [Test Kitchen](https://docs.chef.io/workstation/kitchen/).
 
+Nodes run the community [Cinc Client](https://cinc.sh/) distribution of Chef
+Infra Client instead of the license-gated Chef Infra Client. Everything — the
+`knife zero` workflow and the Test Kitchen suites — is driven by
+[Cinc Workstation](https://cinc.sh/docs/workstation/), and on-node
+configuration lives in `/etc/cinc`.
+
 Note: Manual configuration of servers and applications is highly discouraged,
 and can be overwritten or lost without notice!
 
 ## Setup
 
-### Install Chef Workstation
+### Install Cinc Workstation
 
-* macOS, Windows, RHEL, Ubuntu: https://docs.chef.io/workstation/install_workstation/
+Cinc Workstation replaces Chef Workstation. Install it following
-* Arch Linux: https://aur.archlinux.org/packages/chef-workstation
+https://cinc.sh/docs/workstation/ — or, on Arch Linux, the `cinc-workstation-bin`
+AUR package. It installs to `/opt/cinc-workstation` and reads its user config
+from `~/.cinc-workstation`.
 
 #### rbenv
 
 If you use rbenv to manage Ruby versions on your system, install the
-[rbenv-chef-workstation](https://github.com/docwhat/rbenv-chef-workstation)
+`rbenv-cinc-workstation` plugin (the Cinc port of
-plugin.
+[rbenv-chef-workstation](https://github.com/docwhat/rbenv-chef-workstation)),
+create an empty `$(rbenv root)/versions/cinc-workstation` directory, and run
+`rbenv rehash`. This repository pins the version in `.ruby-version`.
 
-### Install gem dependencies
+### Install knife-zero
 
-Clone this repository, `cd` into it, and run:
+`knife-zero` is not bundled with Cinc Workstation. Install it into Cinc's user
+gem directory:
 
-    bundle install
+    cinc gem install knife-zero
+
+### Configure knife
+
+Copy `.cinc/config.rb.example` to `.cinc/config.rb` and adjust
+`encrypted_data_bag_secret` if your secret lives elsewhere.
 
 ## Common tasks
 
 ### Bootstrap a new host server
 
-    knife zero bootstrap root@server-name.kosmos.org --run-list "role[base],role[kvm_host]" --secret-file .chef/encrypted_data_bag_secret
+    knife zero bootstrap root@server-name.kosmos.org --run-list "role[base],role[kvm_host]" --secret-file .cinc/encrypted_data_bag_secret
 
 ### Bootstrap a new VM
 
-    knife zero bootstrap ubuntu@zerotier-ip-address -x ubuntu --sudo --run-list "role[base],role[kvm_guest]" --secret-file .chef/encrypted_data_bag_secret
+    knife zero bootstrap ubuntu@zerotier-ip-address -x ubuntu --sudo --run-list "role[base],role[kvm_guest]" --secret-file .cinc/encrypted_data_bag_secret
 
 ### Bootstrap a new VM with environment and role/app (postgres replica as example)
 
-    knife zero bootstrap ubuntu@10.1.1.134 -x ubuntu --sudo --environment production --run-list "role[base],role[kvm_guest],role[postgresql_replica]" --secret-file .chef/encrypted_data_bag_secret
+    knife zero bootstrap ubuntu@10.1.1.134 -x ubuntu --sudo --environment production --run-list "role[base],role[kvm_guest],role[postgresql_replica]" --secret-file .cinc/encrypted_data_bag_secret
 
 ### Run Chef Zero on a host server
 
@@ -50,13 +66,17 @@ Clone this repository, `cd` into it, and run:
 
     knife zero converge -a knife_zero.host name:vm-name-23
 
-### Update Chef Client on a server:
+### Update Cinc Client on a server
 
-    knife zero converge name:dev.kosmos.org --client-version 15.3.14
+    knife zero converge name:dev.kosmos.org --client-version 18.2.7
 
-### Upgrade Chef Client on a node:
+### Upgrade Cinc Client on a node
 
-    knife ssh -a knife_zero.host 'name:foo-1' 'curl -L https://chef.io/chef/install.sh | sudo bash -s -- -v 18.10.17'
+    knife ssh -a knife_zero.host 'name:foo-1' 'curl -L https://omnitruck.cinc.sh/install.sh | sudo bash -s -- -v 19'
+
+### Migrate an existing node to Cinc Client
+
+    scripts/knife/migrate_node_to_cinc.sh node-name-1   # then converge as usual
 
 ## Managing cookbooks
 
@@ -70,13 +90,12 @@ Vendor installed cookbooks to the `cookbooks/` dir:
 
     berks vendor cookbooks/ --delete
 
-## "Expired" TLS certificates
+## Integration tests
 
-If you encounter expired TLS certificates during a Chef run (e.g. for remote
+Some cookbooks contain Test Kitchen suites. They are configured to run against
-files), the issue is likely that the certificate has been issued by Let's
+Cinc Client (`provisioner: product_name: cinc`) and use Cinc Workstation's
-Encrypt and Chef is still using its own, outdated CA cert store (see
+bundled `kitchen` (which ships a Cinc-capable `kitchen-dokken`):
-[here](https://github.com/chef/chef/issues/12126#issuecomment-932067530) for
-example).
 
-As a hotfix, you can manually remove the "DST Root CA X3" cert from
+    kitchen test
-`/opt/chef/embedded/ssl/cert.pem` on the machine you're trying to converge.
+
+(use `kitchen converge`/`verify` while iterating on a single suite).

Vagrantfile

@@ -1,92 +0,0 @@
-# -*- mode: ruby -*-
-# vi: set ft=ruby :
-
-# All Vagrant configuration is done below. The "2" in Vagrant.configure
-# configures the configuration version.
-#
-# Documentation for all configuration options can be found here:
-# https://www.vagrantup.com/docs/vagrantfile
-Vagrant.configure(2) do |config|
-
-  config.vm.provider :virtualbox do |vb|
-    vb.memory = "1024"
-    vb.cpus = 2
-  end
-
-  # config.vm.provider :docker do |d, override|
-  #   d.image = "nishidayuya/docker-vagrant-ubuntu:xenial"
-  #   d.has_ssh = true
-  #   override.vm.box = nil
-  #   override.ssh.port = 22
-  # end
-
-  config.vm.define "ldap" do |ldap|
-    ldap.vm.box = "bento/ubuntu-18.04"
-    ldap.vm.network "forwarded_port", guest: 389, host: 389
-    ldap.vm.network "private_network", ip: "192.168.56.5"
-
-    ldap.vm.provision :chef_zero do |chef|
-      chef.version                            = "18.2.7"
-      chef.node_name                          = "vagrant-node-ldap"
-      chef.arguments                          = "--chef-license accept"
-      chef.cookbooks_path                     = ["cookbooks", "site-cookbooks"]
-      chef.data_bags_path                     = "data_bags"
-      chef.roles_path                         = "roles"
-      chef.nodes_path                         = "nodes"
-      chef.environments_path                  = "environments"
-      chef.encrypted_data_bag_secret_key_path = ".chef/encrypted_data_bag_secret"
-      chef.environment                        = "development"
-
-      chef.add_recipe "kosmos-base"
-      chef.add_recipe "kosmos-dirsrv"
-    end
-  end
-
-  config.vm.define "bitcoin" do |bitcoin|
-    bitcoin.vm.box = "generic/ubuntu2004"
-    bitcoin.vm.network "forwarded_port", guest: 4201, host: 4201
-    bitcoin.vm.network "private_network", ip: "192.168.56.6"
-    # bitcoin.vm.synced_folder "../kredits/yap/", "/opt/yap"
-
-    bitcoin.vm.provision :chef_zero do |chef|
-      chef.version                            = "18.2.7"
-      chef.node_name                          = "vagrant-node-bitcoin"
-      chef.arguments                          = "--chef-license accept"
-      chef.cookbooks_path                     = ["cookbooks", "site-cookbooks"]
-      chef.data_bags_path                     = "data_bags"
-      chef.roles_path                         = "roles"
-      chef.nodes_path                         = "nodes"
-      chef.environments_path                  = "environments"
-      chef.encrypted_data_bag_secret_key_path = ".chef/encrypted_data_bag_secret"
-      chef.environment                        = "development"
-
-      chef.add_recipe "kosmos-base"
-      chef.add_recipe "kosmos-bitcoin::source"
-      # chef.add_recipe "kosmos-bitcoin::foo"
-    end
-  end
-
-  config.vm.define "openresty" do |openresty|
-    openresty.vm.box = "bento/ubuntu-20.04"
-    openresty.vm.network "forwarded_port", guest: 6379, host: 6379
-    openresty.vm.network "private_network", ip: "192.168.56.7"
-
-    openresty.vm.provision :chef_zero do |chef|
-      chef.version                            = "18.2.7"
-      chef.node_name                          = "vagrant-openresty"
-      chef.arguments                          = "--chef-license accept"
-      chef.cookbooks_path                     = ["cookbooks", "site-cookbooks"]
-      chef.data_bags_path                     = "data_bags"
-      chef.roles_path                         = "roles"
-      chef.nodes_path                         = "nodes"
-      chef.environments_path                  = "environments"
-      chef.encrypted_data_bag_secret_key_path = ".chef/encrypted_data_bag_secret"
-      chef.environment                        = "development"
-
-      chef.add_recipe "kosmos-base"
-      chef.add_role "openresty"
-      chef.add_role "openresty_proxy"
-    end
-  end
-
-end

doc/ldap.md

@@ -6,7 +6,7 @@ The admin account is `cn=Directory Manager` with the password stored in the
 encrypted data bag `admin_password` item
 
 ```
-$ knife data bag show credentials dirsrv --secret-file .chef/encrypted_data_bag_secret
+$ knife data bag show credentials dirsrv --secret-file .cinc/encrypted_data_bag_secret
 ```
 
 ## Example search (will prompt for password):

doc/test-kitchen-open-tasks.md

@@ -0,0 +1,60 @@
+# Test Kitchen: open tasks
+
+Status of the Test Kitchen integration suites after the migration to Cinc
+Workstation. Suites are run with Cinc Workstation's bundled `kitchen`
+(`kitchen-dokken` 2.25.0), which provisions Cinc Client via
+`provisioner: product_name: cinc`.
+
+## Suites with tests
+
+| Cookbook | Test | Berksfile | State |
+| --- | --- | --- | --- |
+| `kosmos-ejabberd` | `test/integration/default` | complete (path + pinned deps) | untested |
+| `kosmos_email` | `test/integration/default` | added | not green yet |
+| `kosmos_garage` | `test/integration/default` | fixed (local path deps) | untested |
+| `kosmos_rsk` | `test/integration/rskj` | complete | green (verified) |
+
+## Suites without tests
+
+`kosmos_liquor-cabinet` and `liquor_cabinet` declare
+`verifier.inspec_tests: test/integration/default`, but that directory does not
+exist. Either add real InSpec tests or remove the `inspec_tests` entries.
+
+## Fixes applied during the migration
+
+- Cinc provisioning verified: `kosmos_rsk` and `kosmos_email` both reached a
+  running `cinc-client 18.2.7` inside Dokken (no Chef license prompt).
+- `kosmos_rsk` suite passes end to end (9/9) after the fixes below, confirming
+  the Cinc Workstation → `kitchen-dokken` 2.25 → `product_name: cinc` pipeline.
+- `kosmos_email`: added a `Berksfile` (there was none, so declared dependencies
+  like `hostname` could not resolve).
+- `kosmos_garage`: added local `path:` dependencies
+  (`kosmos_openresty`, `kosmos-base`, `openresty`, `kosmos-postfix`) so Berkshelf
+  no longer tries to fetch internal cookbooks from Supermarket.
+- `kosmos_rsk`: bumped the `rskj` pin from `9.0.1~<codename>` to
+  `9.0.4~<codename>` (matching the current version in `ppa:rsksmart/rskj`), and
+  updated the matching assertion in `test/integration/rskj/rskj_test.rb`.
+
+## Remaining tasks
+
+1. **`kosmos_email` is not green yet.** Even with the Berksfile, the default
+   recipe pulls in a lot (`unbound`, `opendkim`, `spamassassin`, `postfix`,
+   `dovecot`, `tls_cert_for` from `kosmos-base`, firewall rules). It likely needs
+   a reduced/lifecycle setup before it converges in Dokken.
+2. **`kosmos_garage`** has not been run since the Berksfile fix; it uses
+   Ubuntu 20.04 and needs its data bag secret.
+3. **`kosmos-ejabberd`** is untested. It has the most complete Berksfile and a
+   `pre_converge` lifecycle that prepares PostgreSQL, so it is the most likely
+   suite to pass — worth doing next.
+4. **Liquor suites**: add tests or drop the dangling `inspec_tests`.
+5. **Benign warning**: knife-zero double-loads `knife ssh` under knife 19,
+   printing `already initialized constant Chef::Knife::Ssh::SUDO_PASSWORD_PROMPT`.
+   Harmless; would need an upstream knife-zero fix to silence.
+
+## Notes
+
+- Suites live in `site-cookbooks/<name>/kitchen.yml`; run them with
+  `kitchen test` from that directory (Cinc Workstation must be on the PATH, see
+  the repository `README.md`).
+- Five cookbooks are git submodules (`blossom`, `deno`, `openresty`,
+  `postgresql`, `strfry`); their suites would need changes in their own repos.

environments/development.json

@@ -1,6 +1,6 @@
 {
   "name": "development",
-  "description": "development environment, for example for Vagrant",
+  "description": "development environment, for example for local Test Kitchen runs",
   "json_class": "Chef::Environment",
   "chef_type": "environment",
   "default_attributes": {

environments/production.json

@@ -48,7 +48,8 @@
         "hostmaster@kosmos.org":  "mail@kosmos.org",
         "postmaster@kosmos.org":  "mail@kosmos.org",
         "abuse@kosmos.org":       "mail@kosmos.org",
-        "mail@kosmos.org":        "foundation@kosmos.org"
+        "mail@kosmos.org":        "foundation@kosmos.org",
+        "l6@kosmos.org":          "hackerhouse@kosmos.org"
       }
     },
     "garage": {

scripts/knife/migrate_node_to_cinc.sh

@@ -0,0 +1,81 @@
+#!/usr/bin/env bash
+#
+# Prepare a node for Cinc Workstation management so that a plain
+# `knife zero converge` works afterwards.
+#
+# Usage: scripts/knife/migrate_node_to_cinc.sh NODE_NAME
+#
+# Idempotent and safe to re-run. Run from the chef repo (uses .cinc/config.rb).
+#
+# What it does on the node (as root, over knife's own SSH connection):
+#   1. installs Cinc Client if the node still runs Chef Infra Client
+#   2. makes /etc/cinc resolve to the existing knife-zero config in /etc/chef
+#   3. drops the deprecated attribute whitelist/blacklist lines from client.rb
+#   4. pre-seeds /etc/cinc/client.d/attributes.rb (the allowed_* lists) so the
+#      first converge does not filter node attributes
+#   5. best-effort removes the leftover upstream chef package
+set -euo pipefail
+
+node="${1:?usage: $0 NODE_NAME}"
+
+repo_root="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)"
+attributes_file="$repo_root/site-cookbooks/kosmos-base/files/default/chef_attributes.rb"
+node_file="$repo_root/nodes/$node.json"
+
+[[ -f "$node_file" ]] || { echo "error: no node file at $node_file" >&2; exit 1; }
+[[ -f "$attributes_file" ]] || { echo "error: no attributes file at $attributes_file" >&2; exit 1; }
+
+# host + port, same convention as scripts/knife/converge_all.sh
+read -r host vm_host < <(python3 -c '
+import json, sys
+d = json.load(open(sys.argv[1]))
+n = d.get("normal", {})
+print(n.get("knife_zero", {}).get("host", ""), n.get("vm_host", ""))
+' "$node_file")
+[[ -n "$host" ]] || { echo "error: $node has no knife_zero.host" >&2; exit 1; }
+port=22; [[ -n "$vm_host" ]] || port=2222
+
+attr_b64="$(base64 -w0 "$attributes_file")"
+
+remote="$(mktemp)"; trap 'rm -f "$remote"' EXIT
+cat >"$remote" <<EOF
+set -euo pipefail
+
+# 1. Cinc Client (Chef -> Cinc migration, or fresh)
+if ! command -v cinc-client >/dev/null 2>&1 && [ ! -x /opt/cinc/bin/cinc-client ]; then
+  echo ">> installing Cinc Client"
+  curl -sL https://omnitruck.cinc.sh/install.sh | bash
+fi
+
+# 2. Point /etc/cinc at the existing knife-zero config in /etc/chef
+if [ -L /etc/cinc ]; then
+  echo ">> /etc/cinc already a symlink"
+elif [ -d /etc/cinc ] && [ -d /etc/chef ]; then
+  echo ">> replacing package /etc/cinc with symlink to /etc/chef"
+  rm -rf /etc/cinc && ln -s /etc/chef /etc/cinc
+elif [ ! -e /etc/cinc ] && [ -d /etc/chef ]; then
+  echo ">> linking /etc/cinc -> /etc/chef"
+  ln -s /etc/chef /etc/cinc
+fi
+
+# 3. Drop deprecated attribute lists so they don't shadow the new ones
+sed -i -E '/^[[:space:]]*(automatic|default|normal|override)_attribute_(white|black)list/d' /etc/cinc/client.rb 2>/dev/null || true
+
+# 4. Pre-seed the allowlist from the repo (source of truth), before first converge
+mkdir -p /etc/cinc/client.d
+echo '$attr_b64' | base64 -d > /etc/cinc/client.d/attributes.rb
+echo ">> wrote /etc/cinc/client.d/attributes.rb"
+
+# 5. Best-effort: drop the leftover upstream chef package
+if command -v dpkg >/dev/null 2>&1 && dpkg -l chef 2>/dev/null | grep -q '^ii'; then
+  echo ">> removing the upstream chef package"
+  apt-get remove -y chef >/dev/null
+fi
+
+echo ">> $node is ready. Converge with:"
+echo ">>   knife zero converge -a knife_zero.host -p$port name:$node"
+EOF
+
+# ship the script (base64) and run it as root over knife's own connection
+b64="$(base64 -w0 "$remote")"
+knife ssh -a knife_zero.host -p"$port" "name:$node" "echo $b64 | base64 -d | sudo bash"

scripts/postgresql/switch_primary.sh

@@ -9,14 +9,14 @@ fi
 
 old_primary_hostname=$1
 new_primary_hostname=$2
-new_primary_ip=`bundle exec knife node show $new_primary_hostname -a knife_zero.host | grep -E "([0-9]{1,3}[\.]){3}[0-9]{1,3}" -o`
+new_primary_ip=`knife node show $new_primary_hostname -a knife_zero.host | grep -E "([0-9]{1,3}[\.]){3}[0-9]{1,3}" -o`
 
 echo "Switching primary to $new_primary_hostname ($new_primary_ip)"
-bundle exec knife ssh name:$new_primary_hostname -a knife_zero.host "sudo su - postgres -c '/usr/lib/postgresql/12/bin/pg_ctl promote -D /var/lib/postgresql/12/main'"
+knife ssh name:$new_primary_hostname -a knife_zero.host "sudo su - postgres -c '/usr/lib/postgresql/12/bin/pg_ctl promote -D /var/lib/postgresql/12/main'"
 echo "Stopping postgresql on $old_primary_hostname"
-bundle exec knife ssh name:$old_primary_hostname -a knife_zero.host "sudo systemctl stop postgresql@12-main.service"
+knife ssh name:$old_primary_hostname -a knife_zero.host "sudo systemctl stop postgresql@12-main.service"
 echo "Replacing pg.kosmos.local host entries with $new_primary_ip"
-bundle exec knife ssh roles:postgresql_client -a knife_zero.host "sudo sed -r \"s/^[0-9]+\.[0-9]+\.[0-9]+\.[0-9]+\s(pg.kosmos.local)/$new_primary_ip\t\1/\" -i /etc/hosts"
+knife ssh roles:postgresql_client -a knife_zero.host "sudo sed -r \"s/^[0-9]+\.[0-9]+\.[0-9]+\.[0-9]+\s(pg.kosmos.local)/$new_primary_ip\t\1/\" -i /etc/hosts"
 
 # TODO
 # 1. Change roles in node configs

site-cookbooks/kosmos-bitcoin/kitchen.yml

@@ -1,32 +0,0 @@
----
-driver:
-  name: vagrant
-
-## The forwarded_port port feature lets you connect to ports on the VM guest via
-## localhost on the host.
-## see also: https://www.vagrantup.com/docs/networking/forwarded_ports.html
-
-#  network:
-#    - ["forwarded_port", {guest: 80, host: 8080}]
-
-provisioner:
-  name: chef_zero
-
-  ## product_name and product_version specifies a specific Chef product and version to install.
-  ## see the Chef documentation for more details: https://docs.chef.io/config_yml_kitchen.html
-  #  product_name: chef
-  #  product_version: 15
-
-verifier:
-  name: inspec
-
-platforms:
-  - name: ubuntu-18.04
-  - name: centos-7
-
-suites:
-  - name: default
-    verifier:
-      inspec_tests:
-        - test/integration/default
-    attributes:

site-cookbooks/kosmos-ejabberd/kitchen.yml

@@ -14,6 +14,7 @@ transport:
 
 provisioner:
   name: dokken
+  product_name: cinc
   client_rb:
     # the ejabberd package starts its service as ejabberd@localhost during
     # installation; keep the chef node name in sync so ERLANG_NODE matches

site-cookbooks/kosmos_email/Berksfile

@@ -0,0 +1,16 @@
+source 'https://supermarket.chef.io'
+
+# community cookbooks pinned to the versions vendored for production (see the
+# root Berksfile.lock), so the integration suite exercises the same cookbook set
+cookbook 'firewall', '= 6.2.16'
+cookbook 'hostname', '= 0.4.2'
+cookbook 'unbound', '~> 3.0.2'
+cookbook 'postfix',
+  git: 'https://gitea.kosmos.org/kosmos/postfix-cookbook.git',
+  ref: 'bugfix/sasl_attributes'
+
+# local cookbooks
+cookbook 'kosmos-base', path: '../kosmos-base'
+cookbook 'kosmos-postfix', path: '../kosmos-postfix'
+
+metadata

site-cookbooks/kosmos_email/kitchen.yml

@@ -25,10 +25,8 @@ provisioner:
   #   always_update_cookbooks: <%= !ENV['CI'] %>
   # always_update_cookbooks: false
 
-  ## product_name and product_version specifies a specific Chef product and version to install.
+  # install Cinc Client instead of the license-gated Chef Infra Client
-  ## see the Chef documentation for more details: https://docs.chef.io/workstation/config_yml_kitchen/
+  product_name: cinc
-  #  product_name: chef
-  #  product_version: 16
 
 verifier:
   name: inspec

site-cookbooks/kosmos_garage/Berksfile

@@ -1,3 +1,9 @@
 source 'https://supermarket.chef.io'
 
+# local cookbooks
+cookbook 'kosmos_openresty', path: '../kosmos_openresty'
+cookbook 'kosmos-base', path: '../kosmos-base'
+cookbook 'openresty', path: '../openresty'
+cookbook 'kosmos-postfix', path: '../kosmos-postfix'
+
 metadata

site-cookbooks/kosmos_garage/kitchen.yml

@@ -1,6 +1,7 @@
 ---
 driver:
   name: dokken
+  chef_version: 18.2.7
   pull_platform_image: false
   pull_chef_image: false
   memory_limit: 2147483648 # 2GB
@@ -10,6 +11,7 @@ transport:
 
 provisioner:
   name: dokken
+  product_name: cinc
   clean_dokken_sandbox: false
   client_rb:
     environment: testing

site-cookbooks/kosmos_liquor-cabinet/kitchen.yml

@@ -1,10 +1,12 @@
 ---
 driver:
   name: dokken
+  chef_version: 18.2.7
   privileged: true  # allows systemd services to start
 
 provisioner:
   name: dokken
+  product_name: cinc
 
 transport:
   name: dokken

site-cookbooks/kosmos_rsk/attributes/default.rb

@@ -1,4 +1,4 @@
-node.default['rskj']['version'] = "9.0.1~#{node['lsb']['codename']}"
+node.default['rskj']['version'] = "9.0.4~#{node['lsb']['codename']}"
 node.default['rskj']['network'] = 'testnet'
 
 node.default['rskj']['nginx']['domain'] = nil

site-cookbooks/kosmos_rsk/kitchen.yml

@@ -25,10 +25,8 @@ provisioner:
   #   always_update_cookbooks: <%= !ENV['CI'] %>
   # always_update_cookbooks: false
 
-  ## product_name and product_version specifies a specific Chef product and version to install.
+  # install Cinc Client instead of the license-gated Chef Infra Client
-  ## see the Chef documentation for more details: https://docs.chef.io/workstation/config_yml_kitchen/
+  product_name: cinc
-  #  product_name: chef
-  #  product_version: 16
 
 verifier:
   name: inspec

site-cookbooks/kosmos_rsk/test/integration/rskj/rskj_test.rb

@@ -9,7 +9,7 @@ end
 
 describe package('rskj') do
   it { should be_installed }
-  its('version') { should eq '9.0.1~noble' }
+  its('version') { should eq '9.0.4~noble' }
 end
 
 describe service('rsk') do

site-cookbooks/liquor_cabinet/kitchen.yml

@@ -1,10 +1,12 @@
 ---
 driver:
   name: dokken
+  chef_version: 18.2.7
   privileged: true  # allows systemd services to start
 
 provisioner:
   name: dokken
+  product_name: cinc
 
 transport:
   name: dokken

site-cookbooks/tor-full/.kitchen.yml

@@ -1,22 +0,0 @@
----
-driver:
-  name: vagrant
-
-provisioner:
-  name: chef_zero
-
-platforms:
-  - name: centos-6.5
-  - name: centos-7.0
-  - name: fedora-20
-  - name: fedora-21
-  - name: ubuntu-10.04
-  - name: ubuntu-12.04
-  - name: ubuntu-14.04
-
-suites:
-  - name: default
-    excludes: []
-    run_list:
-      - recipe[tor-full::default]
-    attributes: {}