Initial Chef repository

2015-07-21 19:45:23 +02:00
parent 7e5401fc71
commit ee4079fa85
1151 changed files with 185163 additions and 0 deletions
--- a/cookbooks/ssh_known_hosts/CHANGELOG.md
+++ b/cookbooks/ssh_known_hosts/CHANGELOG.md
@@ -0,0 +1,73 @@
+v2.0.0 (2014-12-02)
+-------------------
+- [#36] Fix the way keys are rendered
+- [#22] Update to README
+- [#32] Clean up logging
+- [#23] Do not hash public keys
+- [#34] Serverspec updates
+- [#28] Add data bag caching option
+- [#20] Add checspec matchers
+- [#33] Add test to verify chefspec matcher
+
+v1.3.2 (2014-04-23)
+-------------------
+- [COOK-4579] - Do not use ssh-keyscan stderr
+
+
+v1.3.0 (2014-04-09)
+-------------------
+- [COOK-4489] Updated ssh-keyscan to include -t type
+
+
+v1.2.0 (2014-02-18)
+-------------------
+### Bug
+- **[COOK-3453](https://tickets.opscode.com/browse/COOK-3453)** - ssh_known_hosts cookbook ruby block executes on every chef run
+
+
+v1.1.0
+------
+[COOK-3765] - support ssh-keyscan using an alternative port number
+
+
+v1.0.2
+------
+### Bug
+- **[COOK-3113](https://tickets.opscode.com/browse/COOK-3113)** - Use empty string when result is `nil`
+
+v1.0.0
+------
+This is a major release because it requires a server that supports the partial search feature.
+
+- Opscode Hosted Chef
+- Opscode Private Chef
+- Open Source Chef 11
+
+### Improvement
+
+- [COOK-830]: uses an inordinate amount of RAM when running exception handlers
+
+v0.7.4
+------
+- [COOK-2440] - `ssh_known_hosts` fails to use data bag entries, doesn't grab items
+
+v0.7.2
+------
+- [COOK-2364] - Wrong LWRP name used in recipe
+
+v0.7.0
+------
+- [COOK-2320] - Merge `known_host` LWRP into `ssh_known_hosts`
+
+v0.6.0
+------
+- [COOK-2268] - Allow to run with chef-solo
+
+v0.5.0
+------
+- [COOK-1077] - allow adding arbitrary host keys from a data bag
+
+v0.4.0
+------
+- COOK-493: include fqdn
+- COOK-721: corrected permissions
--- a/cookbooks/ssh_known_hosts/README.md
+++ b/cookbooks/ssh_known_hosts/README.md
@@ -0,0 +1,202 @@
+ssh_known_hosts Cookbook
+========================
+The Chef `ssh_known_hosts` cookbook exposes resource and default recipe for adding hosts and keys to the `/etc/ssh/ssh_known_hosts` file.
+
+- The default recipe builds `/etc/ssh/ssh_known_hosts` based either on search indexes using `rsa,dsa` key types and ohai data **or**, when `['ssh_known_hosts']['use_data_bag_cache']` is `true`, on the contents of a data bag that is maintained by the `cacher` recipe running on a worker node.
+- The cacher recipe builds and maintains a data bag based on search indexes using `rsa,dsa` key types and ohai data.
+- The LWRP provides a way to add custom entries in your own recipes.
+
+You can also optionally put other host keys in a data bag called "`ssh_known_hosts`". See below for details.
+
+
+Requirements
+------------
+Should work on any operating system that supports `/etc/ssh/ssh_known_hosts`.
+
+The Opscode `partial_search` cookbook is required for the default recipe, as well as a Chef Server that supports partial search:
+
+- Opscode Hosted Chef
+- Opscode Private Chef
+- Open Source Chef Server 11
+
+
+Usage
+-----
+### LWRP
+
+Use the LWRP `ssh_known_hosts_entry` to append an entry for the specified host in `/etc/ssh/ssh_known_hosts`. For example:
+
+```ruby
+ssh_known_hosts_entry 'github.com'
+```
+
+This will append an entry in `/etc/ssh/ssh_known_hosts` like this:
+
+```text
+# github.com SSH-2.0-OpenSSH_5.5p1 Debian-6+squeeze1+github8
+github.com ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAq2A7hRGmdnm9tUDbO9IDSwBK6TbQa+PXYPCPy6rbTrTtw7PHkccKrpp0yVhp5HdEIcKr6pLlVDBfOLX9QUsyCOV0wzfjIJNlGEYsdlLJizHhbn2mUjvSAHQqZETYP81eFzLQNnPHt4EVVUh7VfDESU84KezmD5QlWpXLmvU31/yMf+Se8xhHTvKSCZIFImWwoG6mbUoWf9nzpIoaSjB+weqqUUmpaaasXVal72J+UX2B+2RPW3RcT0eOzQgqlJL3RKrTJvdsjE3JEAvGq3lGHSZXy28G3skua2SmVi/w4yCE6gbODqnTWlg7+wC604ydGXA8VJiS5ap43JXiUFFAaQ==
+```
+
+You can optionally specify your own key, if you don't want to use `ssh-keyscan`:
+
+```ruby
+ssh_known_hosts_entry 'github.com' do
+  key 'node.example.com ssh-rsa ...'
+end
+```
+
+### Cacher
+
+Use the `cacher` recipe on a single "worker" node somewhere in your cluster to maintain a data bag (`server_data/known_hosts` by default) containing all of your nodes host keys.  The advantage to this approach is that is much faster than running a search of all nodes, and substantially lightens the load on locally hosted Chef servers.  The drawback is that the data is slightly delayed (because the cacher worker must converge first).
+
+To use the cacher, simply include the `ssh_known_hosts::cacher` cookbook in a wrapper cookbook or run list on a designated worker node.
+
+#### Attributes
+
+The following attributes are set on a per-platform basis, see the `attributes/default.rb`.
+
+* `node['ssh_known_hosts']['file']` - Sets up the location of the ssh_known_hosts file for the system. 
+  Defaults to '/etc/ssh/ssh_known_hosts'
+* `node['ssh_known_hosts']['key_type']` - Determines which key type ssh-keyscan will use to determine the 
+  host key, different systems will have different available key types, check your manpage for available 
+  key types for ssh-keyscan. Defaults to 'rsa,dsa'
+* `node['ssh_known_hosts']['use_data_bag_cache']` - Use the data bag maintained by the cacher server to build `/etc/ssh/ssh_known_hosts` instead of a direct search (requires that a node be set up to run the cacher recipe regularly).
+* `node['ssh_known_hosts']['cacher']['data_bag']`/`node['ssh_known_hosts']['cacher']['data_bag_item']` - Data bag where cacher recipe should store its keys.
+
+#### LWRP Attributes
+
+<table>
+  <thead>
+    <tr>
+      <th>Attribute</th>
+      <th>Description</th>
+      <th>Example</th>
+      <th>Default</th>
+    </tr>
+  </thead>
+
+  <tbody>
+    <tr>
+      <td>host</td>
+      <td>the host to add</td>
+      <td><tt>github.com</tt></td>
+      <td></td>
+    </tr>
+    <tr>
+      <td>key</td>
+      <td>(optional) provide your own key</td>
+      <td><tt>ssh-rsa ...</tt></td>
+      <td><tt>ssh-keyscan -H #{host}</tt></td>
+    </tr>
+    <tr>
+      <td>port</td>
+      <td>(optional) the server port that ssh-keyscan will use to gather the public key</td>
+      <td><tt>2222</tt></td>
+      <td><tt>22</tt></td>
+    </tr>
+  </tbody>
+</table>
+
+- - -
+
+### Default Recipe
+
+Searches the Chef Server for all hosts that have SSH host keys using `rsa,dsa` key types and generates an `/etc/ssh/ssh_known_hosts`.
+
+#### Adding custom host keys
+
+There are two ways to add custom host keys. You can either use the provided LWRP (see above), or by creating a data bag called "`ssh_known_hosts`" and adding an item for each host:
+
+```javascript
+{
+  "id": "github",
+  "fqdn": "github.com",
+  "rsa": "github-rsa-host-key"
+}
+```
+
+There are additional optional values you may use in the data bag:
+
+<table>
+  <thead>
+    <tr>
+      <th>Attribute</th>
+      <th>Description</th>
+      <th>Example</th>
+      <th>Default</th>
+    </tr>
+  </thead>
+
+  <tbody>
+    <tr>
+      <td>id</td>
+      <td>a unique id for this data bag entry</td>
+      <td><tt>github</tt></td>
+      <td></td>
+    </tr>
+    <tr>
+      <td>fqdn</td>
+      <td>the fqdn of the host</td>
+      <td><tt>github.com</tt></td>
+      <td></td>
+    </tr>
+    <tr>
+      <td>rsa</td>
+      <td>the rsa key for this server</td>
+      <td><tt>ssh-rsa AAAAB3...</tt></td>
+      <td></td>
+    </tr>
+    <tr>
+      <td>ipaddress</td>
+      <td>the ipaddress of the node (if fqdn is missing)</td>
+      <td><tt>1.1.1.1</tt></td>
+      <td></td>
+    </tr>
+    <tr>
+      <td>hostname</td>
+      <td>local hostname of the server (if not a fqdn)</td>
+      <td><tt>myserver.local</tt></td>
+      <td></td>
+    </tr>
+    <tr>
+      <td>dsa</td>
+      <td>the dsa key for this server</td>
+      <td><tt>ssh-dsa ABAAC3...</tt></td>
+      <td></td>
+    </tr>
+  </tbody>
+</table>
+
+###ChefSpec matchers
+
+A custom matcher is available for you to use in recipe tests.
+
+``` 
+describe 'my_cookbook::my_recipe' do
+	let(:chef_run) { ChefSpec::Runner.new.converge(described_recipe) }
+	it { expect(chef_run).to append_to_ssh_known_hosts 'github.com' }
+end
+```
+
+
+License and Authors
+-------------------
+- Author: Scott M. Likens (<scott@likens.us>)
+- Author: Seth Vargo (<sethvargo@gmail.com>)
+- Author: Lamont Granquist (<lamont@opscode.com>)
+
+```text
+Copyright:: 2011-2013, Opscode, Inc
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+```
--- a/cookbooks/ssh_known_hosts/attributes/default.rb
+++ b/cookbooks/ssh_known_hosts/attributes/default.rb
@@ -0,0 +1,23 @@
+#
+# Author:: Seth Vargo (<sethvargo@gmail.com>)
+# Attributes:: default
+#
+# Copyright 2013, Seth Vargo
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+default['ssh_known_hosts']['file'] = '/etc/ssh/ssh_known_hosts'
+default['ssh_known_hosts']['key_type'] = 'rsa,dsa'
+default['ssh_known_hosts']['cacher']['data_bag'] = 'server_data'
+default['ssh_known_hosts']['cacher']['data_bag_item'] = 'known_hosts'
--- a/cookbooks/ssh_known_hosts/libraries/matchers.rb
+++ b/cookbooks/ssh_known_hosts/libraries/matchers.rb
@@ -0,0 +1,7 @@
+if defined?(ChefSpec)
+
+    def append_to_ssh_known_hosts(resource)
+      ChefSpec::Matchers::ResourceMatcher.new(:ssh_known_hosts_entry, :create, resource)
+    end
+
+end
--- a/cookbooks/ssh_known_hosts/metadata.json
+++ b/cookbooks/ssh_known_hosts/metadata.json
--- a/cookbooks/ssh_known_hosts/providers/entry.rb
+++ b/cookbooks/ssh_known_hosts/providers/entry.rb
@@ -0,0 +1,78 @@
+#
+# Author:: Seth Vargo (<sethvargo@gmail.com>)
+# Provider:: entry
+#
+# Copyright 2013, Seth Vargo
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+use_inline_resources if defined?(use_inline_resources)
+
+def whyrun_supported?
+  true
+end
+
+action :create do
+
+  if new_resource.key
+
+    if new_resource.key_type == 'rsa' || new_resource.key_type == 'dsa'
+      key_type = "ssh-#{new_resource.key_type}"
+    else
+      key_type = new_resource.key_type
+    end
+
+    key = "#{new_resource.host} #{key_type} #{new_resource.key}"
+    
+  else
+
+    key = `ssh-keyscan -t#{node['ssh_known_hosts']['key_type']} -p #{new_resource.port} #{new_resource.host}`
+  end
+    comment = key.split("\n").first || ""
+
+  if key_exists?(key, comment)
+    Chef::Log.debug "Known hosts key for #{new_resource.name} already exists - skipping"
+  else
+    new_keys = (keys + [key]).uniq.sort
+    file "ssh_known_hosts-#{new_resource.name}" do
+      path node['ssh_known_hosts']['file']
+      action :create
+      backup false
+      content "#{new_keys.join($/)}#{$/}"
+    end
+  end
+end
+
+private
+  def keys
+    unless @keys
+      if key_file_exists?
+        lines = ::File.readlines(node['ssh_known_hosts']['file'])
+        @keys = lines.map {|line| line.chomp}.reject {|line| line.empty?}
+      else
+        @keys = []
+      end
+    end
+    @keys
+  end
+
+  def key_file_exists?
+    ::File.exists?(node['ssh_known_hosts']['file'])
+  end
+
+  def key_exists?(key, comment)
+    keys.any? do |line|
+      line.match(/#{Regexp.escape(comment)}|#{Regexp.escape(key)}/)
+    end
+  end
--- a/cookbooks/ssh_known_hosts/recipes/cacher.rb
+++ b/cookbooks/ssh_known_hosts/recipes/cacher.rb
@@ -0,0 +1,60 @@
+# Gather a list of all nodes, warning if using Chef Solo
+if Chef::Config[:solo]
+  fail 'ssh_known_hosts::cacher requires Chef search - Chef Solo does ' \
+    'not support search!'
+else
+  all_host_keys = partial_search(
+    :node, 'keys:*',
+    :keys => {
+      'hostname'        => [ 'hostname' ],
+      'fqdn'            => [ 'fqdn' ],
+      'ipaddress'       => [ 'ipaddress' ],
+      'host_rsa_public' => [ 'keys', 'ssh', 'host_rsa_public' ],
+      'host_dsa_public' => [ 'keys', 'ssh', 'host_dsa_public' ]
+    }
+  ).collect do |host|
+    {
+      'fqdn' => host['fqdn'] || host['ipaddress'] || host['hostname'],
+      'key' => host['host_rsa_public'] || host['host_dsa_public']
+    }
+  end
+  Chef::Log.debug("Partial search got: #{all_host_keys.inspect}")
+end
+
+new_data_bag_content = {
+  "id" => node['ssh_known_hosts']['cacher']['data_bag_item'],
+  "keys" => all_host_keys
+}
+
+Chef::Log.debug('New data bag content: ' \
+  "#{new_data_bag_content.inspect}")
+
+if Chef::DataBag.list.key?(node['ssh_known_hosts']['cacher']['data_bag'])
+  # Check to see if there are actually any changes to be made (so we don't save
+  # data bags unnecessarily)
+  existing_data_bag_content = data_bag_item(
+    node['ssh_known_hosts']['cacher']['data_bag'],
+    node['ssh_known_hosts']['cacher']['data_bag_item']
+  ).raw_data
+  Chef::Log.debug('Existing data bag content: ' \
+    "#{existing_data_bag_content.inspect}")
+else
+  Chef::Log.debug('Data bag ' \
+    "\"#{node['ssh_known_hosts']['cacher']['data_bag']}\" not found.  " \
+    'Creating.')
+  new_databag = Chef::DataBag.new
+  new_databag.name(node['ssh_known_hosts']['cacher']['data_bag'])
+  new_databag.save
+end
+
+unless (defined? existing_data_bag_content) &&
+  new_data_bag_content == existing_data_bag_content
+
+  Chef::Log.debug('Data bag contents differ.  Saving updates.')
+
+  host_key_db_item = Chef::DataBagItem.new
+  host_key_db_item.data_bag(node['ssh_known_hosts']['cacher']['data_bag'])
+  host_key_db_item.raw_data = new_data_bag_content
+
+  host_key_db_item.save
+end
--- a/cookbooks/ssh_known_hosts/recipes/default.rb
+++ b/cookbooks/ssh_known_hosts/recipes/default.rb
@@ -0,0 +1,87 @@
+#
+# Cookbook Name:: ssh_known_hosts
+# Recipe:: default
+#
+# Author:: Scott M. Likens (<scott@likens.us>)
+# Author:: Joshua Timberman (<joshua@opscode.com>)
+# Author:: Seth Vargo (<sethvargo@gmail.com>)
+#
+# Copyright 2009, Adapp, Inc.
+# Copyright 2011-2013, Opscode, Inc.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+if node['ssh_known_hosts']['use_data_bag_cache']
+  # Load hosts from the ssh known hosts cacher (if the data bag exists)
+  unless Chef::DataBag.list.key?(node['ssh_known_hosts']['cacher']['data_bag'])
+    fail 'use_data_bag_cache is set but the configured data bag was not found'
+  end
+  
+  hosts = data_bag_item(
+    node['ssh_known_hosts']['cacher']['data_bag'],
+    node['ssh_known_hosts']['cacher']['data_bag_item']
+  )['keys']
+  Chef::Log.info "hosts data bag: #{hosts.inspect}"
+else
+  # Gather a list of all nodes, warning if using Chef Solo
+  if Chef::Config[:solo]
+    Chef::Log.warn 'ssh_known_hosts requires Chef search - Chef Solo does not support search!'
+
+    # On Chef Solo, we still want the current node to be in the ssh_known_hosts
+    hosts = [node]
+  else
+    hosts = partial_search(:node, "keys_ssh:* NOT name:#{node.name}",
+                           :keys => {
+                             'hostname' => [ 'hostname' ],
+                             'fqdn'     => [ 'fqdn' ],
+                             'ipaddress' => [ 'ipaddress' ],
+                             'host_rsa_public' => [ 'keys', 'ssh', 'host_rsa_public' ],
+                             'host_dsa_public' => [ 'keys', 'ssh', 'host_dsa_public' ]
+                           }
+                          ).collect do |host|
+                            {
+                              'fqdn' => host['fqdn'] || host['ipaddress'] || host['hostname'],
+                              'key' => host['host_rsa_public'] || host['host_dsa_public']
+                            }
+    end
+  end
+end
+
+# Add the data from the data_bag to the list of nodes.
+# We need to rescue in case the data_bag doesn't exist.
+if Chef::DataBag.list.key?('ssh_known_hosts')
+  begin
+    hosts += data_bag('ssh_known_hosts').collect do |item|
+      entry = data_bag_item('ssh_known_hosts', item)
+      {
+        'fqdn' => entry['fqdn'] || entry['ipaddress'] || entry['hostname'],
+        'key'  => entry['rsa'] || entry['dsa']
+      }
+    end
+  rescue
+    Chef::Log.info "Could not load data bag 'ssh_known_hosts'"
+  end
+end
+
+# Loop over the hosts and add 'em
+hosts.each do |host|
+  unless host['key'].nil?
+    # The key was specified, so use it
+    ssh_known_hosts_entry host['fqdn'] do
+      key host['key']
+    end
+  else
+    # No key specified, so have known_host perform a DNS lookup
+    ssh_known_hosts_entry host['fqdn']
+  end
+end
--- a/cookbooks/ssh_known_hosts/resources/entry.rb
+++ b/cookbooks/ssh_known_hosts/resources/entry.rb
@@ -0,0 +1,30 @@
+#
+# Author:: Seth Vargo (<sethvargo@gmail.com>)
+# Resource:: entry
+#
+# Copyright 2013, Seth Vargo
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+actions :create
+default_action :create
+
+attribute :host, :kind_of => String, :name_attribute => true
+attribute :key, :kind_of => String
+attribute :key_type, :kind_of => String, :default => 'rsa'
+attribute :port, :kind_of => Fixnum, :default => 22
+
+def initialize(*args)
+  super
+  @action = :create
+end