kosmos/chef
kosmos/chef
8
3
Fork 0
Code Issues 30 Pull Requests 3 Projects 2 Activity

Update the elasticsearch cookbook

Browse Source
This commit is contained in:
Greg Karékinian 2023-06-29 16:39:09 +02:00
parent 68ce3c4834
commit 7683de09a2
48 changed files with 2016 additions and 2065 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
Berksfile
View File

@ -34,7 +34,7 @@ cookbook 'mariadb', '= 0.3.1'
cookbook 'ipfs', cookbook 'ipfs',
git: 'https://gitea.kosmos.org/kosmos/ipfs-cookbook.git', git: 'https://gitea.kosmos.org/kosmos/ipfs-cookbook.git',
ref: 'v0.7.0' ref: 'v0.7.0'
cookbook 'elasticsearch', '= 4.3.0' cookbook 'elasticsearch', '~> 5.1.3'
cookbook 'java', '~> 4.3.0' cookbook 'java', '~> 4.3.0'
cookbook 'ulimit', '~> 1.0.0' cookbook 'ulimit', '~> 1.0.0'
cookbook 'golang', '~> 5.3.1' cookbook 'golang', '~> 5.3.1'

7
Berksfile.lock
View File

@ -5,7 +5,7 @@ DEPENDENCIES
chef-sugar (= 5.1.12) chef-sugar (= 5.1.12)
compat_resource (= 12.19.0) compat_resource (= 12.19.0)
composer (~> 2.7.0) composer (~> 2.7.0)
elasticsearch (= 4.3.0) elasticsearch (~> 5.1.3)
fail2ban (~> 7.0.4) fail2ban (~> 7.0.4)
firewall (~> 6.2.16) firewall (~> 6.2.16)
git (~> 10.0.0) git (~> 10.0.0)
@ -54,11 +54,8 @@ GRAPH
apt (>= 0.0.0) apt (>= 0.0.0)
php (>= 0.0.0) php (>= 0.0.0)
windows (>= 0.0.0) windows (>= 0.0.0)
elasticsearch (4.3.0) elasticsearch (5.1.3)
apt (>= 0.0.0)
ark (>= 0.0.0) ark (>= 0.0.0)
chef-sugar (>= 0.0.0)
yum (>= 0.0.0)
fail2ban (7.0.4) fail2ban (7.0.4)
yum-epel (>= 0.0.0) yum-epel (>= 0.0.0)
firewall (6.2.16) firewall (6.2.16)

1
cookbooks/elasticsearch/.foodcritic
View File

@ -1 +0,0 @@
~FC057

5
cookbooks/elasticsearch/.markdownlint-cli2.yaml Normal file
View File

@ -0,0 +1,5 @@
config:
ul-indent: false # MD007
line-length: false # MD013
no-duplicate-heading: false # MD024
reference-links-images: false # MD052

997
cookbooks/elasticsearch/CHANGELOG.md
View File

File diff suppressed because it is too large Load Diff

111
cookbooks/elasticsearch/CONTRIBUTING.md
View File

@ -1,111 +0,0 @@
Contributing to cookbook-elasticsearch
======================================
### General philosophy
We try, as much as possible, to mirror the upstream experience of installing, configuring, and running Elasticsearch. Sometimes, the upstream project won't expose certain settings, or provide defaults that everyone agrees with. Wherever possible, we will:
- follow the upstream standards for specific config files
- include any files shipped with packages upstream in this cookbook as-is
- expose a template and cookbook setting for any file that you might want to modify
- recommend major changes to standard files be sent upstream first
- minimize the number of exceptions and maintained "workarounds" in this cookbook
### Workflow for contributing
1. Create a branch directly in this repo or a fork (if you don't have push access). Please name branches within this repository `<github username>/<description>`. For example, something like karmi/install_from_deb.
1. Create an issue or open a PR. If you aren't sure your PR will solve the issue, or may be controversial, we commend opening an issue separately and linking to it in your PR, so that if the PR is not accepted, the issue will remain and be tracked.
1. Close (and reference) issues by the `closes #XXX` or `fixes #XXX` notation in the commit message. Please use a descriptive, useful commit message that could be used to understand why a particular change was made.
1. Keep pushing commits to the initial branch, `--amend`-ing if necessary. Please don't mix fixing unrelated issues in a single branch.
1. When everything is ready for merge, clean up the branch (rebase with master to synchronize, squash, edit commits, etc) to prepare for it to be merged.
### Merging contributions
1. After reviewing commits for documentation, passing CI tests, and good descriptive commit messages, merge it with --no-ff switch, so it's indicated in the Git history
1. Do not use the Github "merge button", since it doesn't do a fast-forward merge (see previous item).
### Testing
This cookbook is equipped with both unit tests (chefspec) and integration tests
(test-kitchen and serverspec). It also comes with rubocop and foodcritic tasks
in the supplied Rakefile. Contributions to this cookbook should include tests
for new features or bugfixes, with a preference for unit tests over integration
tests to ensure speedy testing runs. ***All tests and most other commands here
should be run using bundler*** and our standard Gemfile. This ensures that
contributions and changes are made in a standardized way against the same
versions of gems. We recommend installing rubygems-bundler so that bundler is
automatically inserting `bundle exec` in front of commands run in a directory
that contains a Gemfile.
A full test run of all tests and style checks would look like:
```bash
$ bundle exec rake style
$ bundle exec rake spec
$ bundle exec rake integration
$ bundle exec rake destroy
```
The final destroy is intended to clean up any systems that failed a test, and is
mostly useful when running with kitchen drivers for cloud providers, so that no
machines are left orphaned and costing you money.
#### Fixtures
This cookbook supplies a few different test fixtures (under `test/fixtures/`)
that can be shared amongst any number of unit or integration tests: cookbooks,
environments, and nodes. Environments and nodes are automatically loaded into
chef-zero for both chefspec tests that run locally and serverspec tests that run
from test-kitchen.
It also contains 'platform data' that can be used to drive unit testing, for
example, you might read `httpd` for some platforms and `apache2` for others,
allowing you to write a single test for the Apache webserver. Unfortunately,
without further modifications to `busser` and `busser-serverspec`, the platform
data will not be available to serverspec tests.
#### Style and Best Practices
Rubocop and Foodcritic evaluations may be made by running `rake style`. There
are no overrides for foodcritic rules, however the adjustments to
rubocop are made using the supplied `.rubocop.yml` file and have been documented
by comments within. Most notably, rubocop has been restricted to only apply to
`.rb` files.
Rubocop and foodcritic tests can be executed using `rake style`.
#### Unit testing
Unit testing is done using the latest versions of Chefspec. The current default
test layout includes running against all supported platforms, as well as
stubbing data into chef-zero. This allows us to also test against chef search.
As is currently a best practice in the community, we will avoid the use of
chef-solo, but not create barriers to explicitly fail for chef-solo.
Unit tests can be executed using `rake spec`.
#### Integration testing
Integration testing is accomplished using the latest versions of test-kitchen
and serverspec. Currently, this cookbook uses the busser-serverspec plugin for
copying serverspec files to the system being tested. There is some debate in the
community about whether this should be done using busser-rspec instead, and each
busser plugin has a slightly different feature set.
While the default test-kitchen configuration uses the vagrant driver, you may
override this using `~/.kitchen/config.yml` or by placing a `.kitchen.local.yml`
in the current directory. This allows you to run these integration tests using
any supported test-kitchen driver (ec2, rackspace, docker, etc).
Integration tests can be executed using `rake integration` or `kitchen test`.
### Releasing
1. Create/update the changelog. We are using the `github_changelog_generator`
gem.
1. We highly recommend using the `stove` project, which pushes cookbooks to
Supermarket and tags to Github.

137
cookbooks/elasticsearch/FAQ.md Normal file
View File

@ -0,0 +1,137 @@
# Frequently asked questions
## Versions and Support
### Does this cookbook install [Java](https://www.java.com/en/)? What version?
This cookbook requires java, but does not provide it. Please install Java before using any recipe in this cookbook. Please also note that Elasticsearch itself has [specific minimum Java version requirements](https://www.elastic.co/guide/en/elasticsearch/reference/current/setup.html#jvm-version). We recommend [this cookbook](https://github.com/agileorbit-cookbooks/java) to install Java.
### What version of [Chef](https://www.chef.io/) does this cookbook require/support?
This cookbook follows the [recommended Chef community cookbook policy](https://github.com/chef/chef-rfc/blob/master/rfc092-dependency-update-cadence.md#cookbook-and-ecosystem-tooling-support) regarding Chef support; specifically, we support at least the last 6 months of Chef Client versions. We explicitly don't support anything less than Chef 12.5 and greater. We run CI as well as testing with chefspec and test-kitchen.
### What versions of [Elasticsearch](https://www.elastic.co/products/elasticsearch) does this cookbook support?
This cookbook is being written and tested to support Elasticsearch 6.x and greater. If you must have a cookbook that works with older versions of Elasticsearch, please test and then pin to a specific, older `major.minor` version of this cookbook and only leave the patch release to float. Older versions can be found via [Git tags](https://github.com/elastic/cookbook-elasticsearch/tags) or on [Chef Supermarket](https://supermarket.chef.io/cookbooks/elasticsearch). We also maintain bugfix branches for major released lines (0.x, 1.x, 2.x, 3.x) of this cookbook so that we can still release fixes for older cookbooks. Previous versions of this cookbook may be found using the git tags on this repository.
## How do I
### How do I set the JVM heap size?
The [allocated_memory](https://github.com/elastic/cookbook-elasticsearch/blob/master/libraries/provider_configure.rb#L27-L32) parameter controls this.
If you do not set this parameter, the heap size will be set to 50% of system memory or 31g, whatever is smaller.
### How should I discover other Elasticsearch nodes?
We recommend using [chef search](https://docs.chef.io/chef_search.html) in your wrapper cookbook, or using one of the contributing plugins that leverage cloud-specific features (e.g. `discovery-ec2`).
### How do I create EBS block devices or other block devices?
We recommend [the aws cookbook](https://github.com/chef-cookbooks/aws).
### How do I upgrade Elasticsearch in place?
Upgrading Elasticsearch in place is not recommended, and generally not supported by this cookbook. We strongly recommend you pin versions of Elasticsearch and spin up new servers to migrate to a new version, one node at a time. This cookbook does not generally set destructive options like asking the package manager to overwrite configuration files without prompting, either.
See also: <https://github.com/elastic/cookbook-elasticsearch/issues/533>
### How do I override settings in the systemd unit file?
If you'd like to modify the system unit file, you have two supported options:
1. [Specify a different source template](https://github.com/elastic/cookbook-elasticsearch/blob/master/libraries/resource_service.rb#L26-L27)
1. Use an override file (see "Unit File Load Path" in the [systemd documentation](https://www.freedesktop.org/software/systemd/man/systemd.unit.html))
Typically, the override file should go in something like: `/etc/systemd/system/elasticsearch.service.d/elasticsearch.conf`.
Check out <https://github.com/elastic/elasticsearch/issues/23848#issuecomment-290836681> for more information.
### How do I avoid running various elasticsearch_* resource?
If you're running this cookbook inside docker, or manually performing some of the steps to install, configure, or run Elasticsearch, you will notice immediately that this cookbook complains about any missing resources. In order to provide the cookbook will appropriate settings (some resources _need_ information from others, e.g. configuring elasticsearch requires knowing where it is installed), you should simply use the missing resource but specify `action :none`. See #573 for more information
For example, `elasticsearch_plugin` needs to source the environment file used by `elasticsearch_service` in order to be sure it uses the same settings. If you're running in a container, you may not want to use a service. Therefore, do something like this:
```ruby
elasticsearch_service 'elasticsearch' do
args '-d' # let other resources know we need to use -d, but don't touch the service
action :none
end
```
## Specific errors and messages
### Elasticsearch complains about data paths on startup
Per 5.3.1 release notes, Elasticsearch now fails to start if you provide default.path.data and an array of path.data in order to correct a bug from 5.3.0 that merged the default into the array instead of ignoring it. However, default values for cookbook attributes that set those values are also preventing ES from starting, even though path.data isn't an array.
TL;DR -- you should upgrade and get the bugfix (of the original bugfix). See <https://github.com/elastic/cookbook-elasticsearch/issues/563> for more information.
### Java "trust anchors" error when installing Elasticseach plugins
If you're using OpenJDK, installing Elasticsearch plugins might fail with a Java SSL exception:
```text
Exception in thread "main" javax.net.ssl.SSLException: java.lang.RuntimeException: Unexpected error: java.security.InvalidAlgorithmParameterException: the trustAnchors parameter must be non-empty
```
This can be fixed by running the configuration for the `ca-certificates-java` package before installing any plugins:
```ruby
# http://phutchins.com/blog/2017/03/14/java-trust-anchors-error-when-installing-es-plugins/
execute "ca-certificates-configure" do
command "sudo /var/lib/dpkg/info/ca-certificates-java.postinst configure"
end
```
See [this article](http://phutchins.com/blog/2017/03/14/java-trust-anchors-error-when-installing-es-plugins/) for more details.
### Chef::Exceptions::Package: Installed package is newer than candidate package
You may be trying to downgrade Elasticsearch, or the newer package has gone missing from their repos. Depending on what you'd like to do next, you may [provide package_options arguments](https://github.com/elastic/cookbook-elasticsearch/blob/master/libraries/resource_install.rb#L27) to yum or apt to tell it what you'd like to do more specifically. In #571, someone else has figured out how to direct apt/dpkg to upgrade the way they want, but we didn't want to prescribe what end users want their package manager to do.
Alternately, you can add some logic to skip the install if the correct version is already installed (e.g. add `not_if "rpm -qa | grep -s elasticsearch"` to your `elasticsearch_install` resource).
### Elasticsearch is installed in the wrong directory name; the version is incorrect
If you install by URL, and don't provide the version attribute to the `elasticsearch_install` resource, this cookbook can't tell what version you've provided (any arbitrary filename works, so there's no guarantee we can even figure it out). You will get the default version included in the directory name in this case, unless you specify which version you're installing as well. See #535 for more information.
### Elasticsearch won't start with configuration it doesn't recognize
There's a chicken-and-egg issue with installing a plugin and then configuring it. It would be nice if Elasticsearch allowed configuration settings that didn't do anything, and emitted a warning instead of a fatal error.
You have two options to workaround this -- (a) Don't start Elasticsearch until the plugin is installed; in other words, use one elasticsearch_configure and don't issue a :start action to elasticsearch_service until the plugin resource runs its own actions. Alternately, (b) check for whether or not x-pack is installed at the start of a Chef run, and don't configure any x-pack settings unless it's installed (this will require 2 chef runs to fully configure x-pack, as the ::File.exists? is evaluated very early in the Chef run), e.g.:
```ruby
x_pack_installed = ::File.exists?("#{es_conf.path_plugins}/x-pack")
settings = {
'http.port' => port,
'cluster.name' => cluster_name,
'node.name' => node_name,
'bootstrap.memory_lock' => false,
'discovery.zen.minimum_master_nodes' => 1
}
if x_pack_installed
settings['xpack.monitoring.enabled'] = true
...
end
es_conf = elasticsearch_configure 'elasticsearch' do
allocated_memory '512m'
configuration settings
end
es_conf.path_data data_location if data_location
...
```
### .deb package installs fail inside containers
This is a known issue upstream and the packaging folks have been working to resolve it. You can follow along at:
<https://github.com/elastic/elasticsearch/issues/25846>
### How do I test multiple Elasticsearch nodes in test-kitchen?
Check out <https://github.com/elastic/cookbook-elasticsearch/pull/648/files> for an example of one possible solution.

15
cookbooks/elasticsearch/LICENSE Normal file
View File

@ -0,0 +1,15 @@
This software is licensed under the Apache 2 license, quoted below.
Copyright (c) 2015 Elasticsearch <https://www.elastic.co/>
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.

88
cookbooks/elasticsearch/README.md
View File

@ -1,6 +1,6 @@
# Elasticsearch Chef Cookbook # Elasticsearch Chef Cookbook
[![Build Status](https://travis-ci.org/elastic/cookbook-elasticsearch.svg?branch=master)](https://travis-ci.org/elastic/cookbook-elasticsearch) [![Cookbook Version](https://img.shields.io/cookbook/v/elasticsearch.svg)](https://supermarket.chef.io/cookbooks/elasticsearch)[![Build Status](https://jenkins-01.eastus.cloudapp.azure.com/job/elasticsearch-cookbook/badge/icon)](https://jenkins-01.eastus.cloudapp.azure.com/job/elasticsearch-cookbook/) [![Cookbook Version](https://img.shields.io/cookbook/v/elasticsearch.svg)](https://supermarket.chef.io/cookbooks/elasticsearch)
**Please** review the [frequently asked questions](FAQ.md) and [contributing guidelines](CONTRIBUTING.md) before opening issues or submitting pull requests. **Please** review the [frequently asked questions](FAQ.md) and [contributing guidelines](CONTRIBUTING.md) before opening issues or submitting pull requests.
@ -8,59 +8,6 @@
Please [check out the previous 3.x.x releases](https://github.com/elastic/cookbook-elasticsearch/tree/3.x.x) of this cookbook. Please consider pinning your cookbook to '~> 3.0' for support for Elasticsearch 6 and earlier, or '~> 4.0' release for Elasticsearch 6 and beyond. Please [check out the previous 3.x.x releases](https://github.com/elastic/cookbook-elasticsearch/tree/3.x.x) of this cookbook. Please consider pinning your cookbook to '~> 3.0' for support for Elasticsearch 6 and earlier, or '~> 4.0' release for Elasticsearch 6 and beyond.
## Attributes
Please consult [attributes/default.rb](attributes/default.rb) for a large list
of checksums for many different archives and package files of different
elasticsearch versions. Both recipes and resources/providers here use those
default values.
You may use `%s` in your URL and this cookbook will use sprintf/format to insert
the version parameter as a string into your download_url.
|Name|Default|Other values|
|----|-------|------------|
| For Elasticsearch < 7: ||
|`default['elasticsearch']['download_urls']['debian']`|[See values](attributes/default.rb).|`%s` will be replaced with the version attribute above|
|`default['elasticsearch']['download_urls']['rhel']`|[See values](attributes/default.rb).|`%s` will be replaced with the version attribute above|
|`default['elasticsearch']['download_urls']['tarball']`|[See values](attributes/default.rb).|`%s` will be replaced with the version attribute above|
| For Elasticsearch >= 7: ||
|`default['elasticsearch']['download_urls_v7']['debian']`|[See values](attributes/default.rb).|`%s` will be replaced with the version attribute above|
|`default['elasticsearch']['download_urls_v7']['rhel']`|[See values](attributes/default.rb).|`%s` will be replaced with the version attribute above|
|`default['elasticsearch']['download_urls_v7']['tarball']`|[See values](attributes/default.rb).|`%s` will be replaced with the version attribute above|
This cookbook's `elasticsearch::default` recipe also supports setting any `elasticsearch_` resource using attributes:
```
default['elasticsearch']['user'] = {}
default['elasticsearch']['install'] = {}
default['elasticsearch']['configure'] = {}
default['elasticsearch']['service'] = {}
default['elasticsearch']['plugin'] = {}
```
For example, this will pass a username 'foo' to `elasticsearch_user` and set a uid to `1234`:
```
default['elasticsearch']['user']['username'] = 'foo'
default['elasticsearch']['user']['uid'] = '1234'
```
## Recipes
Resources are the intended way to consume this cookbook, however we have
provided a single recipe that configures Elasticsearch by downloading an archive
containing a distribution of Elasticsearch, and extracting that into `/usr/share`.
See the attributes section above to for what defaults you can adjust.
### default
The default recipe creates an elasticsearch user, group, package installation,
configuration files, and service with all of the default options.
Please note that there are [additional examples within the test fixtures](test/fixtures/cookbooks/elasticsearch_test),
including a demonstration of how to configure two instances of Elasticsearch on a single server.
## Resources ## Resources
## Notifications and Service Start/Restart ## Notifications and Service Start/Restart
@ -72,8 +19,6 @@ actions to the underlying service resource if you wish to notify it.
You **must** supply your desired notifications when using each resource if you want Chef to automatically restart services. Again, we don't recommend this unless you know what you're doing. You **must** supply your desired notifications when using each resource if you want Chef to automatically restart services. Again, we don't recommend this unless you know what you're doing.
We are supporting whyrun mode in this cookbook, simply because we're using all builtin resources from core Chef, and these also already support whyrun. If you contribute to this cookbook, please be sure to maintain that or guard dangerous Ruby code with something like `if !whyrun_mode? || nested_resource.whyrun_supported?`.
### Resource names ### Resource names
Many of the resources provided in this cookbook need to share configuration Many of the resources provided in this cookbook need to share configuration
@ -103,6 +48,7 @@ elasticsearch_plugin 'x-pack'
``` ```
### elasticsearch_user ### elasticsearch_user
Actions: `:create`, `:remove` Actions: `:create`, `:remove`
Creates a user and group on the system for use by elasticsearch. Here is an Creates a user and group on the system for use by elasticsearch. Here is an
@ -127,6 +73,7 @@ end
``` ```
### elasticsearch_install ### elasticsearch_install
Actions: `:install`, `:remove` Actions: `:install`, `:remove`
Downloads the elasticsearch software, and unpacks it on the system. There are Downloads the elasticsearch software, and unpacks it on the system. There are
@ -155,29 +102,29 @@ elasticsearch_install 'elasticsearch'
```ruby ```ruby
elasticsearch_install 'my_es_installation' do elasticsearch_install 'my_es_installation' do
type 'package' # type of install type 'package'
version '7.4.2' version '7.8.0'
action :install # could be :remove as well action :install
end end
``` ```
```ruby ```ruby
elasticsearch_install 'my_es_installation' do elasticsearch_install 'my_es_installation' do
type 'tarball' # type of install type 'tarball'
dir '/usr/local' # where to install dir '/usr/local' # where to install
download_url "https://download.elasticsearch.org/elasticsearch/elasticsearch/elasticsearch-1.7.2.tar.gz" download_url "https://download.elasticsearch.org/elasticsearch/elasticsearch/elasticsearch-1.7.2.tar.gz"
# sha256 # sha256
download_checksum "6f81935e270c403681e120ec4395c28b2ddc87e659ff7784608b86beb5223dd2" download_checksum "6f81935e270c403681e120ec4395c28b2ddc87e659ff7784608b86beb5223dd2"
action :install # could be :remove as well action :install
end end
``` ```
```ruby ```ruby
elasticsearch_install 'my_es_installation' do elasticsearch_install 'my_es_installation' do
type 'tarball' # type of install type 'tarball'
version '7.4.2' version '7.8.0'
action :install # could be :remove as well action :install # could be :remove as well
end end
``` ```
@ -194,6 +141,7 @@ end
``` ```
### elasticsearch_configure ### elasticsearch_configure
Actions: `:manage`, `:remove` Actions: `:manage`, `:remove`
Configures an elasticsearch instance; creates directories for configuration, Configures an elasticsearch instance; creates directories for configuration,
@ -205,7 +153,7 @@ which is a hash of any elasticsearch configuration directives. The
other important attribute is `default_configuration` -- this contains the other important attribute is `default_configuration` -- this contains the
minimal set of required defaults. minimal set of required defaults.
Note that these are both _not_ a Chef mash, everything must be in a single level Note that these are both *not* a Chef mash, everything must be in a single level
of keys and values. Any settings you pass in configuration will be merged into of keys and values. Any settings you pass in configuration will be merged into
(and potentially overwrite) any default settings. (and potentially overwrite) any default settings.
@ -215,11 +163,13 @@ for more.
Examples: Examples:
With all defaults - With all defaults -
```ruby ```ruby
elasticsearch_configure 'elasticsearch' elasticsearch_configure 'elasticsearch'
``` ```
With mostly defaults - With mostly defaults -
```ruby ```ruby
elasticsearch_configure 'elasticsearch' do elasticsearch_configure 'elasticsearch' do
allocated_memory '512m' allocated_memory '512m'
@ -232,6 +182,7 @@ end
``` ```
Very complicated - Very complicated -
```ruby ```ruby
elasticsearch_configure 'my_elasticsearch' do elasticsearch_configure 'my_elasticsearch' do
# if you override one of these, you probably want to override all # if you override one of these, you probably want to override all
@ -269,6 +220,7 @@ end
``` ```
### elasticsearch_service ### elasticsearch_service
Actions: `:configure`, `:remove` Actions: `:configure`, `:remove`
Writes out a system service configuration of the appropriate type, and enables Writes out a system service configuration of the appropriate type, and enables
@ -286,6 +238,7 @@ entirely skip trying to setup those scripts. Combined with changing the default
service actions, this will have the same effect as `action :nothing`. service actions, this will have the same effect as `action :nothing`.
### elasticsearch_plugin ### elasticsearch_plugin
Actions: `:install`, `:remove` Actions: `:install`, `:remove`
Installs or removes a plugin to a given elasticsearch instance and plugin Installs or removes a plugin to a given elasticsearch instance and plugin
@ -295,7 +248,7 @@ Furthermore, there isn't a way to determine if a plugin is compatible with ES or
even what version it is. So once we install a plugin to a directory, we even what version it is. So once we install a plugin to a directory, we
generally assume that is the desired one and we don't touch it further. generally assume that is the desired one and we don't touch it further.
See https://github.com/elastic/cookbook-elasticsearch/issues/264 for more info. See <https://github.com/elastic/cookbook-elasticsearch/issues/264> for more info.
NB: You [may encounter issues on certain distros](http://blog.backslasher.net/java-ssl-crash.html) with NSS 3.16.1 and OpenJDK 7.x. NB: You [may encounter issues on certain distros](http://blog.backslasher.net/java-ssl-crash.html) with NSS 3.16.1 and OpenJDK 7.x.
Officially supported or commercial plugins require just the plugin name: Officially supported or commercial plugins require just the plugin name:
@ -324,6 +277,7 @@ end
``` ```
Plugins from Maven Central or Sonatype require 'groupId/artifactId/version': Plugins from Maven Central or Sonatype require 'groupId/artifactId/version':
```ruby ```ruby
elasticsearch_plugin 'mapper-attachments' do elasticsearch_plugin 'mapper-attachments' do
url 'org.elasticsearch/elasticsearch-mapper-attachments/2.6.0' url 'org.elasticsearch/elasticsearch-mapper-attachments/2.6.0'
@ -332,6 +286,7 @@ end
``` ```
Plugins can be installed from a custom URL or file location as follows: Plugins can be installed from a custom URL or file location as follows:
```ruby ```ruby
elasticsearch_plugin 'mapper-attachments' do elasticsearch_plugin 'mapper-attachments' do
url 'http://some.domain.name//my-plugin-1.0.0.zip' url 'http://some.domain.name//my-plugin-1.0.0.zip'
@ -347,6 +302,7 @@ end
The plugin resource respects the `https_proxy` or `http_proxy` (non-SSL) The plugin resource respects the `https_proxy` or `http_proxy` (non-SSL)
[Chef settings](https://docs.chef.io/config_rb_client.html) unless explicitly [Chef settings](https://docs.chef.io/config_rb_client.html) unless explicitly
disabled using `chef_proxy false`: disabled using `chef_proxy false`:
```ruby ```ruby
elasticsearch_plugin 'kopf' do elasticsearch_plugin 'kopf' do
url 'lmenezes/elasticsearch-kopf' url 'lmenezes/elasticsearch-kopf'
@ -377,6 +333,7 @@ end
## License ## License
```text
This software is licensed under the Apache 2 license, quoted below. This software is licensed under the Apache 2 license, quoted below.
Copyright (c) 2015 Elasticsearch <https://www.elastic.co/> Copyright (c) 2015 Elasticsearch <https://www.elastic.co/>
@ -392,3 +349,4 @@ This software is licensed under the Apache 2 license, quoted below.
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and See the License for the specific language governing permissions and
limitations under the License. limitations under the License.
```

213
cookbooks/elasticsearch/attributes/default.rb
View File

@ -1,213 +0,0 @@
# empty settings (populate these for the elasticsearch::default recipe)
# see the resources or README.md to see what you can pass here.
default['elasticsearch']['user'] = {}
default['elasticsearch']['install'] = {}
default['elasticsearch']['configure'] = {}
default['elasticsearch']['service'] = {}
default['elasticsearch']['plugin'] = {}
# platform_family keyed download URLs
default['elasticsearch']['download_urls'] = {
'debian' => 'https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-%s.deb',
'rhel' => 'https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-%s.rpm',
'tarball' => 'https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-%s.tar.gz',
}
default['elasticsearch']['download_urls_v7'] = {
'debian' => 'https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-%s-amd64.deb',
'rhel' => 'https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-%s-x86_64.rpm',
'tarball' => 'https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-%s-linux-x86_64.tar.gz',
}
# platform_family keyed download sha256 checksums
default['elasticsearch']['checksums']['6.0.0']['debian'] = '28f38779156387c1db274d8d733429e574b54b4f518da6f0741f6276f8229939'
default['elasticsearch']['checksums']['6.0.0']['rhel'] = '823fa8aa24e9948dea30f0a468f0403b34a62180e02ed752443d5964334c29a1'
default['elasticsearch']['checksums']['6.0.0']['tarball'] = '0420e877a8b986485244f603770737e9e4e47186fdfa1093768a11e391e3d9f4'
default['elasticsearch']['checksums']['6.0.1']['debian'] = 'ebe6c14638a4909155fe409fb46a7f52bcd3ad2151bfd2f400ab6f8f61c45b3e'
default['elasticsearch']['checksums']['6.0.1']['rhel'] = '25e7a8b152ea85886688398f48bc995d47cd2a12a7d98487748c6166f0732b85'
default['elasticsearch']['checksums']['6.0.1']['tarball'] = 'efaf32aba41e1b7fd086639c0f062c39e1f28b360a78d5c2b8deed797a4c5c57'
default['elasticsearch']['checksums']['6.1.0']['debian'] = '102be4439b1df7f7606003b3c839dbf69d3827c3e996563c98c0d54560b4fc16'
default['elasticsearch']['checksums']['6.1.0']['rhel'] = 'ebac1b4e1fc8ae3f7266cba93ef113510cba13435ada2c92480506d16cf6b865'
default['elasticsearch']['checksums']['6.1.0']['tarball'] = 'c879fe2698635a2f132db4a02d84f657bc0ccdb5c5f68dda5102f9b2afa508d7'
default['elasticsearch']['checksums']['6.1.1']['debian'] = '8b6e65dce742c733aa61da24f9c8c0d4d4b7f53ae11d7f4168e98b5a0ed58b45'
default['elasticsearch']['checksums']['6.1.1']['rhel'] = '9820555c72b61b54686bcf0697cdabace28b02315bb5a156999495a16b103d5a'
default['elasticsearch']['checksums']['6.1.1']['tarball'] = '0cadc90c2ab1bd941e3965eef96fbc2c08b12b832ae81f5882e81505333b74b6'
default['elasticsearch']['checksums']['6.1.2']['debian'] = '64d8bd2bd895904bb91daff656764b10da93531f2011c94d7c779124e53dd5f5'
default['elasticsearch']['checksums']['6.1.2']['rhel'] = 'bfa6809ac94bda92a4ec1bf601c8266f82a4c7842a7702da4dede8f7d5c6a2ec'
default['elasticsearch']['checksums']['6.1.2']['tarball'] = '9c0eae0bdab78c59dac0ba3a9c054e6785dc0f5ce4666e284f42010a326abc0f'
default['elasticsearch']['checksums']['6.2.0']['debian'] = 'eeb974247ea4360e37798888a5f49227d5ea33e11665a060c9b01b25140f9554'
default['elasticsearch']['checksums']['6.2.0']['rhel'] = '869b6506a35aad6b6d82fe987130402cef9b40c33ff7d98eeaa186eb2a628964'
default['elasticsearch']['checksums']['6.2.0']['tarball'] = '7be4a6580aca7d17b2fe1a1e589aa0a005b6240ef5dce6a5288a56f68021f8f6'
default['elasticsearch']['checksums']['6.2.1']['debian'] = '2b145aa11bccd2fe99256b9715ff665fe091b62fc699bc77cd07c528dcbf2391'
default['elasticsearch']['checksums']['6.2.1']['rhel'] = '47b97342821cbd1805826a18a7559a59bd045b9aef66e45c3b293b02aeaaeba8'
default['elasticsearch']['checksums']['6.2.1']['tarball'] = '0ccd13c53d23dcb2aea5c0f71dcbe81283e1e31d6ae5d40dec03656852cb468b'
default['elasticsearch']['checksums']['6.2.2']['debian'] = 'e0a694dcbbac993a4039978ca60e6c05b0bd78ec7eef20a1e95b98979579a47a'
default['elasticsearch']['checksums']['6.2.2']['rhel'] = 'a31277bb89b93da510bf40261882f710a448178ec5430c7a78ac77e91f733cf9'
default['elasticsearch']['checksums']['6.2.2']['tarball'] = 'b26e3546784b39ce3eacc10411e68ada427c5764bcda3064e9bb284eca907983'
default['elasticsearch']['checksums']['6.2.3']['debian'] = 'b54a1b685656a1424d4956e48daed923752fc268b79bb1b8616cc91f6a78e3bb'
default['elasticsearch']['checksums']['6.2.3']['rhel'] = 'd513a6f82436914c35e774529686e5fdfed1af77264e39228e5d64eee22c78ce'
default['elasticsearch']['checksums']['6.2.3']['tarball'] = '01dd8dec5f0acf04336721e404bf4d075675a3acae9f2a9fdcdbb5ca11baca76'
default['elasticsearch']['checksums']['6.2.4']['debian'] = 'f9c889bf1e2e897a881f624dab6a4aa3be5ec1f9d65b6d115021095496351969'
default['elasticsearch']['checksums']['6.2.4']['rhel'] = '9c13e0fb8bc90b7bd6e09c8ba922c2d89c2183bb6161047ec6549c0cbcccee72'
default['elasticsearch']['checksums']['6.2.4']['tarball'] = '91e6f1ea1e1dd39011e7a703d2751ca46ee374665b08b0bfe17e0c0c27000e8e'
default['elasticsearch']['checksums']['6.3.0']['debian'] = '659c3f9a90ae5fe4568d5103de3bbd37e8a03143447118ee61fe72d514b8f4d6'
default['elasticsearch']['checksums']['6.3.0']['rhel'] = 'b6d7951b8ad72c06e7a23263227e1b90b8aa6e5d5ee798a6c849934516121109'
default['elasticsearch']['checksums']['6.3.0']['tarball'] = '0464127140820d82b24bd2830232131ea85bcd49267a8bc7365e4fa391dee2a3'
default['elasticsearch']['checksums']['6.3.1']['debian'] = '350cc95201814ec39b1c80bbb8329faf1dae20a3386b28b626144c6f67ced333'
default['elasticsearch']['checksums']['6.3.1']['rhel'] = '073e2bc52a82bc099cca1534391a057c6506dbcb06e849e24ff4d60187ade4b3'
default['elasticsearch']['checksums']['6.3.1']['tarball'] = '0b4d1c3c66894c9bf54284b53c1448d0e6def89a09c9fe4446e7167e51b8040e'
default['elasticsearch']['checksums']['6.4.0']['debian'] = 'dc7b4b5583ad4689f57369fb8bf8c9b708cfc278701c895762378b010460f658'
default['elasticsearch']['checksums']['6.4.0']['rhel'] = 'b7b8e6265ec4a3aec509fe1e3fb054e6c64256ce7f4ab64d1d5b1629b24883c2'
default['elasticsearch']['checksums']['6.4.0']['tarball'] = 'e9786efb5cecd12adee2807c7640ba9a1ab3b484d2e87497bb8d0b6df0e24f01'
default['elasticsearch']['checksums']['6.4.1']['debian'] = 'ba71dbe16bfb6111dbe7d49ffd447ed1092a754636b30214067d31ff3059e100'
default['elasticsearch']['checksums']['6.4.1']['rhel'] = '2d63f9814ff67edc5c8a8e247ed359a8ef7e23cfbe3a438041a72b44b0be09b2'
default['elasticsearch']['checksums']['6.4.1']['tarball'] = '6fce529465975b2cd2f5aef09b459006636e4b903e623fd6361ae9feeebf1542'
default['elasticsearch']['checksums']['6.4.2']['debian'] = '4ca2cc982db8e46743a98e2c3d678e60c24331b28f79ea424bb8a323e611f678'
default['elasticsearch']['checksums']['6.4.2']['rhel'] = '8987285d92d0d94f0e842e51ba8ed04fb9ad5e078cc68f63ef2ff8ca1986ad08'
default['elasticsearch']['checksums']['6.4.2']['tarball'] = '32a9c0305faed1cbd4e22c85a8c333ca2b6704f1d0006e92709fada10cec3f45'
default['elasticsearch']['checksums']['6.4.3']['debian'] = 'cbf62cf228e556ca89d94562cb747127da69f5a4a04198d1a19230455671d0d4'
default['elasticsearch']['checksums']['6.4.3']['rhel'] = 'd059c695eadefc26788106a7d3d4c6b300bc50ede36dfbcb04b1caa9211d5bff'
default['elasticsearch']['checksums']['6.4.3']['tarball'] = '861bc4d4188ce65df34a8876f104bbb2cddff3586990e57233e0847eb961b15e'
default['elasticsearch']['checksums']['6.5.0']['debian'] = '5fddd2759f7a96f884b0bfb863e04624ba50db39fa6455a87bef2b7bc4bff16e'
default['elasticsearch']['checksums']['6.5.0']['rhel'] = 'b78a6d3ddb59d086f0da28a2e25d9e5f0b5af61f0fd2898ce972f7ca9e84f961'
default['elasticsearch']['checksums']['6.5.0']['tarball'] = 'b158c02dddccc265d25855282015a7b09207c216be99c08114514ed283969b81'
default['elasticsearch']['checksums']['6.5.1']['debian'] = '7de183122a9dbb7b07ee56e18edf0365dc27e413a5962363b944a9aa63ffbb4a'
default['elasticsearch']['checksums']['6.5.1']['rhel'] = 'fc38dcce176a96665c0801054794f62d8a9a23c9f930919248cfbe2622c7255e'
default['elasticsearch']['checksums']['6.5.1']['tarball'] = '17cf2f58c7cbb36c4419a8143321660defe2c492b3a6eaa9a6dcd5e36347d124'
default['elasticsearch']['checksums']['6.5.2']['debian'] = '9cb0997dc6d2be16c988c0ee43ccafd19a8b2e757326af84f4cead40f74c614f'
default['elasticsearch']['checksums']['6.5.2']['rhel'] = 'ddab10c3be94fb89bc7057bdb20624b5e65e3ba8244d655e243119a5103e07c7'
default['elasticsearch']['checksums']['6.5.2']['tarball'] = 'b7b73b871957bdcff75d333eba8c1c6adf3731470f943b159365bde413e45dfc'
default['elasticsearch']['checksums']['6.5.3']['debian'] = '38b30461201fe8d126d124f04d961e7c037bea7a6fb9ca485c08e681d8d30456'
default['elasticsearch']['checksums']['6.5.3']['rhel'] = '2f3eb7682e06211061bea90a0314a515f0c4ef683f45c8e57bfb1dfb14679c3a'
default['elasticsearch']['checksums']['6.5.3']['tarball'] = 'cee4340dfaadf27d73ccaef5a8019e177021e1ffea3b7e622ca5ab9267725cac'
default['elasticsearch']['checksums']['6.5.4']['debian'] = 'c0a062ffb45f989cd3091c66f62605178c41c3735991d95506a6986a90924833'
default['elasticsearch']['checksums']['6.5.4']['rhel'] = 'aa4006f754bd1a0bfaa338ba40d93a1762917c1862951577c62b1f073026b5ba'
default['elasticsearch']['checksums']['6.5.4']['tarball'] = '762e25c036fa2e882cb7e91d83868fa15a1cad8549d279a8c5826a005bb31877'
default['elasticsearch']['checksums']['6.6.0']['debian'] = '0f34cd050bc2a6b4f3a4a8ea71755a03c638e1d929141424a246792e067534a9'
default['elasticsearch']['checksums']['6.6.0']['rhel'] = '8fdaaefa22cef3370bdf353df09bf07695d0e3764ac0c9b6ad8a1412076f9566'
default['elasticsearch']['checksums']['6.6.0']['tarball'] = '4993a17b88268a053042510404bcb808f86d6438fec458be4e0ef1c412681655'
default['elasticsearch']['checksums']['6.6.1']['debian'] = '505d5aae1c2ba66e36be41fe94ec9e4c444cbdd7926aaaa9392356a2c40d2102'
default['elasticsearch']['checksums']['6.6.1']['rhel'] = '3ba9ef562c951eccd070704db498dcf703f05b567d459f93f144e9189e1ee21a'
default['elasticsearch']['checksums']['6.6.1']['tarball'] = 'a4fd359facf03272013e3f2bffc846c1e1f8c60b7fb0daf856831f14947ed297'
default['elasticsearch']['checksums']['6.6.2']['debian'] = 'e5aa8b1449fb08b56ec5d9a52324cadd7439c29533fa7662badede051a9a9778'
default['elasticsearch']['checksums']['6.6.2']['rhel'] = '8a42533764f0332e09cb6d2ed2af02a641766d73a3669458b20fe37615047872'
default['elasticsearch']['checksums']['6.6.2']['tarball'] = 'f32d6fa73b466e29f30a5b6e0bcc6352a5508329f765c948c80c242ddfb97770'
default['elasticsearch']['checksums']['6.7.0']['debian'] = 'fdf31c8774b779cf5bf3c3f6e01371d11b5a3889ca3d9c3dbd2dc6142d96a633'
default['elasticsearch']['checksums']['6.7.0']['rhel'] = 'e09010ab7cbf5c81f8821e25a6617de29bcc658a3c79750e337471ef4711d8b4'
default['elasticsearch']['checksums']['6.7.0']['tarball'] = 'bed3587b892319d2e6b25d68c03930c95cf6d74210e3b368a638ca9f837a2315'
default['elasticsearch']['checksums']['6.7.1']['debian'] = '807223a278ffd29630f01a1c21ff268aa5b51ff36c66a2c56a7a8e275d102831'
default['elasticsearch']['checksums']['6.7.1']['rhel'] = '88b6f4de17ea943ac996648783cd07471ee37b894d17bf17939d9a0cedb30237'
default['elasticsearch']['checksums']['6.7.1']['tarball'] = '7be3f94882130c769e5a6b95dbccdc3cab9f101cbc68edfa661f7188c78f06e2'
default['elasticsearch']['checksums']['6.7.2']['debian'] = '8ad69655c22eabe94295ffb23434ae26ef4d839555ee9cb65261a6fe48e0dc12'
default['elasticsearch']['checksums']['6.7.2']['rhel'] = 'aabd389a4098fb99113907ae9665d454291dd32641cd33056f26ec713d017972'
default['elasticsearch']['checksums']['6.7.2']['tarball'] = 'd0ed638c825914d3c5fd8b34f69b5bcf9cfa5cf85af25145ea7a910ea8f708a9'
default['elasticsearch']['checksums']['6.8.0']['debian'] = 'cb5556c6b75ef2e2d3b0399625aaabae0bbd84229ffc08e8475fdbec1e1b66c0'
default['elasticsearch']['checksums']['6.8.0']['rhel'] = '702bde8def4d51788b49477225d345b693e34ede35c1fbf6fd1421860d3451f5'
default['elasticsearch']['checksums']['6.8.0']['tarball'] = '92c67cf48a145a60579857f190ed03a56e3ae42e599d0c84fed0747af866fffd'
default['elasticsearch']['checksums']['6.8.1']['debian'] = 'e7711bfb68e82f4538067da03634ef1937d71a1ab448d7d4eb3f20bc1c4854d1'
default['elasticsearch']['checksums']['6.8.1']['rhel'] = '4663f759cade5d1613842f86af3f78d064ce3c8e81de9615a353cc2eb413aed0'
default['elasticsearch']['checksums']['6.8.1']['tarball'] = 'ad33d463884acde2d8682cb46927f03603ea66133b84c12f53dc9edbc02447e3'
default['elasticsearch']['checksums']['6.8.2']['debian'] = '9e6b591bef75d042abb4888d914e37aab19bf8afb6eda9efaa6b270ac7edd151'
default['elasticsearch']['checksums']['6.8.2']['rhel'] = '3561e6c1bbac324448928e752b9d6c0d2a413c569993625f6dd255b343f61a6c'
default['elasticsearch']['checksums']['6.8.2']['tarball'] = '2dc3107475659dfd21afa74612a43d5ccbcf4942ab679e8e4e77167cafe27c94'
default['elasticsearch']['checksums']['6.8.3']['debian'] = '65220ba62793445dc0a6fc41078f6020dcfe5b3c9d5ee2ba75d17866b5e09d40'
default['elasticsearch']['checksums']['6.8.3']['rhel'] = '9c3c7d18f12db3e27fb8c751ffae1c5a5dd8db59266a5ebb69d620e3ae363de2'
default['elasticsearch']['checksums']['6.8.3']['tarball'] = '824078e421c9f7e5ab9c875e4019d9ebfe3ada99db286b54dec090f97d1cbe25'
default['elasticsearch']['checksums']['6.8.4']['debian'] = '55b4248b38c72d254230c8469847649489fe4abf1b2e14e3a82f6ae5214f8a8f'
default['elasticsearch']['checksums']['6.8.4']['rhel'] = '1d618c51a263984da1992230d35611b012aff8ade4fee0a16338f9919a9a4a9b'
default['elasticsearch']['checksums']['6.8.4']['tarball'] = 'b8a545cc6d5fc3bd4be7adbe7057faa68fc548cf1923b30a2d90f328c6da96c9'
default['elasticsearch']['checksums']['6.8.5']['debian'] = '115e5649962793fbcc628b0eb660eafe570ff67676ca88cf98e7cec830fd1dd4'
default['elasticsearch']['checksums']['6.8.5']['rhel'] = '96520e7c3b49bdad753ccfad76ce2899dbcecf6a1531f20a59b5087305a1c557'
default['elasticsearch']['checksums']['6.8.5']['tarball'] = '5fe84fa84a0ca0136aad9bbdfc2053f8dda9a3e166ddf34e947bb1fe24e4ce6d'
default['elasticsearch']['checksums']['6.8.6']['debian'] = '82dce29bb3c9108f44e936c3fc6200ce7264bb1a27c1a1cc6dde39b6eac03487'
default['elasticsearch']['checksums']['6.8.6']['rhel'] = '4880396d1a78046efe4a6ec45c1cc2f1f9f0d328466aa32355e95f9834d9d0af'
default['elasticsearch']['checksums']['6.8.6']['tarball'] = '67b9b07a649831eb30779cf8264548a0828414ba9abb1be7bc592a8db91fdf7c'
default['elasticsearch']['checksums']['7.0.0']['debian'] = '491d36af33bc896f8c44c46e1ed550e304220c9eedee991091864f5f3434e2f4'
default['elasticsearch']['checksums']['7.0.0']['rhel'] = '99ed1d22c128189ccb07c99babe7f437180e42c94af15782acf4b80d2520abc4'
default['elasticsearch']['checksums']['7.0.0']['tarball'] = '68deee5e9c961ad48cc83859970ac7a79029fe5f5e6de7e58ea0f7c7713f25ba'
default['elasticsearch']['checksums']['7.0.1']['debian'] = 'b84419e213539f5af2a0b3d73ebc11cf2da7e7061e6308e83333a9282bf920e4'
default['elasticsearch']['checksums']['7.0.1']['rhel'] = '52aba98ea0165492fa899ed8487af5cab10f5e452611995ebac31b5f44facd50'
default['elasticsearch']['checksums']['7.0.1']['tarball'] = '33866cbc61383d1ead01622ff54cadacec0b9fe81224af9b77104be614a56eca'
default['elasticsearch']['checksums']['7.1.0']['debian'] = '9cdb867ebfda5550590e2dc98e37020d9aa6bfd508b1b8be9f11f5683c807349'
default['elasticsearch']['checksums']['7.1.0']['rhel'] = '3facf261e9cbb88c8d792d2ff97e2aeb79c44d2df5fd83591ce4f2d5378eb7a4'
default['elasticsearch']['checksums']['7.1.0']['tarball'] = '91b4bd3722086e83dbe5ee7abf179de19b0cf9dc8a639438bde2721f184cc42e'
default['elasticsearch']['checksums']['7.1.1']['debian'] = '2ef15cb7e37d32b93c51ad537959831bd72cac2627f255d22cc574cec5de6aef'
default['elasticsearch']['checksums']['7.1.1']['rhel'] = '7950c7d97407d8948ca164d81866b389fe734e3ec775452a3e407e49cbd0d5ed'
default['elasticsearch']['checksums']['7.1.1']['tarball'] = 'd8cd7df1232d931b9ff8fda1e960d94b03db21b7f33162d6865ebcbe69a45f95'
default['elasticsearch']['checksums']['7.2.0']['debian'] = '1ff7b88c4bc38438a67719df499b17d4f7082a77eda89f39016f83414554ea95'
default['elasticsearch']['checksums']['7.2.0']['rhel'] = 'a854decb443631a0031a9492c1d5acbed00222381cb63cba68ae6d5deee3994c'
default['elasticsearch']['checksums']['7.2.0']['tarball'] = '24a2c31ff3ef34624752f266c39e40e84b103ab529dae7e062f1cb4b614cd1c5'
default['elasticsearch']['checksums']['7.2.1']['debian'] = '41f507b83fc49a3da5109afd18cc626ec5458acf567f00a80ac3f1c34b6d4b7f'
default['elasticsearch']['checksums']['7.2.1']['rhel'] = '96fdac0a8e6c74182d920b39e3f4830b722731a646126222c189e12a95302e6e'
default['elasticsearch']['checksums']['7.2.1']['tarball'] = 'd5d660895a39755ad6af16ec32cff5e2b645d5faf7150c69acc0fe013c604ae7'
default['elasticsearch']['checksums']['7.3.0']['debian'] = '72ae24cf0f5d97a307f35d116a41e1165e80f58f08b0ca4e6de3ec5228f06f9c'
default['elasticsearch']['checksums']['7.3.0']['rhel'] = 'f49dc809cf48369b70546f13dfb28b43e1a07387b681ca786c794762d52847ca'
default['elasticsearch']['checksums']['7.3.0']['tarball'] = '6cddb4d9a3c5e730f8e35e2155937c069d057129808ea135d31f0ed5f7f2ea22'
default['elasticsearch']['checksums']['7.3.1']['debian'] = '570af7456603fd103408ed61ccec4473302976d46e1ff845b74a881122977e02'
default['elasticsearch']['checksums']['7.3.1']['rhel'] = '240f93d16da4c20d2cc377b7c6a61dbf4fb9634d74829ccb5f7cd42c023bc967'
default['elasticsearch']['checksums']['7.3.1']['tarball'] = '69864bcbca495603004c6c4e6f5d69bce4dc89cd6d672d1ebc7f5387a3d13c74'
default['elasticsearch']['checksums']['7.3.2']['debian'] = '690e98653b3dc50ec5f8e65c480ec41c8c4db0d2c63b5ed3f25fef53d6aaaa55'
default['elasticsearch']['checksums']['7.3.2']['rhel'] = 'bdada0a4c7b5574c41726154212b6b25373e2b4d7d2a64e24238b206ad422ecd'
default['elasticsearch']['checksums']['7.3.2']['tarball'] = '8254dc2d98959cb7964a523a7abab3cd40362dde694ce020ae1efd554efb8e82'
default['elasticsearch']['checksums']['7.4.0']['debian'] = '3edf17d9d63a08a0f7eb7d9727a1737e1c770277f64fe44342115e62f752cc51'
default['elasticsearch']['checksums']['7.4.0']['rhel'] = '1bfae41734c77af3bc66084ac0cc04add1190f9311b045d3c184ea7b3e688334'
default['elasticsearch']['checksums']['7.4.0']['tarball'] = 'b712ced4a7feae46c3f966d52ac24f1b3418ee3ffbf0495a97f27ccf40878873'
default['elasticsearch']['checksums']['7.4.1']['debian'] = '55a92288e81856e9bb6c36c0f7149b24cf36432527ca809fc48e25775b0cf584'
default['elasticsearch']['checksums']['7.4.1']['rhel'] = '8ec30fbd95235cb15d0f27cd40f75a43f640f5832e2ee2d44fe8d2983cd5724f'
default['elasticsearch']['checksums']['7.4.1']['tarball'] = '39be609b1051bc92dc8a9481d47c4c16c9efd77e8a423eca97eb45fec5d4d156'
default['elasticsearch']['checksums']['7.4.2']['debian'] = '514a8e21e173481edb9130ebbf33f15209b467df5c2222632d63c4527c16abc6'
default['elasticsearch']['checksums']['7.4.2']['rhel'] = 'af616eed2cd30411f400dee0c993eb8fccd55e510548697d7cc0eb178ac4adec'
default['elasticsearch']['checksums']['7.4.2']['tarball'] = 'f54475eafbf3384f4378002d87badf013018a9238937a4bd780e71271c670772'
default['elasticsearch']['checksums']['7.5.0']['debian'] = '5b167d15461049f6aa58a96d805c9bcd297ad19467392eea125ce91c5eaaf908'
default['elasticsearch']['checksums']['7.5.0']['rhel'] = 'a8e802c74c3163272fb7119a9d23c1e8f7bbe76e6502a3fcc30709705bc57f4a'
default['elasticsearch']['checksums']['7.5.0']['tarball'] = '1e183717071e329591c109a8560a90b66ddd55958a68cc83385ae123d680b4ca'
default['elasticsearch']['checksums']['7.5.1']['debian'] = 'e566a88e15d8f85cf793c8f971b51eeae6465a0aa73f968ae4b1ee6aa71e4c20'
default['elasticsearch']['checksums']['7.5.1']['rhel'] = 'e6202bba2bd8644d23dcbef9ad7780c847dfe4ee699d3dc1804f6f62eed59c2d'
default['elasticsearch']['checksums']['7.5.1']['tarball'] = '37c317efaacf33a1bae250a59e822864750fddd8caf08c4b6a6c235ffa5f47e8'

115
cookbooks/elasticsearch/chefignore Normal file
View File

@ -0,0 +1,115 @@
# Put files/directories that should be ignored in this file when uploading
# to a Chef Infra Server or Supermarket.
# Lines that start with '# ' are comments.
# OS generated files #
######################
.DS_Store
ehthumbs.db
Icon?
nohup.out
Thumbs.db
.envrc
# EDITORS #
###########
.#*
.project
.settings
*_flymake
*_flymake.*
*.bak
*.sw[a-z]
*.tmproj
*~
\#*
REVISION
TAGS*
tmtags
.vscode
.editorconfig
## COMPILED ##
##############
*.class
*.com
*.dll
*.exe
*.o
*.pyc
*.so
*/rdoc/
a.out
mkmf.log
# Testing #
###########
.circleci/*
.codeclimate.yml
.delivery/*
.foodcritic
.kitchen*
.mdlrc
.overcommit.yml
.rspec
.rubocop.yml
.travis.yml
.watchr
.yamllint
azure-pipelines.yml
Dangerfile
examples/*
features/*
Guardfile
kitchen*.yml
mlc_config.json
Procfile
Rakefile
spec/*
test/*
# SCM #
#######
.git
.gitattributes
.gitconfig
.github/*
.gitignore
.gitkeep
.gitmodules
.svn
*/.bzr/*
*/.git
*/.hg/*
*/.svn/*
# Berkshelf #
#############
Berksfile
Berksfile.lock
cookbooks/*
tmp
# Bundler #
###########
vendor/*
Gemfile
Gemfile.lock
# Policyfile #
##############
Policyfile.rb
Policyfile.lock.json
# Documentation #
#############
CODE_OF_CONDUCT*
CONTRIBUTING*
documentation/*
TESTING*
UPGRADING*
# Vagrant #
###########
.vagrant
Vagrantfile

31
cookbooks/elasticsearch/files/elasticsearch.asc Normal file
View File

@ -0,0 +1,31 @@
-----BEGIN PGP PUBLIC KEY BLOCK-----
Version: GnuPG v2.0.14 (GNU/Linux)
mQENBFI3HsoBCADXDtbNJnxbPqB1vDNtCsqhe49vFYsZN9IOZsZXgp7aHjh6CJBD
A+bGFOwyhbd7at35jQjWAw1O3cfYsKAmFy+Ar3LHCMkV3oZspJACTIgCrwnkic/9
CUliQe324qvObU2QRtP4Fl0zWcfb/S8UYzWXWIFuJqMvE9MaRY1bwUBvzoqavLGZ
j3SF1SPO+TB5QrHkrQHBsmX+Jda6d4Ylt8/t6CvMwgQNlrlzIO9WT+YN6zS+sqHd
1YK/aY5qhoLNhp9G/HxhcSVCkLq8SStj1ZZ1S9juBPoXV1ZWNbxFNGwOh/NYGldD
2kmBf3YgCqeLzHahsAEpvAm8TBa7Q9W21C8vABEBAAG0RUVsYXN0aWNzZWFyY2gg
KEVsYXN0aWNzZWFyY2ggU2lnbmluZyBLZXkpIDxkZXZfb3BzQGVsYXN0aWNzZWFy
Y2gub3JnPokBOAQTAQIAIgUCUjceygIbAwYLCQgHAwIGFQgCCQoLBBYCAwECHgEC
F4AACgkQ0n1mbNiOQrRzjAgAlTUQ1mgo3nK6BGXbj4XAJvuZDG0HILiUt+pPnz75
nsf0NWhqR4yGFlmpuctgCmTD+HzYtV9fp9qW/bwVuJCNtKXk3sdzYABY+Yl0Cez/
7C2GuGCOlbn0luCNT9BxJnh4mC9h/cKI3y5jvZ7wavwe41teqG14V+EoFSn3NPKm
TxcDTFrV7SmVPxCBcQze00cJhprKxkuZMPPVqpBS+JfDQtzUQD/LSFfhHj9eD+Xe
8d7sw+XvxB2aN4gnTlRzjL1nTRp0h2/IOGkqYfIG9rWmSLNlxhB2t+c0RsjdGM4/
eRlPWylFbVMc5pmDpItrkWSnzBfkmXL3vO2X3WvwmSFiQbkBDQRSNx7KAQgA5JUl
zcMW5/cuyZR8alSacKqhSbvoSqqbzHKcUQZmlzNMKGTABFG1yRx9r+wa/fvqP6OT
RzRDvVS/cycws8YX7Ddum7x8uI95b9ye1/Xy5noPEm8cD+hplnpU+PBQZJ5XJ2I+
1l9Nixx47wPGXeClLqcdn0ayd+v+Rwf3/XUJrvccG2YZUiQ4jWZkoxsA07xx7Bj+
Lt8/FKG7sHRFvePFU0ZS6JFx9GJqjSBbHRRkam+4emW3uWgVfZxuwcUCn1ayNgRt
KiFv9jQrg2TIWEvzYx9tywTCxc+FFMWAlbCzi+m4WD+QUWWfDQ009U/WM0ks0Kww
EwSk/UDuToxGnKU2dQARAQABiQEfBBgBAgAJBQJSNx7KAhsMAAoJENJ9ZmzYjkK0
c3MIAIE9hAR20mqJWLcsxLtrRs6uNF1VrpB+4n/55QU7oxA1iVBO6IFu4qgsF12J
TavnJ5MLaETlggXY+zDef9syTPXoQctpzcaNVDmedwo1SiL03uMoblOvWpMR/Y0j
6rm7IgrMWUDXDPvoPGjMl2q1iTeyHkMZEyUJ8SKsaHh4jV9wp9KmC8C+9CwMukL7
vM5w8cgvJoAwsp3Fn59AxWthN3XJYcnMfStkIuWgR7U2r+a210W6vnUxU4oN0PmM
cursYPyeV0NX/KQeUeNMwGTFB6QHS/anRaGQewijkrYYoTNtfllxIu9XYmiBERQ/
qPDlGRlOgVTd9xUfHFkzB52c70E=
=92oX
-----END PGP PUBLIC KEY BLOCK-----

81
cookbooks/elasticsearch/libraries/helpers.rb
View File

@ -6,16 +6,32 @@ module ElasticsearchCookbook
instance_name = resource.instance_name instance_name = resource.instance_name
# if we are truly given a specific name to find # if we are truly given a specific name to find
name_match = find_exact_resource(run_context, resource_type, resource_name) rescue nil name_match = begin
find_exact_resource(run_context, resource_type, resource_name)
rescue
nil
end
return name_match if name_match return name_match if name_match
# first try by instance name attribute # first try by instance name attribute
name_instance = find_instance_name_resource(run_context, resource_type, instance_name) rescue nil name_instance = begin
find_instance_name_resource(run_context, resource_type, instance_name)
rescue
nil
end
return name_instance if name_instance return name_instance if name_instance
# otherwise try the defaults # otherwise try the defaults
name_default = find_exact_resource(run_context, resource_type, 'default') rescue nil name_default = begin
name_elasticsearch = find_exact_resource(run_context, resource_type, 'elasticsearch') rescue nil find_exact_resource(run_context, resource_type, 'default')
rescue
nil
end
name_elasticsearch = begin
find_exact_resource(run_context, resource_type, 'elasticsearch')
rescue
nil
end
# if we found exactly one default name that matched # if we found exactly one default name that matched
return name_default if name_default && !name_elasticsearch return name_default if name_default && !name_elasticsearch
@ -58,58 +74,7 @@ module ElasticsearchCookbook
return results.first return results.first
end end
nil # falsey nil
end
def determine_download_url(new_resource, node)
platform_family = node['platform_family']
version_key = 'download_urls'
if Gem::Version.new(new_resource.version) >= Gem::Version.new('7.0.0')
version_key = 'download_urls_v7'
end
url_string = nil
if new_resource.download_url
url_string = new_resource.download_url
elsif new_resource.type == 'tarball'
url_string = node['elasticsearch'][version_key]['tarball']
elsif new_resource.type == 'package' && node['elasticsearch']['download_urls'][platform_family]
url_string = node['elasticsearch'][version_key][platform_family]
end
if url_string && new_resource.version
return format(url_string, new_resource.version)
elsif url_string
return url_string
end
end
def determine_download_checksum(new_resource, node)
platform_family = node['platform_family']
# for the sake of finding correct attribute data, use rhel for amazon too
# See https://github.com/elastic/cookbook-elasticsearch/issues/609
platform_family = 'rhel' if platform_family == 'amazon'
install_type = new_resource.type
version = new_resource.version
if new_resource.download_checksum
new_resource.download_checksum
elsif install_type == 'tarball'
node && version &&
node['elasticsearch'] &&
node['elasticsearch']['checksums'] &&
node['elasticsearch']['checksums'][version] &&
node['elasticsearch']['checksums'][version]['tarball']
elsif install_type == 'package' && node['elasticsearch']['checksums'][version] && node['elasticsearch']['checksums'][version][platform_family]
node && version && platform_family &&
node['elasticsearch'] &&
node['elasticsearch']['checksums'] &&
node['elasticsearch']['checksums'][version] &&
node['elasticsearch']['checksums'][version][platform_family]
end
end end
# proxy helper for chef sets JVM 8 proxy options # proxy helper for chef sets JVM 8 proxy options
@ -134,6 +99,10 @@ module ElasticsearchCookbook
end end
end end
def es_user
find_es_resource(Chef.run_context, :elasticsearch_user, new_resource)
end
class HashAndMashBlender class HashAndMashBlender
attr_accessor :target attr_accessor :target
def initialize(hash_or_mash_or_whatever) def initialize(hash_or_mash_or_whatever)

73
cookbooks/elasticsearch/libraries/matchers.rb
View File

@ -1,73 +0,0 @@
# ChefSpec is a tool to unit test cookbooks in conjunction with rspec
# Learn more on the README or at https://github.com/sethvargo/chefspec.
if defined?(ChefSpec)
ChefSpec.define_matcher(:elasticsearch_configure)
ChefSpec.define_matcher(:elasticsearch_install)
ChefSpec.define_matcher(:elasticsearch_plugin)
ChefSpec.define_matcher(:elasticsearch_service)
ChefSpec.define_matcher(:elasticsearch_user)
def create_elasticsearch_user(resource_name)
ChefSpec::Matchers::ResourceMatcher.new(:elasticsearch_user, :create, resource_name)
end
def remove_elasticsearch_user(resource_name)
ChefSpec::Matchers::ResourceMatcher.new(:elasticsearch_user, :remove, resource_name)
end
def install_elasticsearch(resource_name)
ChefSpec::Matchers::ResourceMatcher.new(:elasticsearch_install, :install, resource_name)
end
def remove_elasticsearch(resource_name)
ChefSpec::Matchers::ResourceMatcher.new(:elasticsearch_install, :remove, resource_name)
end
def manage_elasticsearch_configure(resource_name)
ChefSpec::Matchers::ResourceMatcher.new(:elasticsearch_configure, :manage, resource_name)
end
def remove_elasticsearch_configure(resource_name)
ChefSpec::Matchers::ResourceMatcher.new(:elasticsearch_configure, :remove, resource_name)
end
def configure_elasticsearch_service(resource_name)
ChefSpec::Matchers::ResourceMatcher.new(:elasticsearch_service, :configure, resource_name)
end
def remove_elasticsearch_service(resource_name)
ChefSpec::Matchers::ResourceMatcher.new(:elasticsearch_service, :remove, resource_name)
end
def enable_elasticsearch_service(resource_name)
ChefSpec::Matchers::ResourceMatcher.new(:elasticsearch_service, :enable, resource_name)
end
def disable_elasticsearch_service(resource_name)
ChefSpec::Matchers::ResourceMatcher.new(:elasticsearch_service, :disable, resource_name)
end
def start_elasticsearch_service(resource_name)
ChefSpec::Matchers::ResourceMatcher.new(:elasticsearch_service, :start, resource_name)
end
def stop_elasticsearch_service(resource_name)
ChefSpec::Matchers::ResourceMatcher.new(:elasticsearch_service, :stop, resource_name)
end
def restart_elasticsearch_service(resource_name)
ChefSpec::Matchers::ResourceMatcher.new(:elasticsearch_service, :restart, resource_name)
end
def status_elasticsearch_service(resource_name)
ChefSpec::Matchers::ResourceMatcher.new(:elasticsearch_service, :status, resource_name)
end
def install_elasticsearch_plugin(resource_name)
ChefSpec::Matchers::ResourceMatcher.new(:elasticsearch_plugin, :install, resource_name)
end
def remove_elasticsearch_plugin(resource_name)
ChefSpec::Matchers::ResourceMatcher.new(:elasticsearch_plugin, :remove, resource_name)
end
end

164
cookbooks/elasticsearch/libraries/provider_configure.rb
View File

@ -1,164 +0,0 @@
# Chef Provider for configuring an elasticsearch instance
class ElasticsearchCookbook::ConfigureProvider < Chef::Provider::LWRPBase
include ElasticsearchCookbook::Helpers
provides :elasticsearch_configure
def whyrun_supported?
true # we only use core Chef resources that also support whyrun
end
def action_manage
# lookup existing ES resources
es_user = find_es_resource(Chef.run_context, :elasticsearch_user, new_resource)
es_svc = find_es_resource(Chef.run_context, :elasticsearch_service, new_resource)
es_install = find_es_resource(Chef.run_context, :elasticsearch_install, new_resource)
default_configuration = new_resource.default_configuration.dup
# if a subdir parameter is missing but dir is set, infer the subdir name
# then go and be sure it's also set in the YML hash if it wasn't given there
if new_resource.path_data && default_configuration['path.data'].nil?
default_configuration['path.data'] = new_resource.path_data
end
if new_resource.path_logs && default_configuration['path.logs'].nil?
default_configuration['path.logs'] = new_resource.path_logs
end
# calculation for memory allocation; 50% or 31g, whatever is smaller
unless new_resource.allocated_memory
half = ((node['memory']['total'].to_i * 0.5).floor / 1024)
malloc_str = (half > 30_500 ? '30500m' : "#{half}m")
new_resource.allocated_memory malloc_str
end
# Create ES directories
#
[new_resource.path_conf, "#{new_resource.path_conf}/scripts"].each do |path|
d = directory path do
owner es_user.username
group es_user.groupname
mode '0750'
recursive true
action :nothing
end
d.run_action(:create)
new_resource.updated_by_last_action(true) if d.updated_by_last_action?
end
# Create data path directories
#
data_paths = new_resource.path_data.is_a?(Array) ? new_resource.path_data : new_resource.path_data.split(',')
data_paths = data_paths << new_resource.path_logs
data_paths.each do |path|
d = directory path.strip do
owner es_user.username
group es_user.groupname
mode '0755'
recursive true
action :nothing
end
d.run_action(:create)
new_resource.updated_by_last_action(true) if d.updated_by_last_action?
end
# Create elasticsearch shell variables file
#
# Valid values in /etc/sysconfig/elasticsearch or /etc/default/elasticsearch
# ES_HOME JAVA_HOME ES_PATH_CONF DATA_DIR LOG_DIR PID_DIR ES_JAVA_OPTS
# RESTART_ON_UPGRADE ES_USER ES_GROUP ES_STARTUP_SLEEP_TIME MAX_OPEN_FILES
# MAX_LOCKED_MEMORY MAX_MAP_COUNT
#
# We provide these values as resource attributes/parameters directly
params = {}
params[:ES_HOME] = new_resource.path_home
params[:JAVA_HOME] = new_resource.java_home
params[:ES_PATH_CONF] = new_resource.path_conf
params[:DATA_DIR] = new_resource.path_data
params[:LOG_DIR] = new_resource.path_logs
params[:PID_DIR] = new_resource.path_pid
params[:RESTART_ON_UPGRADE] = new_resource.restart_on_upgrade
params[:ES_USER] = es_user.username if es_install.type == 'tarball'
params[:ES_GROUP] = es_user.groupname if es_install.type == 'tarball'
params[:ES_STARTUP_SLEEP_TIME] = new_resource.startup_sleep_seconds.to_s
params[:MAX_OPEN_FILES] = new_resource.nofile_limit
params[:MAX_LOCKED_MEMORY] = new_resource.memlock_limit
params[:MAX_MAP_COUNT] = new_resource.max_map_count
default_config_name = es_svc.service_name || es_svc.instance_name || new_resource.instance_name || 'elasticsearch'
shell_template = template "elasticsearch.in.sh-#{default_config_name}" do
path %w[rhel amazon].include?(node['platform_family']) ? "/etc/sysconfig/#{default_config_name}" : "/etc/default/#{default_config_name}"
source new_resource.template_elasticsearch_env
cookbook new_resource.cookbook_elasticsearch_env
mode '0644'
variables(params: params)
action :nothing
end
shell_template.run_action(:create)
new_resource.updated_by_last_action(true) if shell_template.updated_by_last_action?
# Create jvm.options file
#
jvm_options_template = template "jvm_options-#{default_config_name}" do
path "#{new_resource.path_conf}/jvm.options"
source new_resource.template_jvm_options
cookbook new_resource.cookbook_jvm_options
owner es_user.username
group es_user.groupname
mode '0644'
variables(jvm_options: [
"-Xms#{new_resource.allocated_memory}",
"-Xmx#{new_resource.allocated_memory}",
new_resource.jvm_options,
].flatten.join("\n"))
action :nothing
end
jvm_options_template.run_action(:create)
new_resource.updated_by_last_action(true) if jvm_options_template.updated_by_last_action?
# Create ES logging file
#
logging_template = template "log4j2_properties-#{default_config_name}" do
path "#{new_resource.path_conf}/log4j2.properties"
source new_resource.template_log4j2_properties
cookbook new_resource.cookbook_log4j2_properties
owner es_user.username
group es_user.groupname
mode '0640'
variables(logging: new_resource.logging)
action :nothing
end
logging_template.run_action(:create)
new_resource.updated_by_last_action(true) if logging_template.updated_by_last_action?
# Create ES elasticsearch.yml file
#
merged_configuration = default_configuration.merge(new_resource.configuration.dup)
# warn if someone is using symbols. we don't support.
found_symbols = merged_configuration.keys.select { |s| s.is_a?(Symbol) }
unless found_symbols.empty?
Chef::Log.warn("Please change the following to strings in order to work with this Elasticsearch cookbook: #{found_symbols.join(',')}")
end
# workaround for https://github.com/elastic/cookbook-elasticsearch/issues/590
config_vars = ElasticsearchCookbook::HashAndMashBlender.new(merged_configuration).to_hash
yml_template = template "elasticsearch.yml-#{default_config_name}" do
path "#{new_resource.path_conf}/elasticsearch.yml"
source new_resource.template_elasticsearch_yml
cookbook new_resource.cookbook_elasticsearch_yml
owner es_user.username
group es_user.groupname
mode '0640'
helpers(ElasticsearchCookbook::Helpers)
variables(config: config_vars)
action :nothing
end
yml_template.run_action(:create)
new_resource.updated_by_last_action(true) if yml_template.updated_by_last_action?
end
end

238
cookbooks/elasticsearch/libraries/provider_install.rb
View File

@ -1,238 +0,0 @@
# Chef Provider for installing or removing Elasticsearch from package or tarball
# downloaded from elasticsearch.org and installed by package manager or ark resource
class ElasticsearchCookbook::InstallProvider < Chef::Provider::LWRPBase
include ElasticsearchCookbook::Helpers
include Chef::DSL::IncludeRecipe
provides :elasticsearch_install
def whyrun_supported?
true # we only use core Chef resources that also support whyrun
end
def action_install
if new_resource.type == 'tarball'
install_tarball_wrapper_action
elsif new_resource.type == 'package'
install_package_wrapper_action
elsif new_resource.type == 'repository'
install_repo_wrapper_action
else
raise "#{install_type} is not a valid install type"
end
end
def action_remove
if new_resource.type == 'tarball'
remove_tarball_wrapper_action
elsif new_resource.type == 'package'
remove_package_wrapper_action
elsif new_resource.type == 'repository'
remove_repo_wrapper_action
else
raise "#{install_type} is not a valid install type"
end
end
protected
def install_repo_wrapper_action
es_user = find_es_resource(Chef.run_context, :elasticsearch_user, new_resource)
unless es_user && es_user.username == 'elasticsearch' && es_user.groupname == 'elasticsearch'
raise 'Custom usernames/group names is not supported in Elasticsearch 6+ repository installation'
end
if new_resource.enable_repository_actions
if node['platform_family'] == 'debian'
apt_r = apt_repo_resource
apt_r.run_action(:add)
new_resource.updated_by_last_action(true) if apt_r.updated_by_last_action?
else
yr_r = yum_repo_resource
yr_r.run_action(:create)
new_resource.updated_by_last_action(true) if yr_r.updated_by_last_action?
end
end
if !new_resource.version.nil? && %w[rhel amazon].include?(node['platform_family']) && !new_resource.version.include?('-')
# NB: yum repo packages are broken in Chef if you don't specify a release
# https://github.com/chef/chef/issues/4103
new_resource.version = "#{new_resource.version}-1"
end
pkg_r = package 'elasticsearch' do
options new_resource.package_options
version new_resource.version
action :nothing
end
pkg_r.run_action(:install)
new_resource.updated_by_last_action(true) if pkg_r.updated_by_last_action?
end
def remove_repo_wrapper_action
if new_resource.enable_repository_actions
if node['platform_family'] == 'debian'
apt_r = apt_repo_resource
apt_r.run_action(:remove)
new_resource.updated_by_last_action(true) if apt_r.updated_by_last_action?
else
yr_r = yum_repo_resource
yr_r.run_action(:delete)
new_resource.updated_by_last_action(true) if yr_r.updated_by_last_action?
end
end
pkg_r = package 'elasticsearch' do
options new_resource.package_options
version new_resource.version
action :nothing
end
pkg_r.run_action(:remove)
new_resource.updated_by_last_action(true) if pkg_r.updated_by_last_action?
end
def install_package_wrapper_action
es_user = find_es_resource(Chef.run_context, :elasticsearch_user, new_resource)
unless es_user && es_user.username == 'elasticsearch' && es_user.groupname == 'elasticsearch'
raise 'Custom usernames/group names is not supported in Elasticsearch 6+ package installation'
end
found_download_url = determine_download_url(new_resource, node)
unless found_download_url
raise 'Could not determine download url for package on this platform'
end
filename = found_download_url.split('/').last
checksum = determine_download_checksum(new_resource, node)
package_options = new_resource.package_options
unless checksum
Chef::Log.warn("No checksum was provided for #{found_download_url}, this may download a new package on every chef run!")
end
remote_file_r = remote_file "#{Chef::Config[:file_cache_path]}/#{filename}" do
source found_download_url
checksum checksum
mode '0644'
action :nothing
end
remote_file_r.run_action(:create)
new_resource.updated_by_last_action(true) if remote_file_r.updated_by_last_action?
pkg_r = if node['platform_family'] == 'debian'
dpkg_package "#{Chef::Config[:file_cache_path]}/#{filename}" do
options package_options
action :nothing
end
else
package "#{Chef::Config[:file_cache_path]}/#{filename}" do
options package_options
action :nothing
end
end
pkg_r.run_action(:install)
new_resource.updated_by_last_action(true) if pkg_r.updated_by_last_action?
end
def remove_package_wrapper_action
package_url = determine_download_url(new_resource, node)
filename = package_url.split('/').last
pkg_r = if node['platform_family'] == 'debian'
dpkg_package "#{Chef::Config[:file_cache_path]}/#{filename}" do
action :nothing
end
else
package "#{Chef::Config[:file_cache_path]}/#{filename}" do
action :nothing
end
end
pkg_r.run_action(:remove)
new_resource.updated_by_last_action(true) if pkg_r.updated_by_last_action?
end
def install_tarball_wrapper_action
include_recipe 'ark'
es_user = find_es_resource(Chef.run_context, :elasticsearch_user, new_resource)
found_download_url = determine_download_url(new_resource, node)
unless found_download_url
raise 'Could not determine download url for tarball on this platform'
end
ark_r = ark 'elasticsearch' do
url found_download_url
owner es_user.username
group es_user.groupname
version new_resource.version
has_binaries ['bin/elasticsearch', 'bin/elasticsearch-plugin']
checksum determine_download_checksum(new_resource, node)
prefix_root new_resource.dir
prefix_home new_resource.dir
not_if do
link = "#{new_resource.dir}/elasticsearch"
target = "#{new_resource.dir}/elasticsearch-#{new_resource.version}"
binary = "#{target}/bin/elasticsearch"
::File.directory?(link) && ::File.symlink?(link) && ::File.readlink(link) == target && ::File.exist?(binary)
end
action :nothing
end
ark_r.run_action(:install)
new_resource.updated_by_last_action(true) if ark_r.updated_by_last_action?
# destroy the sample config directory for tarball installs, or it will
# take precedence beyond the default stuff in /etc/elasticsearch and within
# /etc/sysconfig or /etc/default
sample_r = directory "#{new_resource.dir}/elasticsearch/config" do
action :nothing
recursive true
end
sample_r.run_action(:delete)
new_resource.updated_by_last_action(true) if sample_r.updated_by_last_action?
end
def remove_tarball_wrapper_action
# remove the symlink to this version
link_r = link "#{new_resource.dir}/elasticsearch" do
only_if do
link = "#{new_resource.dir}/elasticsearch"
target = "#{new_resource.dir}/elasticsearch-#{new_resource.version}"
::File.directory?(link) && ::File.symlink?(link) && ::File.readlink(link) == target
end
action :nothing
end
link_r.run_action(:delete)
new_resource.updated_by_last_action(true) if link_r.updated_by_last_action?
# remove the specific version
d_r = directory "#{new_resource.dir}/elasticsearch-#{new_resource.version}" do
recursive true
action :nothing
end
d_r.run_action(:delete)
new_resource.updated_by_last_action(true) if d_r.updated_by_last_action?
end
def yum_repo_resource
yum_repository "elastic-#{new_resource.version.to_i}.x" do
baseurl "https://artifacts.elastic.co/packages/#{new_resource.version.to_i}.x/yum"
gpgkey 'https://artifacts.elastic.co/GPG-KEY-elasticsearch'
action :nothing # :add, remove
end
end
def apt_repo_resource
apt_repository "elastic-#{new_resource.version.to_i}.x" do
uri "https://artifacts.elastic.co/packages/#{new_resource.version.to_i}.x/apt"
key 'https://artifacts.elastic.co/GPG-KEY-elasticsearch'
components ['main']
distribution 'stable'
action :nothing # :create, :delete
end
end
end

108
cookbooks/elasticsearch/libraries/provider_plugin.rb
View File

@ -1,108 +0,0 @@
# Chef Provider for installing an elasticsearch plugin
class ElasticsearchCookbook::PluginProvider < Chef::Provider::LWRPBase
include ElasticsearchCookbook::Helpers
include Chef::Mixin::ShellOut
provides :elasticsearch_plugin
def whyrun_supported?
true # we only use core Chef resources that also support whyrun, or guard
end
def action_install
return if plugin_exists(new_resource.plugin_name)
# since install can take a URL argument instead
url_or_name = new_resource.url || new_resource.plugin_name
manage_plugin("install #{url_or_name}")
end # action
def action_remove
return unless plugin_exists(new_resource.plugin_name)
manage_plugin("remove #{new_resource.plugin_name}")
end # action
def manage_plugin(arguments)
es_user = find_es_resource(Chef.run_context, :elasticsearch_user, new_resource)
es_install = find_es_resource(Chef.run_context, :elasticsearch_install, new_resource)
es_conf = find_es_resource(Chef.run_context, :elasticsearch_configure, new_resource)
assert_state_is_valid(es_user, es_install, es_conf) unless whyrun_mode?
# shell_out! automatically raises on error, logs command output
# required for package installs that show up with parent dir owned by root
plugin_dir_exists = ::File.exist?(es_conf.path_plugins)
unless plugin_dir_exists
cmd_str = "mkdir -p #{es_conf.path_plugins}"
if whyrun_mode?
Chef::Log.info("Would run command: #{cmd_str}")
else
shell_out_as_user!(cmd_str, Chef.run_context)
new_resource.updated_by_last_action(true)
end
end
unless plugin_exists(new_resource.plugin_name)
cmd_str = "#{es_conf.path_bin}/elasticsearch-plugin #{arguments.chomp(' ')} #{new_resource.options}".chomp(' ')
if whyrun_mode?
Chef::Log.info("Would run command: #{cmd_str}")
else
command_array = cmd_str.split(' ')
shell_out_as_user!(command_array, Chef.run_context)
new_resource.updated_by_last_action(true)
end
end
end
def plugin_exists(name)
es_conf = find_es_resource(Chef.run_context, :elasticsearch_configure, new_resource)
path = es_conf.path_plugins
Dir.entries(path).any? do |plugin|
next if plugin =~ /^\./
name == plugin
end
rescue
false
end
def assert_state_is_valid(_es_user, _es_install, es_conf)
unless es_conf.path_plugins # we do not check existence (may not exist if no plugins installed)
raise "Could not determine the plugin directory (#{es_conf.path_plugins}). Please check elasticsearch_configure[#{es_conf.name}]."
end
unless es_conf.path_bin && ::File.exist?(es_conf.path_bin)
raise "Could not determine the binary directory (#{es_conf.path_bin}). Please check elasticsearch_configure[#{es_conf.name}]."
end
true
end
def shell_out_as_user!(command, run_ctx)
es_install = find_es_resource(run_ctx, :elasticsearch_install, new_resource)
es_conf = find_es_resource(run_ctx, :elasticsearch_configure, new_resource)
es_svc = find_es_resource(run_ctx, :elasticsearch_service, new_resource)
# we need to figure out the env file path to set environment for plugins
default_config_name = es_svc.service_name || es_svc.instance_name || es_conf.instance_name || 'elasticsearch'
include_file_resource = find_exact_resource(run_ctx, :template, "elasticsearch.in.sh-#{default_config_name}")
env = { 'ES_INCLUDE' => include_file_resource.path }
# Add HTTP Proxy vars unless explicitly told not to
if new_resource.chef_proxy
env['ES_JAVA_OPTS'] = "#{ENV['ES_JAVA_OPTS']} #{get_java_proxy_arguments}"
end
# See this link for an explanation:
# https://www.elastic.co/guide/en/elasticsearch/plugins/2.1/plugin-management.html
if es_install.type == 'package' || es_install.type == 'repository'
# package installations should install plugins as root
shell_out!(command, :env => env, :timeout => 1200)
else
# non-package installations should install plugins as the ES user
es_user = find_es_resource(run_ctx, :elasticsearch_user, new_resource)
shell_out!(command, user: es_user.username, group: es_user.groupname, :env => env, :timeout => 1200)
end
end
end # provider

140
cookbooks/elasticsearch/libraries/provider_service.rb
View File

@ -1,140 +0,0 @@
# Chef Provider for configuring an elasticsearch service in the init system
class ElasticsearchCookbook::ServiceProvider < Chef::Provider::LWRPBase
provides :elasticsearch_service
include ElasticsearchCookbook::Helpers
def whyrun_supported?
true # we only use core Chef resources that also support whyrun
end
def action_remove
raise "#{new_resource} remove not currently implemented"
end
def action_configure
es_user = find_es_resource(Chef.run_context, :elasticsearch_user, new_resource)
es_install = find_es_resource(Chef.run_context, :elasticsearch_install, new_resource)
es_conf = find_es_resource(Chef.run_context, :elasticsearch_configure, new_resource)
default_config_name = new_resource.service_name || new_resource.instance_name || es_conf.instance_name || 'elasticsearch'
d_r = directory "#{es_conf.path_pid}-#{default_config_name}" do
path es_conf.path_pid
owner es_user.username
group es_user.groupname
mode '0755'
recursive true
action :nothing
end
d_r.run_action(:create)
new_resource.updated_by_last_action(true) if d_r.updated_by_last_action?
# Create service for init and systemd
#
if new_resource.init_source
init_r = template "/etc/init.d/#{new_resource.service_name}" do
source new_resource.init_source
cookbook new_resource.init_cookbook
owner 'root'
mode '0755'
variables(
# we need to include something about #{progname} fixed in here.
program_name: new_resource.service_name,
install_type: es_install.type
)
only_if { ::File.exist?('/etc/init.d') }
action :nothing
end
init_r.run_action(:create)
new_resource.updated_by_last_action(true) if init_r.updated_by_last_action?
end
if new_resource.systemd_source
systemd_parent_r = directory "/usr/lib/systemd/system-#{default_config_name}" do
path '/usr/lib/systemd/system'
action :nothing
only_if { ::File.exist?('/usr/lib/systemd') }
end
systemd_parent_r.run_action(:create)
new_resource.updated_by_last_action(true) if systemd_parent_r.updated_by_last_action?
default_conf_dir = %w[rhel amazon].include?(node['platform_family']) ? '/etc/sysconfig' : '/etc/default'
systemd_r = template "/usr/lib/systemd/system/#{new_resource.service_name}.service" do
source new_resource.systemd_source
cookbook new_resource.systemd_cookbook
owner 'root'
mode '0644'
variables(
# we need to include something about #{progname} fixed in here.
program_name: new_resource.service_name,
default_dir: default_conf_dir,
path_home: es_conf.path_home,
es_user: es_user.username,
es_group: es_user.groupname,
nofile_limit: es_conf.nofile_limit,
install_type: es_install.type
)
only_if 'which systemctl'
action :nothing
end
systemd_r.run_action(:create)
# special case here -- must reload unit files if we modified one
if systemd_r.updated_by_last_action?
new_resource.updated_by_last_action(systemd_r.updated_by_last_action?)
reload_r = execute "reload-systemd-#{new_resource.service_name}" do
command 'systemctl daemon-reload'
action :nothing
only_if 'which systemctl'
end
reload_r.run_action(:run)
end
end
# flatten in an array here, in case the service_actions are a symbol vs. array
[new_resource.service_actions].flatten.each do |act|
passthrough_action(act)
end
end
# Passthrough actions to service[service_name]
#
def action_enable
passthrough_action(:enable)
end
def action_disable
passthrough_action(:disable)
end
def action_start
passthrough_action(:start)
end
def action_stop
passthrough_action(:stop)
end
def action_restart
passthrough_action(:restart)
end
def action_status
passthrough_action(:status)
end
def passthrough_action(action)
svc_r = lookup_service_resource
svc_r.run_action(action)
new_resource.updated_by_last_action(true) if svc_r.updated_by_last_action?
end
def lookup_service_resource
rc = Chef.run_context.resource_collection
rc.find("service[#{new_resource.service_name}]")
rescue
service new_resource.service_name do
supports status: true, restart: true
action :nothing
end
end
end

48
cookbooks/elasticsearch/libraries/provider_user.rb
View File

@ -1,48 +0,0 @@
# Chef Provider for creating a user and group for Elasticsearch
class ElasticsearchCookbook::UserProvider < Chef::Provider::LWRPBase
include ElasticsearchCookbook::Helpers
provides :elasticsearch_user
def whyrun_supported?
true # we only use core Chef resources that also support whyrun
end
def action_create
group_r = group new_resource.groupname do
gid new_resource.gid
action :nothing
system true
end
group_r.run_action(:create)
new_resource.updated_by_last_action(true) if group_r.updated_by_last_action?
user_r = user new_resource.username do
comment new_resource.comment
shell new_resource.shell
uid new_resource.uid
gid new_resource.groupname
manage_home false
action :nothing
system true
end
user_r.run_action(:create)
new_resource.updated_by_last_action(true) if user_r.updated_by_last_action?
end
def action_remove
# delete user before deleting the group
user_r = user new_resource.username do
action :nothing
end
user_r.run_action(:remove)
new_resource.updated_by_last_action(true) if user_r.updated_by_last_action?
group_r = group new_resource.groupname do
action :nothing
end
group_r.run_action(:remove)
new_resource.updated_by_last_action(true) if group_r.updated_by_last_action?
end
end

92
cookbooks/elasticsearch/libraries/resource_configure.rb
View File

@ -1,92 +0,0 @@
# Chef Resource for configuring an Elasticsearch node
class ElasticsearchCookbook::ConfigureResource < Chef::Resource::LWRPBase
resource_name :elasticsearch_configure
provides :elasticsearch_configure
actions(:manage, :remove)
default_action :manage
# this is what helps the various resources find each other
attribute(:instance_name, kind_of: String, default: nil)
# if you override one of these, you should probably override them all
attribute(:path_home, kind_of: String, default: '/usr/share/elasticsearch')
attribute(:path_conf, kind_of: String, default: '/etc/elasticsearch')
attribute(:path_data, kind_of: String, default: '/var/lib/elasticsearch')
attribute(:path_logs, kind_of: String, default: '/var/log/elasticsearch')
attribute(:path_pid, kind_of: String, default: '/var/run/elasticsearch')
attribute(:path_plugins, kind_of: String, default: '/usr/share/elasticsearch/plugins')
attribute(:path_bin, kind_of: String, default: '/usr/share/elasticsearch/bin')
attribute(:template_elasticsearch_env, kind_of: String, default: 'elasticsearch.in.sh.erb')
attribute(:cookbook_elasticsearch_env, kind_of: String, default: 'elasticsearch')
attribute(:template_jvm_options, kind_of: String, default: 'jvm_options.erb')
attribute(:cookbook_jvm_options, kind_of: String, default: 'elasticsearch')
attribute(:template_elasticsearch_yml, kind_of: String, default: 'elasticsearch.yml.erb')
attribute(:cookbook_elasticsearch_yml, kind_of: String, default: 'elasticsearch')
attribute(:template_log4j2_properties, kind_of: String, default: 'log4j2.properties.erb')
attribute(:cookbook_log4j2_properties, kind_of: String, default: 'elasticsearch')
attribute(:logging, kind_of: Hash, default: {}.freeze)
attribute(:java_home, kind_of: String, default: nil)
# other settings in /etc/default or /etc/sysconfig
attribute(:memlock_limit, kind_of: String, default: 'unlimited')
attribute(:max_map_count, kind_of: String, default: '262144')
attribute(:nofile_limit, kind_of: String, default: '65536')
attribute(:startup_sleep_seconds, kind_of: [String, Integer], default: 5)
attribute(:restart_on_upgrade, kind_of: [TrueClass, FalseClass], default: false)
# Calculations for this are done in the provider, as we can't do them in the
# resource definition. default is 50% of RAM or 31GB, which ever is smaller.
attribute(:allocated_memory, kind_of: String)
attribute(:jvm_options, kind_of: Array, default:
%w(
-XX:+UseConcMarkSweepGC
-XX:CMSInitiatingOccupancyFraction=75
-XX:+UseCMSInitiatingOccupancyOnly
-XX:+AlwaysPreTouch
-server
-Xss1m
-Djava.awt.headless=true
-Dfile.encoding=UTF-8
-Djna.nosys=true
-XX:-OmitStackTraceInFastThrow
-Dio.netty.noUnsafe=true
-Dio.netty.noKeySetOptimization=true
-Dio.netty.recycler.maxCapacityPerThread=0
-Dlog4j.shutdownHookEnabled=false
-Dlog4j2.disable.jmx=true
-XX:+HeapDumpOnOutOfMemoryError
).freeze)
# These are the default settings. Most of the time, you want to override
# the `configuration` attribute below. If you do override the defaults, you
# must supply ALL needed defaults, and don't use nil as a value in the hash.
attribute(:default_configuration, kind_of: Hash, default: {
# === NAMING
'cluster.name' => 'elasticsearch',
# can't access node.name, so expect to have to set set this
'node.name' => Chef::Config[:node_name],
# if omitted or nil, these will be populated from attributes above
'path.data' => nil, # see path_data above
'path.logs' => nil, # see path_logs above
# Refer to ES documentation on how to configure these to a
# specific node role/type instead of using the defaults
#
# 'node.data' => ?,
# 'node.master' => ?,
}.freeze)
# These settings are merged with the `default_configuration` attribute,
# allowing you to override and set specific settings. Unless you intend to
# wipe out all default settings, your configuration items should go here.
#
attribute(:configuration, kind_of: Hash, default: {}.freeze)
end

31
cookbooks/elasticsearch/libraries/resource_install.rb
View File

@ -1,31 +0,0 @@
# Chef Resource for installing or removing Elasticsearch from package or source
class ElasticsearchCookbook::InstallResource < Chef::Resource::LWRPBase
resource_name :elasticsearch_install
provides :elasticsearch_install
actions(:install, :remove)
default_action :install
# this is what helps the various resources find each other
attribute(:instance_name, kind_of: String)
# if this version parameter is not set by the caller, we look at
# `attributes/default.rb` for a default value to use, or we raise
attribute(:version, kind_of: String, default: '7.4.2')
# we allow a string or symbol for this value
attribute(:type, kind_of: String, equal_to: %w(package tarball repository), default: 'repository')
# these use `attributes/default.rb` for default values per platform and install type
attribute(:download_url, kind_of: String)
attribute(:download_checksum, kind_of: String) # sha256
# where to install?
attribute(:dir, kind_of: String, default: '/usr/share')
# attributes used by the package-flavor provider
attribute(:package_options, kind_of: String)
# attributes for the repository-option install
attribute(:enable_repository_actions, kind_of: [TrueClass, FalseClass], default: true)
end

19
cookbooks/elasticsearch/libraries/resource_plugin.rb
View File

@ -1,19 +0,0 @@
# Chef Resource for installing an elasticsearch plugin
class ElasticsearchCookbook::PluginResource < Chef::Resource::LWRPBase
resource_name :elasticsearch_plugin
provides :elasticsearch_plugin
include ElasticsearchCookbook::Helpers
actions(:install, :remove)
default_action :install
# if the name or url are different from the resource name
attribute(:plugin_name, kind_of: String, name_attribute: true)
attribute(:url, kind_of: String, default: nil)
attribute(:chef_proxy, kind_of: [TrueClass, FalseClass], default: false)
attribute(:options, kind_of: String, default: '')
# this is what helps the various resources find each other
attribute(:instance_name, kind_of: String, default: nil)
end

28
cookbooks/elasticsearch/libraries/resource_service.rb
View File

@ -1,28 +0,0 @@
# Chef Resource for declaring a service for Elasticsearch
class ElasticsearchCookbook::ServiceResource < Chef::Resource::LWRPBase
resource_name :elasticsearch_service
provides :elasticsearch_service
actions(
:configure, :remove, # our custom actions
:enable, :disable, :start, :stop, :restart, :status # passthrough to service resource
)
default_action :configure
# this is what helps the various resources find each other
attribute(:instance_name, kind_of: String, default: nil)
attribute(:service_name, kind_of: String, name_attribute: true)
attribute(:args, kind_of: String, default: '-d')
# service actions
attribute(:service_actions, kind_of: [Symbol, String, Array], default: [:enable, :start].freeze)
# allow overridable init script
attribute(:init_source, kind_of: String, default: 'initscript.erb')
attribute(:init_cookbook, kind_of: String, default: 'elasticsearch')
# allow overridable systemd unit
attribute(:systemd_source, kind_of: String, default: 'systemd_unit.erb')
attribute(:systemd_cookbook, kind_of: String, default: 'elasticsearch')
end

19
cookbooks/elasticsearch/libraries/resource_user.rb
View File

@ -1,19 +0,0 @@
# Chef Resource for declaring a user and group for Elasticsearch
class ElasticsearchCookbook::UserResource < Chef::Resource::LWRPBase
resource_name :elasticsearch_user
provides :elasticsearch_user
actions(:create, :remove)
default_action :create
# this is what helps the various resources find each other
attribute(:instance_name, kind_of: String, default: nil)
attribute(:username, kind_of: String, name_attribute: true) # default to resource name
attribute(:uid, kind_of: Integer)
attribute(:shell, kind_of: String, default: '/bin/bash')
attribute(:comment, kind_of: String, default: 'Elasticsearch User')
attribute(:groupname, kind_of: String, name_attribute: true) # default to resource name
attribute(:gid, kind_of: Integer)
end

220
cookbooks/elasticsearch/libraries/versions.rb Normal file
View File

@ -0,0 +1,220 @@
module ElasticsearchCookbook
module VersionHelpers
def default_download_url(version)
platform_family = node['platform_family']
machine = node['kernel']['machine']
case platform_family
when 'debian'
arch = machine.include?('x86_64') ? 'amd64' : 'arm64'
file_type = 'deb'
when 'rhel', 'fedora', 'amazon'
arch = machine.include?('x86_64') ? 'x86_64' : 'aarch64'
file_type = 'rpm'
else
raise "Unsupported platform family: #{platform_family}"
end
base_url = 'https://artifacts.elastic.co/downloads/elasticsearch'
"#{base_url}/elasticsearch-#{version}-#{arch}.#{file_type}"
end
def checksum_platform
platform_family = node['platform_family']
arch = if arm?
platform_family == 'debian' ? 'arm64' : 'aarch64'
else
'x86_64'
end
"#{platform_family == 'debian' ? 'debian' : 'rpm'}_#{arch}"
end
def default_download_checksum(version)
case version
when '6.5.0'
{
'rpm_x86_64' => 'e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855',
'debian_x86_64' => 'e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855',
}
when '6.5.1'
{
'rpm_x86_64' => 'e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855',
'debian_x86_64' => 'e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855',
}
when '6.5.2'
{
'rpm_x86_64' => 'e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855',
'debian_x86_64' => '9cb0997dc6d2be16c988c0ee43ccafd19a8b2e757326af84f4cead40f74c614f',
}
when '6.5.3'
{
'rpm_x86_64' => '2f3eb7682e06211061bea90a0314a515f0c4ef683f45c8e57bfb1dfb14679c3a',
'debian_x86_64' => '38b30461201fe8d126d124f04d961e7c037bea7a6fb9ca485c08e681d8d30456',
}
when '6.5.4'
{
'rpm_x86_64' => 'aa4006f754bd1a0bfaa338ba40d93a1762917c1862951577c62b1f073026b5ba',
'debian_x86_64' => 'c0a062ffb45f989cd3091c66f62605178c41c3735991d95506a6986a90924833',
}
when '6.6.0'
{
'rpm_x86_64' => 'e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855',
'debian_x86_64' => 'e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855',
}
when '6.6.1'
{
'rpm_x86_64' => 'e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855',
'debian_x86_64' => 'e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855',
}
when '6.6.2'
{
'rpm_x86_64' => 'e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855',
'debian_x86_64' => 'e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855',
}
when '6.7.0'
{
'rpm_x86_64' => 'e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855',
'debian_x86_64' => 'e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855',
}
when '6.7.1'
{
'rpm_x86_64' => 'e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855',
'debian_x86_64' => 'e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855',
}
when '6.7.2'
{
'rpm_x86_64' => 'e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855',
'debian_x86_64' => 'e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855',
}
when '6.8.2'
{
'rpm_x86_64' => 'e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855',
'debian_x86_64' => 'e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855',
}
when '6.8.3'
{
'rpm_x86_64' => 'e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855',
'debian_x86_64' => 'e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855',
}
when '6.8.4'
{
'rpm_x86_64' => 'e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855',
'debian_x86_64' => 'e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855',
}
when '6.8.5'
{
'rpm_x86_64' => 'e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855',
'debian_x86_64' => 'e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855',
}
when '6.8.6'
{
'rpm_x86_64' => '4880396d1a78046efe4a6ec45c1cc2f1f9f0d328466aa32355e95f9834d9d0af',
'debian_x86_64' => '82dce29bb3c9108f44e936c3fc6200ce7264bb1a27c1a1cc6dde39b6eac03487',
}
when '7.0.1'
{
'x86_64rpm_x86_64' => 'e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855',
'debian_arm64' => 'e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855',
}
when '7.1.0'
{
'debian_arm64' => 'e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855',
'x86_64rpm_x86_64' => 'e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855',
}
when '7.1.1'
{
'x86_64rpm_x86_64' => 'e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855',
'debian_arm64' => '2ef15cb7e37d32b93c51ad537959831bd72cac2627f255d22cc574cec5de6aef',
}
when '7.2.0'
{
'debian_x86_64' => '1ff7b88c4bc38438a67719df499b17d4f7082a77eda89f39016f83414554ea95',
'rpm_x86_64' => 'a854decb443631a0031a9492c1d5acbed00222381cb63cba68ae6d5deee3994c',
}
when '7.2.1'
{
'debian_x86_64' => '41f507b83fc49a3da5109afd18cc626ec5458acf567f00a80ac3f1c34b6d4b7f',
'rpm_x86_64' => '96fdac0a8e6c74182d920b39e3f4830b722731a646126222c189e12a95302e6e',
}
when '7.3.0'
{
'debian_x86_64' => '72ae24cf0f5d97a307f35d116a41e1165e80f58f08b0ca4e6de3ec5228f06f9c',
'rpm_x86_64' => 'f49dc809cf48369b70546f13dfb28b43e1a07387b681ca786c794762d52847ca',
}
when '7.3.1'
{
'debian_x86_64' => '570af7456603fd103408ed61ccec4473302976d46e1ff845b74a881122977e02',
'rpm_x86_64' => '240f93d16da4c20d2cc377b7c6a61dbf4fb9634d74829ccb5f7cd42c023bc967',
}
when '7.3.2'
{
'debian_x86_64' => '690e98653b3dc50ec5f8e65c480ec41c8c4db0d2c63b5ed3f25fef53d6aaaa55',
'rpm_x86_64' => 'bdada0a4c7b5574c41726154212b6b25373e2b4d7d2a64e24238b206ad422ecd',
}
when '7.4.0'
{
'debian_x86_64' => '3edf17d9d63a08a0f7eb7d9727a1737e1c770277f64fe44342115e62f752cc51',
'rpm_x86_64' => '1bfae41734c77af3bc66084ac0cc04add1190f9311b045d3c184ea7b3e688334',
}
when '7.4.1'
{
'debian_x86_64' => '55a92288e81856e9bb6c36c0f7149b24cf36432527ca809fc48e25775b0cf584',
'rpm_x86_64' => '8ec30fbd95235cb15d0f27cd40f75a43f640f5832e2ee2d44fe8d2983cd5724f',
}
when '7.4.2'
{
'debian_x86_64' => '514a8e21e173481edb9130ebbf33f15209b467df5c2222632d63c4527c16abc6',
'rpm_x86_64' => 'af616eed2cd30411f400dee0c993eb8fccd55e510548697d7cc0eb178ac4adec',
}
when '7.5.0'
{
'debian_x86_64' => '5b167d15461049f6aa58a96d805c9bcd297ad19467392eea125ce91c5eaaf908',
'rpm_x86_64' => 'a8e802c74c3163272fb7119a9d23c1e8f7bbe76e6502a3fcc30709705bc57f4a',
}
when '7.5.1'
{
'debian_x86_64' => 'e566a88e15d8f85cf793c8f971b51eeae6465a0aa73f968ae4b1ee6aa71e4c20',
'rpm_x86_64' => 'e6202bba2bd8644d23dcbef9ad7780c847dfe4ee699d3dc1804f6f62eed59c2d',
}
when '7.16.3'
{
'debian_x86_64' => '03992d97930b734155981076b3cd250c22742f3876f5f135f374940d1cb3ae2e',
'debian_arm64' => 'c383e5b45eb070e1b6d53b9dc56218634794e2e0b27ea42a7d4a12650eec2b70',
'rpm_aarch64' => 'f833e86db87240bcdc822ea40fc6103f019c35bafcfd7ac6063ef01d5b588e1c',
'rpm_x86_64' => '9edf142b9a25b9000a9bf8638bc0590916f367b66e4abb3ce80d8f00f9de0c9c',
}
when '7.17.8'
{
'debian_x86_64' => 'd4875477129214519f6150aaf35374103f075886913307d6ed7c138d04ae6fa1',
'debian_arm64' => '7dd69704b8d6d71aa58bb05f86d63fb34c00f2fcabdff244e9dab37226ca48af',
'rpm_aarch64' => 'bb151d40c7979e5c5c6b9b1a227d494bb463642af938a6b21ae46a4eae767c74',
'rpm_x86_64' => 'd1d1cf15143029c658224d39ebf174f8da802bb26800cd88f974ad2a0ee16484',
}
when '7.17.9'
{
'debian_x86_64' => '7832e13c0b67239370058b729d321af1a12f0b329c0a3828c57d2fd4a9cb6555',
'debian_arm64' => 'ec7064982bd3601280478b5d1ea01b8b8d95cbaaffad441e7bef194c53e8cccd',
'rpm_aarch64' => '16a6e97440b0a4542d9d69168287fe143d40db138e9a3fd3e6348e60abe77175',
'rpm_x86_64' => '751beebbe28ebefcd451796c1075208421b109bdae752383122142fd00a04559',
}
when '8.6.1'
{
'debian_x86_64' => 'a4ea8a7409a9c32752688f03f1df628624fa48a1c38bc5d0eee21883d5b34083',
'debian_arm64' => '84fbd0d36e98aff028eac5027e4bcc2cc8b84bf63dc175fc72e4ea3649c5c8b7',
'rpm_aarch64' => '39e80fe8cc3b864601848e008cf8a0b45b76076408abac093bedc14d0c1328bf',
'rpm_x86_64' => '939daa9480693df658d75bd38c75c2cbf5876e31ff74a543ef8a9d45a81ac728',
}
when '8.6.2'
{
'debian_x86_64' => '8bd0b859e7fa7df8d9e632120c327530f088c5b564cd3b5538eda1b92a181676',
'debian_arm64' => '6e0088c9ac8c2d51f3d60360607f344b6511feaf5d0f3931a4c9d81448757ba9',
'rpm_aarch64' => 'f20a70195e807e1b2981ec37960df8fffef5412f89936b0834d9e8d64d2c8cc1',
'rpm_x86_64' => '5fc28cdfd3aeeeb746778ca873ce47d9836eb6d26746a562b98650c655bb8a3b',
}
else
raise "Unsupported version #{version}"
end
end
end
end

42
cookbooks/elasticsearch/metadata.json
View File

@ -1 +1,41 @@
{"name":"elasticsearch","version":"4.3.0","description":"Installs and configures Elasticsearch","long_description":"","maintainer":"Karel Minarik","maintainer_email":"karel.minarik@elasticsearch.org","license":"Apache-2.0","platforms":{"amazon":">= 0.0.0","centos":">= 0.0.0","debian":">= 0.0.0","fedora":">= 0.0.0","redhat":">= 0.0.0","ubuntu":">= 0.0.0"},"dependencies":{"apt":">= 0.0.0","yum":">= 0.0.0","chef-sugar":">= 0.0.0","ark":">= 0.0.0"},"recommendations":{},"suggestions":{},"conflicting":{},"providing":{},"replacing":{},"attributes":{},"groupings":{},"recipes":{},"source_url":"https://github.com/elastic/cookbook-elasticsearch","issues_url":"https://github.com/elastic/cookbook-elasticsearch/issues","chef_version":[[">= 12.5"]],"ohai_version":[]} {
"name": "elasticsearch",
"description": "Installs and configures Elasticsearch",
"long_description": "",
"maintainer": "Sous Chefs",
"maintainer_email": "help@sous-chefs.org",
"license": "Apache-2.0",
"platforms": {
"amazon": ">= 0.0.0",
"centos": ">= 0.0.0",
"debian": ">= 0.0.0",
"fedora": ">= 0.0.0",
"redhat": ">= 0.0.0",
"ubuntu": ">= 0.0.0"
},
"dependencies": {
"ark": ">= 0.0.0"
},
"providing": {
},
"recipes": {
},
"version": "5.1.3",
"source_url": "https://github.com/sous-chefs/elasticsearch",
"issues_url": "https://github.com/sous-chefs/elasticsearch/issues",
"privacy": false,
"chef_versions": [
[
">= 15.3"
]
],
"ohai_versions": [
],
"gems": [
],
"eager_load_libraries": true
}

32
cookbooks/elasticsearch/metadata.rb
View File

@ -1,24 +1,18 @@
# Encoding: utf-8
name 'elasticsearch' name 'elasticsearch'
maintainer 'Karel Minarik' maintainer 'Sous Chefs'
maintainer_email 'karel.minarik@elasticsearch.org' maintainer_email 'help@sous-chefs.org'
license 'Apache-2.0' license 'Apache-2.0'
description 'Installs and configures Elasticsearch' description 'Installs and configures Elasticsearch'
version '4.3.0' version '5.1.3'
issues_url 'https://github.com/sous-chefs/elasticsearch/issues'
source_url 'https://github.com/sous-chefs/elasticsearch'
chef_version '>= 15.3'
supports 'amazon' supports 'amazon'
supports 'centos' supports 'centos'
supports 'debian' supports 'debian'
supports 'fedora' supports 'fedora'
supports 'redhat' supports 'redhat'
supports 'ubuntu' supports 'ubuntu'
depends 'apt' depends 'ark'
depends 'yum'
depends 'chef-sugar'
depends 'ark'
issues_url 'https://github.com/elastic/cookbook-elasticsearch/issues'
source_url 'https://github.com/elastic/cookbook-elasticsearch'
chef_version '>= 12.5' if respond_to?(:chef_version)

46
cookbooks/elasticsearch/recipes/default.rb
View File

@ -1,46 +0,0 @@
# Encoding: utf-8
#
# Cookbook Name:: elasticsearch
# Recipe:: default
#
include_recipe 'chef-sugar'
# see README.md and test/fixtures/cookbooks for more examples!
elasticsearch_user 'elasticsearch' do
node['elasticsearch']['user'].each do |key, value|
# Skip nils, use false if you want to disable something.
send(key, value) unless value.nil?
end
end
elasticsearch_install 'elasticsearch' do
node['elasticsearch']['install'].each do |key, value|
# Skip nils, use false if you want to disable something.
send(key, value) unless value.nil?
end
end
elasticsearch_configure 'elasticsearch' do
node['elasticsearch']['configure'].each do |key, value|
# Skip nils, use false if you want to disable something.
send(key, value) unless value.nil?
end
end
elasticsearch_service 'elasticsearch' do
node['elasticsearch']['service'].each do |key, value|
# Skip nils, use false if you want to disable something.
send(key, value) unless value.nil?
end
end
# by default, no plugins
node['elasticsearch']['plugin'].each do |plugin_name, plugin_value|
elasticsearch_plugin plugin_name do
plugin_value.each do |key, value|
# Skip nils, use false if you want to disable something.
send(key, value) unless value.nil?
end
end
end

17
cookbooks/elasticsearch/renovate.json Normal file
View File

@ -0,0 +1,17 @@
{
"$schema": "https://docs.renovatebot.com/renovate-schema.json",
"extends": ["config:base"],
"packageRules": [{
"groupName": "Actions",
"matchUpdateTypes": ["patch", "pin", "digest"],
"automerge": true,
"addLabels": ["Release: Patch", "Skip: Announcements"]
},
{
"groupName": "Actions",
"matchUpdateTypes": ["major"],
"automerge": false,
"addLabels": ["Release: Patch", "Skip: Announcements"]
}
]
}

230
cookbooks/elasticsearch/resources/configure.rb Normal file
View File

@ -0,0 +1,230 @@
unified_mode true
# this is what helps the various resources find each other
property :instance_name, String
# If you override one of these, you should probably override them all
property :path_home, String, default: '/usr/share/elasticsearch'
property :path_conf, String, default: '/etc/elasticsearch'
property :path_data, [String, Array], default: '/var/lib/elasticsearch'
property :path_logs, String, default: '/var/log/elasticsearch'
property :path_pid, String, default: '/var/run/elasticsearch'
property :path_plugins, String, default: '/usr/share/elasticsearch/plugins'
property :path_bin, String, default: '/usr/share/elasticsearch/bin'
property :template_elasticsearch_env, String, default: 'elasticsearch.in.sh.erb'
property :cookbook_elasticsearch_env, String, default: 'elasticsearch'
property :template_jvm_options, String, default: 'jvm_options.erb'
property :cookbook_jvm_options, String, default: 'elasticsearch'
property :template_elasticsearch_yml, String, default: 'elasticsearch.yml.erb'
property :cookbook_elasticsearch_yml, String, default: 'elasticsearch'
property :template_log4j2_properties, String, default: 'log4j2.properties.erb'
property :cookbook_log4j2_properties, String, default: 'elasticsearch'
property :logging, Hash, default: {}.freeze
property :java_home, String
# other settings in /etc/default or /etc/sysconfig
property :memlock_limit, String, default: 'unlimited'
property :max_map_count, String, default: '262144'
property :nofile_limit, String, default: '65535'
property :startup_sleep_seconds, [String, Integer], default: 5
property :restart_on_upgrade, [true, false], default: false
# Calculations for this are done in the provider, as we can't do them in the
# resource definition. default is 50% of RAM or 31GB, which ever is smaller.
property :allocated_memory, String
property :jvm_options, Array, default:
%w(
8-13:-XX:+UseConcMarkSweepGC
8-13:-XX:CMSInitiatingOccupancyFraction=75
8-13:-XX:+UseCMSInitiatingOccupancyOnly
14-:-XX:+UseG1GC
-Djava.io.tmpdir=${ES_TMPDIR}
-XX:+HeapDumpOnOutOfMemoryError
9-:-XX:+ExitOnOutOfMemoryError
-XX:ErrorFile=/var/log/elasticsearch/hs_err_pid%p.log
8:-XX:+PrintGCDetails
8:-XX:+PrintGCDateStamps
8:-XX:+PrintTenuringDistribution
8:-XX:+PrintGCApplicationStoppedTime
8:-Xloggc:/var/log/elasticsearch/gc.log
8:-XX:+UseGCLogFileRotation
8:-XX:NumberOfGCLogFiles=32
8:-XX:GCLogFileSize=64m
9-:-Xlog:gc*,gc+age=trace,safepoint:file=/var/log/elasticsearch/gc.log:utctime,pid,tags:filecount=32,filesize=64m
).freeze
# These are the default settings. Most of the time, you want to override
# the `configuration` attribute below. If you do override the defaults, you
# must supply ALL needed defaults, and don't use nil as a value in the hash.
property :default_configuration, Hash, default: {
'cluster.name' => 'elasticsearch',
'node.name' => Chef::Config[:node_name],
}
# These settings are merged with the `default_configuration` attribute,
# allowing you to override and set specific settings. Unless you intend to
# wipe out all default settings, your configuration items should go here.
#
property :configuration, Hash, default: {}
include ElasticsearchCookbook::Helpers
action :manage do
# lookup existing ES resources
es_user = find_es_resource(Chef.run_context, :elasticsearch_user, new_resource)
es_svc = find_es_resource(Chef.run_context, :elasticsearch_service, new_resource)
es_install = find_es_resource(Chef.run_context, :elasticsearch_install, new_resource)
default_configuration = new_resource.default_configuration.dup
# if a subdir parameter is missing but dir is set, infer the subdir name
# then go and be sure it's also set in the YML hash if it wasn't given there
if new_resource.path_data && default_configuration['path.data'].nil?
default_configuration['path.data'] = new_resource.path_data
end
if new_resource.path_logs && default_configuration['path.logs'].nil?
default_configuration['path.logs'] = new_resource.path_logs
end
# Calculation for memory allocation; 50% or 31g, whatever is smaller
#
unless new_resource.allocated_memory
half = ((node['memory']['total'].to_i * 0.5).floor / 1024)
malloc_str = (half > 30_500 ? '30500m' : "#{half}m")
new_resource.allocated_memory malloc_str
end
# Create ES directories
#
[new_resource.path_conf, "#{new_resource.path_conf}/scripts"].each do |path|
directory path do
owner es_user.username
group es_user.groupname
mode '0750'
recursive true
action :create
end
end
directory new_resource.path_logs do
owner es_user.username
group es_user.groupname
mode '0750'
recursive true
action :create
end
if new_resource.path_data.is_a?(String)
directory new_resource.path_data do
owner es_user.username
group es_user.groupname
mode '0750'
recursive true
action :create
end
else
new_resource.path_data.each do |path|
directory path.strip do
owner es_user.username
group es_user.groupname
mode '0750'
recursive true
action :create
end
end
end
# Create elasticsearch shell variables file
#
# Valid values in /etc/sysconfig/elasticsearch or /etc/default/elasticsearch
# ES_HOME JAVA_HOME ES_PATH_CONF DATA_DIR LOG_DIR PID_DIR ES_JAVA_OPTS
# RESTART_ON_UPGRADE ES_USER ES_GROUP ES_STARTUP_SLEEP_TIME MAX_OPEN_FILES
# MAX_LOCKED_MEMORY MAX_MAP_COUNT
#
# We provide these values as resource attributes/parameters directly
params = {}
params[:ES_HOME] = new_resource.path_home
params[:JAVA_HOME] = new_resource.java_home
params[:ES_PATH_CONF] = new_resource.path_conf
params[:DATA_DIR] = new_resource.path_data
params[:LOG_DIR] = new_resource.path_logs
params[:PID_DIR] = new_resource.path_pid
params[:RESTART_ON_UPGRADE] = new_resource.restart_on_upgrade
params[:ES_USER] = es_user.username if es_install.type == 'tarball'
params[:ES_GROUP] = es_user.groupname if es_install.type == 'tarball'
params[:ES_STARTUP_SLEEP_TIME] = new_resource.startup_sleep_seconds.to_s
params[:MAX_OPEN_FILES] = new_resource.nofile_limit
params[:MAX_LOCKED_MEMORY] = new_resource.memlock_limit
params[:MAX_MAP_COUNT] = new_resource.max_map_count
default_config_name = es_svc.service_name || es_svc.instance_name || new_resource.instance_name || 'elasticsearch'
with_run_context :root do
template "elasticsearch.in.sh-#{default_config_name}" do
path platform_family?('rhel', 'amazon') ? "/etc/sysconfig/#{default_config_name}" : "/etc/default/#{default_config_name}"
source new_resource.template_elasticsearch_env
cookbook new_resource.cookbook_elasticsearch_env
mode '0644'
variables(params: params)
action :create
end
end
template "jvm_options-#{default_config_name}" do
path "#{new_resource.path_conf}/jvm.options"
source new_resource.template_jvm_options
cookbook new_resource.cookbook_jvm_options
owner es_user.username
group es_user.groupname
mode '0644'
variables(jvm_options: [
"-Xms#{new_resource.allocated_memory}",
"-Xmx#{new_resource.allocated_memory}",
new_resource.jvm_options,
].flatten.join("\n"))
action :create
end
template "log4j2_properties-#{default_config_name}" do
path "#{new_resource.path_conf}/log4j2.properties"
source new_resource.template_log4j2_properties
cookbook new_resource.cookbook_log4j2_properties
owner es_user.username
group es_user.groupname
mode '0640'
variables(logging: new_resource.logging)
action :create
end
# Create ES elasticsearch.yml file
#
merged_configuration = default_configuration.merge(new_resource.configuration.dup)
# Warn if someone is using symbols. We don't support.
found_symbols = merged_configuration.keys.select { |s| s.is_a?(Symbol) }
unless found_symbols.empty?
Chef::Log.warn("Please change the following to strings in order to work with this Elasticsearch cookbook: #{found_symbols.join(',')}")
end
# workaround for https://github.com/sous-chefs/elasticsearch/issues/590
config_vars = ElasticsearchCookbook::HashAndMashBlender.new(merged_configuration).to_hash
with_run_context :root do
template "elasticsearch.yml-#{default_config_name}" do
path "#{new_resource.path_conf}/elasticsearch.yml"
source new_resource.template_elasticsearch_yml
cookbook new_resource.cookbook_elasticsearch_yml
owner es_user.username
group es_user.groupname
mode '0640'
helpers(ElasticsearchCookbook::Helpers)
variables(config: config_vars)
action :create
end
end
end

47
cookbooks/elasticsearch/resources/install.rb Normal file
View File

@ -0,0 +1,47 @@
unified_mode true
use 'partial/_common'
use 'partial/_package'
use 'partial/_repository'
property :type,
String,
equal_to: %w(package tarball repository),
default: 'repository'
action :install do
case new_resource.type
when 'tarball'
raise 'Tarball method is not currently supported, due to no supporting systemd service'
when 'package'
elasticsearch_install_package "ElasticSearch #{new_resource.version}" do
version new_resource.version
instance_name new_resource.instance_name
download_url download_url
download_checksum download_checksum
end
when 'repository'
elasticsearch_install_repository "ElasticSearch #{new_resource.version}" do
version new_resource.version
instance_name new_resource.instance_name
enable_repository_actions new_resource.enable_repository_actions
package_options new_resource.package_options
end
else
raise "#{new_resource.type} is not a valid install type"
end
end
action :remove do
case new_resource.type
when 'package'
elasticsearch_install_package "ElasticSearch #{new_resource.version}" do
action :remove
end
when 'repository'
elasticsearch_install_repository "ElasticSearch #{new_resource.version}" do
action :remove
end
else
raise "#{install_type} is not a valid install type"
end
end

41
cookbooks/elasticsearch/resources/install_package.rb Normal file
View File

@ -0,0 +1,41 @@
include ElasticsearchCookbook::Helpers
unified_mode true
use 'partial/_common'
use 'partial/_package'
action :install do
remote_file "#{Chef::Config[:file_cache_path]}/#{filename_from_url}" do
source new_resource.download_url
checksum new_resource.download_checksum
mode '0644'
action :create
end
if platform_family?('debian')
dpkg_package filename_from_url do
options new_resource.package_options
source "#{Chef::Config[:file_cache_path]}/#{filename_from_url}"
action :install
end
else
package filename_from_url do
options new_resource.package_options
source "#{Chef::Config[:file_cache_path]}/#{filename_from_url}"
action :install
end
end
end
action :remove do
package "#{Chef::Config[:file_cache_path]}/#{filename_from_url}" do
action :temove
end
end
action_class do
include ElasticsearchCookbook::Helpers
def filename_from_url
new_resource.download_url.split('/').last
end
end

63
cookbooks/elasticsearch/resources/install_repository.rb Normal file
View File

@ -0,0 +1,63 @@
unified_mode true
use 'partial/_common'
use 'partial/_repository'
include ElasticsearchCookbook::Helpers
action :install do
major_version = new_resource.version.split('.')[0]
es_user = find_es_resource(Chef.run_context, :elasticsearch_user, new_resource)
unless es_user && es_user.username == 'elasticsearch' && es_user.groupname == 'elasticsearch'
raise 'Custom usernames/group names is not supported in Elasticsearch 6+ repository installation'
end
if new_resource.enable_repository_actions
if platform_family?('debian')
apt_repository "elastic-#{major_version}.x" do
uri 'https://artifacts.elastic.co/packages/7.x/apt'
key 'elasticsearch.asc'
cookbook 'elasticsearch'
components ['main']
distribution 'stable'
end
else
yum_repository "elastic-#{major_version}.x" do
baseurl "https://artifacts.elastic.co/packages/#{major_version}.x/yum"
gpgkey 'https://artifacts.elastic.co/GPG-KEY-elasticsearch'
action :create
end
end
end
package 'elasticsearch' do
options new_resource.package_options
version new_resource.version
action :install
end
end
action :remove do
if new_resource.enable_repository_actions
if platform_family?('debian')
apt_repository "elastic-#{new_resource.version}.x" do
action :remove
end
else
yum_repository "elastic-#{new_resource.version}.x" do
action :remove
end
end
end
package 'elasticsearch' do
options new_resource.package_options
version new_resource.version
action :remove
end
end
action_class do
include ElasticsearchCookbook::Helpers
end

12
cookbooks/elasticsearch/resources/partial/_common.rb Normal file
View File

@ -0,0 +1,12 @@
include ElasticsearchCookbook::Helpers
include ElasticsearchCookbook::VersionHelpers
property :instance_name,
String
property :version,
String,
default: '7.17.9'
property :package_options,
String

7
cookbooks/elasticsearch/resources/partial/_package.rb Normal file
View File

@ -0,0 +1,7 @@
property :download_url,
String,
default: lazy { default_download_url(new_resource.version) }
property :download_checksum,
String,
default: lazy { default_download_checksum(new_resource.version)[checksum_platform] }

3
cookbooks/elasticsearch/resources/partial/_repository.rb Normal file
View File

@ -0,0 +1,3 @@
property :enable_repository_actions,
[true, false],
default: true

84
cookbooks/elasticsearch/resources/plugin.rb Normal file
View File

@ -0,0 +1,84 @@
unified_mode true
include ElasticsearchCookbook::Helpers
property :plugin_name,
String,
name_property: true
property :url,
String
property :options,
String,
default: ''
# this is what helps the various resources find each other
property :instance_name,
String
action :install do
execute "Install plugin #{new_resource.plugin_name}" do
command "#{es_conf.path_bin}/elasticsearch-plugin install #{new_resource.options} #{config[:plugin_name]}".chomp(' ')
not_if { plugin_exists? }
environment env
user config[:user] unless config[:install_type] == 'package' || config[:install_type] == 'repository'
group config[:group] unless config[:install_type] == 'package' || config[:install_type] == 'repository'
end
end
action :remove do
execute "Remove plugin #{new_resource.plugin_name}" do
command "#{es_conf.path_bin}/elasticsearch-plugin remove #{new_resource.options} #{config[:plugin_name]}".chomp(' ')
only_if { plugin_exists? }
environment env
user config[:user] unless config[:install_type] == 'package' || config[:install_type] == 'repository'
group config[:group] unless config[:install_type] == 'package' || config[:install_type] == 'repository'
end
end
action_class do
def es_user
find_es_resource(Chef.run_context, :elasticsearch_user, new_resource)
end
def es_install
find_es_resource(Chef.run_context, :elasticsearch_install, new_resource)
end
def es_conf
find_es_resource(Chef.run_context, :elasticsearch_configure, new_resource)
end
def env
include_file_resource = find_exact_resource(Chef.run_context, :template, "elasticsearch.in.sh-#{config[:name]}")
{ 'ES_INCLUDE' => include_file_resource.path }
end
def config
{
name: new_resource.instance_name || es_conf.instance_name || 'elasticsearch',
plugin_name: new_resource.url || new_resource.plugin_name,
install_type: es_install.type,
user: es_user.username,
group: es_user.groupname,
path_conf: es_conf.path_conf,
path_plugins: es_conf.path_plugins,
path_bin: es_conf.path_bin,
}
end
def plugin_exists?
# This is quicker than shelling out to the plugin list command
# The plugin install command checks for the existsance of the plugin directory anyway
es_conf = find_es_resource(Chef.run_context, :elasticsearch_configure, new_resource)
path = es_conf.path_plugins
Dir.entries(path).any? do |plugin|
next if plugin =~ /^\./
config[:plugin_name] == plugin
end
rescue
false
end
end

126
cookbooks/elasticsearch/resources/service.rb Normal file
View File

@ -0,0 +1,126 @@
unified_mode true
include ElasticsearchCookbook::Helpers
# this is what helps the various resources find each other
property :instance_name,
String
property :service_name,
String,
name_property: true
property :service_actions,
[Symbol, String, Array],
default: [:enable, :start]
action :configure do
es_user = find_es_resource(Chef.run_context, :elasticsearch_user, new_resource)
es_conf = find_es_resource(Chef.run_context, :elasticsearch_configure, new_resource)
default_config_name = new_resource.service_name || new_resource.instance_name || es_conf.instance_name || 'elasticsearch'
directory "#{es_conf.path_pid}-#{default_config_name}" do
path es_conf.path_pid
owner es_user.username
group es_user.groupname
mode '0755'
recursive true
action :create
end
default_conf_dir = platform_family?('rhel', 'amazon') ? '/etc/sysconfig' : '/etc/default'
systemd_unit new_resource.service_name do
content(
Unit: {
Description: 'Elasticsearch',
Documentation: 'https://www.elastic.co',
Wants: 'network-online.target',
After: 'network-online.target',
},
Service: {
Type: 'notify',
RuntimeDirectory: 'elasticsearch',
PrivateTmp: 'true',
Environment: [
"ES_HOME=#{es_conf.path_home}",
'ES_PATH_CONF=/etc/elasticsearch',
"PID_DIR=#{es_conf.path_pid}",
'ES_SD_NOTIFY=true',
],
EnvironmentFile: "-#{default_conf_dir}/#{new_resource.service_name}",
WorkingDirectory: "#{es_conf.path_home}",
User: es_user.username,
Group: es_user.groupname,
ExecStart: "#{es_conf.path_home}/bin/systemd-entrypoint -p ${PID_DIR}/elasticsearch.pid --quiet",
StandardOutput: 'journal',
StandardError: 'inherit',
LimitNOFILE: '65535',
LimitNPROC: '4096',
LimitAS: 'infinity',
LimitFSIZE: 'infinity',
TimeoutStopSec: '0',
KillSignal: 'SIGTERM',
KillMode: 'process',
SendSIGKILL: 'no',
SuccessExitStatus: '143',
TimeoutStartSec: '900',
},
Install: {
WantedBy: 'multi-user.target',
}
)
verify false
action :create
unit_name "#{new_resource.service_name}.service"
end
# flatten in an array here, in case the service_actions are a symbol vs. array
[new_resource.service_actions].flatten.each do |act|
passthrough_action(act)
end
end
# Passthrough actions to service[service_name]
#
action :enable do
passthrough_action(:enable)
end
action :disable do
passthrough_action(:disable)
end
action :start do
passthrough_action(:start)
end
action :stop do
passthrough_action(:stop)
end
action :restart do
passthrough_action(:restart)
end
action :status do
passthrough_action(:status)
end
action_class do
def passthrough_action(action)
svc_r = lookup_service_resource
svc_r.run_action(action)
new_resource.updated_by_last_action(true) if svc_r.updated_by_last_action?
end
def lookup_service_resource
rc = Chef.run_context.resource_collection
rc.find("service[#{new_resource.service_name}]")
rescue
service new_resource.service_name do
supports status: true, restart: true
action :nothing
end
end
end

73
cookbooks/elasticsearch/resources/user.rb Normal file
View File

@ -0,0 +1,73 @@
include ElasticsearchCookbook::Helpers
unified_mode true
property :instance_name,
String
property :username,
String,
name_property: true
property :groupname,
String,
default: lazy { username }
property :shell,
String,
default: '/bin/bash'
property :uid,
Integer
property :comment,
String,
default: 'Elasticsearch User'
property :gid,
Integer
action :create do
group_r = group new_resource.groupname do
gid new_resource.gid
action :nothing
system true
end
group_r.run_action(:create)
new_resource.updated_by_last_action(true) if group_r.updated_by_last_action?
user_r = user new_resource.username do
comment new_resource.comment
shell new_resource.shell
uid new_resource.uid
gid new_resource.groupname
manage_home false
action :nothing
system true
end
user_r.run_action(:create)
new_resource.updated_by_last_action(true) if user_r.updated_by_last_action?
end
action :remove do
user_r = user new_resource.username do
action :nothing
end
user_r.run_action(:remove)
new_resource.updated_by_last_action(true) if user_r.updated_by_last_action?
group_r = group new_resource.groupname do
action :nothing
end
group_r.run_action(:remove)
new_resource.updated_by_last_action(true) if group_r.updated_by_last_action?
end

18
cookbooks/elasticsearch/templates/amazon/initscript.erb
View File

@ -33,7 +33,7 @@ fi
# Sets the default values for elasticsearch variables used in this script # Sets the default values for elasticsearch variables used in this script
ES_HOME="/usr/share/elasticsearch" ES_HOME="/usr/share/elasticsearch"
MAX_OPEN_FILES=65536 MAX_OPEN_FILES=65535
MAX_MAP_COUNT=262144 MAX_MAP_COUNT=262144
ES_PATH_CONF="/etc/elasticsearch" ES_PATH_CONF="/etc/elasticsearch"
@ -69,21 +69,7 @@ if [ ! -x "$exec" ]; then
exit 1 exit 1
fi fi
checkJava() {
if [ -x "$JAVA_HOME/bin/java" ]; then
JAVA="$JAVA_HOME/bin/java"
else
JAVA=`which java`
fi
if [ ! -x "$JAVA" ]; then
echo "Could not find any executable java binary. Please install java in your PATH or set JAVA_HOME"
exit 1
fi
}
start() { start() {
checkJava
[ -x $exec ] || exit 5 [ -x $exec ] || exit 5
if [ -n "$MAX_OPEN_FILES" ]; then if [ -n "$MAX_OPEN_FILES" ]; then
@ -92,7 +78,7 @@ start() {
if [ -n "$MAX_LOCKED_MEMORY" ]; then if [ -n "$MAX_LOCKED_MEMORY" ]; then
ulimit -l $MAX_LOCKED_MEMORY ulimit -l $MAX_LOCKED_MEMORY
fi fi
if [ -n "$MAX_MAP_COUNT" -a -f /proc/sys/vm/max_map_count ]; then if [ -n "$MAX_MAP_COUNT" -a -f /proc/sys/vm/max_map_count ] && [ "$MAX_MAP_COUNT" -gt $(cat /proc/sys/vm/max_map_count) ]; then
sysctl -q -w vm.max_map_count=$MAX_MAP_COUNT sysctl -q -w vm.max_map_count=$MAX_MAP_COUNT
fi fi

18
cookbooks/elasticsearch/templates/centos/initscript.erb
View File

@ -33,7 +33,7 @@ fi
# Sets the default values for elasticsearch variables used in this script # Sets the default values for elasticsearch variables used in this script
ES_HOME="/usr/share/elasticsearch" ES_HOME="/usr/share/elasticsearch"
MAX_OPEN_FILES=65536 MAX_OPEN_FILES=65535
MAX_MAP_COUNT=262144 MAX_MAP_COUNT=262144
ES_PATH_CONF="/etc/elasticsearch" ES_PATH_CONF="/etc/elasticsearch"
@ -69,21 +69,7 @@ if [ ! -x "$exec" ]; then
exit 1 exit 1
fi fi
checkJava() {
if [ -x "$JAVA_HOME/bin/java" ]; then
JAVA="$JAVA_HOME/bin/java"
else
JAVA=`which java`
fi
if [ ! -x "$JAVA" ]; then
echo "Could not find any executable java binary. Please install java in your PATH or set JAVA_HOME"
exit 1
fi
}
start() { start() {
checkJava
[ -x $exec ] || exit 5 [ -x $exec ] || exit 5
if [ -n "$MAX_OPEN_FILES" ]; then if [ -n "$MAX_OPEN_FILES" ]; then
@ -92,7 +78,7 @@ start() {
if [ -n "$MAX_LOCKED_MEMORY" ]; then if [ -n "$MAX_LOCKED_MEMORY" ]; then
ulimit -l $MAX_LOCKED_MEMORY ulimit -l $MAX_LOCKED_MEMORY
fi fi
if [ -n "$MAX_MAP_COUNT" -a -f /proc/sys/vm/max_map_count ]; then if [ -n "$MAX_MAP_COUNT" -a -f /proc/sys/vm/max_map_count ] && [ "$MAX_MAP_COUNT" -gt $(cat /proc/sys/vm/max_map_count) ]; then
sysctl -q -w vm.max_map_count=$MAX_MAP_COUNT sysctl -q -w vm.max_map_count=$MAX_MAP_COUNT
fi fi

18
cookbooks/elasticsearch/templates/debian/initscript.erb
View File

@ -39,7 +39,7 @@ ES_HOME=/usr/share/$NAME
#ES_JAVA_OPTS= #ES_JAVA_OPTS=
# Maximum number of open files # Maximum number of open files
MAX_OPEN_FILES=65536 MAX_OPEN_FILES=65535
# Maximum amount of locked memory # Maximum amount of locked memory
#MAX_LOCKED_MEMORY= #MAX_LOCKED_MEMORY=
@ -82,22 +82,8 @@ if [ ! -x "$DAEMON" ]; then
exit 1 exit 1
fi fi
checkJava() {
if [ -x "$JAVA_HOME/bin/java" ]; then
JAVA="$JAVA_HOME/bin/java"
else
JAVA=`which java`
fi
if [ ! -x "$JAVA" ]; then
echo "Could not find any executable java binary. Please install java in your PATH or set JAVA_HOME"
exit 1
fi
}
case "$1" in case "$1" in
start) start)
checkJava
log_daemon_msg "Starting $DESC" log_daemon_msg "Starting $DESC"
@ -124,7 +110,7 @@ case "$1" in
ulimit -l $MAX_LOCKED_MEMORY ulimit -l $MAX_LOCKED_MEMORY
fi fi
if [ -n "$MAX_MAP_COUNT" -a -f /proc/sys/vm/max_map_count ]; then if [ -n "$MAX_MAP_COUNT" -a -f /proc/sys/vm/max_map_count ] && [ "$MAX_MAP_COUNT" -gt $(cat /proc/sys/vm/max_map_count) ]; then
sysctl -q -w vm.max_map_count=$MAX_MAP_COUNT sysctl -q -w vm.max_map_count=$MAX_MAP_COUNT
fi fi

145
cookbooks/elasticsearch/templates/default/log4j2.properties.erb
View File

@ -7,14 +7,16 @@ logger.action.level = debug
appender.console.type = Console appender.console.type = Console
appender.console.name = console appender.console.name = console
appender.console.layout.type = PatternLayout appender.console.layout.type = PatternLayout
appender.console.layout.pattern = [%d{ISO8601}][%-5p][%-25c{1.}] %marker%m%n appender.console.layout.pattern = [%d{ISO8601}][%-5p][%-25c{1.}] [%node_name]%marker %m%n
######## Server JSON ############################
appender.rolling.type = RollingFile appender.rolling.type = RollingFile
appender.rolling.name = rolling appender.rolling.name = rolling
appender.rolling.fileName = ${sys:es.logs.base_path}${sys:file.separator}${sys:es.logs.cluster_name}.log appender.rolling.fileName = ${sys:es.logs.base_path}${sys:file.separator}${sys:es.logs.cluster_name}_server.json
appender.rolling.layout.type = PatternLayout appender.rolling.layout.type = ESJsonLayout
appender.rolling.layout.pattern = [%d{ISO8601}][%-5p][%-25c{1.}] %marker%.-10000m%n appender.rolling.layout.type_name = server
appender.rolling.filePattern = ${sys:es.logs.base_path}${sys:file.separator}${sys:es.logs.cluster_name}-%d{yyyy-MM-dd}-%i.log.gz
appender.rolling.filePattern = ${sys:es.logs.base_path}${sys:file.separator}${sys:es.logs.cluster_name}-%d{yyyy-MM-dd}-%i.json.gz
appender.rolling.policies.type = Policies appender.rolling.policies.type = Policies
appender.rolling.policies.time.type = TimeBasedTriggeringPolicy appender.rolling.policies.time.type = TimeBasedTriggeringPolicy
appender.rolling.policies.time.interval = 1 appender.rolling.policies.time.interval = 1
@ -29,60 +31,147 @@ appender.rolling.strategy.action.condition.type = IfFileName
appender.rolling.strategy.action.condition.glob = ${sys:es.logs.cluster_name}-* appender.rolling.strategy.action.condition.glob = ${sys:es.logs.cluster_name}-*
appender.rolling.strategy.action.condition.nested_condition.type = IfAccumulatedFileSize appender.rolling.strategy.action.condition.nested_condition.type = IfAccumulatedFileSize
appender.rolling.strategy.action.condition.nested_condition.exceeds = 2GB appender.rolling.strategy.action.condition.nested_condition.exceeds = 2GB
################################################
######## Server - old style pattern ###########
appender.rolling_old.type = RollingFile
appender.rolling_old.name = rolling_old
appender.rolling_old.fileName = ${sys:es.logs.base_path}${sys:file.separator}${sys:es.logs.cluster_name}.log
appender.rolling_old.layout.type = PatternLayout
appender.rolling_old.layout.pattern = [%d{ISO8601}][%-5p][%-25c{1.}] [%node_name]%marker %m%n
appender.rolling_old.filePattern = ${sys:es.logs.base_path}${sys:file.separator}${sys:es.logs.cluster_name}-%d{yyyy-MM-dd}-%i.log.gz
appender.rolling_old.policies.type = Policies
appender.rolling_old.policies.time.type = TimeBasedTriggeringPolicy
appender.rolling_old.policies.time.interval = 1
appender.rolling_old.policies.time.modulate = true
appender.rolling_old.policies.size.type = SizeBasedTriggeringPolicy
appender.rolling_old.policies.size.size = 128MB
appender.rolling_old.strategy.type = DefaultRolloverStrategy
appender.rolling_old.strategy.fileIndex = nomax
appender.rolling_old.strategy.action.type = Delete
appender.rolling_old.strategy.action.basepath = ${sys:es.logs.base_path}
appender.rolling_old.strategy.action.condition.type = IfFileName
appender.rolling_old.strategy.action.condition.glob = ${sys:es.logs.cluster_name}-*
appender.rolling_old.strategy.action.condition.nested_condition.type = IfAccumulatedFileSize
appender.rolling_old.strategy.action.condition.nested_condition.exceeds = 2GB
################################################
rootLogger.level = info rootLogger.level = info
rootLogger.appenderRef.console.ref = console rootLogger.appenderRef.console.ref = console
rootLogger.appenderRef.rolling.ref = rolling rootLogger.appenderRef.rolling.ref = rolling
rootLogger.appenderRef.rolling_old.ref = rolling_old
######## Deprecation JSON #######################
appender.deprecation_rolling.type = RollingFile appender.deprecation_rolling.type = RollingFile
appender.deprecation_rolling.name = deprecation_rolling appender.deprecation_rolling.name = deprecation_rolling
appender.deprecation_rolling.fileName = ${sys:es.logs.base_path}${sys:file.separator}${sys:es.logs.cluster_name}_deprecation.log appender.deprecation_rolling.fileName = ${sys:es.logs.base_path}${sys:file.separator}${sys:es.logs.cluster_name}_deprecation.json
appender.deprecation_rolling.layout.type = PatternLayout appender.deprecation_rolling.layout.type = ESJsonLayout
appender.deprecation_rolling.layout.pattern = [%d{ISO8601}][%-5p][%-25c{1.}] %marker%.-10000m%n appender.deprecation_rolling.layout.type_name = deprecation
appender.deprecation_rolling.filePattern = ${sys:es.logs.base_path}${sys:file.separator}${sys:es.logs.cluster_name}_deprecation-%i.log.gz appender.deprecation_rolling.layout.esmessagefields=x-opaque-id
appender.deprecation_rolling.filePattern = ${sys:es.logs.base_path}${sys:file.separator}${sys:es.logs.cluster_name}_deprecation-%i.json.gz
appender.deprecation_rolling.policies.type = Policies appender.deprecation_rolling.policies.type = Policies
appender.deprecation_rolling.policies.size.type = SizeBasedTriggeringPolicy appender.deprecation_rolling.policies.size.type = SizeBasedTriggeringPolicy
appender.deprecation_rolling.policies.size.size = 1GB appender.deprecation_rolling.policies.size.size = 1GB
appender.deprecation_rolling.strategy.type = DefaultRolloverStrategy appender.deprecation_rolling.strategy.type = DefaultRolloverStrategy
appender.deprecation_rolling.strategy.max = 4 appender.deprecation_rolling.strategy.max = 4
#################################################
######## Deprecation - old style pattern #######
appender.deprecation_rolling_old.type = RollingFile
appender.deprecation_rolling_old.name = deprecation_rolling_old
appender.deprecation_rolling_old.fileName = ${sys:es.logs.base_path}${sys:file.separator}${sys:es.logs.cluster_name}_deprecation.log
appender.deprecation_rolling_old.layout.type = PatternLayout
appender.deprecation_rolling_old.layout.pattern = [%d{ISO8601}][%-5p][%-25c{1.}] [%node_name]%marker %m%n
appender.deprecation_rolling_old.filePattern = ${sys:es.logs.base_path}${sys:file.separator}${sys:es.logs.cluster_name}\
_deprecation-%i.log.gz
appender.deprecation_rolling_old.policies.type = Policies
appender.deprecation_rolling_old.policies.size.type = SizeBasedTriggeringPolicy
appender.deprecation_rolling_old.policies.size.size = 1GB
appender.deprecation_rolling_old.strategy.type = DefaultRolloverStrategy
appender.deprecation_rolling_old.strategy.max = 4
#################################################
logger.deprecation.name = org.elasticsearch.deprecation logger.deprecation.name = org.elasticsearch.deprecation
logger.deprecation.level = warn logger.deprecation.level = warn
logger.deprecation.appenderRef.deprecation_rolling.ref = deprecation_rolling logger.deprecation.appenderRef.deprecation_rolling.ref = deprecation_rolling
logger.deprecation.appenderRef.deprecation_rolling_old.ref = deprecation_rolling_old
logger.deprecation.additivity = false logger.deprecation.additivity = false
######## Search slowlog JSON ####################
appender.index_search_slowlog_rolling.type = RollingFile appender.index_search_slowlog_rolling.type = RollingFile
appender.index_search_slowlog_rolling.name = index_search_slowlog_rolling appender.index_search_slowlog_rolling.name = index_search_slowlog_rolling
appender.index_search_slowlog_rolling.fileName = ${sys:es.logs.base_path}${sys:file.separator}${sys:es.logs.cluster_name}_index_search_slowlog.log appender.index_search_slowlog_rolling.fileName = ${sys:es.logs.base_path}${sys:file.separator}${sys:es.logs\
appender.index_search_slowlog_rolling.layout.type = PatternLayout .cluster_name}_index_search_slowlog.json
appender.index_search_slowlog_rolling.layout.pattern = [%d{ISO8601}][%-5p][%-25c] %marker%.-10000m%n appender.index_search_slowlog_rolling.layout.type = ESJsonLayout
appender.index_search_slowlog_rolling.filePattern = ${sys:es.logs.base_path}${sys:file.separator}${sys:es.logs.cluster_name}_index_search_slowlog-%d{yyyy-MM-dd}.log appender.index_search_slowlog_rolling.layout.type_name = index_search_slowlog
appender.index_search_slowlog_rolling.policies.type = Policies appender.index_search_slowlog_rolling.layout.esmessagefields=message,took,took_millis,total_hits,types,stats,search_type,total_shards,source,id
appender.index_search_slowlog_rolling.policies.time.type = TimeBasedTriggeringPolicy
appender.index_search_slowlog_rolling.policies.time.interval = 1
appender.index_search_slowlog_rolling.policies.time.modulate = true
appender.index_search_slowlog_rolling.filePattern = ${sys:es.logs.base_path}${sys:file.separator}${sys:es.logs\
.cluster_name}_index_search_slowlog-%i.json.gz
appender.index_search_slowlog_rolling.policies.type = Policies
appender.index_search_slowlog_rolling.policies.size.type = SizeBasedTriggeringPolicy
appender.index_search_slowlog_rolling.policies.size.size = 1GB
appender.index_search_slowlog_rolling.strategy.type = DefaultRolloverStrategy
appender.index_search_slowlog_rolling.strategy.max = 4
#################################################
######## Search slowlog - old style pattern ####
appender.index_search_slowlog_rolling_old.type = RollingFile
appender.index_search_slowlog_rolling_old.name = index_search_slowlog_rolling_old
appender.index_search_slowlog_rolling_old.fileName = ${sys:es.logs.base_path}${sys:file.separator}${sys:es.logs.cluster_name}\
_index_search_slowlog.log
appender.index_search_slowlog_rolling_old.layout.type = PatternLayout
appender.index_search_slowlog_rolling_old.layout.pattern = [%d{ISO8601}][%-5p][%-25c{1.}] [%node_name]%marker %m%n
appender.index_search_slowlog_rolling_old.filePattern = ${sys:es.logs.base_path}${sys:file.separator}${sys:es.logs.cluster_name}\
_index_search_slowlog-%i.log.gz
appender.index_search_slowlog_rolling_old.policies.type = Policies
appender.index_search_slowlog_rolling_old.policies.size.type = SizeBasedTriggeringPolicy
appender.index_search_slowlog_rolling_old.policies.size.size = 1GB
appender.index_search_slowlog_rolling_old.strategy.type = DefaultRolloverStrategy
appender.index_search_slowlog_rolling_old.strategy.max = 4
#################################################
logger.index_search_slowlog_rolling.name = index.search.slowlog logger.index_search_slowlog_rolling.name = index.search.slowlog
logger.index_search_slowlog_rolling.level = trace logger.index_search_slowlog_rolling.level = trace
logger.index_search_slowlog_rolling.appenderRef.index_search_slowlog_rolling.ref = index_search_slowlog_rolling logger.index_search_slowlog_rolling.appenderRef.index_search_slowlog_rolling.ref = index_search_slowlog_rolling
logger.index_search_slowlog_rolling.appenderRef.index_search_slowlog_rolling_old.ref = index_search_slowlog_rolling_old
logger.index_search_slowlog_rolling.additivity = false logger.index_search_slowlog_rolling.additivity = false
######## Indexing slowlog JSON ##################
appender.index_indexing_slowlog_rolling.type = RollingFile appender.index_indexing_slowlog_rolling.type = RollingFile
appender.index_indexing_slowlog_rolling.name = index_indexing_slowlog_rolling appender.index_indexing_slowlog_rolling.name = index_indexing_slowlog_rolling
appender.index_indexing_slowlog_rolling.fileName = ${sys:es.logs.base_path}${sys:file.separator}${sys:es.logs.cluster_name}_index_indexing_slowlog.log appender.index_indexing_slowlog_rolling.fileName = ${sys:es.logs.base_path}${sys:file.separator}${sys:es.logs.cluster_name}\
appender.index_indexing_slowlog_rolling.layout.type = PatternLayout _index_indexing_slowlog.json
appender.index_indexing_slowlog_rolling.layout.pattern = [%d{ISO8601}][%-5p][%-25c] %marker%.-10000m%n appender.index_indexing_slowlog_rolling.layout.type = ESJsonLayout
appender.index_indexing_slowlog_rolling.filePattern = ${sys:es.logs.base_path}${sys:file.separator}${sys:es.logs.cluster_name}_index_indexing_slowlog-%d{yyyy-MM-dd}.log appender.index_indexing_slowlog_rolling.layout.type_name = index_indexing_slowlog
appender.index_indexing_slowlog_rolling.layout.esmessagefields=message,took,took_millis,doc_type,id,routing,source
appender.index_indexing_slowlog_rolling.filePattern = ${sys:es.logs.base_path}${sys:file.separator}${sys:es.logs.cluster_name}\
_index_indexing_slowlog-%i.json.gz
appender.index_indexing_slowlog_rolling.policies.type = Policies appender.index_indexing_slowlog_rolling.policies.type = Policies
appender.index_indexing_slowlog_rolling.policies.time.type = TimeBasedTriggeringPolicy appender.index_indexing_slowlog_rolling.policies.size.type = SizeBasedTriggeringPolicy
appender.index_indexing_slowlog_rolling.policies.time.interval = 1 appender.index_indexing_slowlog_rolling.policies.size.size = 1GB
appender.index_indexing_slowlog_rolling.policies.time.modulate = true appender.index_indexing_slowlog_rolling.strategy.type = DefaultRolloverStrategy
appender.index_indexing_slowlog_rolling.strategy.max = 4
#################################################
######## Indexing slowlog - old style pattern ##
appender.index_indexing_slowlog_rolling_old.type = RollingFile
appender.index_indexing_slowlog_rolling_old.name = index_indexing_slowlog_rolling_old
appender.index_indexing_slowlog_rolling_old.fileName = ${sys:es.logs.base_path}${sys:file.separator}${sys:es.logs.cluster_name}\
_index_indexing_slowlog.log
appender.index_indexing_slowlog_rolling_old.layout.type = PatternLayout
appender.index_indexing_slowlog_rolling_old.layout.pattern = [%d{ISO8601}][%-5p][%-25c{1.}] [%node_name]%marker %m%n
appender.index_indexing_slowlog_rolling_old.filePattern = ${sys:es.logs.base_path}${sys:file.separator}${sys:es.logs.cluster_name}\
_index_indexing_slowlog-%i.log.gz
appender.index_indexing_slowlog_rolling_old.policies.type = Policies
appender.index_indexing_slowlog_rolling_old.policies.size.type = SizeBasedTriggeringPolicy
appender.index_indexing_slowlog_rolling_old.policies.size.size = 1GB
appender.index_indexing_slowlog_rolling_old.strategy.type = DefaultRolloverStrategy
appender.index_indexing_slowlog_rolling_old.strategy.max = 4
#################################################
logger.index_indexing_slowlog.name = index.indexing.slowlog.index logger.index_indexing_slowlog.name = index.indexing.slowlog.index
logger.index_indexing_slowlog.level = trace logger.index_indexing_slowlog.level = trace
logger.index_indexing_slowlog.appenderRef.index_indexing_slowlog_rolling.ref = index_indexing_slowlog_rolling logger.index_indexing_slowlog.appenderRef.index_indexing_slowlog_rolling.ref = index_indexing_slowlog_rolling
logger.index_indexing_slowlog.appenderRef.index_indexing_slowlog_rolling_old.ref = index_indexing_slowlog_rolling_old
logger.index_indexing_slowlog.additivity = false logger.index_indexing_slowlog.additivity = false
<% @logging.each do |k,v| %>
<%= k %>=<%= v %>
<% end %>

13
cookbooks/elasticsearch/templates/default/systemd_unit.erb
View File

@ -1,20 +1,23 @@
[Unit] [Unit]
Description=Elasticsearch Description=Elasticsearch
Documentation=http://www.elastic.co Documentation=https://www.elastic.co
Wants=network-online.target Wants=network-online.target
After=network-online.target After=network-online.target
[Service] [Service]
Type=notify
RuntimeDirectory=elasticsearch RuntimeDirectory=elasticsearch
PrivateTmp=true
Environment=ES_HOME=/usr/share/elasticsearch Environment=ES_HOME=/usr/share/elasticsearch
Environment=ES_PATH_CONF=/etc/elasticsearch Environment=ES_PATH_CONF=/etc/elasticsearch
Environment=PID_DIR=/var/run/elasticsearch Environment=PID_DIR=/var/run/elasticsearch
Environment=ES_SD_NOTIFY=true
EnvironmentFile=-<%= @default_dir %>/<%= @program_name %> EnvironmentFile=-<%= @default_dir %>/<%= @program_name %>
WorkingDirectory=<%= @path_home %> WorkingDirectory=<%= @path_home %>
User=elasticsearch User=<%= @es_user %>
Group=elasticsearch Group=<%= @es_group %>
ExecStart=<%= @path_home %>/bin/elasticsearch -p ${PID_DIR}/elasticsearch.pid --quiet ExecStart=<%= @path_home %>/bin/elasticsearch -p ${PID_DIR}/elasticsearch.pid --quiet
@ -28,7 +31,7 @@ StandardOutput=journal
StandardError=inherit StandardError=inherit
# Specifies the maximum file descriptor number that can be opened by this process # Specifies the maximum file descriptor number that can be opened by this process
LimitNOFILE=65536 LimitNOFILE=65535
# Specifies the maximum number of processes # Specifies the maximum number of processes
LimitNPROC=4096 LimitNPROC=4096
@ -57,4 +60,4 @@ SuccessExitStatus=143
[Install] [Install]
WantedBy=multi-user.target WantedBy=multi-user.target
# Built for distribution-6.0.0 (distribution) # Built by elasticsearch chef cookbook for elasticsearch version <%= @version %>

18
cookbooks/elasticsearch/templates/oracle/initscript.erb
View File

@ -33,7 +33,7 @@ fi
# Sets the default values for elasticsearch variables used in this script # Sets the default values for elasticsearch variables used in this script
ES_HOME="/usr/share/elasticsearch" ES_HOME="/usr/share/elasticsearch"
MAX_OPEN_FILES=65536 MAX_OPEN_FILES=65535
MAX_MAP_COUNT=262144 MAX_MAP_COUNT=262144
ES_PATH_CONF="/etc/elasticsearch" ES_PATH_CONF="/etc/elasticsearch"
@ -69,21 +69,7 @@ if [ ! -x "$exec" ]; then
exit 1 exit 1
fi fi
checkJava() {
if [ -x "$JAVA_HOME/bin/java" ]; then
JAVA="$JAVA_HOME/bin/java"
else
JAVA=`which java`
fi
if [ ! -x "$JAVA" ]; then
echo "Could not find any executable java binary. Please install java in your PATH or set JAVA_HOME"
exit 1
fi
}
start() { start() {
checkJava
[ -x $exec ] || exit 5 [ -x $exec ] || exit 5
if [ -n "$MAX_OPEN_FILES" ]; then if [ -n "$MAX_OPEN_FILES" ]; then
@ -92,7 +78,7 @@ start() {
if [ -n "$MAX_LOCKED_MEMORY" ]; then if [ -n "$MAX_LOCKED_MEMORY" ]; then
ulimit -l $MAX_LOCKED_MEMORY ulimit -l $MAX_LOCKED_MEMORY
fi fi
if [ -n "$MAX_MAP_COUNT" -a -f /proc/sys/vm/max_map_count ]; then if [ -n "$MAX_MAP_COUNT" -a -f /proc/sys/vm/max_map_count ] && [ "$MAX_MAP_COUNT" -gt $(cat /proc/sys/vm/max_map_count) ]; then
sysctl -q -w vm.max_map_count=$MAX_MAP_COUNT sysctl -q -w vm.max_map_count=$MAX_MAP_COUNT
fi fi

18
cookbooks/elasticsearch/templates/redhat/initscript.erb
View File

@ -33,7 +33,7 @@ fi
# Sets the default values for elasticsearch variables used in this script # Sets the default values for elasticsearch variables used in this script
ES_HOME="/usr/share/elasticsearch" ES_HOME="/usr/share/elasticsearch"
MAX_OPEN_FILES=65536 MAX_OPEN_FILES=65535
MAX_MAP_COUNT=262144 MAX_MAP_COUNT=262144
ES_PATH_CONF="/etc/elasticsearch" ES_PATH_CONF="/etc/elasticsearch"
@ -69,21 +69,7 @@ if [ ! -x "$exec" ]; then
exit 1 exit 1
fi fi
checkJava() {
if [ -x "$JAVA_HOME/bin/java" ]; then
JAVA="$JAVA_HOME/bin/java"
else
JAVA=`which java`
fi
if [ ! -x "$JAVA" ]; then
echo "Could not find any executable java binary. Please install java in your PATH or set JAVA_HOME"
exit 1
fi
}
start() { start() {
checkJava
[ -x $exec ] || exit 5 [ -x $exec ] || exit 5
if [ -n "$MAX_OPEN_FILES" ]; then if [ -n "$MAX_OPEN_FILES" ]; then
@ -92,7 +78,7 @@ start() {
if [ -n "$MAX_LOCKED_MEMORY" ]; then if [ -n "$MAX_LOCKED_MEMORY" ]; then
ulimit -l $MAX_LOCKED_MEMORY ulimit -l $MAX_LOCKED_MEMORY
fi fi
if [ -n "$MAX_MAP_COUNT" -a -f /proc/sys/vm/max_map_count ]; then if [ -n "$MAX_MAP_COUNT" -a -f /proc/sys/vm/max_map_count ] && [ "$MAX_MAP_COUNT" -gt $(cat /proc/sys/vm/max_map_count) ]; then
sysctl -q -w vm.max_map_count=$MAX_MAP_COUNT sysctl -q -w vm.max_map_count=$MAX_MAP_COUNT
fi fi

18
cookbooks/elasticsearch/templates/ubuntu/initscript.erb
View File

@ -39,7 +39,7 @@ ES_HOME=/usr/share/$NAME
#ES_JAVA_OPTS= #ES_JAVA_OPTS=
# Maximum number of open files # Maximum number of open files
MAX_OPEN_FILES=65536 MAX_OPEN_FILES=65535
# Maximum amount of locked memory # Maximum amount of locked memory
#MAX_LOCKED_MEMORY= #MAX_LOCKED_MEMORY=
@ -82,22 +82,8 @@ if [ ! -x "$DAEMON" ]; then
exit 1 exit 1
fi fi
checkJava() {
if [ -x "$JAVA_HOME/bin/java" ]; then
JAVA="$JAVA_HOME/bin/java"
else
JAVA=`which java`
fi
if [ ! -x "$JAVA" ]; then
echo "Could not find any executable java binary. Please install java in your PATH or set JAVA_HOME"
exit 1
fi
}
case "$1" in case "$1" in
start) start)
checkJava
log_daemon_msg "Starting $DESC" log_daemon_msg "Starting $DESC"
@ -124,7 +110,7 @@ case "$1" in
ulimit -l $MAX_LOCKED_MEMORY ulimit -l $MAX_LOCKED_MEMORY
fi fi
if [ -n "$MAX_MAP_COUNT" -a -f /proc/sys/vm/max_map_count ]; then if [ -n "$MAX_MAP_COUNT" -a -f /proc/sys/vm/max_map_count ] && [ "$MAX_MAP_COUNT" -gt $(cat /proc/sys/vm/max_map_count) ]; then
sysctl -q -w vm.max_map_count=$MAX_MAP_COUNT sysctl -q -w vm.max_map_count=$MAX_MAP_COUNT
fi fi

9
site-cookbooks/kosmos-mastodon/recipes/default.rb
View File

@ -13,7 +13,12 @@ elasticsearch_user 'elasticsearch'
elasticsearch_install 'elasticsearch' do elasticsearch_install 'elasticsearch' do
type 'package' type 'package'
version '7.17.7' # The current version of the elasticsearch cookbook doesn't like versions
# it doesn't know about. This would still be installing the default (7.17.9)
# on a new machine, but it doesn't upgrade the package
download_url 'https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-7.17.7-amd64.deb'
# SHA256
download_checksum '5c588d779023672ba4e315e7cd4db068ac60a38873a35973574a1cae858c2030'
action :install action :install
end end
@ -21,8 +26,6 @@ elasticsearch_configure 'elasticsearch' do
allocated_memory node["kosmos-mastodon"]["elasticsearch"]["allocated_memory"] allocated_memory node["kosmos-mastodon"]["elasticsearch"]["allocated_memory"]
jvm_options %w( jvm_options %w(
-Xms1536m
-Xmx1536m
-XX:+AlwaysPreTouch -XX:+AlwaysPreTouch
-server -server
-Xss1m -Xss1m