openstack/openstack-ansible-rabbitmq_server - openstack-ansible-rabbitmq_server - OpenDev: Free Software Needs Free Tools

'with_' clauses are evaluated before 'when' clauses. In Ansible 1.9 the
task is silently skipped when a variable within a 'with_' clause is
undefined, Ansible 2 provides a deprecation warning.
Separate the task deploying a user provided ssl cert and key into two
and check them individually for 'rabbitmq_user_ssl_cert' or
'rabbitmq_user_ssl_key' being defined.
Change-Id: I68afa3066ad88ecae95fcb29c78764d02dc38627

The RabbitMQ role no longer does any python package installation,
so the pip_install role dependency is no longer required.
Change-Id: I55f3749e680ea9ff496be7322c776d516ee75c0f

The pycrypto package no longer needs to be installed. It was
previously necessary due to the use of memcache for moving
SSL certificates around. As we're no longer using this
method the package installation and its binary deps may be
removed.
Change-Id: I805edeb47dc888411a785aacdb7d93d0cb05c393

With the implementation of https://review.openstack.org/321331 the
human_log callback plugin is now part of the plugins repo.
This patch removes the retrofitted version in tox in favor of using
the version from the plugins repo instead.
Change-Id: I1aad3b4e4fecc9e2b030c50b09e83b2b57ab2eb3

This patch removes the duplicated releasenotes/build
lines from .gitignore.
Change-Id: I64c52fb1a27bcc2ab2fc266dfc91cb2146a4110b

Change-Id: I124d7bbd52bdee571acd33522b2148c5baf8069f

The default value of "rabbitmq_all" remains, but this
change should allow deployers to target the role to multiple
groups now.
Change-Id: Idc1bc44b005f09f9dc7eb9778719a8ba92756c06
Implements: blueprint multi-rabbitmq-clusters

In order to prepare for the move of detailed configuration
content from the install guide to the roles, doc8 checks
are being implemented in the roles to ensure that any new
content submitted is properly checked.
Change-Id: I6576b47799b74b7930dc1ebefefed51e3de23526
Partially-Implements: blueprint osa-install-guide-overhaul

This patch updates the sphinx configuration for docs and releasenotes
to make it easier to replicate across repositories and to comply with
pep8 testing without exceptions.
Change-Id: I389cf6c3490adb516cf90305393471b47f83a937

Change the default install version of RabbitMQ Server to 3.6.3.
The 3.6.3 Debian package adds support for systemd and no longer leaves a
rabbitmq-server process running for signal handling to the erlang VM.
The `pgrep` command checking that rabbitmq is running has been updated
to search entire process names to account for this change.
Change-Id: I742b03f64cd0232835b7143507e3d3cd14d79f23

In order to prepare for implementing requirements management by the
OpenStack requirements management process, and to improve the
reliability and effectiveness of test execution, this patch implements
some changes to the tox configuration:
- The minimum tox version is increased in order to be able to use
 constraints for the python packages.
- The OpenStack upper-constraints are used when preparing the test
 venv for the linters checks.
- Any proxy environment variables set on the test host are passed
 into the venv to enable testing from behind a proxy.
- The environment variables used by Ansible tests are moved into
 a new venv called 'ansible' and this environment is inherited
 by all Ansible-related tests.
- The docs test will clean-up an existing build directory before
 executing the docs build.
- The releasenotes build cannot use upper-constraints at this point,
 so it doesn't.
- The Ansible role download will no longer ignore errors so that any
 problems discovered will result in a failed test.
- The human readable logging callback plugin is implemented for
 functional testing.
- The ansible test requirements are moved into tox.ini to ensure
 compliance for requirements.txt/test-requirements.txt for the
 global-requirements management contract.
- The ~/.ansible directory as a whole is not deleted. Instead only
 the plugins and roles folders are deleted to ensure that zuul's
 Ansible artifacts are left in-place.
- The ansible-lint version is updated to support execution against a
 folder, and the test now executes against the entire role to ensure
 that it captures all applicable files for lint testing.
This is a combined port of the following:
- https://review.openstack.org/323507
- https://review.openstack.org/338193
- https://review.openstack.org/332443
- https://review.openstack.org/338193
- https://review.openstack.org/339493
Change-Id: Ie08d1c8c82220e496bfa6d8b6743b5b3318c72ed

To ensure that the rabbitmq-server service is registered, reload the
systemd daemon after installing the rabbitmq package.
Change-Id: I33f7e6d5647509371a29c368f5cf0a21ab2a5ef7

This patch cleans up the container cache preparation in the tests
due to the following patches:
- https://review.openstack.org/307856 (new image build process)
- https://review.openstack.org/315114 (new apt config process)
- https://review.openstack.org/322188 (new resolver config process)
Change-Id: I6732c158197a686c36a02075595111e00183d934

A new release of flake8 is causing lint faiures. Our requirements
should match OpenStack requirements anyway.
This patch pins to the current master requirements from the
OpenStack requirements repository using global-requirements.
Change-Id: I8a4a8ad41af91e12d50343fddf399097f3388155

When executing the role with Ansible 2.1, the following
deprecation warning is issued in the output for some tasks.
[DEPRECATION WARNING]: Using bare variables is deprecated.
This patch addresses the tasks to fix the behaviour appropriately.
Change-Id: Ie48209425474a8a1623d8cdf2e3323beee7c6696

This makes the include_vars consistent accross all the
openstack-ansible-.* repos.
Change-Id: If636200ea7701e5eac047ba41bf4d11fd3bd91f9

Change the default install version of RabbitMQ Server to 3.6.2.
Tests have been updated to upgrade from 3.6.1 to 3.6.2 and an unused
test vars folder has been removed.
socat has been added as a new dependency to work around a known issue
with systemd: https://github.com/rabbitmq/rabbitmq-server/issues/664
Change-Id: I2a4d127a852e7780dd95f905f3e1d5d1df60028a

When ansible-lint runs it triggers an error for tasks that call
out to the package manager instead of using the package module.
These tasks call out to the dpkg and rpm package applications
to query if a version of rabbitmq is already installed and in
need of an upgrade.
Change-Id: I0ca5b0403562537d2ecfdba0466a87fb0b874933
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>

Separate files have been created for vars and tasks related to a
specific package manager. The 'vars_files_var' variable has been created
to store a list of files to search for distro specific variables.
The 'rabbitmq_apt_packages' variable has been deprecated and renamed to
the more generalized 'rabbitmq_dependencies' to better describe its
purpose and to simplify reuse of existing install tasks between multiple
distros.
Change-Id: I1940593978b733501daf5fe25edd393f2f6bee0c

In order to prevent race conditions with nodes joining the cluster
simultaneously when the cluster is first formed, we move the rabbitmq
installation play to be 'serial: 1'. However, when the nodes are being
upgraded, it cannot be done in serial so in this case we set 'serial: 0'
The tests are removed from a post_task include in the install play, and
moved to their own play as they need to be run after the entire cluster
has been formed. As well as moving a few generic vars into the
test-vars.yml include, we also pass in the specific version of rabbitmq
to be tested against in the test play.
Fixes-Bug: #1573030
Change-Id: Id119ff9f20ddfd8e1f29598c8c5ce862d2e7fab4

Since the refactor in 1b9035aa79 this
play is no longer used, so it can be removed.
Change-Id: Ibc8bb7df529b28c4edf04a7efb33ea7a3f69c47f

Change-Id: Ibde80f270698cc93f4089724deffca375291c953

Currently all pip install tasks only require the package to be
present. This means that when an environment undergoes a minor
upgrade the package is not upgraded to the same version that
was tested with. This ultimately results in a deployed
environment that does not match the tested environment.
While for the services installed into venvs this is not an
issue, it does affect those which do not use venvs and any
packages which are installed outside of a venv or on top
of a venv.
This patch changes the behaviour to ensure that the install
task will always use the latest available package. In
developer_mode this will mean using the version specified
in upper-constraints, and in an integrated build this will
mean the version which is available in the wheel repo's
folder for the tag.
Change-Id: Ibeaebc906262a7e5816048a002fe69c6bdd622e5

The following changes have been made to reorganize the structure of this
role's test playbooks, bringing them into line with other
OpenStack-Ansible roles:
 - move each playbook to an individual file
 - rename playbooks descriptively
 - define hosts and groups directly in the inventory file
 - include group vars required by containers
 - store extra variables in a single shared test-vars file
 - reduce lxc network dhcp network range to avoid conflicts with static
 container addresses
 - include callback plugin for human readable logging of test tasks
Change-Id: I6aa877b10945a265d7484c9aacb3c232d82d8ebb

Paramiko version 2.0 has been released. It now uses the Python library
cryptography. Installing this requires additional system packages. This
commit adds in the appropriate packages required by cryptography based
on its documentation [1].
An alternative approach would have been to constrain the version of
Paramiko however the project describes the 1.x versions as relying on
insecure dependencies [2].
[1] https://cryptography.io/en/latest/installation/
[2] http://www.paramiko.org/installing.html
Change-Id: Ibda3558158f5fd57c1da05cf29e88b6075c0827e

Modify run_tests.sh file permissions to make the script executable to
increase the convenience of using the script.
Change-Id: Ie71596b028b1765083818692c7485eaed2bd88a2

Change-Id: I582a5c6d7283df438527858b3ec3f17920361431

This change ensures that the RabbitMQ ServerName is being set to
the **ansible_hostname** fact. This ensures the RabbitMQ server
has an appropriate server name which is compliant to the RFC.
Change-Id: I1f6a161b3af8fd0edd677ac1c3f396e6e8b5d02e
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>

This version of ansible is broken due to the following upstream
Ansible issue:
 * https://github.com/ansible/ansible-modules-extras/issues/2042
Change-Id: I78e1923a100b3d8bbaad4881f694c429dc849d95
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>

Change-Id: I6683e5227bf1b9f438efc2612f84311acfbd7b51

A task made mention of nova and was tagged with tags applicable
to nova. This removes them.
Change-Id: I3c66560445b50ede2bdb95138da7dbb7c0c9abd3

Remove cluster formation configuration from rabbitmq.config. Cluster
formation is managed by the task files in the role, see
tasks/rabbitmq_cluster.yml for details. The entries in the config
template are unnecessary and I believe they may be causing clustering
failures when the clustering tasks run.
Remove testing for cluster configuration in rabbitmq.config.
Change-Id: Ia65da05a5c967965143e8167657688d6906ad1f0

The git source for python2_lxc was necessary before the package was
available on pypi, which it now is:
 - https://pypi.python.org/pypi/lxc-python2
This patch removes the py_from_git role from the functional testing
playbook.
Change-Id: I09be5e131344592123ffa689f2e4ab0971470b90

This patch sets the test to check if the test is running in
OpenStack-CI, and if it is - then make the containers use
the host's pip.conf and some of the apt configuration.
All containers should make use of the hosts's apt sources
configuration. The tasks detect and implement this.
The intent is to make use of OpenStack-CI resources better
and consequently reduce the test run time.
Change-Id: I6dd30753f88c7876f78d49abbdfc4861d779a11c

Latest release of ansible-lint 2.4.0 has broken
role linter tasks by introducing a dependency on
openstack-ansible-plugins for all linting
While we may choose to adapt the linter tox env
in the future, for now we want to return to previous
linter behavior.
Change-Id: I4eea5b03321bb10ac5a367aa8c6c5ff460813682

Utilizing RST includes to reduce duplication of documentation
effort. All contents of the Ansible defaults/main.yml are
now included in the documentation directly.
Change-Id: Ib5224208259bc50b80cff2613f487ecbe461719a

Change-Id: Ia277f45197f8faaf3ebc159e3321519bd0469687

Change-Id: I52be3d82e11d3c74f5ec974e0dcc7a6bb4b628b8

Change the default install version of RabbitMQ Server to 3.6.1-1.
Additionally, the functional test playbook file has been split into
multiple, more focused, files. Tests to ensure that the expected version
of RabbitMQ Server is running and to upgrade from 3.5.7 to 3.6.1 have
been added.
Change-Id: I8907a3a71334be7b89ad743c3f2031e0a42a00f2

Change-Id: Iec90783490208160caca075abdb9b02e0148411c

Change-Id: I16268bd66221ca1770da8474d6ec24e83106be88

This fix will add ERLANG VM tuning parameters with increased
default settings to support larger installations.
Following overrides have been added:
 rabbitmq_async_threads defaults to 128 (from 32)
 rabbitmq_process_limit defaults to 1048576 (from 256k)
Closes-Bug: #1549940
DocImpact
Change-Id: Ia0fab288db8aa287e667dfc843f02d7ec318a816

Workarounding the upstream ansible apt module bug
documented here:
https://github.com/ansible/ansible-modules-core/pull/1517
For the next versions of ansible we'll be using, we should
check if the apt bug is fixed. When it's fixed, we could
abandon this change and use the standard apt module
with correct cache handling.
Change-Id: I4b27e630757ca228ed3543bb1d30187d2ee4e73b