openstack/openstack-ansible-rabbitmq_server - openstack-ansible-rabbitmq_server - OpenDev: Free Software Needs Free Tools

Currently all pip install tasks only require the package to be
present. This means that when an environment undergoes a minor
upgrade the package is not upgraded to the same version that
was tested with. This ultimately results in a deployed
environment that does not match the tested environment.
While for the services installed into venvs this is not an
issue, it does affect those which do not use venvs and any
packages which are installed outside of a venv or on top
of a venv.
This patch changes the behaviour to ensure that the install
task will always use the latest available package. In
developer_mode this will mean using the version specified
in upper-constraints, and in an integrated build this will
mean the version which is available in the wheel repo's
folder for the tag.
Change-Id: Ibeaebc906262a7e5816048a002fe69c6bdd622e5

The following changes have been made to reorganize the structure of this
role's test playbooks, bringing them into line with other
OpenStack-Ansible roles:
 - move each playbook to an individual file
 - rename playbooks descriptively
 - define hosts and groups directly in the inventory file
 - include group vars required by containers
 - store extra variables in a single shared test-vars file
 - reduce lxc network dhcp network range to avoid conflicts with static
 container addresses
 - include callback plugin for human readable logging of test tasks
Change-Id: I6aa877b10945a265d7484c9aacb3c232d82d8ebb

Paramiko version 2.0 has been released. It now uses the Python library
cryptography. Installing this requires additional system packages. This
commit adds in the appropriate packages required by cryptography based
on its documentation [1].
An alternative approach would have been to constrain the version of
Paramiko however the project describes the 1.x versions as relying on
insecure dependencies [2].
[1] https://cryptography.io/en/latest/installation/
[2] http://www.paramiko.org/installing.html
Change-Id: Ibda3558158f5fd57c1da05cf29e88b6075c0827e

Modify run_tests.sh file permissions to make the script executable to
increase the convenience of using the script.
Change-Id: Ie71596b028b1765083818692c7485eaed2bd88a2

Change-Id: I582a5c6d7283df438527858b3ec3f17920361431

This change ensures that the RabbitMQ ServerName is being set to
the **ansible_hostname** fact. This ensures the RabbitMQ server
has an appropriate server name which is compliant to the RFC.
Change-Id: I1f6a161b3af8fd0edd677ac1c3f396e6e8b5d02e
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>

This version of ansible is broken due to the following upstream
Ansible issue:
 * https://github.com/ansible/ansible-modules-extras/issues/2042
Change-Id: I78e1923a100b3d8bbaad4881f694c429dc849d95
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>

Change-Id: I6683e5227bf1b9f438efc2612f84311acfbd7b51

A task made mention of nova and was tagged with tags applicable
to nova. This removes them.
Change-Id: I3c66560445b50ede2bdb95138da7dbb7c0c9abd3

Remove cluster formation configuration from rabbitmq.config. Cluster
formation is managed by the task files in the role, see
tasks/rabbitmq_cluster.yml for details. The entries in the config
template are unnecessary and I believe they may be causing clustering
failures when the clustering tasks run.
Remove testing for cluster configuration in rabbitmq.config.
Change-Id: Ia65da05a5c967965143e8167657688d6906ad1f0

The git source for python2_lxc was necessary before the package was
available on pypi, which it now is:
 - https://pypi.python.org/pypi/lxc-python2
This patch removes the py_from_git role from the functional testing
playbook.
Change-Id: I09be5e131344592123ffa689f2e4ab0971470b90

This patch sets the test to check if the test is running in
OpenStack-CI, and if it is - then make the containers use
the host's pip.conf and some of the apt configuration.
All containers should make use of the hosts's apt sources
configuration. The tasks detect and implement this.
The intent is to make use of OpenStack-CI resources better
and consequently reduce the test run time.
Change-Id: I6dd30753f88c7876f78d49abbdfc4861d779a11c

Latest release of ansible-lint 2.4.0 has broken
role linter tasks by introducing a dependency on
openstack-ansible-plugins for all linting
While we may choose to adapt the linter tox env
in the future, for now we want to return to previous
linter behavior.
Change-Id: I4eea5b03321bb10ac5a367aa8c6c5ff460813682

Utilizing RST includes to reduce duplication of documentation
effort. All contents of the Ansible defaults/main.yml are
now included in the documentation directly.
Change-Id: Ib5224208259bc50b80cff2613f487ecbe461719a

Change-Id: Ia277f45197f8faaf3ebc159e3321519bd0469687

Change-Id: I52be3d82e11d3c74f5ec974e0dcc7a6bb4b628b8

Change the default install version of RabbitMQ Server to 3.6.1-1.
Additionally, the functional test playbook file has been split into
multiple, more focused, files. Tests to ensure that the expected version
of RabbitMQ Server is running and to upgrade from 3.5.7 to 3.6.1 have
been added.
Change-Id: I8907a3a71334be7b89ad743c3f2031e0a42a00f2

Change-Id: Iec90783490208160caca075abdb9b02e0148411c

Change-Id: I16268bd66221ca1770da8474d6ec24e83106be88

This fix will add ERLANG VM tuning parameters with increased
default settings to support larger installations.
Following overrides have been added:
 rabbitmq_async_threads defaults to 128 (from 32)
 rabbitmq_process_limit defaults to 1048576 (from 256k)
Closes-Bug: #1549940
DocImpact
Change-Id: Ia0fab288db8aa287e667dfc843f02d7ec318a816

Workarounding the upstream ansible apt module bug
documented here:
https://github.com/ansible/ansible-modules-core/pull/1517
For the next versions of ansible we'll be using, we should
check if the apt bug is fixed. When it's fixed, we could
abandon this change and use the standard apt module
with correct cache handling.
Change-Id: I4b27e630757ca228ed3543bb1d30187d2ee4e73b

This patch updates the tox.ini the same bashate exceptions as are
currently in the OpenStack-Ansible playbook repo.
It also ensures that the linters and all lint targets work
appropriately and normalises the tox.ini configuration to use
uniform formatting.
The use of ansible.cfg is removed as there is no way of being
certain which paths can be used without reverting to an ugly
sed hack in the commands. This is why it is preferred to make
use of environment variables which make use of tox's default
substitutions instead. It's a more reliable way of achieving
the goal for the purpose of gating and testing.
The switch to using a git clone instead of ansible-galaxy to
download the plugins is due to the path spec not being able to
work in Ansible 2.x. [1]
[1] https://github.com/ansible/ansible/issues/13563
Change-Id: I19d0a88574ca7bcb4d8709e14fbb5ffec5a8d021

OpenStack-Infra has requested that lint tests be pipelined into the 'linters'
tox target in order to make more efficient use of nodepool resources.
This patch:
- Ensures that a complete set of lints tests is available in individual
 tox targets.
- Adds a 'linters' target which executes all the other lints tests.
Change-Id: I98feb85a9a3a530f291c50bc46ec9c639327162a

Change-Id: I1529ef969605dab7ec6adfe23f02a1637d6a9ff0

This changes the test for rabbitmq process checking to use
a command that would actually be useful.
Change-Id: I2b48df8dce4b77c0359ad8f28f544559c3e54f50
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>

Currently, the key and cert files are not being distributed to
non-primary nodes. This commit does the following:
- references the primary node in rabbitmq_ssl_key_distribute.yml when
 referencing the key and cert facts
- uses a slurp in rabbitmq_ssl_key_distribute.yml so we can get exact
 copies for the key / cert files (previously the files would get
 created without trailing newlines)
- adds some basic tests to ensure these files are consistent across
 the cluster
- removes unnecessary with_items loops in
 rabbitmq_ssl_key_distribute.yml
Change-Id: I98bcd17dac9a47cc19e0d9ec90176945a6d7e355
Co-Authored-By: Jesse Pretorius <jesse.pretorius@rackspace.co.uk>

Currently the contents of the key file are 'rabbitmq_ssl_key_fact'
instead of the actual key and the same goes for the cert.
This patch ensures that the contents of the file are the actual
contents of the fact, not just the fact name.
Change-Id: I3ce18697fa1d6e25678ec624f4e554ed1ff904d6

This brings our tox.ini files inline with other OpenStack projects.
Change-Id: I6dd5066b6e6649b1e0ffca22c913fdc776fbc042
Depends-On: I376e51824c46f9b5eb4a4cff01aaa176632087dd
Signed-off-by: Paul Belanger <pabelanger@redhat.com>

Instead of launching two separate build jobs in our CI infrastructure,
one for bashate and one for pep8, launch a single one. While the jobs
are short, the setup of the images takes a long time, so this is
friendlier to our CI infrastructure.
Use the pep8 environment as common lint target and merge the bashate job
into it.
This patch also renamed dev-requirements to test-requirements to bring it
into line with all other OpenStack projects.
Change-Id: I19c4d9577ff86325fc87b9c03825f40defeeb273

Change-Id: I68d7daf1fda3d8b5cd538f684e973366a3fd61c8
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>

Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>