openstack/openstack-ansible-rabbitmq_server - openstack-ansible-rabbitmq_server - OpenDev: Free Software Needs Free Tools

With tox release of 4.0, some parameters were deprecated and are ignored now
which causes tox failures. One of the most spread issues we have is using
`whitelist_externals` isntead of `allowlist_externals`
Change-Id: I55cade793fc74fd5555c0f520bc132916343330e

This repo is now testing only with Python 3, so let's make
a few cleanups:
- Remove setup.* files, those are not needed for this repo.
- Cleanup */source/conf.py to remove now obsolete content.
- Remove install_command from tox.ini, the default is fine,
 move constraints into deps
- Enable warnings for docs build
Change-Id: I07da37d6f15aefc7b2fa558a6ead7028d6c9c556

OpenStack is dropping the py2.7 support in ussuri cycle.
Only update required is in tox to define common baspython as py3 os that
all tox env including [testenv:functional] will use py3 instead of py2.
Complete discussion & schedule can be found in
- http://lists.openstack.org/pipermail/openstack-discuss/2019-October/010142.html
- https://etherpad.openstack.org/p/drop-python2-support
Ussuri Communtiy-wide goal:
https://governance.openstack.org/tc/goals/selected/ussuri/drop-py27.html
Change-Id: I657f13a54e794a5e58cee5ad03d74ab266a06ca7

1. Sphinx 1.8 introduced [1] the '--keep-going' argument which, as its name
suggests, keeps the build running when it encounters non-fatal errors.
This is exceptionally useful in avoiding a continuous edit-build loop
when undertaking large doc reworks where multiple errors may be
introduced.
2. Remove the unnecessary blank line for tox.ini
[1] https://github.com/sphinx-doc/sphinx/commit/e3483e9b045
Depends-On: Ie837d376bb1a1d93367369d25140dd8a9d5ac14e
Change-Id: If83813f29a349783b6003545fb2457f9eb9f3352

This patch adds a `pdf-docs` tox target that will build
PDF versions of our docs. As per the Train community goal:
 https://governance.openstack.org/tc/goals/selected/train/pdf-doc-generation.html
Add sphinxcontrib-svg2pdfconverter to doc/requirements.txt
to convert our SVGs.
Change-Id: I0b200d57d7c92c959dc90fc540653aed71a6778c
Story: 2006105

Change-Id: Id294850bce5c6485cd704c595812630b98a9c572

We want to default to running all tox environments under python 3, so
set the basepython value in each environment.
We do not want to specify a minor version number, because we do not
want to have to update the file every time we upgrade python.
We do not want to set the override once in testenv, because that
breaks the more specific versions used in default environments like
py35 and py36.
Change-Id: I7704b6d9620eb3bd4c98a57f82268422fcb1852a
Closes-Bug: #1801657 

We want to default to running all tox environments under python 3, so
set the basepython value in each environment.
We do not want to specify a minor version number, because we do not
want to have to update the file every time we upgrade python.
We do not want to set the override once in testenv, because that
breaks the more specific versions used in default environments like
py35 and py36.
Change-Id: I22102abc36569501e256facb6ade9fdb28c3d47b
Signed-off-by: Doug Hellmann <doug@doughellmann.com>

Now that we're using the general templates, we can slim down
the role test definitions. We can also remove tests which are
not being watched, or which are fundamentally broken. With
this we can also remove unused scripts/plays.
We do the following:
1. We remove the 'upgrade' job, given that it's been broken
 all cycle due to the way the job executes and we need to
 regroup to figure out another way to do it.
Change-Id: I473b4162e141e05692e89f0420ea74c745dfc9e5

The TESTING_BRANCH environment variable is provided by the
run_tests.sh script and is derived from the .gitreview file.
This ensures that once the master branch becomes a stable
branch, the constraints from the stable branch in the
integrated repository will automatically get used once the
.gitreview file is updated.
To ensure that the required environment variables are present
we export them appropriately in run_tests.sh and modify the
tox configuration to pass them into the tox test.
Change-Id: Iff747fcfc56de833fa2fd43f807543896ced9527
Needed-By: https://review.openstack.org/579371 

We want to default to running all tox environments under python 3, so
set the basepython value in each environment.
We do not want to specify a minor version number, because we do not
want to have to update the file every time we upgrade python.
We do not want to set the override once in testenv, because that
breaks the more specific versions used in default environments like
py35 and py36.
Change-Id: I95c87fb0f374feebb9137548359e5da87fc27dc2

Using tox for requirements management requires in-repo
requirements files for all our repositories. Rather than
do that, we make use of the tests repo to capture our
common requirements and use this to install them.
This reduces our review requirement rate and simplifies
maintenance for us for the tox config. It also makes it
usable with 'Depends-On', which is marvellous!
The tox requirements definitions for docs/releasenotes
builds are left in-place as those are standard entries
across the community. If that changes at some point, we
can re-assess those entries too.
Depends-On: https://review.openstack.org/579208
Change-Id: I062311624046af061100f7ae8cccf10984b27c68

The version of erlang we're using, while functional, is not recommended.
This change updates our version of erlang for rabbitmq to be the
recommended version by changing the pinned version of erlang.
The apt package pinning meta dependency has been removed and put into
the apt task file as an include role. To ensure a more uniform process
for setting the version of erlang used with rabbitmq the new variable
`rabbitmq_erlang_version_spec` has been added. This option has a default
of "null" and is set by distro specific variables when applicable.
Erlang version support matrix: https://www.rabbitmq.com/which-erlang.html
Change-Id: Iced12d4533eec068bd11d8bd235d81308ef40427
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>

Now that run_tests.sh handles the tests repo clone, we can
remove the use of the older tests-repo-clone.sh script.
Change-Id: I04b2c0614b3a328b8d805f64f8ed517435bf0ddc

In order to allow the use of the environment variable which informs
Ansible which user executed the playbook, we pass the USER env var
into the environment that tox builds.
Change-Id: I426e3356583a8f9d394e3e7721c8a502a7fe4adb
Implements: blueprint python-build-install-simplification
Signed-off-by: Jesse Pretorius <jesse.pretorius@rackspace.co.uk>

For compliance with the Project Testing Interface as described in:
https://governance.openstack.org/tc/reference/project-testing-interface.html
For more detials information, please refer to:
http://lists.openstack.org/pipermail/openstack-dev/2017-December/125710.html
Change-Id: Ide1ffe82a4b49c700a91e271163f73816e08542f

The current functional test does a full upgrade process,
but uses the same current role for both the previous
version and the new version.
Ideally we want to test the initial deploy using the
previous series' role, so that we're actually testing
a series upgrade, not just a rabbitmq upgrade.
This patch separates the upgrade test out and makes the
default test just be a deployment. This allows us to
more clearly see where something breaks.
Depends-On: https://review.openstack.org/543245
Change-Id: I43e1e224cd4237ac37b40fde5ce7c279331a012d

Currently the role tests use whatever versions of pip,
setuptools and wheel are already installed on the host.
When a version of these tools changes it often causes
problems for our testing.
This will ensure that we use a known good set of pins
which is maintained in the general SHA bumping process.
Change-Id: I8a14c63d82956406f63b6ae810f3dd4eb2fb1cd2

When executing the tests repo clone in OpenStack-CI,
use zuul-cloner instead of git to enable cross-repo
testing. This ensures that if a dependent patch from
the tests repo is noted using 'Depends-On: <change-id>'
in the commit message, that patch will be included.
Depends-On: Idce7abebf32f24c356a27e099fbca954d917402b
Depends-On: I5da7802d61d2ab6b03908138e3a3ed2db22e3d29
Change-Id: I469bef1a5aad2b1c31972ec8f8ac3d7c0cdf4cc7

Trivialfix
Change-Id: I96d954332991745dcbaf6d5f0ec4ef50c41aba72

Some hacking have been removed,so we can delete them.
More details:
 https://github.com/openstack-dev/hacking/blob/master/setup.cfg
Change-Id: Id822074fbde57c6a02ee5eb9e5acc288b95a947b

With the implementation of https://review.openstack.org/388087 all
tox targets may now use upper constraints.
Change-Id: Idee4dc3e35fb0f67a3f12e641999753baee0182f

This patch consumes the centralised Ansible test scripts
implemented in https://review.openstack.org/381853
Change-Id: Icbfcc4e2f6584803458c0eb7eb1239c459b83492

With https://review.openstack.org/381479 merged, setting this
variable in tox.ini via an extra var is unnecessary.
Change-Id: I1aa87d5ddfaefb1365c6eb42dca413ada2b71a35

This patch consumes the test scripts implemented by
https://review.openstack.org/375061 to ensure that
the tests and test preparation is consistent and
more maintainable.
Change-Id: Iea6a5793509df959e877a8974e3322a0b92ba360

This commit also moves the upgrade test to its own
scenario using tox environment func_upgrade
Closes-Bug: #1623592
Change-Id: Ib79d9c261c3bf5e018806867ebe804b5853908e9
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>

Change-Id: Id013e531bad0d7938e726b26e2e005f028435507

The volume of logs we collect in CI jobs are extensive. This
patch ensures that the logs collected are compressed to reduce
the space taken in CI systems.
PYTHONUNBUFFERED is also set to ensure that the console log
from the CI jobs is recorded in the exact order of execution.
Change-Id: I5eb6b16deaf3e0b01d46dae86ee40b1f8e90581d
Related-Bug: #1620849 

This change enables log collection within the gate so that further analysis
on gate tasks can be performed post build. This is very useful when
debugging problems.
Change-Id: I64ba550208a690a784dbad8a3b871d0c1d703612
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>

As per [1] all linting tests can now use upper-constraints. This patch
removes all instances of the install_command override relating to lint
testing which were needed to negate the use of upper-constraints.
[1] http://lists.openstack.org/pipermail/openstack-dev/2016-August/101474.html
Change-Id: I4cbd9a5baa0d4ae96c600af104146c47813ebc90

This adds python packages required for SNI support to ensure that tox
installs the packages into the venv when running tests.
Change-Id: I5277e8f7b172a2ab847e368d8780e9992b5b6a88

The plugin repo needs to be cloned for ansible-lint to
understand ansible plugins. The commands currently
reside in tox.ini under the ansible section and are not
currently included. This commit fixes that error.
Change-Id: Ic998e7f82a03a5008b6b15976c094b833a2a177e

CentOS 7 comes with Python 2.7.5, which does not have support for
servers that use Server Name Indication (SNI) to host more than
one SSL certificate on the same IP address. Three Python modules
are required on CentOS 7 systems to add this support:
* pyasn1
* pyOpenSSL
* ndg-httpsclient
This patch should prevent the SSL verification issues that occur
when the RabbitMQ RPMs are downloaded with get_url.
Closes-bug: 1604922
Change-Id: Ia14f25ed0ae399cfea6177adb6d34582a40c9249

The 'docs' tox target executes the doc8 lint test which may result in
failures when testing documentation builds, but OpenStack-CI does not
execute that tox target.
In order to ensure that we catch all standard documentation syntax
errors and prevent them from merging, this patch includes the docs
target in the 'linters' chain of tests.
Fixes for any failures which result from executing this test are also
included in the patch.
Change-Id: I0f3d0585da4e9d946627bb78dc7314149e8dee9f

With the implementation of https://review.openstack.org/321331 the
human_log callback plugin is now part of the plugins repo.
This patch removes the retrofitted version in tox in favor of using
the version from the plugins repo instead.
Change-Id: I1aad3b4e4fecc9e2b030c50b09e83b2b57ab2eb3

In order to prepare for the move of detailed configuration
content from the install guide to the roles, doc8 checks
are being implemented in the roles to ensure that any new
content submitted is properly checked.
Change-Id: I6576b47799b74b7930dc1ebefefed51e3de23526
Partially-Implements: blueprint osa-install-guide-overhaul

In order to prepare for implementing requirements management by the
OpenStack requirements management process, and to improve the
reliability and effectiveness of test execution, this patch implements
some changes to the tox configuration:
- The minimum tox version is increased in order to be able to use
 constraints for the python packages.
- The OpenStack upper-constraints are used when preparing the test
 venv for the linters checks.
- Any proxy environment variables set on the test host are passed
 into the venv to enable testing from behind a proxy.
- The environment variables used by Ansible tests are moved into
 a new venv called 'ansible' and this environment is inherited
 by all Ansible-related tests.
- The docs test will clean-up an existing build directory before
 executing the docs build.
- The releasenotes build cannot use upper-constraints at this point,
 so it doesn't.
- The Ansible role download will no longer ignore errors so that any
 problems discovered will result in a failed test.
- The human readable logging callback plugin is implemented for
 functional testing.
- The ansible test requirements are moved into tox.ini to ensure
 compliance for requirements.txt/test-requirements.txt for the
 global-requirements management contract.
- The ~/.ansible directory as a whole is not deleted. Instead only
 the plugins and roles folders are deleted to ensure that zuul's
 Ansible artifacts are left in-place.
- The ansible-lint version is updated to support execution against a
 folder, and the test now executes against the entire role to ensure
 that it captures all applicable files for lint testing.
This is a combined port of the following:
- https://review.openstack.org/323507
- https://review.openstack.org/338193
- https://review.openstack.org/332443
- https://review.openstack.org/338193
- https://review.openstack.org/339493
Change-Id: Ie08d1c8c82220e496bfa6d8b6743b5b3318c72ed

The following changes have been made to reorganize the structure of this
role's test playbooks, bringing them into line with other
OpenStack-Ansible roles:
 - move each playbook to an individual file
 - rename playbooks descriptively
 - define hosts and groups directly in the inventory file
 - include group vars required by containers
 - store extra variables in a single shared test-vars file
 - reduce lxc network dhcp network range to avoid conflicts with static
 container addresses
 - include callback plugin for human readable logging of test tasks
Change-Id: I6aa877b10945a265d7484c9aacb3c232d82d8ebb

Change-Id: I6683e5227bf1b9f438efc2612f84311acfbd7b51

This patch updates the tox.ini the same bashate exceptions as are
currently in the OpenStack-Ansible playbook repo.
It also ensures that the linters and all lint targets work
appropriately and normalises the tox.ini configuration to use
uniform formatting.
The use of ansible.cfg is removed as there is no way of being
certain which paths can be used without reverting to an ugly
sed hack in the commands. This is why it is preferred to make
use of environment variables which make use of tox's default
substitutions instead. It's a more reliable way of achieving
the goal for the purpose of gating and testing.
The switch to using a git clone instead of ansible-galaxy to
download the plugins is due to the path spec not being able to
work in Ansible 2.x. [1]
[1] https://github.com/ansible/ansible/issues/13563
Change-Id: I19d0a88574ca7bcb4d8709e14fbb5ffec5a8d021

OpenStack-Infra has requested that lint tests be pipelined into the 'linters'
tox target in order to make more efficient use of nodepool resources.
This patch:
- Ensures that a complete set of lints tests is available in individual
 tox targets.
- Adds a 'linters' target which executes all the other lints tests.
Change-Id: I98feb85a9a3a530f291c50bc46ec9c639327162a

This brings our tox.ini files inline with other OpenStack projects.
Change-Id: I6dd5066b6e6649b1e0ffca22c913fdc776fbc042
Depends-On: I376e51824c46f9b5eb4a4cff01aaa176632087dd
Signed-off-by: Paul Belanger <pabelanger@redhat.com>

Instead of launching two separate build jobs in our CI infrastructure,
one for bashate and one for pep8, launch a single one. While the jobs
are short, the setup of the images takes a long time, so this is
friendlier to our CI infrastructure.
Use the pep8 environment as common lint target and merge the bashate job
into it.
This patch also renamed dev-requirements to test-requirements to bring it
into line with all other OpenStack projects.
Change-Id: I19c4d9577ff86325fc87b9c03825f40defeeb273

Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>