openstack/openstack-ansible-memcached_server - openstack-ansible-memcached_server - OpenDev: Free Software Needs Free Tools

With tox release of 4.0, some parameters were deprecated and are ignored now
which causes tox failures. One of the most spread issues we have is using
`whitelist_externals` isntead of `allowlist_externals`
Change-Id: I92e651bdcf6bdc6da8d90f2934238b51a5534cca

This repo is now testing only with Python
3, so let's make a few cleanups:
- remove setup.* files, we don't need them for this repo
- Cleanup */source/conf.py to remove now
 obsolete content.
- Remove install_cmd, the default is fine,
 move constraints into deps.
 Cleanup tox.ini a bit
Change-Id: I00d81ef2ce142faadc4064e3fe15ca020783a7b4

OpenStack is dropping the py2.7 support in ussuri cycle.
Only update required is in tox to define common baspython as py3 os that
all tox env including [testenv:functional] will use py3 instead of py2.
Complete discussion & schedule can be found in
- http://lists.openstack.org/pipermail/openstack-discuss/2019-October/010142.html
- https://etherpad.openstack.org/p/drop-python2-support
Ussuri Communtiy-wide goal:
https://governance.openstack.org/tc/goals/selected/ussuri/drop-py27.html
Change-Id: I9130b0b4b937223285ea0c5eaf2074acb6b459cd

1. Sphinx 1.8 introduced [1] the '--keep-going' argument which, as its name
suggests, keeps the build running when it encounters non-fatal errors.
This is exceptionally useful in avoiding a continuous edit-build loop
when undertaking large doc reworks where multiple errors may be
introduced.
2. Remove the unnecessary blank line for tox.ini
[1] https://github.com/sphinx-doc/sphinx/commit/e3483e9b045
Depends-On: Ie837d376bb1a1d93367369d25140dd8a9d5ac14e
Change-Id: I1105e54b43eb286f89a7e498cdc9491c31b43add

This patch adds a `pdf-docs` tox target that will build
PDF versions of our docs. As per the Train community goal:
 https://governance.openstack.org/tc/goals/selected/train/pdf-doc-generation.html
Add sphinxcontrib-svg2pdfconverter to doc/requirements.txt
to convert our SVGs.
Change-Id: I0ea6477cb62773a1c9c82e77204c9bc72255a892
Story: 2006105

Change-Id: I26f9ce0de55a719df7c1330c01f48aa36056eb1a

We want to default to running all tox environments under python 3, so
set the basepython value in each environment.
We do not want to specify a minor version number, because we do not
want to have to update the file every time we upgrade python.
We do not want to set the override once in testenv, because that
breaks the more specific versions used in default environments like
py35 and py36.
Change-Id: I76d4aeeb50299a43164dc32fe1f91b47ec194bb1
Closes-Bug: #1801657 

We want to default to running all tox environments under python 3, so
set the basepython value in each environment.
We do not want to specify a minor version number, because we do not
want to have to update the file every time we upgrade python.
We do not want to set the override once in testenv, because that
breaks the more specific versions used in default environments like
py35 and py36.
Change-Id: I067942ea9572e31da71c9f79dd4f94fa426c7566
Signed-off-by: Doug Hellmann <doug@doughellmann.com>

The TESTING_BRANCH environment variable is provided by the
run_tests.sh script and is derived from the .gitreview file.
This ensures that once the master branch becomes a stable
branch, the constraints from the stable branch in the
integrated repository will automatically get used once the
.gitreview file is updated.
To ensure that the required environment variables are present
we export them appropriately in run_tests.sh and modify the
tox configuration to pass them into the tox test.
Change-Id: I23a7b2e060cde571ae5b1103616c209246814bb2
Needed-By: https://review.openstack.org/579371 

Using tox for requirements management requires in-repo
requirements files for all our repositories. Rather than
do that, we make use of the tests repo to capture our
common requirements and use this to install them.
This reduces our review requirement rate and simplifies
maintenance for us for the tox config. It also makes it
usable with 'Depends-On', which is marvellous!
The tox requirements definitions for docs/releasenotes
builds are left in-place as those are standard entries
across the community. If that changes at some point, we
can re-assess those entries too.
Depends-On: https://review.openstack.org/579208
Change-Id: If2b1244e6c4f8ba2d43c9499d3861bc1cf2b21b6

Now that run_tests.sh handles the tests repo clone, we can
remove the use of the older tests-repo-clone.sh script.
Change-Id: I2ccaa1df08aaf772058607653059ba3ccfc3fc86

For compliance with the Project Testing Interface as described in:
https://governance.openstack.org/tc/reference/project-testing-interface.html
For more detials information, please refer to:
http://lists.openstack.org/pipermail/openstack-dev/2017-December/125710.html
Change-Id: I12655940f683caec7cc0708a42ddf52720de6dbe

In order to allow the use of the environment variable which informs
Ansible which user executed the playbook, we pass the USER env var
into the environment that tox builds.
Change-Id: Ie5a013db37bcfd65cfe4dbe9589fbcb9b8c231e2
Implements: blueprint python-build-install-simplification
Signed-off-by: Jesse Pretorius <jesse.pretorius@rackspace.co.uk>

Currently the role tests use whatever versions of pip,
setuptools and wheel are already installed on the host.
When a version of these tools changes it often causes
problems for our testing.
This will ensure that we use a known good set of pins
which is maintained in the general SHA bumping process.
Change-Id: I445e5b8ec4e650871586b2777c4f25699405c7bd

When executing the tests repo clone in OpenStack-CI,
use zuul-cloner instead of git to enable cross-repo
testing. This ensures that if a dependent patch from
the tests repo is noted using 'Depends-On: <change-id>'
in the commit message, that patch will be included.
Change-Id: I6de4c78710f655b3a8a8be070e6dec4ad2c65e7e
Depends-On: Idce7abebf32f24c356a27e099fbca954d917402b

TrivialFix
Change-Id: I7edc7077ba702418ec09805e897474784a4a9f5a

Some hacking have been removed,so we can delete them.
More details:
 https://github.com/openstack-dev/hacking/blob/master/setup.cfg
Change-Id: I4b21a0c05f2e37e689f1d7e544e407c899c2fd72

With the implementation of https://review.openstack.org/388087 all
tox targets may now use upper constraints.
Change-Id: I4fb541946db159cffe425fa3569beb8c68ed9a91

This patch consumes the centralised Ansible test scripts
implemented in https://review.openstack.org/381853
Change-Id: I486582d6793f61a08e09da8fe04203fac9c214bd

This patch consumes the test scripts implemented by
https://review.openstack.org/375061 to ensure that
the tests and test preparation is consistent and
more maintainable.
Change-Id: Iece5efb32143979f4d4a0322e3b26aaa5f10d66a

Change-Id: I3cf79aceb879f1873484ada3b9dacf0c3429f4b0
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>

Change-Id: I321cd3e794edbef9bf21055115bdc8bec747392d
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>

The volume of logs we collect in CI jobs are extensive. This
patch ensures that the logs collected are compressed to reduce
the space taken in CI systems.
PYTHONUNBUFFERED is also set to ensure that the console log
from the CI jobs is recorded in the exact order of execution.
Change-Id: Ice645e23ba4e505b54ff441f577400220e658754
Related-Bug: #1620849 

Change-Id: I811d23c4dc8d855e51d28b1fba82d6def8f0d623

This change enables log collection within the gate so that further analysis
on gate tasks can be performed post build. This is very useful when
debugging problems.
Change-Id: I457910535f3f324e124b182f55dcf8b1eea5f725
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>

As per [1] all linting tests can now use upper-constraints. This patch
removes all instances of the install_command override relating to lint
testing which were needed to negate the use of upper-constraints.
[1] http://lists.openstack.org/pipermail/openstack-dev/2016-August/101474.html
Change-Id: I9038c25b607255d05716acb15f4a7e971e7fc3b4

The plugin repo needs to be cloned for ansible-lint to
understand ansible plugins. The commands currently
reside in tox.ini under the ansible section and are not
currently included. This commit fixes that error.
Change-Id: I8a9a24cb5e6ae902d5dd6fcc78b0f03952467bc6

The 'docs' tox target executes the doc8 lint test which may result in
failures when testing documentation builds, but OpenStack-CI does not
execute that tox target.
In order to ensure that we catch all standard documentation syntax
errors and prevent them from merging, this patch includes the docs
target in the 'linters' chain of tests.
Fixes for any failures which result from executing this test are also
included in the patch.
Change-Id: I2794185c0fce4c5fb027cb63d20850c72c4a91a4

With the implementation of https://review.openstack.org/321331 the
human_log callback plugin is now part of the plugins repo.
This patch removes the retrofitted version in tox in favor of using
the version from the plugins repo instead.
Change-Id: I5671dcb28277a3942b9c75ab218dd7d476d48bf0

In order to prepare for the move of detailed configuration
content from the install guide to the roles, doc8 checks
are being implemented in the roles to ensure that any new
content submitted is properly checked.
Change-Id: I50a62d136ac771887d684385fd2b2043a607ee15
Partially-Implements: blueprint osa-install-guide-overhaul

As zuul uses Ansible the ~/.ansible directory is populated with
artifacts which zuul uses to facilitate the job execution.
This patch ensures that the execution of tests does not remove
the whole ~/.ansible directory, but instead only removes the
parts which we populate.
Change-Id: I299f19cd999869dd3f41d24fbdad044816998866

In order to prepare for implementing requirements management by the
OpenStack requirements management process, all requirements in
requirements.txt and test-requirements.txt need to be in the
global-requirements file.
As ansible and ansible-lint are not in those files, this patch moves
them into explicit deps for the ansible environments in tox.ini.
Change-Id: If661d8d5aff86f3bb163624052e41160dd3c9a18

This should make it easier to understand check and gate
failures by analyzing the logs.
Change-Id: I0783e109f34494580658705a0962ab74da5cc20a

In order to prepare for implementing requirements management by the
OpenStack requirements management process this patch implements
some changes to the tox configuration:
- The minimum tox version is increased in order to be able to use
 constraints for the python packages.
- The OpenStack upper-constraints are used when preparing the test
 venv for the linters checks.
- Any proxy environment variables set on the test host are passed
 into the venv to enable testing from behind a proxy.
- The environment variables used by Ansible tests are moved into
 a new venv called 'ansible' and this environment is inherited
 by all Ansible-related tests.
- The docs test will clean-up an existing build directory before
 executing the docs build.
- The releasenotes build cannot use upper-constraints at this point,
 so it doesn't.
- The Ansible role download will no longer ignore errors so that any
 problems discovered will result in a failed test.
Change-Id: I2ef319e48a4106d911e2ceaddce2c17094ac1aa7

ansible-lint 2.7.0 was recently released, update the role's test
requirements to use it.
The tox test has been changed to run ansible-lint against this role,
rather than the test playbook which would only apply to this role's
dependencies.
Also include a skip tag for the 'yum clean all' task since the yum
module currently does not have a way of executing this action.
Change-Id: Ie7c8a2c1635ba64ca64e71a6d80dc6545f86f4e5

This commit updates the memcached_server role to work on Trusty,
Xenial, and CentOS 7.
NOTES:
1. This role no longer creates the memcache user since both Ubuntu and
 CentOS already install a suitable user
2. We have temporarily disabled testing of the log file since CentOS and
 Xenial do not log to file
3. On Ubuntu we drop ulimits into /etc/defaults/memcached, we need to
 figure out how to do the equivalent on CentOS
4. We update tasks/memcached_config.yml to use the correct memcached
 user in limits.conf, however neither these limits or the ones in
 templates/memcached.debian.j2 actually seem to be taking effect.
 More work in an additional review will need to be done to clean this
 all up.
Implements: blueprint multi-platform-host
Change-Id: I4c32f3d60939615c5d0c6fb202e96aacb35ab9b4
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>

Change-Id: Ib13e9c50ea0f96e97ef0eb8f92521ff68892d0cc

This patch updates the tox.ini the same bashate exceptions as are
currently in the OpenStack-Ansible playbook repo.
It also ensures that the linters and all lint targets work
appropriately and normalises the tox.ini configuration to use
uniform formatting.
The use of ansible.cfg is removed as there is no way of being
certain which paths can be used without reverting to an ugly
sed hack in the commands. This is why it is preferred to make
use of environment variables which make use of tox's default
substitutions instead. It's a more reliable way of achieving
the goal for the purpose of gating and testing.
The switch to using a git clone instead of ansible-galaxy to
download the plugins is due to the path spec not being able to
work in Ansible 2.x. [1]
[1] https://github.com/ansible/ansible/issues/13563
Change-Id: I7c4b2e14487dcfa63e3b85e1cd4e7ff7c4113d23

OpenStack-Infra has requested that lint tests be pipelined into the 'linters'
tox target in order to make more efficient use of nodepool resources.
This patch:
- Ensures that a complete set of lints tests is available in individual
 tox targets.
- Adds a 'linters' target which executes all the other lints tests.
Change-Id: Ie6f8f38e1d21863b2cfbb3cd2075a4f4bf8375e1

This brings our tox.ini files inline with other OpenStack projects.
Change-Id: I05ff0d53f8e5ac7ec40e6649b3c2fc865f45092d
Depends-On: I376e51824c46f9b5eb4a4cff01aaa176632087dd
Signed-off-by: Paul Belanger <pabelanger@redhat.com>

Instead of launching two separate build jobs in our CI infrastructure,
one for bashate and one for pep8, launch a single one. While the jobs
are short, the setup of the images takes a long time, so this is
friendlier to our CI infrastructure.
Use the pep8 environment as common lint target and merge the bashate job
into it.
This patch also renamed dev-requirements to test-requirements to bring it
into line with all other OpenStack projects.
Change-Id: Ieeffb7719b88213aaafa02073f931c491d3deee3

Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>