openstack/openstack-ansible-os_trove - openstack-ansible-os_trove - OpenDev: Free Software Needs Free Tools

To use openstackdocstheme 1.11.0 properly, this patch fixes
some settings according to follow[0].
[0]https://docs.openstack.org/openstackdocstheme/latest/
Change-Id: Ie9bb4dc796983546e648adb7a805fc2ddb875c5c

Change-Id: I1e3c28018e91ba5507110d94ea497d149b9df337

As part of the docs migration work[0] for Pike we need to switch to use the
openstackdocstheme.
[0]https://review.openstack.org/#/c/472275/
Change-Id: I585c3432830346e323f38215e43c24b44f927901

Change-Id: I35128bbaddd54b18f6ebf913e675282533e2f523

Change-Id: I35a002a9b97c22494d4c8967a75d06ac52f28ce5

Change-Id: I3d9172a1c4bf1fdfc0f68292c8ad7f45291a96f9

The keystonemiddleware library recently switched to using the
cryptography library over pycrypto, which was unmaintained. See
Iced7f5115e49ccf4f7f5bf6813cb5988b95c248b
Change-Id: I006bce0c07002fa495b09b0603ea60b5ddf4839b
Co-Authored-By: Nolan Brubaker <nolan.brubaker@rackspace.com>

Change-Id: I735a5e2e6d07813ad34c5203a4ef8cf29b17939c

Currently the role tests use whatever versions of pip,
setuptools and wheel are already installed on the host.
When a version of these tools changes it often causes
problems for our testing.
This will ensure that we use a known good set of pins
which is maintained in the general SHA bumping process.
Change-Id: Ifee6e3aff26d2eae641d1248474b4e55c74c3f94

Change-Id: If132e13a82bd99f612ea4e786a76633b6fbb8616

Option "rpc_backend" from group "DEFAULT" is deprecated for removal
(Replaced by [DEFAULT]/transport_url). Its value may be silently
ignored in the future.
Change-Id: Ia0c175746a5bdd0f8c722ab214719e77160a0e12
Implements: blueprint deprecate-rpc-backend

In order to do a developer mode that allows installation of packages
from local file, which take precedence over the developer_mode
constraints, we need to allow the order of the constraints to be
changed.
This patch adds a "pip_install_developer_constraints" var which is used
to set the developer mode constraints. By default this will leave the
same behaviour but will allow additional constraints to be added, or the
developermode constraints file to be overriden altogether.
Change-Id: I64affcd1b3d5580e1cee5047df465e2ab8144211

Option "rabbit_use_ssl" from group "oslo_messaging_rabbit" is deprecated.
Use option "ssl" from group "oslo_messaging_rabbit".
Change-Id: Ibbd06cb3e65bf9970b0b3de62c9e12ce5fa4f678
Implements: blueprint deprecate-rabbit-use-ssl

In order to make it easier to detect the currently deployed
venv for a service, and therefore allow smarter decisions
for things like upgrading, we implement the venv tag as a
local fact.
The file used to store facts will be the same for all
OpenStack services, with each service using its own section.
Example:
"ansible_local": {
 "openstack_ansible": {
 "trove": {
 "venv_tag": "14.2.1"
 }
 }
}
Change-Id: Id2545ad38d78c5c28310b67a8a3eb74b8b51b026

When openstack_networks is empty list, an error is seen while
setting trove service network id. The error seen was "list object
has no element 0". Added a validation task to fail if trove network
is not created yet.
Change-Id: I9acfc8e06f461073777469712fd6ce50e3187b30

Change-Id: Ic5d80e8b077b0bb59cf23e582724ee417b5e2ee3

We use an SSH bastion host which we do our deployment through. The
deployment host doesn't have direct access to the same network as the
host. As a result the venv local checksum lookup fails.
I have described this here:
https://bugs.launchpad.net/openstack-ansible/+bug/1689283
This is a simple fix for this problem, assuming everything is good it
will need repeating in multiple places in the code base.
Change-Id: I681491539aa4aa3b3c88ce059ae6d1f7e56f651d

When executing the tests repo clone in OpenStack-CI,
use zuul-cloner instead of git to enable cross-repo
testing. This ensures that if a dependent patch from
the tests repo is noted using 'Depends-On: <change-id>'
in the commit message, that patch will be included.
Change-Id: Idd2c2e750c001a3ece7b778520dd60cf9e649acf
Depends-On: Idce7abebf32f24c356a27e099fbca954d917402b

Change-Id: I57c8542f6275fa1db4a68df6f0904b4d943d2939

Change-Id: I7d7fa5fddf08f4551af1bffb1b6d58f3d48eba56

The systemd unit 'TimeoutSec' value which controls the time
between sending a SIGTERM signal and a SIGKILL signal when
stopping or restarting the service has been reduced from 300
seconds to 120 seconds. This provides 2 minutes for long-lived
sessions to drain while preventing new ones from starting
before a restart or a stop.
The 'RestartSec' value which controls the time between the
service stop and start when restarting has been reduced from
150 seconds to 2 seconds to make the restart happen faster.
These values can be adjusted by using the *_init_config_overrides
variables which use the config_template task to change template
defaults.
Change-Id: I7f23a74fd644aa0d631e9b0f7d96083b645bc032

The redis cluster port was missing from the trove sample
config files. A fix to trove is proposed here [1].
[1] https://review.openstack.org/#/c/455889
Change-Id: I15268af320dc1734c51a8c17f19335105cd95e8d

The update of the apt cache and the package installation
can all be handled in a single task by providing the
package action plugin with the right parameters. This
removes an extra task to optimise execution.
The minimum Ansible version is raised to 2.2 due to a
known bug [1] in Ansible's apt module which does not
update the cache properly if the cache update and the
install are combined in a single task.
[1] https://github.com/ansible/ansible-modules-core/issues/1497
Change-Id: Ieaf67c9a776c43e9fd39c63a92642b55a49bb9bc

Move test host vars from the inventory to individual files for each
host. 'ansible_become' has been removed from localhost's vars since it
should be handled on a playbook basis.
Change-Id: I1dfcc4c98e67970ac8c302a7e3ebe7c6bdf82295

To operate properly the trove guest agent needs access to
rabbitmq and also the neutron network for trove to use must
be created and defined in the trove.conf file.
This changeset adds documentation, tasks and
configuration defaults to setup the networking for
trove.
Change-Id: Idcf87c2eef0af475c02412f03433d22d7b08643f

Testing revealed that the trove api service needs the task
manager's encryption key.
Change-Id: I4d34b7e68b69ce30ee1fb08e7495cd22fa157495

In the Ocata release, trove added support for encrypting the rpc
communication between the guest instances and the control plane.
These settings allow the user to specify installation specific
keys versus using the default keys.
Change-Id: Ie42d754d58e983a15b553ad8a399813c9a700344

This creates a specific slice which all OpenStack services will operate
from. By creating an independent slice these components will be governed
away from the system slice allowing us to better optimise resource
consumption.
See the following for more information on slices:
* https://www.freedesktop.org/software/systemd/man/systemd.slice.html
See for following for more information on resource controls:
* https://www.freedesktop.org/software/systemd/man/systemd.resource-control.html
Tools like ``systemd-cgtop`` and ``systemd-cgls`` will now give us
insight into specific processes, process groups, and resouce consumption
in ways that we've not had access to before. To enable some of this reporting
the accounting options have been added to the [Service] section of the unit
file.
Change-Id: I6dccb6eeabceea30922cbcf3a60e32e841612fd0
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>

The openstack service uris protocol variables were not being used
to set the Trove specific uris. This resulted in 'http' always being
used for the public, admin, and internal uris even when 'https' was
intended.
Change-Id: I8a1fd0a851b932f8ee853866155e968824ed5fb8

Option "auth_plugin" is deprecated, use option "auth_type" instead.
Change-Id: I77f5757a4d12c8320702bcf3685c5de7417ceda6
Implements: blueprint deprecate-auth-plugin

Package pyOpenSSL is obsoleted by python2-pyOpenSSL.
Change-Id: I75d22a59b2114c000d2ab16156ee3ee6d7198275

Change-Id: I5b9b14fe41f49106c8b1833e8b4c59ecae8ec181

Change-Id: I5a1bff26ac381b276c85b9f1a7d98ef06bf3c99a

Configure trove_conductor_workers by calculating the default value.
Similar to other services worker threads, the default value for
trove_conductor_workers is half the number of vcpus on the machine.
Also added capping of worker threads for trove.
Change-Id: If853b0e1e45b7137572df8a1123273539343c83e
Closes-Bug: #1664639 

Change-Id: I553e815d2989ce2fbd83c86f1c2b0169ece3c5e3

Change-Id: I1a5608052deb5326e63e73b47b00b8e2096b27c5

Triple-DES is a vulnerable cipher and should be disabled by default.
https://sweet32.info/
https://www.openssl.org/blog/blog/2016/08/24/sweet32/
Change-Id: Id2d22e4dafb6bf34add554a5346ead5a9552e828

Install python2 for Ubuntu 16.04 and CentOS 7 before executing test
script.
Change-Id: Ib11f17e1d980656e3b01571f154aa8ed871dfa41

Trivialfix
Change-Id: I8b61c02777910c65bd56d608bc9d9141e2a7c2aa

Change-Id: I9a49f2a8c21befb02469a02223252b52cc8fa4dd

Updating the trove-post-install to generate the
trove-guestagent.conf during the os_trove deployment.
Closes-Bug: #1658694
Change-Id: I4d8ac1b40d36b188718a02b89a1ea70fb89e7111

Fixes the ability to deploy a venv in cases where:
1) developer_mode is not enabled
2) A cached venv is not downloaded from the repo server
Additional cleanup to the developer_mode venv deployment
logic is implemented by adding a *_venv_download var
which is used to decouple developer_mode from the
cached venv extraction process so that a deployer
can force venv builds in-place (disable cached
venv usage) without enabling developer mode
constraints.
Change-Id: Id4e911dd28bf3e22fe5927b06464ee15c57f010e