openstack/openstack-ansible-os_nova - openstack-ansible-os_nova - OpenDev: Free Software Needs Free Tools

Since ansible-core 2.10 it is recommended to use modules via FQCN
In order to align with recommendation, we perform migration
by applying suggestions made by `ansible-lint --fix=fqcn`
Change-Id: I335f0f1bcdb5e5564ce3f82f44eec7d8c6ab4e0e

In order to reduce divergance with ansible-lint rules, we apply
auto-fixing of violations.
In current patch we replace all kind of truthy variables with
`true` or `false` values to align with recommendations along with
alignment of used quotes.
Change-Id: Ie1737a7f88d783e39492c704bb6805c89a199553

This change matches an earlier modification to os_neutron
Currently we symlink /etc/<service> to empty directory at pre-stage,
and filling it with config only during post_install. This means,
that policies and rootwrap filters are not working properly until
playbook execution finish. Additionally, we replace sudoers file
with new path in it, which makes current operations impossible for
the service, since rootwrap can not gain sudo privileges.
With this change we move symlinking and rootwrap steps to handlers,
which means that we will do replace configs while service is stopped.
During post_install we place all of the configs inside the venv,
which is versioned at the moment.
This way we minimise downtime of the service while performing upgrades
Closes-Bug: #2056180
Change-Id: I9c8212408c21e09895ee5805011aecb40b689a13

in cases when non-standard path to nova instances is configured with nova_system_home_folder variable there may be problems with instances spawning due to libvirt virt-aa-helper missing permission in apparmor profile, this commit resolves this
Change-Id: I3d37eb5a9635044570690370dfcbc060ff4d9e49

At the moment we don't restart services if systemd unit file is changed.
We knowingly prevent systemd_service role handlers to execute
by providing `state: started` as otherwise service will be restarted twice.
With that now we ensure that role handlers will also listen for systemd
unit changes.
Change-Id: I4273d2fbcbff3028e693e3274093c1afebdcfca2

Keystone role was never migrated to usage of haproxy-endpoints role
and included task was used instead the whole time.
With that to reduce complexity and to have unified approach, all mention
of the role and handler are removed from the code.
Change-Id: I3693ee3a9a756161324e3a79464f9650fb7a9f1a

libvirtd.socket does monitor libvirtd.service and trigger service restart
when it spot that service is down.
However in order to enable tcp and tls sockets, we need libvirt
to be stopped.
Currently race condition can happen, when we stop libvirt, but it's
started by socket before we enable tls one.
To overcome this we stop socket along with service.
Change-Id: Iacc093311036fb8d6559a0e32252579303a639ba

Since all supported distros have libvirt version >= 5.7 there's
no reason to ensure that it is true.
So we remove corresponsive code and simplify logic.
Change-Id: I281829214df8affec7774a45a3ca0405a866b5c0

Change-Id: Ic587e1a55b6f15c66e01176dac7b6acdb0abd240

To secure communications from the proxy server to the compute
nodes using VeNCrypt authentication scheme.
In a previous patch a TLS server certificate was deployed to
compute nodes, this patch makes use of this same server cert for
securing VNC sessions on compute nodes. It is recommended that
this certificate be issued by a dedicated certificate authority
solely for the VNC service, as libvirt does not currently have a
mechanism to restrict what certificates can be presented by the
proxy server. This has not been implemented to reduce complexity.
In addition the noVNC proxy needs to present a client certificate
so only approved VNC proxy servers can connect to the Compute nodes.
The PKI role has been used to create a client certificate for the
nova console nodes.
Related Nova docs:
https://docs.openstack.org/nova/latest/admin/remote-console-access.html
To help with the transition from from unencrypted VNC to VeNCrypt,
initially compute nodes auth scheme allows for both encrypted and
unencrypted sessions using the variable `nova_vencrypt_auth_scheme`, this
will be removed in future releases.
Change-Id: Iafb788f80fd401c6ce6e4576bafd06c92431bd65

Instead of using SSH to live migrate VM's use TLS as this is more
secure and SSH migrations are deprecated.
https://docs.openstack.org/nova/xena/admin/secure-live-migration-with-qemu-native-tls.html
A pre-existing PKI (Public Key Infrastruture) setup is required.
TLS live migrations require that all compute hosts can communcate
with each other on port 16514 and port range 49152 to 49261.
To enable TLS live migrations, both libvirt and QEMU require server
and client certificates, the server certicicates is used to verify
servers and the client cert is used by servers to authenticate
clients. A single cert is created by the pki role, that can be
used by both libvirt and QEMU for both client and server auth.
The client, server and CA certifcates need to installed in a
number of locations on each compute host:
* For Libvirt https://libvirt.org/tlscerts.html
* For QEMU https://github.com/libvirt/libvirt/blob/master/src/qemu/qemu.conf
Depends-On: https://review.opendev.org/c/openstack/ansible-role-pki/+/815007
Depends-On: https://review.opendev.org/c/openstack/ansible-role-pki/+/815849
Depends-On: https://review.opendev.org/c/openstack/ansible-role-pki/+/816857
Change-Id: Iddbe8764bb6d3cd3eaee122b2d5ddc02fa3f7662

As per the community goal of migrating the policy file
the format from JSON to YAML[1], we need to replace policy.json to
policy.yaml and remove deprecated policy.json.
config_template has been choosen instead of the copy, since it can
properly handle content that has been lookuped.
We make a separate task not to restart service when it's not needed.
[1] https://governance.openstack.org/tc/goals/selected/wallaby/migrate-policy-format-from-json-to-yaml.html
Change-Id: I3ff3ce4f150854ece690f1bcbd7183f511fccf2e

Since libvirt 5.7 we should not use traditional mode. Instead systemd
mode should be choosen. Sockets are provided by libvirt package.
We just need to conditionally enable or disable them.
As addition we changed nova_libvirtd_listen_tls to 0 by default
because listen_tls requires certificate to be provided which is not
the case in the current role state. So we also fix behaviour of the role
when nova_libvirtd_listen_tls is 1, supposing that deployer has manually
distributed certificates across compute hosts.
Change-Id: Id73cb67de26c305908d0245551fa57a7e6448784
Closes-Bug: #1903846 

The files and templates we carry are almost always in a state of
maintenance. The upstream services are maintaining these files and
there's really no reason we need to carry duplicate copies of them. This
change removes all of the files we expect to get from the upstream
service. while the focus of this change is to remove configuration file
maintenance burdens it also allows the role to execute faster.
 * Source installs have the configuration files within the venv at
 "<<VENV_PATH>>/etc/<<SERVICE_NAME>>". The role will now link the
 default configuration path to this directory. When the service is
 upgraded the link will move to the new venv path.
 * Distro installs package all of the required configuration files.
To maintain our current capabilities to override configuration the
role will fetch files from the disk whenever an override is provided and
then push the fetched file back to the target using `config_template`.
Depends-On: https://review.openstack.org/636162
Change-Id: Ib7d8039513bc2581cf7bc0e2e73aa8ab5da82235
Signed-off-by: Kevin Carter <kevin@cloudnull.com>

In order to radically simplify how we prepare the service
venvs, we use a common role to do the wheel builds and the
venv preparation. This makes the process far simpler to
understand, because the role does its own building and
installing. It also reduces the code maintenance burden,
because instead of duplicating the build processes in the
repo_build role and the service role - we only have it all
done in a single place.
We also change the role venv tag var to use the integrated
build's common venv tag so that we can remove the role's
venv tag in group_vars in the integrated build. This reduces
memory consumption and also reduces the duplication.
This is by no means the final stop in the simplification
process, but it is a step forward. The will be work to follow
which:
1. Replaces 'developer mode' with an equivalent mechanism
 that uses the common role and is simpler to understand.
 We will also simplify the provisioning of pip install
 arguments when doing this.
2. Simplifies the installation of optional pip packages.
 Right now it's more complicated than it needs to be due
 to us needing to keep the py_pkgs plugin working in the
 integrated build.
3. Deduplicates the distro package installs. Right now the
 role installs the distro packages twice - just before
 building the venv, and during the python_venv_build role
 execution.
Depends-On: https://review.openstack.org/598957
Change-Id: I182bde29c049a97bc2b55193aee0b5b3d8532916
Implements: blueprint python-build-install-simplification
Signed-off-by: Jesse Pretorius <jesse.pretorius@rackspace.co.uk>

In order to reduce the packages required to pip install on to the hosts,
we allow the service setup to be delegated to a specific host, defaulting
to the deploy host. We also switch as many tasks as possible to using the
built-in Ansible modules which make use of the shade library.
The 'virtualenv' package is now installed appropriately by the openstack_hosts
role, so there's no need to install it any more. The 'httplib2' package is a
legacy Ansible requirement for the get_url/get_uri module which is no longer
needed. The keystone client library is not required any more now that we're
using the upstream modules. As there are no required packages left, the task
to install them is also removed.
Unfortunately we need to use the openstack client to wait for a compute host
to register, so we add it into the nova venv and implement a change in the
way we do the wait so that openrc/clouds.yaml is only implemented on a single
compute host and the wait task is executed there.
Depends-On: https://review.openstack.org/582359
Change-Id: I702480a5188a583a03f66bb39609f7d25a996e4a

Change-Id: I993181a2d352a83d25bcddf5b39f4be016f0018d

With the more recent versions of ansible, we should now use
"is" instead of the "|" sign for the tests.
This should fix it.
Change-Id: If3e4366c22e900557e4730a7e8838f55ffe30ecc

All operating systems supported by the role have systemd and these
conditionals are no longer needed.
Change-Id: I35500f7eec993b2bcdb245a995a05cacf2c596f8

Also, use nova_services dict to get group name
Change-Id: Iec090937b0213120854847eebf099df4ffc03528

Based on conversation on an ansible issue[1], I implemented
a LB orchestration role[2] similar to the POC here[3].
This will allow external loadbalancer management roles to hook
into a universal notify listener "Manage LB" to perform before/
after endpoint management actions when the service is being
restarted.
[1]: https://github.com/ansible/ansible/issues/27813
[2]: https://github.com/Logan2211/ansible-haproxy-endpoints
[3]: https://github.com/Logan2211/tmp-ansible-27813
Change-Id: I5aecc26606f41bc6b27fbe9a5f600914a88ff2c7

The placement service is already setup to use UWsgi, we need
to move the other Nova services to follow suit as part of our community
goal for Pike.
Additionally, we need to clean up the nginx configuration as we are
moving away from fronting uWSGI with nginx inside the roles.
Depends-On: Ib66b9709fb88205eaf3f133c87357a4dbbdde5ae
Change-Id: If6c30e00c1c753692c970457b75e3ae7f5cc066c
Implements: blueprint goal-deploy-api-in-wsgi

In order to cater for artifact-based installed, and
rolling upgrades, this patch implements a set of local
facts to inform the online migrations task.
The 'nova_all_software_updated' variable will be
set by the playbook on each run to ensure that the
online migrations only happen once all venvs are
homogenous. This ensures that the playbook can be
executed in a serialised fashion and the data will
not be corrupted.
The ``upgrade_levels`` setting for ``compute`` is set
to ``auto`` to ensure that a mixed RPC version
deployment can operate properly when doing a rolling
upgrade as suggested by [1].
Additional changes are made to improve the role's
ability to be executed using serialised playbooks.
Finally, the nova-manage command references to the
config file location have been removed as they refer
to the default location.
[1] https://docs.openstack.org/developer/nova/upgrade.html
Change-Id: I08e5a7f0ce526b11aa52c35ee29c458954a5f22d

The policy.json file is currently read continually by the
services and is not only read on service start. We therefore
cannot template directly to the file read by the service
(if the service is already running) because the new policies
may not be valid until the service restarts. This is
particularly important during a major upgrade. We therefore
only put the policy file in place after the service restart.
This patch also tidies up the handlers and some of the install
tasks to simplify them and reduce the tasks/code a little.
Change-Id: Icba9df7be6012576eca0afb040a6953809cc9a5f

Change-Id: I3f9cde33af0a5f2aaeb0aedc964cb03a40ccbb9f

On CentOS the default is to have service disabled, this ensure nginx is enabled.
Closes-bug: 1681533
Change-Id: I98018fca9c277248b77b60081ea560c012b370af

During an upgrade new service files are added, but systemd is not
reloaded during restart of nova services to pick up these file
changes. This performs a daemon-reload when restarting nova
services.
Change-Id: I98b3f66429ee045f052ad491847cf82d2f5d4efc
Closes-Bug: #1673889 

We don't need to restart nginx - we can instead just reload the
service. Additionally, as more services move to use nginx frontend, it
would be bad to restart all NGinx services at the same time.
For now we need to investigate the impact of reloads in UWsgi before
moving over to a "reload" on UWsgi.
Change-Id: I60e370e784a1ff3a0f5bf8551be804bf05d8bb43

Use specific ordering for nova service restarts.
Change-Id: I29e17c09c6aa1b626aead8e4916cc89604a371d6

A race condition is caused when nova-compute is started for the
first time because it takes a period of time for nova-compute
to spin up, register itself with nova API, and become available for
cell enrollment.
Prior to this there was no wait condition when nova-compute
restarts occurred, so the first time nova-compute started, often
the compute service was not registered in the database and available
for cell enrollment when the enrollment tasks ran.
Change-Id: I510f0a957f53d15affa1fc23f809abff52208438

This patch adds Nova requirements for Ocata:
* Nova Placement API running as uwsgi with Nginx.
* cell_v2 setup for cell0 and cell1
* All required settings for these services with sane defaults
It fixes up some ordering for DB operations:
* online_db_migrations should only happen after a full upgrade.
* Cell setup needs to happen after api_db sync but before db sync.
* Discover_hosts for cell_v2 needs to happen after compute is restarted
This adds functionality to allow uwsgi apps in the init scripts:
* Allowing the "--log" line to be adjusted.
* Setting the condition value so that only enabled services are deployed
* Fixes a bug for program_override which mean this value was never being
used.
Depends-On: I082f37bb3ce61a900e06a58f21c7882f83671355
Change-Id: I282d25988377d18257b708859f89a7ae4260ac07

Using ansible systemd module to daemon reload and service reload is the solution for the future.
Change-Id: I3f9142357379a548b1e1f4190e61157596f750fa
Co-Authored-By: Jean-Philippe Evrard <Jean-Philippe.Evrard@rackspace.co.uk>

Change-Id: Ib0747040d6b53cbb7aec67cfaceae6cc1efb1abc
Implements: blueprint trusty-removal

* only kvm host are supported right now.
Depends-On: Iff4a5999be0263a2c1843d7ca29843468cbc0ccc
Depends-On: I78fb85d44b5b0e1643bd07af3e15462c02041c89
Change-Id: Ie05c243daa7d2d46b5e8779371a363d95cc990e9

Preparing this role for the ansible-lint version bump
Change-Id: Ia5d254d43f9541c82c700080aafee276dafad0a7

Change the 'nova_service_names' from a list to a dictionary mapping
of services, groups that install those services. This brings the
method into line with that used in the os_neutron role in order to
implement a more standardised method.
The init tasks have been updated to run once and loop through this
mapping rather than being included multiple times and re-run against
each host. This may potentially reduce role run times.
Currently the reload of upstart/systemd scripts may not happen if
only one script changes as the task uses a loop with only one result
register. This patch implements handlers to reload upstart/systemd
scripts to ensure that this happens when any one of the scripts
change.
The handler to reload the services now only tries to restart the
service if the host is in the group for the service according to the
service group mapping. This allows us to ensure that handler
failures are no longer ignored and that no execution time is wasted
trying to restart services which do not exist on the host.
Finally:
- Common variables shared by each service's template files have
 been updated to use the service namespaced variables.
- Unused handlers have been removed.
- Unused variables have been removed.
Change-Id: I53fb0ab1cc5762e3559d4ee2635d4cca532df7e3

When executing the role with Ansible 2.1, the following
deprecation warning is issued in the output for some tasks.
[DEPRECATION WARNING]: Using bare variables is deprecated.
This patch addresses the tasks to fix the behaviour appropriately.
Change-Id: I7ef4e446d6fc509420d5b297378f4fa91a519fc8

This change implements the blueprint to convert all roles and plays into
a more generic setup, following upstream ansible best practices.
Items Changed:
* All tasks have tags.
* All roles use namespaced variables.
* All redundant tasks within a given play and role have been removed.
* All of the repetitive plays have been removed in-favor of a more
 simplistic approach. This change duplicates code within the roles but
 ensures that the roles only ever run within their own scope.
* All roles have been built using an ansible galaxy syntax.
* The `*requirement.txt` files have been reformatted follow upstream
 Openstack practices.
* Dynamically generated inventory is now more organized, this should assist
 anyone who may want or need to dive into the JSON blob that is created.
 In the inventory a properties field is used for items that customize containers
 within the inventory.
* The environment map has been modified to support additional host groups to
 enable the seperation of infrastructure pieces. While the old infra_hosts group
 will still work this change allows for groups to be divided up into seperate
 chunks; eg: deployment of a swift only stack.
* The LXC logic now exists within the plays.
* etc/openstack_deploy/user_variables.yml has all password/token
 variables extracted into the separate file
 etc/openstack_deploy/user_secrets.yml in order to allow seperate
 security settings on that file.
Items Excised:
* All of the roles have had the LXC logic removed from within them which
 should allow roles to be consumed outside of the `os-ansible-deployment`
 reference architecture.
Note:
* the directory rpc_deployment still exists and is presently pointed at plays
 containing a deprecation warning instructing the user to move to the standard
 playbooks directory.
* While all of the rackspace specific components and variables have been removed
 and or were refactored the repository still relies on an upstream mirror of
 Openstack built python files and container images. This upstream mirror is hosted
 at rackspace at "http://rpc-repo.rackspace.com" though this is
 not locked to and or tied to rackspace specific installations. This repository
 contains all of the needed code to create and/or clone your own mirror.
DocImpact
Co-Authored-By: Jesse Pretorius <jesse.pretorius@rackspace.co.uk>
Closes-Bug: #1403676
Implements: blueprint galaxy-roles
Change-Id: I03df3328b7655f0cc9e43ba83b02623d038d214e