openstack/openstack-ansible-os_nova - openstack-ansible-os_nova - OpenDev: Free Software Needs Free Tools

UEFI booting support was removed on Debian/Ubuntu was broken by the
removal of ovmf and qemu-efi package installs during a package list
cleanup[1].
It looks like suse/redhat did not reecive the same cleanup effects,
as the UEFI bootloader packages are still installed there, so no
fix is needed on those OS families.
[1] https://review.opendev.org/#/c/662585/
Change-Id: I8ce72e4a0c754a745b1abf42030e41d6084f1361

This removes superfluous packages and moves others to align with
what vars/{debian,suse}.yml does.
Change-Id: I88bc42eaa30d59250003339241380e3e50b64e3b

Unfortunatelly CentOS 7 do not have libvirt and guestfs libraries
for py3, so isntead of symlinking them, we have to install devel package
and build inside venv.
Change-Id: Ie678e44c2369347cf0816cbcc4269de8eba7b963

Return required CentOS packages to build corret venv and
to have virsh CLI tool installed.
Change-Id: I9656f9cf65f371a9640c387b5e5d815bf2062a5f

Had issues where QEMU packages were not in sync and got an error:
... Note: only modules from the same build can be loaded. ...
The qemu package no longer lists dependencies on qemu-system, qemu-user,
qemu-utils in apt show.
Qemu-utils is recommended by other packages but I am unsure as to whether
this will guarantee an upgrade in all circumstances.
While qemu-system does list dependencies on the arm, mips, ppc... packages
they are unversioned and therefore do not keep in step.
Change-Id: I8ea4681a58a54bdc6a24ff053b11dfee5cc7af59

In openSUSE 15.x genisoimage was replaced by mkisofs. Although
some projects have added autodetection, nova needs to be explicitly
told.
Also remove deprecated qemu-kvm (wrapper around qemu-system-x86_64).
Change-Id: If0cbd60790935141d52465abe61b40058f1829b1

In [1] many spurious packages were removed, and libxml2 was probably
a dependancy of a removed package. Add libxml2 as an explicit
requirement.
[1] https://review.opendev.org/662585
Change-Id: I6160b53ccfb161b8b8a161971795fd4962d86201

In [1] this package was moved to the 'devel' list mistakenly, it is
required on the compute host always.
In addition, due to infra. issues with Debian, this patch marks the
jobs as non-voting[2].
[1] https://review.opendev.org/662585
[2] https://review.opendev.org/686396
Change-Id: Iaa08e18d9da1a55a27241e4b5629eccb7969e2e0

- Created nova_compute_packages for compute common packages
All operating systems:
- bridge-utils because os-vif uses iproute2
- postgres headers because OpenStack doesn't support it
- python libraries as they are already installed
- python-openstackclient as it's not used in run-time
Debian removals:
- qemu-kvm because it's just a legacy wrapper for qemu-system-x86
- qemu-utils recommended by qemu-system-x86
- qemu-system-{arm,mips,misc,ppc,sparc,x86} required by qemu-system
- qemu-{system,user,utils} required by qemu
- vlan as nova no longer uses vconfig
- python3-{guestfs,libvirt} as python3 support is not there yet
RedHat removals:
- libvirt-daemon-config-{nwfilter,qemu} required by libvirt-daemon-kvm
- qemu-img-ev required by qemu-kvm-ev
- iscsi-initiator-utils required by libvirt-daemon-driver-storage-iscsi
SUSE removals:
- open-iscsi required by libvirt-daemon-driver-storage-iscsi
- libvirt-daemon-driver-storage-core required by libvirt-daemon-driver-storage-*
- device-mapper required by libvirt-daemon-driver-storage-disk
- nfs-utils required by libvirt-daemon-driver-storage-core
- qemu-block-{curl,dmg,ssh} as that block storage driver is not used
Change-Id: If81939f95042460dd83234160f44426692189376

The RedHat-based operating systems such as CentOS have a pretty
stable list of packages, therefore, we don't need to pin it by
version and we can instead move to a much more generic redhat.yml
which will support a bigger range of systems.
Change-Id: I5eaa3cb6b3bcae96ff8ebebb368eb4a6fdfd1bae

For distro package based deployments, install the packaged nova console
proxies and console clients packages instead of git repos.
For debian/ubuntu installs, use the console client packages directly so
that the service configs of console proxy services are managed by OSA.
Change-Id: I27577ea263d38555e73ead078526cff1c3114b5a

Change-Id: Ibc5f8f9f73905c4c04a254e589cd13fe2254521b

Move service to use uWSGI role instead of iternal task for uwsgi
deployment. This aims to ease the maintenance of uWSGI and speedup
metal deployments as the same uwsgi environment will be used
across all services.
Depends-On: https://review.opendev.org/678200/
Depends-On: https://review.opendev.org/678055/
Change-Id: If5aeeefb93c8ba3c1368970de61ea1300218f637

This driver has been retired [1] and tests are now failing becasue the
nova-lxd repo master branch is now empty.
[1] https://review.opendev.org/#/c/672283/
Change-Id: I9906ede54f6b41972a03bfa1d39ba5f99c6235ed

The qemu-efi package will pull in the x86_64 and aarch64 uefi firmware
ensuring both architectures are supported on debian/ubuntu
Change-Id: Id2f266def07e0ba6967746738efd07cff628a19a

This patch aims to migrate service from usage of regular syslog files
to journald. We also disable uwsgi logging, since it dublicates
requests that are logged by service itself.
Change-Id: If1eda4d803661a0b924941aecd1867302391a5f4

Compute hosts require netcat in order to perform live migrations.
Closes-Bug: #1833644
Change-Id: Ie0caab0b10e878722f1804f67808f1d83d504c59

Depends-On: https://review.opendev.org/666058
Depends-On: https://review.opendev.org/666298
Change-Id: I0ac8593477cd127c33594fd8f1c209f08b138794

The debian packages for individual nova services provide service
configs which automatically start after install, leading to all nova
services running on each compute host. Instead only install the
python3-nova library package and rely on the service files OSA manages.
The nova_service_distro_packages can be cleaned up further since most
packages are provided as dependencies. uwsgi packages are only required
on hosts running nova-api-os-compute or nova-api-metadata, so an
additional key, 'nova_api' has been added for these hosts under
'nova_service_extra_distro_packages'.
Beginning in the Stein release, Ubuntu distro packages are now using
Python3. This requires additionally installing and using the uwsgi python3
plugin.
Change-Id: Id0c7b57f0119c5213abdf3505e4644680a78d55e

The PowerVM driver has been untested for a few years as well
as broken due to using the wrong driver as well. It's not possible
to test it within our infrastructure neither has there been any
contributions to improve/fix it.
This patch drops the support for it.
Change-Id: Iea84648c7ff65b5a4d8b97957fb92716578f714d

In order to boot UEFI instances successfully, the OVMF image
must be installed and located at the path that is hardcoded
inside Nova.
This patch takes care of installing the OVMF images and sets
up links to point towards the hardcoded path for operating
systems that don't install at the expected path.
Change-Id: I62f4aa26ffd0d2f3ba4aa19fbdb72578ef0223e4
Closes-Bug: #1825386
Closes-Bug: #1831279 

Since we no longer install python-keystoneclient[1], there is no
longer a need to install python-httplib2.
Change-Id: I7ef1dac07656bec527b631fdcb621e66b769ccd2

Placement is now a separate service and has been removed from nova[1]
and implemented as a separate service in OSA[2]. As such, the nova
in-tree placement API is now disabled and can be removed.
[1] https://review.opendev.org/#/c/618215/
[2] https://review.opendev.org/#/c/656891/
Depends-On: I01df855d8b9255d24efe16ab053ffa491fda351f
Depends-On: I25ce79b9e3492cb075c50beed90ea2b78055e474
Change-Id: I2f5d4c2f1fc1418bc4f4c4baa1ce3c90b88208fc

As far as we are linking libvirt-python inside venv, we do not need
libvirt-devel anymore and it might not be installed.
Change-Id: I289e04f5fe4f441e82cdd236b2ff865476a4f8d2

This patch adds support for this role to be able to deploy on
Debian Stretch.
Change-Id: I1058238e2467cc257ec48ba84fb89995079e9be7
Needed-By: I9a92b73c419a0dc1cca40dacfef75de61a61db94

The nova-consoleauth service was deprecated in Rocky and unless
you have the workarounds section enabled (which we don't), it will
not be used.
This patch drops it and adds tasks to turn it off and remove alll
the unit files.
Change-Id: I14b4870fd652d33a22c0d8480d847570ace8a3a3

The variable is useless and it only breaks things. The js-jquery
packages don't even exist on RedHat. All those dependencies are
pulled in and vendored in anyways, the Python ones are built, so
noVNC can run with no problem without any of those.
Change-Id: I59ad2014ed001a318ae4b865f0e218b6f8e18d92

The variables nova_developer_mode and nova_venv_download
no longer carry any meaning. This review changes nova to
do the equivalent of what developer_mode was all the time,
meaning that it always builds the venv and never requires
the repo server, but it will use a repo server when available.
We also change include_tasks to import_tasks and include_role
to import_role so that the tags in the python_venv_build role
will work.
Change-Id: I1a1ac5948f3bd647eb0ee5e10e9e7ae06b740bb7

The files and templates we carry are almost always in a state of
maintenance. The upstream services are maintaining these files and
there's really no reason we need to carry duplicate copies of them. This
change removes all of the files we expect to get from the upstream
service. while the focus of this change is to remove configuration file
maintenance burdens it also allows the role to execute faster.
 * Source installs have the configuration files within the venv at
 "<<VENV_PATH>>/etc/<<SERVICE_NAME>>". The role will now link the
 default configuration path to this directory. When the service is
 upgraded the link will move to the new venv path.
 * Distro installs package all of the required configuration files.
To maintain our current capabilities to override configuration the
role will fetch files from the disk whenever an override is provided and
then push the fetched file back to the target using `config_template`.
Depends-On: https://review.openstack.org/636162
Change-Id: Ib7d8039513bc2581cf7bc0e2e73aa8ab5da82235
Signed-off-by: Kevin Carter <kevin@cloudnull.com>

This patch removes the conditional inclusion of the notification
section of the service configuration. This ensures that oslo.messaging
notifications use the correct transport for deployments that have
separate rpc and notify messaging backends. For example, if the
transport_url is not provided in the notification section of the
service configuration, the transport_url specified in the default
section will be used instead.
Change-Id: Ia66fe9a4c5c18f31fa50e06017389489be8c66a4

Change-Id: I27e4856e1f7a6cd48fab146c308f9dab850feb2b

Currently the devel packages are installed everywhere,
but they only need to be where the wheels are built.
Also, there is already a task to install the packages
needed on the target hosts when installing - so we do
not need to give the same list to the venv install role
because they will already have been installed.
Change-Id: Iab434e8c91463d9b0c34eba23b0550fa2af46481

Nova requires this package at runtime and currently assumes
that it is present in the environment. This patch ensures that
assumption is correct.
Change-Id: Ic74ffffb0e3f06ed1392d4006f1a511f66b4a30b

The distro install package lists included virtual packages to install
everything nova, packages for deprecated services (cells v1,
nova-network, nova-volume), and packages specific
to hypervisors currently unsupported by OSA (xen).
nova-compute and serialconsole specific packages been moved out to
nova_service_extra_distro_packages.
Depends-On: https://review.openstack.org/612614
Change-Id: I8fb5e17598b24c2399fbff7ac29957b73042329d

This support allows deployers to easily use NFS and automatically
have mountpoints during deployment.
Change-Id: Ifb15cb7815487f047c0249f340a79c67971f5411

This also removes python-keystonemiddleware from the distro packages
since anything that needs it should pull it as a dependency.
Depends-On: https://review.openstack.org/604473
Depends-On: https://review.openstack.org/604286
Change-Id: I8398b70fc7e371d163bed8bed8fc11f50c521e28

Change-Id: I9f593b04024fc0bbd6d57d4d959044f0dcffb01c

To make the transition between versions easier,
we rename the vars file. This also resolves
issues when meta-dependent role inclusions do
not pick up the correct file when using the
include_vars task with multiple search paths.
Depends-On: https://review.openstack.org/602924
Depends-On: https://review.openstack.org/602920
Depends-On: https://review.openstack.org/602926
Depends-On: https://review.openstack.org/603056
Depends-On: https://review.openstack.org/603205
Change-Id: Idc082d44f456ca506300de4fab138ee9f8da6181

Now that we no longer use our own keystone module, but
instead make use of the ansible runtime venv's shade
library and upstream ansible modules, we can eliminate
this package/library being installed on the host.
Change-Id: I7c885974295fa6df3005b68210fc9e3442296ac1

These vars work for both Xenial and Bionic
Change-Id: I8ab82f2b063621bae904510b0648fabc2b90d7c9

This packages was added in [1] without an explanation of why the
debug symbols are required. The symbols package is not built for
bionic [2] so we remove it allowing the common vars file to work
for both Xenial and Bionic releases.
[1] e771983f2c
[2] http://archive.ubuntu.com/ubuntu/pool/main/d/dosfstools/dosfstools_4.1-1.dsc
Change-Id: I283628e6d42c6708934975d272712353472e066b

Currently everything is wired in the venv to work on python2.
If the virtualenv was built during repo build with python3, the
linking of the library will fail, as the lib folder for python2
doesn't exist.
This should solve the issue by ensuring the existing lib folder
of the env is used.
A problem arises then: Different distros organise packages
differently, in different locations. This makes the approach
compatible with multi-distros and multiple python versions.
Change-Id: I02b296089d5351d68bb14e87e458a4f5d18a4776

Distributions provide packages for the OpenStack services so we add
support for using these instead of the pip ones.
Depends-On: https://review.openstack.org/#/c/579151/
Depends-On: https://review.openstack.org/#/c/579071/
Change-Id: Iff373b5402c664206b940b8e95a0b8845050213b
Implements: blueprint openstack-distribution-packages