openstack/openstack-ansible-os_nova - openstack-ansible-os_nova - OpenDev: Free Software Needs Free Tools

Based on https://patchwork.kernel.org/patch/42605/.
Change-Id: I51f6cc953e25b632853996ad18c274063e12d441

This driver has been retired [1] and tests are now failing becasue the
nova-lxd repo master branch is now empty.
[1] https://review.opendev.org/#/c/672283/
Change-Id: I9906ede54f6b41972a03bfa1d39ba5f99c6235ed

The files and templates we carry are almost always in a state of
maintenance. The upstream services are maintaining these files and
there's really no reason we need to carry duplicate copies of them. This
change removes all of the files we expect to get from the upstream
service. while the focus of this change is to remove configuration file
maintenance burdens it also allows the role to execute faster.
 * Source installs have the configuration files within the venv at
 "<<VENV_PATH>>/etc/<<SERVICE_NAME>>". The role will now link the
 default configuration path to this directory. When the service is
 upgraded the link will move to the new venv path.
 * Distro installs package all of the required configuration files.
To maintain our current capabilities to override configuration the
role will fetch files from the disk whenever an override is provided and
then push the fetched file back to the target using `config_template`.
Depends-On: https://review.openstack.org/636162
Change-Id: Ib7d8039513bc2581cf7bc0e2e73aa8ab5da82235
Signed-off-by: Kevin Carter <kevin@cloudnull.com>

This patch aims to provide the ability for the user
to enable nested kvm virtualization in a kvm compute node
through nova_nested_virt_enabled variable, which its defaults is False.
Change-Id: I64417221fb3d74453d979b7198a0e916e7f4dd23

We do not have a maintainer at the moment for SELinux and hopefully
we will adopt the upstream openstack-selinux package, but for now
in order to let deploys in environments where SELinux is set to
permissive work, we'll have to remove these bits.
This change can be reverted whenever we have a maintainer that's
available to do the work required.
Change-Id: I968937bcc7730faf75750971f8c72b0ea037cbd9

`/etc/kernel/postinst.d/nova-kernel-permissions.sh` (introduced to fix Bug #1507915) is supposed to make newly installed kernels readable to the nova user, as kernels on an Ubuntu system are otherwise only readable to the root user [0].
This script didn't work for a few reasons:
- It never ran, because scripts in `/etc/kernel/postinst.d` are called by `run-parts`, and run-parts skips any script with a period in the name [1].
- Its shebang was missing its bang
- If installation of the same kernel is installed more than once (e.g. reinstallation), `dpkg-statoverride` (and the whole kernel installation) would exit with error, complaining about an override already existing [2].
Fixed with these changes respectively:
- Renamed script to remove the period
- Fixed typo in shebang
- Added `--force` flag to `dpkg-statoverride`
[0] https://bugs.launchpad.net/ubuntu/+source/linux/+bug/759725
[1] https://bugs.launchpad.net/ubuntu/+source/debianutils/+bug/38022
[2] https://bugs.launchpad.net/openstack-manuals/+bug/1275080
Change-Id: I0e130e3c3ecf2171dbdc0e9a809f8066c30d4bc9
Closes-Bug: 1763479

Depends-On: https://review.openstack.org/#/c/559190
Change-Id: Id7ea7c92a473d1c88de795b512cabfde849f1a44

Installing openstack-selinux brings in a *ton* of policies and the
vast majority do not apply to an OpenStack-Ansible deployment. We
can bring in the individual policies that we need in each role.
The openstack-selinux package takes 2-3 minutes to install and it
brings in container-selinux (which is mainly for Docker) and that
adds another 30-45 seconds.
The patch also adds some required SELinux policies for virtlogd to
work and for the non-KVM qemu gate jobs to function properly.
Closes-Bug: 1746602
Change-Id: Ib79cd5f8ebd9cb535c8051a29126262ede2b17d3

Change-Id: I7f256ea7f0e2068b42fa76b0e7c82b9f87f29647

This patch updates the role static files in tree
Change-Id: Ie840b3e06a8c6be6a0afcac48fb831ff437af9b2

This also updated the nova-lxd filters.
Change-Id: I9674b3c159adf4a8caa39a98d9d6090a6e2ce754
Closes-Bug: #1716411 

Change-Id: Idffbe4347cb93880e28803304af99f65fcf9f808

Change-Id: I4eb24ec16146ebad5927459201742569164e5070

Change-Id: I636adafd66fb8a8a0cb551d5757a2728c5e8bb6b

Change-Id: I776f908faff27c97f592e6a4880209d8cd6d0f90

Problem: libvirt password/key injection uses libguestfs to mount the
guest filesystem. libguestfs uses a supermin appliance, and in order to
create this appliance, libguestfs (running as nova user) must read the
host's kernel. Unfortunately, Ubuntu sets file permissions which make
compressed kernels non-readable to non-root users, and this breaks
libvirt password/key injection on compute hosts running Ubuntu.
Solution: When compute hosts are running Ubuntu AND the deployer has
enabled libvirt password or SSH key injection, do the following:
- Run `dpkg-statoverride` to set file permissions on compressed
 kernel (/boot/vmlinuz-*), readable to group 'nova'
- Install a script which does same for each new kernel installed via
 system updates in the future
Related-Bug: #1507915
Change-Id: Ic96b69bb80ce11001b2ee5d63324a12b0f68456d

Change-Id: I30fced77382e55dd8f2ceabefc01ea4a72758670

The existing nova_post_install.yml does not retrieve the rootwrap
filter file for nova-lxd. This change adds the rootwrap file from
the nova-lxd repository.
Change-Id: I0193f150fa802214903ec4532bc1b119d5b84cfe
Closes-Bug: #1656070 

On ppc64le KVM hypervisor SMT is enabled by default.
Adding the task to disable SMT to allow launching the instance in
running state.
Setting the default console type as noVNC for ppc64 arch.
Change-Id: I119455a499255725dd616eb488a1c67f828d925a

Change-Id: Ib724f5fb3062f207fce1e669c614a833beb27ada

The baremetal rootwrap filters were removed from nova over a year ago in
change I952e484cf0b7b6526dced74769ed00a1b7541711. Remove them from this
repository as well. Also update the 'Copy nova rootwrap filter config'
task to handle looking up rootwrap filter files using 'with_fileglob' to
avoid having to maintain the task with each addition or removal of these
files.
Change-Id: I9c7df5d29f9557fbc467402166cec7546a3e79c7

Change-Id: If49ea9ce30e081af2d6d662361a7d35d3ab5a60b

Change-Id: I5912e586c9d620369e84b8d1811d46dfa1677047

Change-Id: Ib04b0a0d62b5c012db2eab1e64497f2dbfbf2691

This updates the repository SHA's to use stable/mitaka where
available and updated SHA's where not.
It also updates all paste, policy and rootwrap configurations
to match the current contents found in stable/mitaka.
Change-Id: I51a8ade20150192ce3a8e3f0dfbf59d389a895e0

This patch does the following:
- updates the Master SHAs for new development work.
- includes updates to policy, paste and rootwrap files as required
- moves the Aodh repository to openstack_services as it now has
 implemented a stable branch
- Updated the keystone-wsgi file as it was still running the code from
 liberty
- add 2 package requirements to keystone which must be present for the
 new wsgi file.
- updates tempest.conf.j2 to replace ssh_auth_method with auth_method,
 and change auth_method to 'keypair' (configured is no longer an
 a valid option)
Change-Id: I933c24c03518865d9d40519dafb2ba46769a5453
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>

The libvirtd.conf file has never been used, and there already
have libvirtd.conf.j2 template in templates directory.
Change-Id: Ie297db1d7974f74abd01f4096ee900adb74199fa

This patch includes the following updates based on the updated
source in Nova's Liberty release:
 - api-paste.ini
 - policy.json
 - rootwrap.d/compute.filters
 - rootwrap.d/network.filters
The Nova S3 and v3 API's have been removed in Liberty, so all
related variables and configuration file entries have been
removed.
The Nova EC2 API is deprecated in Liberty. All related variables in
OpenStack-Ansible and configuration files have been removed as all
deployers are recommended to make use of the actively developed
replacement: https://github.com/stackforge/ec2-api
The Nova v2 and v1.1 API's are enabled using the upstream default
compatibility layer. Neither of these versions will be registered in
the service catalog.
The default API version is set to v2.1. For new environments, no
other API versions are registered in the service catalog.
The following variables have been removed:
 - S3 API
 - nova_s3_service_name
 - nova_s3_service_type
 - nova_s3_service_proto
 - nova_s3_service_publicuri_proto
 - nova_s3_service_adminuri_proto
 - nova_s3_service_internaluri_proto
 - nova_s3_service_port
 - nova_s3_service_description
 - nova_s3_service_publicuri
 - nova_s3_service_publicurl
 - nova_s3_service_adminuri
 - nova_s3_service_adminurl
 - nova_s3_service_internaluri
 - nova_s3_service_internalurl
 - nova_s3_program_name
 - nova_s3_deprecated_but_enabled
 - EC2 API
 - nova_ec2_service_name
 - nova_ec2_service_type
 - nova_ec2_service_proto
 - nova_ec2_service_publicuri_proto
 - nova_ec2_service_adminuri_proto
 - nova_ec2_service_internaluri_proto
 - nova_ec2_service_port
 - nova_ec2_service_description
 - nova_ec2_service_publicuri
 - nova_ec2_service_publicurl
 - nova_ec2_service_adminuri
 - nova_ec2_service_adminurl
 - nova_ec2_service_internaluri
 - nova_ec2_service_internalurl
 - nova_ec2_program_name
 - nova_ec2_deprecated_but_enabled
 - v3 API
 - nova_v3_service_name
 - nova_v3_service_type
 - nova_v3_service_proto
 - nova_v3_service_publicuri_proto
 - nova_v3_service_adminuri_proto
 - nova_v3_service_internaluri_proto
 - nova_v3_service_port
 - nova_v3_service_description
 - nova_v3_service_publicuri
 - nova_v3_service_publicurl
 - nova_v3_service_adminuri
 - nova_v3_service_adminurl
 - nova_v3_service_internaluri
 - nova_v3_service_internalurl
 - nova_v3_deprecated_but_enabled
 - v2.1 API
 - nova_v21_service_name -> nova_service_name
 - nova_v21_service_type -> nova_service_type
 - nova_v21_service_proto -> nova_service_proto
 - nova_v21_service_publicuri_proto -> nova_service_publicuri_proto
 - nova_v21_service_adminuri_proto -> nova_service_adminuri_proto
 - nova_v21_service_internaluri_proto -> nova_service_internaluri_proto
 - nova_v21_service_port -> nova_service_port
 - nova_v21_service_description -> nova_service_description
 - nova_v21_service_publicuri -> nova_service_publicuri
 - nova_v21_service_publicurl -> nova_service_publicurl
 - nova_v21_service_adminuri -> nova_service_adminuri
 - nova_v21_service_adminurl -> nova_service_adminurl
 - nova_v21_service_internaluri -> nova_service_internaluri
 - nova_v21_service_internalurl -> nova_service_internalurl
 - nova_v21_enabled
DocImpact
UpgradeImpact
Implements: blueprint liberty-release
Change-Id: Ie5a42059c10e7fd0bfc4dba8d87dea3f32db968e

Change-Id: Ib3f95497549d8d5f341a5caed02d703570a2b6c8

The change modifies the nova template tasks such that it's now
using the config_template action plugin. This change will make so that
config files can be dynamically updated, by a deployer, at run time,
without requiring the need to modify the in tree templates or defaults.
Partially implements: blueprint tunable-openstack-configuration
Change-Id: I9842ed3fcb2cc4aa379a582359b1ca5d0747f714

This PR replaces the copy_update module with a proper Ansible action
plugin. This change allows for dynamic updates to configuration files
that are ini, json, and yaml.
All of the policy files have been moved to the role templates directories
and the task syntax has been updated to facilitate the new action plugin.
An entry has been added to the ansible.cfg file to inform Ansible to look
into the new directory. In order for the action plugin to work as a
"module" a virtual module was added to the library directory.
Change-Id: I80331628b2c3d426a95c89d9c1b766e2e3f70e6d
Partially implements: blueprint tunable-openstack-configuration

The paste.ini has been rebased on upstream master and a conditional
was added to the nova.conf file.
This change makes it possible for a deployer to consume the
deprecated apis for EC2 and NovaV3. While The endpoints will not be
"automatically" created the paste config has been rebased to support
the apis if needed.
Partially implements: blueprint master-kilofication
Change-Id: I061d743b569ebc0753a47d183545ed185bad854e

* API Versions 1.1 and 3 have been deprecated from nova, plays
 have been modified to completely remove v1.1 and make v3
 optional via nova_v3_deprecated_but_enabled boolean.
* Addition of v2.1 api configuration.
* Elimination of the unused nova_api_ec2 container.
* nova_spice_console has been renamed to nova_console and
 nova_spice_console_container has been renamed to
 nova_console_container to facilitate different consoles in
 the future.
* Spice has been made the default console.
* A standalone task and init scripts for nova_spice.
- Fixed some typos
- Modified HAProxy role to remove nova_api_ec2 and rename
 nova_spice_console to nova_console
- Updated user_secrets.yml
- Unbroke things that I broke
Partially Implements Blueprint: master-kilofication
Change-Id: Ia87dfb1e8c0316103a30e2121f11996a9ca87c25

* Updated Keystone wsgi and paste files from upstream.
* Updated all clients in the openstack_client.yml file.
* Kilo services are tracking the head of master.
* Removed pinned middleware because they're pinned else where.
* Added additional service references for neutron vpnaas, fwaas, and
 lbaas which have now been moved into their own repos and no longer
 exist within the core neutron repository.
* The neutron vpnaas, fwaas, and lbaas have been removed from the
 basic plugins being loaded and a comment has been added to describe
 how one might add them back in.
* Updated rootwrap filters for neutron dhcp and l3.
* Updated heat policy.json
* Added the `python-libguestfs` to the nova-compute installation
 packages.
* Updates all services to point to the latest kilo tag
Services updated due to deprecated configs:
* Keystone
* Glance
* Nova
* Neutron (is still using the deprecated nova auth plugin)
* Heat
* Tempest
Items for future work post initial release:
* roles/os_neutron/files/post-up-checksum-rules:25:
 TODO(cloudnull) remove this script once the bug is fixed.
* roles/rabbitmq_server/tasks/rabbitmq_cluster_join.yml:17:
 TODO(someone): implement a more robust way of checking
Implements: blueprint minimal-kilo
Closes-Bug: 1428421
Closes-Bug: 1428431
Closes-Bug: 1428437
Closes-Bug: 1428445
Closes-Bug: 1428451
Closes-Bug: 1428469
Closes-Bug: 1428639
Change-Id: I28a305d9e40a9cf70148ef7d7b00d467a65ca076

This change implements the blueprint to convert all roles and plays into
a more generic setup, following upstream ansible best practices.
Items Changed:
* All tasks have tags.
* All roles use namespaced variables.
* All redundant tasks within a given play and role have been removed.
* All of the repetitive plays have been removed in-favor of a more
 simplistic approach. This change duplicates code within the roles but
 ensures that the roles only ever run within their own scope.
* All roles have been built using an ansible galaxy syntax.
* The `*requirement.txt` files have been reformatted follow upstream
 Openstack practices.
* Dynamically generated inventory is now more organized, this should assist
 anyone who may want or need to dive into the JSON blob that is created.
 In the inventory a properties field is used for items that customize containers
 within the inventory.
* The environment map has been modified to support additional host groups to
 enable the seperation of infrastructure pieces. While the old infra_hosts group
 will still work this change allows for groups to be divided up into seperate
 chunks; eg: deployment of a swift only stack.
* The LXC logic now exists within the plays.
* etc/openstack_deploy/user_variables.yml has all password/token
 variables extracted into the separate file
 etc/openstack_deploy/user_secrets.yml in order to allow seperate
 security settings on that file.
Items Excised:
* All of the roles have had the LXC logic removed from within them which
 should allow roles to be consumed outside of the `os-ansible-deployment`
 reference architecture.
Note:
* the directory rpc_deployment still exists and is presently pointed at plays
 containing a deprecation warning instructing the user to move to the standard
 playbooks directory.
* While all of the rackspace specific components and variables have been removed
 and or were refactored the repository still relies on an upstream mirror of
 Openstack built python files and container images. This upstream mirror is hosted
 at rackspace at "http://rpc-repo.rackspace.com" though this is
 not locked to and or tied to rackspace specific installations. This repository
 contains all of the needed code to create and/or clone your own mirror.
DocImpact
Co-Authored-By: Jesse Pretorius <jesse.pretorius@rackspace.co.uk>
Closes-Bug: #1403676
Implements: blueprint galaxy-roles
Change-Id: I03df3328b7655f0cc9e43ba83b02623d038d214e