openstack/openstack-ansible-os_nova - openstack-ansible-os_nova - OpenDev: Free Software Needs Free Tools

These values have been shown to work well with larger cluster sizes.
Closes-bug: 1478071
Change-Id: Iec44ef4b2c1d89537ffd1237322a44be5c357699

This allows us to override the default settings, which is useful for
large deployments or deploying a large number of instances. It also
uses an unused variable in neutron for setting the rpc_backend.
Change-Id: I83d11eb79b30dda51c6f738433ca960a0f63246e
Closes-bug: 1471926

This patch ensures that the authorized_keys ansible module, as well as
the built in "generate_ssh_keys" flag for user creation, so that we can
avoid using shell out commands.
Additionally, this moves the key synchronisation to use ansible
variables instead of the memcache server.
Change-Id: Icd97ebd44f6065fc60fdce1b61e9dc2daa45faa0
Closes-Bug: #1477512 

Add all remaining configurable default quota options for nova, neutron,
and cinder.
Change-Id: I3bbbba2f2ca9495c7060ffa5ac5d70e865758518
Closes-Bug: #1446431 

In order to enable and deploy federated Keystone, we need to use version
3 of the Keystone API and the v3 Keystone Client. This work begins that
transition by having a set of backwards compatible library commands.
Specifically, this commit updates the keystone library to use v3
Keystone Client and the usage of ensure_tenant in the os_keystone tasks
to use the v3 admin url.
In version 3 of Keystone's Endpoints (Catalog) API each endpoint only
has one URL and has separate interface types (public, internal, admin).
This change updates all uses of ensure_endpoint to structure the
endpoint data in a better way for the ensure_endpoint command in the
keystone module. As a result, some incidents where internalurl and
adminurl were swapped have been fixed.
Note:
In new deployments the endpoints will be created using the v3 API and
will therefore not be available via the v2 API. This will be a breaking
change to legacy CLI clients. The openstack CLI should be used instead.
DocImpact
Related-Bug: #1470635
Partially-implements: blueprint keystone-federation
Change-Id: I2cd4f505e850b4b113452abc25ee00d486b1637d

This patch introduces an insecure flag for the Keystone internal
 and admin endpoints:
* keystone_service_adminuri_insecure
* keystone_service_internaluri_insecure
Both values default to false. If you have setup SSL endpoints
for Keystone using an untrusted certificate then you should
set the appropriate flag to true in your user_variables.
This patch is used to enable testing and development with
Keystone SSL endpoints without having to make use of SSL
certificates signed by a trusted, public CA.
The patch introduces a new optional argument (insecure) to the
keystone, glance and neutron Ansible libraries. This is a
boolean value which, when true, enables these libraries to
access Keystone endpoints 'insecurely'. When these libraries
are used in plays, the appropriate value is set automatically
as per the above conditions.
Implements: blueprint keystone-federation
Change-Id: Ia07e7e201f901042dd06a86efe5c6f6725e9ce13

This patch implements the implement-ceilometer blueprint.
It addes the necessary role/variables to deploy ceilometer
with a Mongodb backend. The Monogdb backend is assumed to
be up and configured and the playbooks only require a few
values to be set in user_variables to establish a connection.
Change-Id: I2164a1f27f632ce254cc2711ada2c449a9961fed
Implements: blueprint implement-ceilometer

The glance_api_servers points to a list of glance_api_servers for both
cinder and nova. This causes "nova image-list" to fail when glance api
servers are unavailable. Pointing to the LB VIP works as intended, so
removing this var in favour of the "host/port" vars ensures that only
the glance servers that are available are used.
glance_api_servers is still available and if specified will be used in
favour of host/port, but default it is commented out and the host/port
will be used - which uses the internal_lb_vip_address and default
glance_api_service_port.
Change-Id: I6794a1a266d22944be8d5634ee0c0ce6cd9f2c59
Closes-Bug: #1461245 

The nova.conf and tempest.conf option for `enable_instance_password` has been
added as a default. This option has a default of True in nova but false in tempest. 
This causes temptest to fail scheme validation on newer versions of temptest. 
To fix this issue the option being added with a default value of True for both
tempest and nova.
Change-Id: I19f5da9820f2367b3d8dd0a7f215aa3f3ea5f611
Partial-Bug: #1468061 

nova has the configuration option [cinder]/cross_az_attach with a
default of True. This option allows attaching between instances and
volumes in different availability zones.
This commit makes this option configurable in the nova.conf template
and uses a default of True.
Change-Id: Ia95f3d4447b026a8e93c74a8c65a63dcea89994f
Closes-bug: 1457140

This allows you to set the endpoint-type protocol globally for all
services, e.g. internaluri can be http, and publicuri can be https.
You will no longer have to specify it per service, although those
settings already exist and have not changed.
This patch changes no functionality for existing installs or deployments
and the values are defaulted to be the same as before, but allows these
values to be adjusted on a per-endpoint type basis.
Change-Id: I4854216726491f6ea4e265694e702f980fddc5a6
Closes-Bug: #1399383 

This change adds a specific update task to all tasks that all the
apt ansible module. This change was done to ensure that the cache
is updated as expected when instructed to do so. The reason that
the cache update is being removed from the grouping is because
there is an upstream bug that is effecting the process by which
the apt cache is updated when there is a package list to process
within the same task. The work around to make this function as
expected is to move the update into its own task without a package
list.
Upstream Ansible bug:
 - https://github.com/ansible/ansible-modules-core/issues/1497
Change-Id: Ic06d89a76d772c12888b4bc4bbf147be58b0c150
Related-Bug: 1464771

If services are running behind an SSL terminating LB you will want to
differentiate between protocol on internalURL and publicURL endpoints.
This patch allows you to set the values of protocol per endpoint type,
but doesn't change the default behaviour which is to have it set in one
var.
Change-Id: I7a74c85a8841499623746586ae27103a71c6fec0
Partial-Bug: #1399383 

A number of the init templates in the OpenStack roles have the word
runlevel incorrectly spelt as runlelvl. This commit correct those
spelling mistakes.
Change-Id: I0d1b7d5c5cf088fecf07cf0e1bd676b4e4088e2b
Closes-bug: #1464603 

To enable partitioning of DB traffic by-service, each service needs to
use a custom connection string. Defaulting the service address to a
common galera_address makes things continue to work by default.
While the galera_address could be overridden on a container or host
basis this requires repeating that behavior across each infra node in
the inventory. Providing service-specific connection address variables
simplifies the management somewhat for large deployments and may reduce
error rates.
The service install playbooks now default the service-specific variables
instead of galera_address to the internal lb vip from inventory to
maintain the ease-of-use currently available.
Any value for a service-specific variable set in user_variables.yml will
override the value in the playbook's vars to provide selective
customization as needed.
Change-Id: I4c98bf906a0c1cb11ddd41277a855dce22ff646a
Closes-Bug: 1462529

This changes the log directory within the upstart init scripts
such that they are being set throughout the roles and removes an
assumed variable name as the name of the log directory.
Change-Id: I32a2a76a4a73b609999886427b494f380850f12a
Closes-Bug: #1460469 

This patch adds handler flushing as the last task in each role to ensure
that there are log files present when the rsyslog client configuration
task is executed a little later in the playbook that consumes the role.
Closes-Bug: #1458822
Change-Id: I92a26b620aa7bc0fbe33175594d37da7d5aca7df

Add variables for the following 3 nova.conf vars:
max_overflow (default 10)
max_pool_size (default 5)
pool_timeout (default 30)
This allows for sql tuning to better support bulk operations
(boot/delete) with the ability to define custom values in nova.conf
based on business needs.
Change-Id: Ic427e6822f636a304cbbfaab5ac74a13e912da0f
Closes-Bug: #1447389 

This new role is now providing the ability for a user to pin apt
packages as they see fit. The idea is to allow someone to implement
pinning in a generic way that can be represented as a global variable
or as a hostvar. The new role has been added to all install roles as
a dependency which will allow it to ensure that packages are pinned
everywhere as would be expected.
Change-Id: I354e8515570fa7174366ba57d57aece3c304568e

Adding support for dynamically updating the policy files for
nova, glance, neutron, cinder and heat. Uses the copy_update
plugin to detect any updates and applies the changes to the default
policy.json
Implements: blueprint dynamically-manage-policy.json
Change-Id: I573229d6f18a5fe32460b2373ab8b2c36ac722b4

In the kilo release the nova v2.1 API is tied to the v3 API, so v3 needs
to be enabled for v2.1 to be enabled as well. This change adds a setting
to control whether the v2.1 API should be enabled or disabled. If v2.1
is enabled then v3 will be enabled as well, but without registering it
with the keystone catalog.
Change-Id: I1e80189bbcbef1dd712cd6a527b5b59aa939e9e1
Closes-Bug: #1445524 

The paste.ini has been rebased on upstream master and a conditional
was added to the nova.conf file.
This change makes it possible for a deployer to consume the
deprecated apis for EC2 and NovaV3. While The endpoints will not be
"automatically" created the paste config has been rebased to support
the apis if needed.
Partially implements: blueprint master-kilofication
Change-Id: I061d743b569ebc0753a47d183545ed185bad854e

Update keystone authentication middleware in neutron to
support the v3 API in Kilo.
Partially implements: blueprint master-kilofication
Change-Id: Icf692cfbdf4df83be21a87adfb434ef9a3c8ff49

Update keystone authentication middleware in nova to
support the v3 API in Kilo.
Partially implements: blueprint master-kilofication
Change-Id: I2f38ed9a5ad82d98596835a59f6852f1bd3d8ffc

* API Versions 1.1 and 3 have been deprecated from nova, plays
 have been modified to completely remove v1.1 and make v3
 optional via nova_v3_deprecated_but_enabled boolean.
* Addition of v2.1 api configuration.
* Elimination of the unused nova_api_ec2 container.
* nova_spice_console has been renamed to nova_console and
 nova_spice_console_container has been renamed to
 nova_console_container to facilitate different consoles in
 the future.
* Spice has been made the default console.
* A standalone task and init scripts for nova_spice.
- Fixed some typos
- Modified HAProxy role to remove nova_api_ec2 and rename
 nova_spice_console to nova_console
- Updated user_secrets.yml
- Unbroke things that I broke
Partially Implements Blueprint: master-kilofication
Change-Id: Ia87dfb1e8c0316103a30e2121f11996a9ca87c25

* Updated Keystone wsgi and paste files from upstream.
* Updated all clients in the openstack_client.yml file.
* Kilo services are tracking the head of master.
* Removed pinned middleware because they're pinned else where.
* Added additional service references for neutron vpnaas, fwaas, and
 lbaas which have now been moved into their own repos and no longer
 exist within the core neutron repository.
* The neutron vpnaas, fwaas, and lbaas have been removed from the
 basic plugins being loaded and a comment has been added to describe
 how one might add them back in.
* Updated rootwrap filters for neutron dhcp and l3.
* Updated heat policy.json
* Added the `python-libguestfs` to the nova-compute installation
 packages.
* Updates all services to point to the latest kilo tag
Services updated due to deprecated configs:
* Keystone
* Glance
* Nova
* Neutron (is still using the deprecated nova auth plugin)
* Heat
* Tempest
Items for future work post initial release:
* roles/os_neutron/files/post-up-checksum-rules:25:
 TODO(cloudnull) remove this script once the bug is fixed.
* roles/rabbitmq_server/tasks/rabbitmq_cluster_join.yml:17:
 TODO(someone): implement a more robust way of checking
Implements: blueprint minimal-kilo
Closes-Bug: 1428421
Closes-Bug: 1428431
Closes-Bug: 1428437
Closes-Bug: 1428445
Closes-Bug: 1428451
Closes-Bug: 1428469
Closes-Bug: 1428639
Change-Id: I28a305d9e40a9cf70148ef7d7b00d467a65ca076

The logic within the pre_tasks was too loose. The pre_tasks need
to have a bit more restrictive conditionals. Additionall the
`lxc-device add` command which is being used always exits 0
even when the device add fails. While the task does not have
issues with running we should be making sure that our changes
are being recoreded correctly.
Closes-Bug: 1434945
Change-Id: Icb1707db6bde6c0a26af6bca21a1f88dc3c315a8

Introduced namespaced variables for all OpenStack services supporting
this setting as defined through oslo libraries. Default value is False
in each case. Gating commit checks now enable the fatal_deprecations
setting for each supporting service.
Closes Bug: 1428412
Change-Id: I5f41d3fdfa1cc876efc0c33c657c9dad18a8ba51

We are currently not dropping the included config file, which is
preventing nova from being able to resize/migrate from one host to
another. This change simply drops the config file which we already
have under the os_nova role to the correct location.
Change-Id: I2d054f095bc76c821aa7b87f5f2890037260066a
Closes-Bug: #1429188 

Add variables for the following 2 nova.conf vars:
image_cache_manager_interval (default 0)
remove_unused_resized_minimum_age_seconds (default 3600)
Change-Id: Ifa52ab226bfe695b64c32c94dd3ffccf465345a0
Closes-Bug: #1424808 

While testing glance + swift, I noticed that the glance-api.conf and
glance-registry.conf being dropped had incorrect auth_uri /
identity_uri values set. This change updates auth_uri / identity_uri
throughout to point to the correct keystone_service_adminuri /
keystone_service_internalurl variables.
Change-Id: I3cbbfefe7da54b08bb9a55e4a2ca3a8bd786577d
Closes-Bug: 1425099

This change implements the blueprint to convert all roles and plays into
a more generic setup, following upstream ansible best practices.
Items Changed:
* All tasks have tags.
* All roles use namespaced variables.
* All redundant tasks within a given play and role have been removed.
* All of the repetitive plays have been removed in-favor of a more
 simplistic approach. This change duplicates code within the roles but
 ensures that the roles only ever run within their own scope.
* All roles have been built using an ansible galaxy syntax.
* The `*requirement.txt` files have been reformatted follow upstream
 Openstack practices.
* Dynamically generated inventory is now more organized, this should assist
 anyone who may want or need to dive into the JSON blob that is created.
 In the inventory a properties field is used for items that customize containers
 within the inventory.
* The environment map has been modified to support additional host groups to
 enable the seperation of infrastructure pieces. While the old infra_hosts group
 will still work this change allows for groups to be divided up into seperate
 chunks; eg: deployment of a swift only stack.
* The LXC logic now exists within the plays.
* etc/openstack_deploy/user_variables.yml has all password/token
 variables extracted into the separate file
 etc/openstack_deploy/user_secrets.yml in order to allow seperate
 security settings on that file.
Items Excised:
* All of the roles have had the LXC logic removed from within them which
 should allow roles to be consumed outside of the `os-ansible-deployment`
 reference architecture.
Note:
* the directory rpc_deployment still exists and is presently pointed at plays
 containing a deprecation warning instructing the user to move to the standard
 playbooks directory.
* While all of the rackspace specific components and variables have been removed
 and or were refactored the repository still relies on an upstream mirror of
 Openstack built python files and container images. This upstream mirror is hosted
 at rackspace at "http://rpc-repo.rackspace.com" though this is
 not locked to and or tied to rackspace specific installations. This repository
 contains all of the needed code to create and/or clone your own mirror.
DocImpact
Co-Authored-By: Jesse Pretorius <jesse.pretorius@rackspace.co.uk>
Closes-Bug: #1403676
Implements: blueprint galaxy-roles
Change-Id: I03df3328b7655f0cc9e43ba83b02623d038d214e