openstack/nova - nova - OpenDev: Free Software Needs Free Tools

This fix minors comments that have been noticed.
Signed-off-by: Sahid Orentino Ferdjaoui <sahid.ferdjaoui@industrialdiscipline.com>
Change-Id: Iee6c224aa0f26d8550b38a8f69b28d8648b1da70

Start to v2.95 any evacuated instances will be stopped a destination
Implements: bp/allowing-target-state-for-evacuate
Signed-off-by: Sahid Orentino Ferdjaoui <sahid.ferdjaoui@industrialdiscipline.com>
Change-Id: I141b6f057cc4eb9c541c2bc6eddae27270ede08d

Extend microversion 2.90 to allow FQDNs in the hostname parameter.
Multi-create with --hostname continues to be refused, returning error
400 to the user. This simplifies the code by not needing to handle any
sort of suffix or prefix mangling of the FQDN to handle multiple
instances. No other changes are made - not Neutron integration,
metadata API changes (the FQDN will appear as-is in places where the
hostname currently appears), etc.
Change-Id: I47e397dc6da8263762479cc8ae4f8777a6d9d811
Implements: bp/fqdn-in-hostname

This adds a microversion and API support for triggering a rebuild
of volume-backed instances by leveraging cinder functionality to
do so.
Implements: blueprint volume-backed-server-rebuild
Closes-Bug: #1482040
Co-Authored-By: Rajat Dhasmana <rajatdhasmana@gmail.com>
Change-Id: I211ad6b8aa7856eb94bfd40e4fdb7376a7f5c358

Change-Id: I20b21f39c46895448b172f79a3e4603d124f2c53

As agreed in the spec, we will both drop the generation support for a keypair
but we'll also accept @ (at) and . (dot) chars in the keyname, all of them in
the same API microversion.
Rebased the work from I5de15935e83823afa545a250cf84f6a7a37036b4
APIImpact
Implements: blueprint keypair-generation-removal
Co-Authored-By: Nicolas Parquet <nicolas.parquet@gandi.net>
Change-Id: I6a7c71fb4385348c87067543d0454f302907395e

This adds support to the REST API, in a new microversion, for specifying
a destination host to unshelve server action when the server
is shelved offloaded.
This patch also supports the ability to unpin the availability_zone of an
instance that is bound to it.
Note that the functional test changes are due to those tests using the
"latest" microversion 2.91.
Implements: blueprint unshelve-to-host
Change-Id: I9e95428c208582741e6cd99bd3260d6742fcc6b7

Add microversion 2.90, which allows allows users to configure the
hostname that will be exposed via the nova metadata service when
creating their instance.
Change-Id: I95047c1689ac14fa73eba48e19dc438988b78aad
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>

This microversion adds attachment_id and bdm_uuid as stored in the
underlying bdm record while also removing the duplicate id field from
the responses of ``GET /servers/{server_id}/os-volume_attachments`` and
``GET /servers/{server_id}/os-volume_attachments/{volume_id}``.
To accomidate this within the _translate_attachment_summary_view helper
is folded into _translate_attachment_detail_view with the remaining
caller in the now deprecated os-volumes API now replaced with a static
dictionary.
Blueprint: add-attachmentid-to-responses-of-the-os-volume-attachments-api
Change-Id: I977c3fd9bbb1e1d42e6979222137e7366d2815e8

Introduce API microversion 2.88, which makes the following changes to
a number of 'os-hypervisors'. Specifically, the following fields are
dropped from both the '/os-hypervisors/detail' (detailed list) and
'/os-hypervisors/{hypervisor_id}' (show) APIs:
- current_workload
- cpu_info
- vcpus
- vcpus_used
- free_disk_gb
- local_gb
- local_gb_used
- disk_available_least
- free_ram_mb
- memory_mb
- memory_mb_used
- running_vms
In addition, the '/os-hypervisors/statistics' API, which provided a
summary of the above stats but for all hypervisors in the deployment, is
dropped entirely.
Finally, the '/os-hypervisors/{hypervisor}/uptime' API, which provided a
similar response to the '/os-hypervisors/{hypervisor}' API but with an
additional 'uptime' field, has been removed in favour of including this
field in the primary '/os-hypervisors/{hypervisor}' API.
A small tweak to 'tox.ini' that allows us to share some venvs is
included.
Part of blueprint modernize-os-hypervisors-api
Change-Id: I515e484ade6c6455f82a3067940a418a0d7d965a
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>

Enable the policy fixture by default, which should yield more realistic
functional tests. We need to update some tests to use admin APIs where
policy dictates they are necessary. Note that we're currently testing
the legacy policy - not the updated, scoped policy - since the legacy
policy is the default one currently.
Note that we also need to modify the 'SingleCellSimple' fixture in this
change to use the same project ID as the 'OSAPIFixture'.
Change-Id: Ia3dea78f16cb3c7081714c4db36e20d5ee76ed7d
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>

Building on Ic2ad1468d31b7707b7f8f2b845a9cf47d9d076d5, when requested
this microversion will allow boot from volume requests to proceed when
the COMPUTE_RESCUE_BFV trait is also reported by the compute the
instance currently resides on.
Implements: blueprint virt-bfv-instance-rescue
Change-Id: I3242fec1547693078cf36c3637116f8c41f1d0bc

Enable support for API-based extra spec validation. Since most of the
hard work has been done in previous patches, all that's necessary here
is to wire up the microversion handling and turn things on.
Part of blueprint flavor-extra-spec-validators
Change-Id: If67f0d924ea372746a6dc440ea7bdc655e4f0bea
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>

Allow PUT /servers/{server_id}/os-volume_attachments/{volume_id}``
to support specifying ``delete_on_termination`` field in the request
body. This allows updating the attached volume's flag that controls
whether or not it is automatically deleted when the instance is deleted.
When we request 'volumeId' and 'delete_on_termination' in the requst
body to swap volume, since the new microversion it will be support
updating the swapping volume's delete flag.
Co-Authored-By: Dan Smith <dansmith@redhat.com>
Blueprint: destroy-instance-with-datavolume
Change-Id: I6ccac4e17f56b40e67c79d40f32558ef414685ea

Implements blueprint action-event-fault-details
Change-Id: I8e8a5ca1efe4a3554038dcddc503c3030956e0b9

This adds a new microversion to expose the instance action
event details in the
GET /servers/{server_id}/os-instance-actions/{request_id} API.
With the new microversion the "details" key is always returned
with each event dict but the value may be null because of old
records or events that did not fail.
The details are not constrained by policy like the traceback
field since the details are like a fault message on the server
resource when the server is in ERROR status and the fault
message is likewise not constraint by policy unlike the fault
details which is a traceback like the event traceback field.
This commit add a SYSTEM_READER ('rule: system_reader_api') role
to the Show Server Action Details API. With this default policy,
events fault details can be displayed. And also add some nova and
non-nova exception functional tests for os-instance-actions API.
Co-Authored-By: Brin Zhang <zhangbailin@inspur.com>
Implements blueprint action-event-fault-details
Change-Id: I6fe4dd265b0030ce12f92771b255a3d795f03d01

Microversion bump to allow non-admin user to use more filters key
when listing instances.
In order to stay coherent, all existing instance filters who are
related to a field readable by default to non admin users when showing
instance details, should be allowed by default without policy
modification.
Implements: blueprint non-admin-filter-instance-by-az
Change-Id: Ia66d3a1ceb74ed521cf44922929b2a502f3ee935

We have a custom of naming the directory after the API. Reinforce that
here.
Change-Id: I5bf68aacc1d987400a91467835c4b55f03c18beb
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>

The API URL is '/os-keypairs', not '/keypairs'. Attempting to use these
pagination links as-is will result in a HTTP 404 (Not Found).
Change-Id: Ic04568caecc138e6016418f6878d031c4a0d3fb4
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
Closes-bug: #1866373 

APIImpact: Adds 2.82 microversion for /os-server-external-events API.
DocImpact: Adds new version to doc/api_samples/versions/.
Change-Id: I7a626544d8221dc0eeb5672986ca897ce4718be8
Blueprint: nova-cyborg-interaction

Hypervisor view builder's _collection_name,
'hypervisors' is used to build next link that
sdk use as uri to do paginated query. But correct
API should be `GET /v2.1/os-hypervisors?<params>`
rather than `GET /v2.1/hypervisors?<params>`.
This patch fixes this bug.
Change-Id: Idc4f3fe54136a6bd3dbc7dc0efd3f62745991199
Closes-Bug: 1864428
Signed-off-by: Fan Zhang <zh.f@outlook.com>

Tests were previously using arbitrary keys that weren't published
anywhere in the codebase. Switch to using meaningful ones.
Change-Id: I8da84b48e4d630eeb91d92346aa2323e25e28e3b
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>

The 'security_group_rules' [1], 'floating_ips' [2], 'fixed_ips' [3] and
'security_groups' [4] quotas are all nova-network only and can be
removed or, more specifically, set to unlimited and ignored until we
eventually bump our minimum API microversion.
[1] https://review.opendev.org/477700
[2] https://review.opendev.org/457862
[3] https://review.opendev.org/457861
[4] https://review.opendev.org/457860
Change-Id: I9a5362fdf29e3680c59f620c585f3d730e4f6adb
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>

Like '/os-networks', we can't remove this in its entirety due to the
fact that some of these are proxy URLs that also work with neutron.
These are retained but everything else is nuked.
Note that this highlights a bug in the API, whereby a missing
'objects.Network.cidr' value results in a value of 'None' being output
over the API. Clearly this is incorrect, but it's probably not worth
fixing for this deprecated API.
Change-Id: I31cb0891144bdd7945479bb6692b0a533de4c5d0
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>

Drop support for most of the 'os-networks' REST APIs excluding those
that proxy through to neutron.
This API now returns a 410 response for the non-proxy routes.
Unit tests are removed for removed APIs and the functional API sample
tests are just asserting the 410 response now same. The latter are also
expanded to cover APIs that weren't previously tested.
The API sample docs are left intact since the API reference still builds
from those and can be considered more or less branchless, so people
looking at the API reference can apply it to older deployments of nova
before these APIs were removed.
Note: yes, the API samples are correct. It really is a useless API when
used with neutron.
Change-Id: I68bfa77a520382317fc490a4f6c12dd62fc6dcda
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>

This adds a new microversion and support for requesting image pre-caching
on an aggregate.
Related to blueprint image-precache-support
Change-Id: I4ab96095106b38737ed355fcad07e758f8b5a9b0

In microversion 2.80, the ``GET /os-migrations`` API will have
optional ``user_id`` and ``project_id`` query parameters for
filtering migrations by user and/or project:
* GET /os-migrations?user_id=ef9d34b4-45d0-4530-871b-3fb535988394
* GET /os-migrations?project_id=011ee9f4-8f16-4c38-8633-a254d420fd54
* GET /os-migrations?user_id=ef9d34b4-45d0-4530-871b-3fb535988394&project_id=011ee9f4-8f16-4c38-8633-a254d420fd54
And expose the ``user_id`` and ``project_id`` fields in the following APIs:
* GET /os-migrations
* GET /servers/{server_id}/migrations
* GET /servers/{server_id}/migrations/{migration_id}
Co-Authored-By: Qiu Fossen <qiujunting>
Part of blueprint add-user-id-field-to-the-migrations-table
Change-Id: I7313d6cde1a5e1dc7dd6f3c0dff9f30bbf4bee2c

This is probably the most involved of all the changes and requires a
significant expansion of the NeutronFixture to mock floating IP
interactions. All the API samples were based off responses pulled from a
DevStack configuration using openstackclient and old versions of the
novaclient (where the 'floating-ip-*' commands were still present) in
debug mode.
Change-Id: Ib2f10a51bebd10cc69b78427b485aeac19f59141
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>

We're going to want to use this for realistic API samples. The samples
we're using here were taken from a DevStack deployment based on pre-RC1
Train code so they should be fairly reflective of what you'd see in a
real deployment.
Note that this effectively undoes a lot of the changes first introduced
in Ibbee7fd11c1aa254e399d302adbae69126e98262, particularly around the
responses for instances in a down cell, where we previously changed
things so a 'security_groups' field was present in the response. This
is okay since we're not creating interfaces and therefore don't expect
to have security groups present.
Change-Id: I3c94b61fc323fefbd1c8790c4a2f60cada29e86f
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>

This has been tying us to nova-network. This should be a one line change
but it has a large knock-on effect due to lots of samples using it. We
just need to suck it up and deal with it, unfortunately.
Change-Id: I09c88e0fdf3635683c56901637fc3c0a9084d482
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>

Another case of there being no nova-network host.
Change-Id: I51df797df4e1b77f70fe3dd87374737e650c27d9
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>

Three tests require modification to specifically request an API that our
NeutronFixture provides. The rest just work (TM).
Change-Id: I423b7ab5c98ea9d559caefec0b49271e28a140fa
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>

For our immediate purposes, this should always return something so we
modify the mock to do just that. That requires some modifications to
existing tests but is otherwise pretty simple. A future change will
address the TODO contained within to properly mock the neutron API.
Change-Id: Ibbee7fd11c1aa254e399d302adbae69126e98262
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>

This one is long because network quotas aren't a thing when using
neutron, which means we need to remove them from all the samples.
Change-Id: I9cb2fb1b3817797a53c1331d666750399a7f5f87
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>

This is pretty simple - a nova-network host obviously won't be present
in nova-network is disabled.
Change-Id: I4872f0bcaabb8e1cf1aad297a1c55993f2a3fe6e
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>

This is a partial revert of commit
9606c80402 which added the 'path' query
parameter to work with noVNC v1.1.0. This broke all other console types
using websockify server (serial, spice) because the websockify server
itself doesn't know how to handle the 'path' query parameter. It is the
noVNC vnc_lite.html file which parses the 'path' variable and uses it
as the url to the websockify server. So, all other console types should
*not* be generating a console access url with a 'path' query parameter,
only noVNC.
Closes-Bug: #1845243
TODO(melwitt): Figure out how to test serial and/or spice console in
the gate
Change-Id: I9521f21a685edc44121d75bdf534c201fa87c2d7

This is a follow up to I04bca162c3a1d4fed7056385dfdca72c07bab9a5
to make test_list_volume_attachments use two attachments for the
list response output and to update the API reference samples.
Change-Id: I6d7cee16e1eed6fa4fdb6389c6d3ff670ac5a7c3

The amount of DB and compute service stubbing in these
functional tests is pretty gross and makes it harder to
maintain/extend them for new microversions which makes
it harder for new contributors to work with these kinds
of tests.
This removes the stubs and uses the CinderFixture. The
only new stub is dealing with detaching a volume with a
device tag since the fake driver does not track device
metadata on instances.
The API reference doc samples are regenerated using:
 tox -e api-samples VolumeAttachmentsSample
Change-Id: I04bca162c3a1d4fed7056385dfdca72c07bab9a5

Add the 'delete_on_termination' field to the volume attach API to support
configuring whether to delete the data volume when the instance is destroyed.
To avoid upgrade impact issues with older computes, the
'delete_on_termination' field is set in the API rather than when the BDM
is created in the compute service.
Implements: blueprint support-delete-on-termination-in-server-attach-volume
Change-Id: I55731b1822a4e32909665a2872d80895cb5b12f7

Add a new server topology API to show server NUMA information:
 - GET /servers/{server_id}/topology
Add new policy to control the default behavior:
 - compute:server:topology:index
 - compute:server:topology:host:index
Change-Id: Ie647ef96597195b0ef00f77cece16c2bef8a78d4
Implements: blueprint show-server-numa-topology
Signed-off-by: Yongli He <yongli.he@intel.com>

This adds support, in a new microversion, for specifying an availability
zone to the unshelve server action when the server is shelved offloaded.
Note that the functional test changes are due to those tests using the
"latest" microversion where an empty dict is not allowed for unshelve
with 2.77 so the value is changed from an empty dict to None.
Implements: blueprint support-specifying-az-when-restore-shelved-server
Closes-Bug: #1723880
Change-Id: I4b13483eef42bed91d69eabf1f30762d6866f957

In a future change, the use of 'hw:cpu_policy' will require a host to
report PCPU inventory. Rather than modify the fake driver used in these
tests to report such inventory, just use a different extra spec,
'hw:numa_nodes'. This has the added bonus of being supported by both the
libvirt and Hyper-V virt drivers, unlike 'hw:cpu_policy' and
'hw:mem_page_size', which are only supported by the libvirt virt driver.
Change-Id: Id203dc07f08557b1b094ec72e1df3493ec9524b1
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>

This patch adds a new external event called "power-update"
through which ironic will convey all (power_off and power_on)
power state changes (running -> shutdown or shutdown -> running
will be the only ones handled by nova and the rest will be ignored)
on a physical instance to nova. The database will be updated
accordingly to reflect the real vm_state and power_state of the
instance. This way nova will not be able to enforce
an incorrect power state on the physical instance during
the periodic "sync_power_states" task.
Implements blueprint nova-support-instance-power-update
Story: 2004969
Task: 29423
Change-Id: I2b292050cc3ce5ef625659f5a1fe56bb76072496