openstack/nova - nova - OpenDev: Free Software Needs Free Tools

The /os-hypervisors/detail API endpoint was experiencing significant
performance issues in environments with many compute nodes when using
microversion 2.88 or higher, as it made sequential RPC calls to gather
uptime information from each compute node.
This change optimizes uptime retrieval by:
* Adding uptime to periodic resource updates sent by nova-compute to the
 database, eliminating synchronous RPC calls during API requests
* Restricting RPC-based uptime retrieval to hypervisor types that support
 it (libvirt and z/VM), avoiding unnecessary calls that would always fail
* Preferring cached database uptime data over RPC calls when available
Closes-Bug: #2122036
Assisted-By: Claude <noreply@anthropic.com>
Change-Id: I5723320f578192f7e0beead7d5df5d7e47d54d2b
Co-Authored-By: Sylvain Bauza <sbauza@redhat.com>
Signed-off-by: Sean Mooney <work@seanmooney.info>

RBAC community wide goal phase-2[1] is to add 'service'
role for the service APIs policy rule. This commit
defaults the service APIs to 'service' role. This way
service APIs will be allowed for service user only.
Tempest tests also modified to simulate the service-to-service
communication. Tempest tests send the user with service
role to nova API.
- https://review.opendev.org/c/openstack/tempest/+/892639>
Partial implement blueprint policy-service-role-default
[1] https://governance.openstack.org/tc/goals/selected/consistent-and-secure-rbac.html#phase-2
Change-Id: I1565ea163fa2c8212f71c9ba375654d2aab28330
Signed-off-by: Ghanshyam Maan <gmaan@ghanshyammann.com>

These all belong in separate files. Make it so.
We also rename the volume_attachment schema file to volume_attachments,
to better link it to the actual API code, and tweak an error message to
fix some capitalization.
Change-Id: Iffefc263bbf19d18137207c0432c16fdb3c513f9
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>

This change tightens the validation around the attachment
update API to ensure that it can only be called if the source
volume has a non empty migration status.
That means it will only accept a request to swap the volume if
it is the result of a cinder volume migration.
This change is being made to prevent the instance domain
XML from getting out of sync with the nova BDM records
and cinder connection info. In the future support for direct
swap volume actions can be re-added if and only if the
nova libvirt driver is updated to correctly modify the domain.
The libvirt driver is the only driver that supported this API
outside of a cinder orchestrated swap volume.
By allowing the domain XML and BDMs to get out of sync
if an admin later live-migrates the VM the host path will not be
modified for the destination host. Normally this results in a live
migration failure which often prompts the admin to cold migrate instead.
however if the source device path exists on the destination the migration
will proceed. This can lead to 2 VMs using the same host block device.
At best this will cause a crash or data corruption.
At worst it will allow one guest to access the data of another.
Prior to this change there was an explicit warning in nova API ref
stating that humans should never call this API because it can lead
to this situation. Now it considered a hard error due to the
security implications.
Closes-Bug: #2112187
Depends-on: https://review.opendev.org/c/openstack/tempest/+/957753
Change-Id: I439338bd2f27ccd65a436d18c8cbc9c3127ee612
Signed-off-by: Sean Mooney <work@seanmooney.info>

We also fix some issues in the api-ref, since it'll be another while
before we can replace that.
Change-Id: If661e3af796475637c0e76b3dfbfd5b7a7f38c24
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>

* Add a note explaining presence of xvpvnc console type
* Make 'url' mandatory in create response
* Remove unnecessary description fields: we will populate these later
* De-deuplcate request body schemas
* Re-add references to the rdp console to the api-ref
Change-Id: I5555b8cf7a83fad689e98522850b5550b49566ed
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>

* Address an off-by-one error: the cpu_info field was modified in v2.28,
 not v2.27,
* Correct the api-ref to indicate that the 'servers' field is not
 actually required and will be missing if '?with_servers=false', while
 the 'name' and 'uuid' fields of servers entries *are* required.
* Clarify a comment about the above in the schemas.
* Uncouple the '_hypervisor_response' and '_hypervisor_detail_response'
 helper schemas. The minor increase in lines of code is worth it for
 the decrease in complexity.
* Add the 'host_ip', 'hypervisor_type', and 'hypervisor_version' fields
 to the list of required fields for "detail"-style responses (show and
 detailed list).
* Make the 'current_workload', 'disk_available_least', 'free_disk_gb',
 'free_ram_mb', 'host_ip' and 'running_vms' fields of the hypervisor
 "detail"-style responses nullable, and the 'current_workload',
 'disk_available_least', 'free_disk_gb', 'free_ram_mb' and
 'running_vms' fields of the deprecated statistics API nullable.
Change-Id: Ibe55b44e65fe17141c63cceae8a003816ffe4f23
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>

These 3 parameters will always be returned [1].
[1] https://github.com/openstack/nova/blob/c4c5134369/nova/api/openstack/compute/server_topology.py#L58-L62
Change-Id: I169750a3bfe2d4a776874b7f5fdce0bc1aa568a1
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>

The user_id and project_id fields can be null. Explain why.
Change-Id: I34353ec7e55845a83f5e2f1dc4b41dead4ac5c81
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>

The request_specs_dict in the _list_view is initialized as a
defaultdict object in order to return empty string as default.
But the request_spec_dict is replaced with a normal dict object in
the v2.96 microversion, then if server list and RequestSpec missmatch
happens by any reason, the List Server API and the List
Server Detail API hit 500 Internal server error because of key error.
This commit updates the req_spec_dict to use normal dict object, then
it returns sentinel object if there is no appropriate
request_spec object.
Closes-Bug: #2095364
Change-Id: If282b8709954f276cb5d48114437809d771a9958

microversion 2.98 changed the GET and GET list details
API response to expand the image field
- https://review.opendev.org/c/openstack/nova/+/939649
but when we change the views/servers.py then update server
and rebuild server API response also gets updated.
- https://review.opendev.org/c/openstack/nova/+/939649/12/nova/api/openstack/compute/views/servers.py#595
The microversion change actually changed the response for
the update and rebuild API but did not update the documentation
about this change. Neither added test for rebuild and update
which verify the response change.
Since microversion 2.75, we have made all these 4 APIs response
consistent.
Implements: blueprint image-properties-in-server-show
Change-Id: Ifc2edc59c601391a31fd8953c742969e5b368767

microversion 2.96 changed the GET and GET list details
APi response to add a new field 'pinned_availability_zone'
- https://review.opendev.org/c/openstack/nova/+/904568
but when we change the views/servers.py then update server
and rebuild server API response also gets updated.
- https://review.opendev.org/c/openstack/nova/+/904568/17/nova/api/openstack/compute/views/servers.py#309
The microversion change actually changed the response for
the update and rebuild API but did not update the documentation
about this change. Neither added test for rebuild and update
which verify the response change.
Since microversion 2.75, we have made all these 4 APIs response
consistent.
Change-Id: I31fbb3c5e689302713bfb7a254c5d0cc12de2dc4

Nova currently lacks a straightforward way to expose scheduler hints
associated with a server. This change extends existing Nova's
API to allow users to retrieve this information when it is available.
It adds a new API microversion to support returning the associated
scheduler_hints in ``GET /servers/{server_id}``, `GET /servers/detail``,
``PUT /servers/{server_id}`` and ``POST /server/{server_id}/action``
(rebuild) responses.
Implements: blueprint show-scheduler-hints-in-server-details
Change-Id: If0070cfbc218e0515bf3b58fc8c5067ae22c7072

This patch adds a new console type, "spice-direct", which provides
the connection information required to talk the native SPICE protocol
directly to qemu on the hypervisor. This is intended to be fronted
by a proxy which will handle authentication separately.
A new microversion is introduced which adds the type "spice-direct"
to the existing "spice" protocol.
An example request:
POST /servers/<uuid>/remote-consoles
{
 "remote_console": {
 "protocol": "spice",
 "type": "spice-direct"
 }
}
An example response:
{
 "remote_console": {
 "protocol": "spice",
 "type": "spice-direct",
 "url": "http://localhost:13200/nova?token=XXX";
 }
}
This token can then be used to lookup connection details for the
console using a request like this:
GET /os-console-auth-tokens/<consoletoken>
Which returns something like this:
{
 "console": {
 "instance_uuid": <uuid>,
 "host": <hypervisor>,
 "port": <a TCP port number>,
 "tls_port": <another TCP port number>,
 "internal_access_path": null
 }
}
APIImpact
Change-Id: I1e701cbabc0e2c435685e31465159eec09e3b1a0

This change adds a new api microversion to add support for
including image properties in ``server show`` and ``server list
--long`` responses as well as in response for ``server rebuild``
instance action.
Implements: blueprint image-properties-in-server-show
Change-Id: Ic135389954e43e6478288c0cdcffd780915cdb40

This is not admin-only as suggested by the api-ref. In addition, there is
no mechanism available to disable flavors so this will always be false.
Change-Id: Ie1906366c8f141c6f6c58f21b0ff67eb827df4c8
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>

This patch introduce the REST API modifications to attach/detach a share
and list/show share mappings.
Manila is the OpenStack Shared Filesystems service.
These series of patches implement changes required in Nova to allow the shares
provided by Manila to be associated with and attached to instances using
virtiofs.
Implements: blueprint libvirt-virtiofs-attach-manila-shares
Change-Id: I0255a5697cd4ea148bd91c4f6fd183841d69a333

After this patch nova rejects the add host to aggregate API action
if the host has instances and the new aggregate for the host would
mean that these instances need to move from one AZ (even from the
default one) to another. Such AZ change is not implemented in nova
and currently leads to stuck instances.
Similarly nova will reject remove host from aggregate API action if the
host has instances and the aggregate removal would mean that the
instances need to change AZ.
Depends-On: https://review.opendev.org/c/openstack/tempest/+/821732
Change-Id: I19c4c6d34aa2cc1f32d81e8c1a52762fa3a18580
Closes-Bug: #1907775 

The addresses field containes delayed (cached) information,
due to internal implementation details.
This used to be considered API contact breakage,
but apperently no plan for fixing it.
Change-Id: I8db2800ede0276d60138369ec2d60c55a1791040

When we added evacuate to STOPPED in microversion 2.95 in
d732ee38a1, we never updated our
api-ref. This patch just adds a sentence explaining how evacuate
behaves starting in 2.95.
Related: bp/allowing-target-state-for-evacuate
Change-Id: I1ed7211bcdd85547d3c7d8d396b6efab36b835c6

As of now, the server show and server list --long output
shows the availability zone, that is, the AZ to which the
host of the instance belongs. There is no way to tell from
this information if the instance create request included an
AZ or not.
This change adds a new api microversion to add support for
including availability zone requested during instance create
in server show and server list --long responses.
Change-Id: If4cf09c1006a3f56d243b9c00712bb24d2a796d3

RDP console was only for HyperV driver so removing the
API. As API url stay same (because same used for other
console types API), RDP console API will return 400.
Cleaning up the related config options as well as moving its
API ref to obsolete seciton.
Keeping RPC method to avoid error when old controller is used
with new compute. It can be removed in next RPC version bump.
Change-Id: I8f5755009da4af0d12bda096d7a8e85fd41e1a8c

The RDP console was only available for HyperV driver, therefore its
connection information via API ``os-console-auth-tokens`` will now return
HTTP ``400 (BadRequest)`` error.
Starting from 2.31 microversion, this API return connection info
for all other console type.
Change-Id: I94e590eb4cbe3b2d8eff7fe881f7b98af8979be2

Cleanup doc/code ref of HyperV driver.
Change-Id: I6cd8fb90829e040bfd356ff6b1c41aa9a1c906d2

this change uses codespell to fix typos in
the api-ref and updates setup.cfg to exclude more
binary types and build files.
Change-Id: Ife9bf9a87c70126969bf43f7aa4a3645617de901

Change-Id: Ie8078739d077a903e3ad83d27b178eda416b0859
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>

We add a new specific policy when a host value is provided for cold-migrate,
but by default it will only be an admin-only rule in order to not change
the behaviour.
Change-Id: I128242d5f689fdd08d74b1dcba861177174753ff
Implements: blueprint cold-migrate-to-host-policy

Extend microversion 2.90 to allow FQDNs in the hostname parameter.
Multi-create with --hostname continues to be refused, returning error
400 to the user. This simplifies the code by not needing to handle any
sort of suffix or prefix mangling of the FQDN to handle multiple
instances. No other changes are made - not Neutron integration,
metadata API changes (the FQDN will appear as-is in places where the
hostname currently appears), etc.
Change-Id: I47e397dc6da8263762479cc8ae4f8777a6d9d811
Implements: bp/fqdn-in-hostname

Change-Id: If94895d0fc4b5a59febd02e127fbf93959084fee

Flavor with special characters in flavor ID is not allowed to create
Closes-Bug: 1920938
Change-Id: I600ae07f7fb039c9ccf461c4bf74ae42d9a3744f

This adds a microversion and API support for triggering a rebuild
of volume-backed instances by leveraging cinder functionality to
do so.
Implements: blueprint volume-backed-server-rebuild
Closes-Bug: #1482040
Co-Authored-By: Rajat Dhasmana <rajatdhasmana@gmail.com>
Change-Id: I211ad6b8aa7856eb94bfd40e4fdb7376a7f5c358

As agreed in the spec, we will both drop the generation support for a keypair
but we'll also accept @ (at) and . (dot) chars in the keyname, all of them in
the same API microversion.
Rebased the work from I5de15935e83823afa545a250cf84f6a7a37036b4
APIImpact
Implements: blueprint keypair-generation-removal
Co-Authored-By: Nicolas Parquet <nicolas.parquet@gandi.net>
Change-Id: I6a7c71fb4385348c87067543d0454f302907395e

This adds support to the REST API, in a new microversion, for specifying
a destination host to unshelve server action when the server
is shelved offloaded.
This patch also supports the ability to unpin the availability_zone of an
instance that is bound to it.
Note that the functional test changes are due to those tests using the
"latest" microversion 2.91.
Implements: blueprint unshelve-to-host
Change-Id: I9e95428c208582741e6cd99bd3260d6742fcc6b7

Rescuing BFV instances became a thing in 2.87. Fix our api-ref.
Change-Id: I9c4fbea53b05490cf05ebd311b25791fa191af16

Change-Id: I48c68e135bdfa7229eac6b242b7f80b005761ee8

This microversion adds attachment_id and bdm_uuid as stored in the
underlying bdm record while also removing the duplicate id field from
the responses of ``GET /servers/{server_id}/os-volume_attachments`` and
``GET /servers/{server_id}/os-volume_attachments/{volume_id}``.
To accomidate this within the _translate_attachment_summary_view helper
is folded into _translate_attachment_detail_view with the remaining
caller in the now deprecated os-volumes API now replaced with a static
dictionary.
Blueprint: add-attachmentid-to-responses-of-the-os-volume-attachments-api
Change-Id: I977c3fd9bbb1e1d42e6979222137e7366d2815e8

Take the opportunity to clean up the docs quite a bit, ultimately
combining two disparate guides on the scheduler into one.
Change-Id: Ia72d39b4774d93793b381359b554c717dc9a6994
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>

Change-Id: I3878fb9f36bb05dbb949f137a5d5c6664a27b69f
Closes-Bug: #1920256 

An empty value for the 'all_tenants' query parameter of '/servers' and
'/servers/detail' means the value defaults to 'True', i.e. requesting
'/servers?all_tenants' is the same as '/servers?all_tenants=1'. Clarify
this, since the current wording is confusing.
Change-Id: Ib5fdd3b73aa5179e0379ee8f465e4118107786be
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
Closes-Bug: #1909745 

Add microversion 2.90, which allows allows users to configure the
hostname that will be exposed via the nova metadata service when
creating their instance.
Change-Id: I95047c1689ac14fa73eba48e19dc438988b78aad
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>