openstack/nova - nova - OpenDev: Free Software Needs Free Tools

This patches adjusts the nova documentation about the extended port
resource request support in nova as the neutron API extension did not
land in Xena.
Change-Id: I3b961426745084bdb4a6d04468f5a3c762be4cfa
blueprint: qos-minimum-guaranteed-packet-rate

The interface attach and detach logic is now fully adapted to the new
extended resource request format, and supports more than one request
group in a single port.
blueprint: qos-minimum-guaranteed-packet-rate
Change-Id: I73e6acf5adfffa9203efa3374671ec18f4ea79eb

Nova re-generates the resource request of an instance for each server
move operation (migrate, resize, evacuate, live-migrate, unshelve) to
find (or validate) a target host for the instance move. This patch
extends the this logic to support the extended resource request from
neutron.
As the changes in the neutron interface code is called from nova-compute
service during the port binding the compute service version is bumped.
And a check is added to the compute-api to reject the move operations
with ports having extended resource request if there are old computes
in the cluster.
blueprint: qos-minimum-guaranteed-packet-rate
Change-Id: Ibcf703e254e720b9a6de17527325758676628d48

This adds the final missing pieces to support creating servers with
ports having extended resource request. As the changes in the neutron
interface code is called from nova-compute service during the port
binding the compute service version is bumped. And a check is added to
the compute-api to reject such server create requests if there are old
computes in the cluster.
Note that some of the negative and SRIOV related interface attach
tests are also started to pass as they are not dependent on any of the
interface attach specific implementation. Still interface attach is
broken here as the failing of the positive tests show.
blueprint: qos-minimum-guaranteed-packet-rate
Change-Id: I9060cc9cb9e0d5de641ade78c5fd7e1cc77ade46

To prepare for the unlikely event that Neutron merges and an operator
enables the port-resource-request-groups neutron API extension before
nova adds support for it, this patch rejects server creation if such
extension is enabled in Neutron. Enabling that extension has zero
benefits without nova support hence the harsh but simple rejection.
A subsequent patch will reject server lifecycle operations in a more
sophisticated way and as soon as we support some operations, like
boot, the deployer might rightfully choose to enable the Neutron
extension.
Change-Id: I2c55d9da13a570efbc1c862116cea31aaa6aa02e
blueprint: qos-minimum-guaranteed-packet-rate

Since Xena release, nova supports creating servers with
SRIOV provisioned with the Cyborg service.
Change-Id: I59afef70b2b38c61b3c58ac90a1a405532b92832

Change-Id: I3ff89b84f6c8caab952dcc0b85836744565f1372

The following logic is added to the ComputeManager attach_interface
flow:
* gather the resource request of the port from neutron
* query allocation candidates restricted to the current compute node
 from placement
* extend the existing allocation of the instance with one of the
 allocation candidates in placement
* update the InstancePCIRequest (if any) to ensure that the PCI claim
 only allocates VF from the PF the placement resources are allocated from
* ensure that during port binding neutron gets the RP UUID, the resources
 are allocated from, in the allocation key of the binding profile
This patch bumps the compute service version to indicate that QoS
interface attach is supported. Also the check that was so far rejected
such attach is now updated to only reject it if the compute service
version is too old.
The "scheduling" during interface attach for PCI backed ports has the
same limitation as normal scheduling for such ports. It always selects
the first allocation candidate returned by placement even if later in
the process it turns out that such allocation candidate points to a PCI
PF that has no free VFs left.
Blueprint: support-interface-attach-with-qos-ports
Change-Id: Id4684093e8bdf3b61667490443e3d2f6ed65f4b3

This change extends the conductor manager to append the cyborg
resource request to the request spec when performing an unshelve.
On shelve offload an instance will be deleted the instance's ARQs
binding info to free up the bound ARQs in Cyborg service.
And this change passes the ARQs to spawn during unshelve an instance.
This change extends the ``shelve_instance``, ``shelve_offload_instance``
and ``unshelve_instance`` rpcapi function to carry the arq_uuids.
Co-Authored-By: Wenping Song <songwenping@inspur.com>
Implements: blueprint cyborg-shelve-and-unshelve
Change-Id: I258df4d77f6d86df1d867a8fe27360731c21d237

The cross-cell resize code does not consider neutron ports with resource
request. To avoid migration failures this patch makes nova to fall back
to same cell resize if the instance has neutron ports with resource
request.
Change-Id: Icaad4b2375b491c8a7e87fb6f4977ae2e13e8190
Closes-Bug: #1907522 

It turned out that during the qos resize work we did not implemented
support of cross cell resize with qos ports. Tempest test coverage for
the resize and migrate is landed recently that made the nova-multi-cell
job to fail.
So this patch disables the qos resize and migrate tempest tests for the
nova-multi-cell job to unblock the gate.
Related-Bug: #1907522
[1] I8c573c4a11a76932057d8415d76114a03269b9f3
Change-Id: I95bc22f7d65454cd9e7b54a0e6d9516f2f204978

Fix list rendering in the accelerator support document.
Change-Id: Ic58c5604b2e63daab7a30a95ad6a3481d96f6f63
Signed-off-by: Takashi Natsume <takanattie@gmail.com>

This change extends the conductor manager
to append the cyborg resource request to the
request spec when performing an evacuate.
This change passes the ARQs to spawn during rebuild
and evacuate. On evacuate the existing ARQs will be deleted
and new ARQs will be created and bound, during rebuild the
existing ARQs are reused.
This change extends the rebuild_instance compute rpcapi
function to carry the arq_uuids. This eliminates the
need to lookup the uuids associated with the arqs assinged
to the instance by quering cyborg.
Co-Authored-By: Wenping Song <songwenping@inspur.com>
Co-Authored-By: Brin Zhang <zhangbailin@inspur.com>
Implements: blueprint cyborg-rebuild-and-evacuate
Change-Id: I147bf4d95e6d86ff1f967a8ce37260730f21d236

Change pygments_style to 'native' since old theme version always used
'native' and the theme now respects the setting and using 'sphinx' can
lead to some strange rendering.
Change-Id: Icd9d529dbe66626ac53a9fdca5bc6821998a0b7d

Switch to openstackdocstheme 2.1.2 and reno 3.1.0 versions. Using
these versions will allow parallelizing building of documents.
Update Sphinx version as well.
openstackdocstheme renames some variables, so follow the renames. A
couple of variables are also not needed anymore, remove them.
Set openstackdocs_auto_version to not version the documents.
Set openstackdocs_auto_name to use project as name.
Depends-On: https://review.opendev.org/728432
Change-Id: I4e3ae3ceabe125ea459ed4baabf2e98686268e50

In 21.0.0 Ussuri we were completed the nova-cyborg interaction feature,
but there are some issue when multiple create instances.
Creating servers with accelerators provisioned with the Cyborg service,
if a flavor asks for resources that are provided by nested Resource
Provider inventories (eg. VGPU) and the user wants multi-create (ie. say
--max 2) then the scheduler could be returning a NoValidHosts exception
even if each nested Resource Provider can support at least one specific
instance, if the total wanted capacity is not supported by only one
nested RP.
For example,creating servers with accelerators provisioned with the
Cyborg service, if two children RP have 4 VGPU inventories each:
 - you can ask for a flavor with 2 VGPU with --max 2
 - but you can't ask for a flavor with 4 VGPU and --max 2
Related-Bug: #1874664
Change-Id: I64647a6ba79c47c891134cedb49f03d3c61e8824

List bullets in the top level have to be left justified,
otherwise we get "<blockquote>" html tags inserted.
Change-Id: I35e80c23c7a6421a648c93767ab954e9a6d1e997

Change-Id: I4b630d755f99863faf6a432b086608d494ef3cc3
Blueprint: nova-cyborg-interaction

Previous patches in the blueprint implemented the support for unshelve
with qos ports and added functional test coverage for the
various scenarios. So this patch changes the API check
that rejected such operation to check for the service version and therefore
conditionally enable the feature.
Change-Id: Iaf70ee41f1bfb1a4964da3f59cd3a0b4b5e20d36
blueprint: support-move-ops-with-qos-ports-ussuri

Microversion bump to allow non-admin user to use more filters key
when listing instances.
In order to stay coherent, all existing instance filters who are
related to a field readable by default to non admin users when showing
instance details, should be allowed by default without policy
modification.
Implements: blueprint non-admin-filter-instance-by-az
Change-Id: Ia66d3a1ceb74ed521cf44922929b2a502f3ee935

Previous patches in the blueprint implemented the support for live
migraton with qos ports and added functional test coverage for the
various live migration scenarios. So this patch removes the API check
that rejected such operation and document the new feature.
Change-Id: Ib9ef18fff28c463c9ffe3607d93428b689dc89fb
blueprint: support-move-ops-with-qos-ports-ussuri

This is to fix the following typos:
 regadless
 users's
 possile
 caculated
Change-Id: If47adefa59f01b7f2ebcee5b7e07e0260df113e4

This is to fix the duplicated words issue like
"if it was specified during during boot time".
Change-Id: I0ffedb8c40399a4b0ecd83b142bdeaf556d1568f

This legacy service is no longer used and was deprecated during the
Stein cycle [1]. It's time to say adios and remove them in their
entirety. This is pretty straightforward, with the sole exception of
schema for the 'remote-consoles' API, which has to continue supporting
requests for type 'xvpvnc' even if we can't fulfil those requests now.
[1] https://review.opendev.org/#/c/610076/
Part of blueprint remove-xvpvncproxy
Depends-On: https://review.opendev.org/695853
Change-Id: I2f7f2379d0cd54e4d0a91008ddb44858cfc5a4cf
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>

The API guide on faults has a section on server actions
but just links to the API reference. There is also the
"Asynchronous faults" section at the bottom which refers
to the server actions API but does not go into details.
This fleshes that section on server actions out a bit more
and includes a walkthrough on the command line of a failed
resize due to NoValidHost. I chose this example since the
server status does not change to ERROR so there is no fault
to see in the server details when the resize action fails.
Change-Id: I95130eeeac1ef8f4676925fe6c0e2d386d88259c

Get excited, people. It's finally dying, for real. There is a lot more
doc work needed here, but this is a start. No need for a release note
modification since we've already said that nova-network has been
removed, so there's no point in saying that the service itself has been
removed since that's implicit.
Change-Id: I18d73212f9d98bc75974a024cf6fd872fdfb1ca4
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>

The todos here are just kind of embarrassing so rather than try
to flesh these out we should just remove them.
The alternative would be writing something at a very high level
like 'the compute service works with the image service to get
the guest operating system image and create snapshots', 'the
compute service works with the block storage service to use
persistent volumes as root volumes with an operating
system image or data volumes', 'the compute service works with
the networking service to provide network access to the server
and firewall rules using security groups' and then link to the
glance, cinder and neutron docs, but I'm not sure it's worth it.
Change-Id: I7f7ca4cc5781a93c0b7b508b5c29e1f6f28194ad

This is pretty basic. As for the todo about using named
personas I've just removed that since I don't think at
this point anyone is going to work on assigning names like
Bob and Sally to roles in the guide and consistently use them.
Change-Id: I88dfb23c0e2750dc37d32dfc431b62baed7238b6

This just gives a high level about how the compute and network
service interact along with the most important networking
resources (ports, networks, security groups and floating IPs).
Change-Id: I05da918c7c7edcca34f54afe80786e8f25f9a029

This adds high level details and links to more detailed docs
in both nova and glance.
This also un-hides the document so it shows up in the table of
contents on the main page. Previously it was being linked from
the general info page and some typos in there are fixed as well.
Change-Id: I495b9ae7855d36cb2597272bcae7ec00360bbed4

When these docs were initially imported [1] the "Considerations"
section was after "Server personality" but [2] refactored the
latter and forgot about the former. This fixes that so
"Considerations" is now just nested into the "Server personality"
section.
[1] I6d5ba6b96772e0f17fc7484312222110ec103bbf
[2] Ibbffdb00d1529a386f338da4776eb614891731c8
Change-Id: Ife6acefd42b9b2f5910fe8f0b1b001eeec80ff06

Change-Id: I14df54db9864e6df8a8bbc0fcc50e2cbb0ba6793

Two things here:
1. The API guide was missing the hyper-v driver which supports
 the suspend operation. Rather than hard-code a list of supported
 drivers in the doc, this change just links to the entry in the
 feature support matrix.
2. The supported hypervisors mention in the API reference is removed
 because the end user using the API should not need to know or care
 what backend hypervisor on which their server is running. They can
 either suspend or not, but we don't need to mention the supporting
 drivers for that in the API reference.
Change-Id: Ib76779a8e34b2c68b0f4af190f71576180360d0f
Related-Bug: #1815403 

The description of "SUSPENDED" server status was misguiding. Rewording
it to make it more accurate.
Change-Id: Ie93b3b38c2000f7e9caa3ca89dea4ec04ed15067
Closes-Bug: #1815403 

Previous patch [1] implemented support for evacuation of servers with
neutron ports having resource request. So this patch removes the API
check that rejected such operations and document the new feature.
[1] Id9ed7a82d42be8ffe760f03e6610b9e6f5a4287b
Change-Id: Id5f2f4f22b856c989e2eef8ed56b9829d1bcefb6
blueprint: support-move-ops-with-qos-ports-ussuri

Add the 'security_groups' parameter as available infomation
in the "List Servers Detailed" API (GET /servers/detail)
when there is a down cell.
Change-Id: I9653a46478c80b343a5a5f8d6b1607688538b1d5
CLoses-Bug: #1846559 

This patch adds functional coverage for such resize. No extra code
changes were needed on top of the migrate patches to make resize work.
Now that the nova code supports the resize this patch removes the
check from the API that rejected the operation.
Note that in the spec [1] we agreed not to introduce new microversion
for this change but treat it as a bugfix. The current change also makes
it possible to accept the resize of these servers with _any_
microversion.
[1] https://specs.openstack.org/openstack/nova-specs/specs/train/approved/support-move-ops-with-qos-ports.html#rest-api-impact
blueprint: support-move-ops-with-qos-ports
Change-Id: Id0ee10e8d323786f4d79c82e3f05b48e76bd0956

Now that the nova code supports such a migration this patch removes the
check from the API that rejected the operation.
Note that in the spec [1] we agreed not to introduce new microversion
for this change but treat it as a bugfix. The current change also makes
it possible to accept the migration of these servers with _any_
microversion.
[1] https://specs.openstack.org/openstack/nova-specs/specs/train/approved/support-move-ops-with-qos-ports.html#rest-api-impact
Change-Id: I4bda569cc7c247e83219276282724c77e760ddcd
blueprint: support-move-ops-with-qos-ports

In a future change, the use of 'hw:cpu_policy' will require a host to
report PCPU inventory. Rather than modify the fake driver used in these
tests to report such inventory, just use a different extra spec,
'hw:numa_nodes'. This has the added bonus of being supported by both the
libvirt and Hyper-V virt drivers, unlike 'hw:cpu_policy' and
'hw:mem_page_size', which are only supported by the libvirt virt driver.
Change-Id: Id203dc07f08557b1b094ec72e1df3493ec9524b1
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>

The section about server actions is talking about the
os-instance-actions API but linking to the server actions
section in the API reference. This retains the link to the
server actions section but changes the "for more details"
link to the actual os-instance-actions reference.
Change-Id: Icb5c4cadd1c552f44bfadefd055c07ea38df0eaa

The api documentation is now published on docs.openstack.org instead
of developer.openstack.org. Update all links that are changed to the
new location.
Note that Neutron publishes to api-ref/network, not networking anymore.
Note that redirects will be set up as well but let's point now to the
new location.
For details, see:
http://lists.openstack.org/pipermail/openstack-discuss/2019-July/007828.html
Change-Id: Id2cf3aa252df6db46575b5988e4937ecfc6792bb

This patch explains what scheduler hints are, gives an example
request, then links to the api-ref for details.
Co-Authored-By: Artom Lifshitz <alifshit@redhat.com>
Change-Id: I1c55b9a17e9d9f57d9ec1383680cccb6ae7fcfaf

Turns out we've a *lot* of disparate metadata systems. Attempt to both
link them somewhat through extensive cross-referencing and extract out
deployment-specific stuff from user-facing docs. Lots of changes here,
but in summary:
- Split out admin-focused content from the metadata API, config drive,
 user data and vendordata docs.
- Merge the config drive, metadata service, vendordata and user-data
 user docs, which are mostly talking about the same thing and are
 fairly barren without the deployment components
- Make use of various oslo.config and Sphinx roles
Side note: I miss when we have tech writers to do this stuff for us :(
Change-Id: I4fb2b628bd93358a752e2397ae353221758e2984
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>

We're going to remove all the code, but first, remove the docs.
Part of blueprint remove-consoleauth
Change-Id: Ie96e18ea7762b93b4116b35d7ebcfcbe53c55527
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>