openstack/nova - nova - OpenDev: Free Software Needs Free Tools

Moves the suspend/resume server functionality out of admin_actions into
its own extension. This part of the blueprint v3-admin-actions-split
allows more selective enablement of features contained in the admin
actions extension.
Note that XML api samples are no longer generated because
bp remove-v3-xml-api has been approved.
Partially implements bp v3-admin-actions-split
DocImpact: Adds os-suspend-server extension and moves suspend/resume
functionality out of os-admin-actions into this new extension
Change-Id: Ie2ad1c6085d65fee397d6ad5b5c9f3bd8e82ad3c

Moves the pause/unpause server functionality out of admin_actions into
its own extension. This part of the blueprint v3-api-admin-actions-split
allows more selective enablement of features contained in the admin
actions extension.
Note that XML api samples are no longer generated because
bp remove-v3-xml-api has been approved.
Partially implements bp v3-api-admin-actions-split
DocImpact: Adds os-pause-server extension and moves pause/unpause
functionality out of os-admin-actions into this new extension
Change-Id: Ib9cce57e2ff1270a82b9d7e39b23ec6b532b9e77

Removes the XML namespace definitions from the V3 API
plugins declarations. Deliberately avoids removing
the XML namespace definition from admin_actions to avoid
yet another rebase of the admin_actions split series.
Fixes up associated tests and requirement for V3 API extensions
to define a namespace property.
Partially implements blueprint remove-v3-xml-api
Change-Id: If3e1314cefdc1abcdd5eca44bcc2282cac664f05

Moves the lock/unlock server functionality out of admin_actions into
its own extension. This part of the larger
blueprint v3-api-admin-actions-split allows more selective enablement of
features contained in the admin actions extension.
Some setup work is done in the tests directory with an
admin_only_action_common.py file. This allows tests which are
split out from test_admin_actions (as their corresponding features
are separated from the admin_actions extension) can continue to
share code.
Note that XML api samples are no longer generated because
bp remove-v3-xml-api has been approved.
Partially implements bp v3-api-admin-actions-split
DocImpact: Adds os-lock-server extension and moves lock/unlock
functionality out of os-admin-actions into this new extension
Change-Id: Ie4b6e856c2f5c33de5575aa8666e0b2784b58d05

This is a step toward removing XML support from the v3 compute API.
First, we need to remove all of the tests that would fail once we start
ripping out the real code.
Part of blueprint remove-v3-xml-api
Change-Id: Ie2c280d5ea1e6b8756535deb05982a92561552db

Allow fake computes to deploy images with hypervisor_type
defined and set to "fake" and by extension to be used in
multi hypervisor_type deployments.
Mixing fake and "real" computes on the same OpenStack
deployment allows to create many fake instances and some
working instances for testing applications on top of
OpenStack (ie: fake instances for stress tests, working
ones for functional tests).
Closes-bug: #1260771
Change-Id: Idc337c4a7ed024f236ca2b60d91e2c30f7d54536

The scheduler filter MetricsFilter filters out those hosts which don't
have the metrics data available as required by metric settings.
This is part of the blueprint utilization-aware-scheduling.
DocImpact: Added a new metrics filter.
Change-Id: Ib4a898774daf683c4496ef3e9953d23027f11ac0

Noted that building the documentation has more dependencies
than merely running Nova.
Noted that the standard development environment is sufficient.
Noted that the standard runtime environment plus oslo.sphinx
is sufficient.
Change-Id: I37b84a1bdfbc906b005c0a53c2c570c0695937b3
Closes-Bug: #1266623 

The preserve_ephemeral option for rebuild will preserve the content of
the ephemeral partition, making stateful golden image based
deployments possible even when clouds haven't deployed Cinder, or the
hypervisor doesn't support Cinder (e.g. BareMetal / Ironic).
Partial-Bug: #1174154
blueprint: baremetal-preserve-ephemeral
Co-Authored-By: Robert Collins <rbtcollins@hp.com>
Change-Id: Id33d5d4107f89814842a3f0b7f33690dd7e3aadc

All pci passthrough code is merged into nova, but the APIs do not support.
this patch extends os-hypervisor to show PCI information for instance and
compute node. This is a part of PCI APIs, for more information see the
blueprint.
Partially implement blueprint pci-api-support
Change-Id: Ia366df257f077c74b1a6639b7078ba4230d3f1a7

As v3 api rules, the extended attribute should be with extension
alias as prefix. So fix this case for flavor-rxtx v3 extension.
Closes-bug: 1252986
DocImpact
Change-Id: I9f5984a406d039245db4bef2df53cfe10a331cc2

Isolates hosts based on image properties and aggregate metadata
 - If a host doesn't belong to any aggregate it can create instances
 from all images.
 - if a host belongs to an aggregate and if this aggregate defines
 metadata that match with the image properties then the host
 is a candidate to boot the instance.
DocImpact
Change-Id: I3b9325e2e103f3bb6eed66789ac2c82941e94397
Implements: blueprint aggregate-host-isolation-based-image-properties

This patch removes disk-config extension for v3 api, and also
remove related tests. And remove resize extension point for
servers core because there isn't any extension using it anymore
and there isn't good way to test resize extension point.
Implements bp api-v3-remove-disk-config
DocImpact
Change-Id: I894ae6f4e0d6c9f956a71e8e061ccb3fa1b039d8

This patch added 2 command line parameters which are used to
connect to an external debugger such as pycharm or eclipse.
This feature is used when you want to connect to a nova
service via a debugger running on a different host.
To use it you start the nova service with the following
command line parameters
--remote_debug-host <where the debugger is running>
--remote_debug-port <port> it's listening on>.
DocImpact
Closes-bug: #1251021
Change-Id: I6ede9bf0813eafbeb91d858c297d4c160aafceba

The ratelimit middleware isn't really useful. That is pointed out
by https://review.openstack.org/#/c/34821/. And it didn't get any
object from mail-list:
http://lists.openstack.org/pipermail/openstack-dev/2013-November/020291.html
This patch remove ratelimit middleware and limits extension. The config option
'api_rate_limit' is only valid in v2 api. The pipeline factory for v3 api won't
check that option anymore. For compatibility, if user is still using old
'api-paste.init', pipeline factory will ignore to load ratelimit middleware, and
print warning message to user for notice ratelimit is deprecated in v3 api.
Closes-bug: 1255471
DocImpact
Change-Id: Ifeae0504e11089f95e4d8af58bcb7372dca87f81

The weight system is being used by the scheduler and the cells code.
Currently this system is using the raw values instead of normalizing them.
This makes difficult to properly use multipliers for establishing the
relative importance between two wheighers (one big magnitude could
shade a smaller one). This change introduces weight normalization so
that:
- From an operator point of view we can prioritize the weighers that
 we are applying. The only way to do this is being sure that all the
 weighers will give a value in a known range, so that it is
 not needed to artificially use a huge multiplier to prioritize a
 weigher.
- From a weigher developer point of view, somebody willing to implement
 one has to care about 1) returning a list of values, 2) setting the
 minimum and maximum values where the weights can range, if they are
 needed and they are significant for the weighing. For a weigher
 developer there are two use cases:
 Case 1: Use of a percentage instead of absolute values (for example, %
 of free RAM). If we compare two nodes focusing on the percentage of free
 ram, the maximum value for the weigher is 100. If we have two nodes one
 with 2048 total/1024 free, and the second one 1024 total/512 free they
 will get both the same weight, since they have the same % of free RAM
 (that is, the 50%).
 Case 2: Use of absolute values. In this case, the maximum of the weigher
 will be the maximum of the values in the list (in the case above, 1024)
 or the maximum value that the magnitude could take (in the case above,
 2048). How this maximum is set, is a decision of the developer. He may
 let the operator choose the behaviour of the weigher though.
- From the point of view of the scheduler we ensure that it is using
 normalized values, and not leveraging the normalization mechanism to the
 weighers.
Changes introduced this commit:
1) it introduces weight normalization so that we can apply multipliers
 easily. All the weights for an object will be normalized between 0.0 and
 1.0 before being sumed up, so that the final weight for a host will be:
 weight = w1_multiplier * norm(w1) + w2_multiplier * norm(w2) + ...
2) weights.BaseWeigher has been changed into an ABC so that we enforce
 that all weighers have the expected methods.
3) weights.BaseWeigher.weigh_objects() does no longer sum up the
 computer weighs to the object, but it rather returns a list that will be
 then normalized and added to the existing weight by BaseWeightHandler
4) Adapt the existing weighers to the above changes. Namely
 - New 'offset_weight_multiplier' for the cell weigher
 nova.cells.weights.weight_offset.WeightOffsetWeigher
 - Changed the name of the existing multiplier methods.
5) unittests for all of the introduced changes.
Implements blueprint normalize-scheduler-weights
DocImpact: Now weights for an object are normalized before suming them
up. This means that each weigher will take a maximum value of 1. This
may have an impact for operators that are using more than one weigher
(currently there is only one weigher: RAMWeiger) and for operators using
cells (where we have several weighers). It is needed to review then the
multipliers used and adjust them properly in case they have been
modified.
Docimpact: There is a new configuration option 'offset_weight_multiplier'
in nova.cells.weights.weight_offset.WeightOffsetWeigher
Change-Id: I81bf90898d3cb81541f4390596823cc00106eb20

The injected_files, injected_file_content_bytes and
injected_file_path_bytes are all related to the
os-personalities extension which has been removed. They
can't be removed from the core quotas code as that has
to still support the V2 API. So for the V3 API we just
hide them.
DocImpact
Partially implements blueprint v3-api-remove-extensions
Change-Id: Iedb6e1427ac22f1926e929f3391e56403c37d088

The new metrics weigher can compute the weight based on the compute
node host's metrics data. The to-be weighed metrics and their
weighing ratio are specified in the configuration file as the
followings:
 metrics_weight_setting = name1=1.0,name2=-1.0
The final weight would be name1.value * 1.0 + name2.value * (-1.0).
This is part of the blueprint utilization-aware-scheduling.
DocImpact
Change-Id: Ib3e68505e6d4d8f6d67b54c5f00de3e1c172738c

All pci passthrough code is merged into nova, but the API does not support.
this patch extends the servers to show PCI information for instance and compute
node. This is a part of PCI APIs, for more information see the blueprint.
Partially implement blueprint pci-api-support
Change-Id: I99eeeca67983c49433ab0079f490a454d250f5e0

Adds api sample tests and api samples for shelve
Partially implements blueprint v3-api-unittests
Partially implements blueprint v3-api-specification
Change-Id: Ib485ab042a340c4672ad01fb35d065fd52889b07

Removes occurences of the personality extension from the V3 API
samples as the extension has been removed. Cleanup of consistency
of image and flavor ref examples for the api samples.
DocImpact
Partially implements blueprint v3-api-remove-extensions
Change-Id: I43a34097904d3d7e280039c3d5d76479a49e92d0

The parameter of agent update API has been changed from "para" to
"agent" since Nova v3 API, and this patch fixes the sample doc.
Partially implements blueprint v3-api-specification
Change-Id: Ic9c0f11a637e48daa9457ab9ae923fefc0fb350d

As discussed at the summit in the QA stream, this removes
the coverage extension from the V2 API.
According to the API Change Guidelines
https://wiki.openstack.org/wiki/APIChangeGuidelines
we should not be doing this. However the coverage
extension is a rather special case. It has security issues
such that it shouldn't be used in a production cloud
environment. It also doesn't work properly so its usefulness
to measure API coverage is limited. And in the past the presence of
this plugin has been used as a reason not to accept
an alternative technique to measure API coverage.
This is the proposed replacement
https://review.openstack.org/#/c/25882/
Change-Id: I07d798129ee277a6f7691c25f88c07a5204c0943

* Add expected_errors decorator.
* Remove 'absolute_limit' from response, that already return by
extension quota_sets. And update api samples.
Partially implements bp nova-v3-api
Change-Id: Iea3ff494b8abed77fdc4b9e5e552784ed3733c34

Adds the flavor-access plugin to the core list for the V3 API.
This is part of the process of adding all of the flavor related
extensions to core, with the exception of rxtx, as per the discussion
here: https://etherpad.openstack.org/p/NovaV3APICore
Note that since the alias changes from os-flavor-access to flavor-access
the api samples move directory and quite a few of other api samples
as well as flavor access is included by default.
Partially implements blueprint nova-v3-api
DocImpact
Change-Id: I9abc7df8f4620d462b0d381a2c0ed180c48f220e

Adds api sample tests and api samples for access_ips
Change-Id: I9d9249a48e7f7ccdff0146fbe5b25463d66a93a7