openstack/devstack - devstack - OpenDev: Free Software Needs Free Tools

Add USE_PYTHON3 and PYTHON3_VERSION variables to allow services to use
python 3 if they indicate support in their python package metadata.
Tested in Heat here -> I837c2fba682ab430d50e9f43913f2fed20325a7a.
Project config change to add a dedicated job to Heat is here -> I0837e62d6ccc66397a5e409f0961edd4be31f467
Change-Id: I079e18b58b214bf8362945c253d6d894ca8b1a6b

valu3 => value3
enviromnet => environment
direcotry => directory
virualenv => virtualenv
editiable => editable
envirnment => environment
Change-Id: I97fb2d44a37b16d02d4fbdb08bfa33414349f651

Change-Id: I368fec44858bd97fc6a314fb20eed2b10932cbb1

Ia0957b47187c3dcadd46154b17022c4213781112 proposes to have bashate
find instances of setting a local value. The issue is that "local"
always returns 0, thus hiding any failure in the commands running to
set the variable.
This is an automated replacement of such instances
Depends-On: I676c805e8f0401f75cc5367eee83b3d880cdef81
Change-Id: I9c8912a8fd596535589b207d7fc553b9d951d3fe

Ia0957b47187c3dcadd46154b17022c4213781112 detects setting local
variables with subshell commands.
Although this is a particuarly benign case, it trips the test. Rather
than putting in an ignore for this, we can easily change it to make
the test pass. This seems better than putting in special work-arounds
to bashate, etc.
Change-Id: I37c3967c0f2d780a636a7d26cda83755085c5c69

When moving to Python 3, we also need to install test-requriements
to allow projects to install any python 3 test dependencies they
might be missing otherwise.
Change-Id: I2d19aa2f7ec8de869a82aa7764ab72cc8693101f

after the glance_store vs. upper-constraints bug, it's probably worth
actually enforcing and sanity checking that devstack is doing what
it's being asked of with LIBS_FROM_GIT. This will hopefully reduce
user generated error.
This *might* not work with the current oslo naming, we'll have to test
and normalize that.
Change-Id: Iffef2007f99a0e932b68c4c897ebbfb748cac2b4

Having behavior on your laptop diverge from behavior in the gate is
confusing. Just use constraints on every devstack run to be consistent.
Users of devstack can edit the requirements repo in order to change
these constraints locally if necessary.
Change-Id: I843208e2e982eb04931b76f5cb4bd219fbcd70de

Libraries were resulting in two edit-constraints runs:
 - one for the library, which adds a non-editable file path
 - then one for the editable servers, but that fails becuse
 pkg-resources couldn't parse the prior entry.
This is fixed in two parts: the dependent patch which supports parsing
file urls that have egg names, and this patch which changes from a
file path to a file url with an egg name.
Change-Id: I0f07858e96ea3baf46f8a453e253b9ed29c7f7e2

Constraints files allow a global view of dependencies for devstack
without the side effect that requirements files have of installing
everything everytime. This is part of the cross project
requirements-management spec.
Change-Id: If089d30146629e6cf817edd634e5c2b80f1366dd

Before the fix, requirements soft-update was used for projects that are
in the file.
Change-Id: I095d42521f54b45a6b13837e2f8375fa04532faa
Closes-Bug: #1469067 

We're adding the ability to have devstack plugins, which should be
much more free to require new things not in global requirements. Our
old thinking of locking down all the requirements doesn't really work
in a plugin model.
Instead, if the project is in projects.txt, continue with the old
behavior. If it is not, do a soft update (update all the requirements
we know about, leave the ones we don't). This was previously the SOFT
requirements update mode, but now it will just be the default.
Change-Id: Ic0c6e01a6d7613d712ac9e7e4a378cc3a8ce75e6

This is to prevent any possible contamination of test results from its
presence.
Change-Id: I5a929854745650cc6a182ffc4d15c50caabdd727

The install_infra() call needs to be done earlier since
pip_install_gr() depends on it. Also the fact that python module names
are supposed to be lower case but some use camel case is a problem
(for example with XenAPI).
Change-Id: I7012d77134fa0d9c746d87e837934e7dcb337e5c
Closes-Bug: #1441820 

This creates a new pip_install_gr that installs from global
requirements allowed versions. Now that stable branches are getting
capped all of devstack needs to be fixed to do things like this.
Change-Id: I8fd0ef2bfc544ca2576fab09d3018f760b8848fe

The original removal is in https://review.openstack.org/#/c/167669/
Change-Id: I3c59f040523d2cd1453465e80280955218880634

Make it possible for someone to config
 PIP_UPGRADE=True
in local.conf and thus force pip_install calls to upgrade. In
automated testing this is probably a bad idea, but in manual testing
or situations where devstack is being used to spin up proof of
concepts having the option to use the latest and greatest Python
modules is a useful way of exploring the health of the ecosystem.
To help with visibility of the setting, and section has been added
in configuration.rst near other similar settings.
Change-Id: I484c954f1e1f05ed02c0b08e8e4a9c18558c05ef

devstack is a development and test environment, but by default we were
only installing the runtime dependencies. We should install all the
testing required packages as well.
Change-Id: I7c95927b9daad15766aac9d1276b10ca62efb24c

If we set mutiple proxy (ip or url), pip install will
treat the second proxy as a command.
Add quotation marks around proxy.
expecially for no_proxy
Change-Id: I38ad3f083ba5155cda0e5e2b8f5df64492b7fecd

We weren't actually using the wheels since PIP_FIND_LINKS environmental
variable was getting lost during the sudo
Change-Id: I4a89a70df63772a16ee5a8c3f1cd86e9c7bb5242

Introduce the tooling to build virtual environments.
* tools/build_venv.sh: build a venv
* introduce lib/stack to house functionality extracted from stack.sh that
 is needed in other places, such as Grenade; start with stack_install_service
 to wrap the venv install mechanics
* declare PROJECT_VENV array to track where project venvs should be installed
* create a venv for each project defined in PROJECT_VENV in stack_install_service()
Change-Id: I508588c0e2541b976dd94569d44b61dd2c35c01c

Instead of specifying the version of a library in devstack, use the version from
global-requirements
Add new function get_from_global_requirements and use it
where it makes sense.
Change-Id: I6b2f062761ac05ef72fc6cc9993bc204faf06fa5

Set PIP_VIRTUAL_ENV to install the package(s) into an existing virtual
environment. This works by simply using the pip command already
in the venv, and not using sudo.
Change-Id: I910e1752e58a666174f83b4f97e547851e66e655

Move Python-related functions into inc/python
Should be transparent to all callers as it is sourced from functions-common
Change-Id: I88043830cef9211b4e0baa91bfcc7a92125afa9f