openstack/devstack - devstack - OpenDev: Free Software Needs Free Tools

Those historic references to port 5000 and 35357 aren't being used
anymore for some time, so let us drop them.
Clean up some python2/3 wording along the way.
No longer mention Identity API v2, which is also a thing of the past.
Change-Id: Iafff097eee082f24ea2ae27ad038ad115aa36c61

Now Apache2 has 5 dedicated processes for Keystone Admin and 5 for
Keystone Public. As each Apache process consumes some memory and
we arbitrarly decided 5 was a good number more than 2 years ago,
maybe now (with the recent memory pressure we feel) is a good time
to reconcider.
With 5 processes our peakmem_tracker.py script reports a max RSS size
for the "wsgi:keystone-ad" and "wsgi:keystone-pu" processes of
2 (public and admin) * 5 (number of processes) * 90 Mo (RSS of each
process) = 900 Mo.
With 3 processes, the overall max RSS for Keystone is
2 * 3 * 90 = 540 Mo.
Note that this is RSS memory, but using the "smem" linux command on
my laptop, I noticed that the USS (Unique set size, i.e RSS excluding
shared memory) is around 80Mo per process. So reducing the number of
processes will actually reduce memory consumption.
Change-Id: Iba72d94aa15ecaa87c0115ad26d6bpeakmem_tracker62d5b3bea0a

The /identity_admin endpoint is the port 80/443 equivalent of the
service that typically runs on port 35357. In v2 some operations must be
performed on the admin endpoint whereas on v3 the services on 5000 and
35357 are exactly the same. This would be why the service was mounted at
/identity_v2_admin however that is misleading because both the v2 and v3
services are present on that endpoint.
This is particularly confusing because we set this as the OS_AUTH_URL
endpoint and it makes it seem like we are doing v2 authentication when
we are not.
Change-Id: If73735026079fb19ca5bd44b3a4dc1f507b5c99d

Add a VirtualHost that defines the necessary options for
enabling SSL. The existing keystone Apache configuration already
does all the location handling.
Change-Id: I836a471a7258f14f051d3dd8bdb428286b5a11aa

When running in httpd, keystone accepts requests on /identity and
/identity_v2_admin.
The path endpoints should be preferred over the ports so keystone
is configured to point applications to the path endpoints by
setting admin_endpoint and public_endpoint.
Change-Id: I34569b9e03c3f36748c92d803349e22a7ee1a633

Remove the microseconds from the apache logs and move back to using
milliseconds. There is no longer any 2.2 workarounds in the keystone
setup process.
Change-Id: I8787eee41fbde1f9794aeffe1e862af0d5117bc3

Commit 2ad1a42ca6 broke entirely the
Apache configuration for Keystone when used without a port on the
/identity URL. This patch fixes that.
Change-Id: I47805138c66456c9c5fa9af1f4ac33b03d0ce5b9

Devstack was setting up a separate directory and copying
http/keystone.py into it for the admin and public endpoints.
Keystone now defines wsgi_scripts entrypoints so that
keystone-wsgi-admin and keystone-wsgi-public are created on
install so devstack can reference these files instead.
See http://httpd.apache.org/docs/2.4/upgrading.html#access for
the apache docs with examples for the Allow|Deny/Require
directives.
Depends-On: Ic9c03e6c00408f3698c10012ca98cfc6ea9b6ace
Change-Id: Ided688be62b64066d90776313c963ec5016363f2

When configured to run under Apache Httpd, keystone will also
handle requests on /identity (public) and /identity_admin (admin).
Change-Id: I4e6eb0cad1603aa0e612d0adc5431565da93870e

Add the response time to keystone's access log for each request. This
will be the last element in the log-line and will be represented in
microseconds.
Change-Id: I19204369af5cdf06df2237550c350dfb3ffc995d

Configure Apache to use the Keystone venv.
Change-Id: I86f1bfdfd800f5b818bfb5c4d2750ff732049107

el6 is shipped with Python 2.6.x which is not expected
to be supported with the openstack kilo release.
For el6 support we need to do lot of thing differently,
which makes the code more complicated.
This change removes el6 and py26 support from devstack.
This change also removed a discontinued (1 year ago)
openSUSE 12.2 code path, which used a similar codepath as el6.
Several comment related to el6 also removed or modified.
Change-Id: Iea0b0c98a5e11fd85bb5e93c099f740fe05d2f3a

Accidentally only added WSGIPassAuthorization to the public port,
like all the other WSGI props, it should be added for both ports.
Change-Id: I4e52e0881df464dfb7b28e22581f462e14e37bdb

For the OS-OAUTH1 Keystone extension to fully work under Apache,
the WSGIPassAuthorization parameter must be set to On, rather
than the default of Off. This will make functional testing of
this extension much easier.
Change-Id: I5dcbdd27e7ef7a60fe3c7cb8b9c3c83b4197dfc1

Configure nova, cinder, glance, swift and neutron to use SSL
on the endpoints using either SSL natively or via a TLS proxy
using stud.
To enable SSL via proxy, in local.conf add
ENABLED_SERVICES+=,tls-proxy
This will create a new test root CA, a subordinate CA and an SSL
server cert. It uses the value of hostname -f for the certificate
subject. The CA certicates are also added to the system CA bundle.
To enable SSL natively, in local.conf add:
USE_SSL=True
Native SSL by default will also use the devstack-generate root and
subordinate CA.
You can override this on a per-service basis by setting
<SERVICE>_SSL_CERT=/path/to/cert
<SERVICE>_SSL_KEY=/path/to/key
<SERVICE>_SSL_PATH=/path/to/ca
You should also set SERVICE_HOST to the FQDN of the host. This
value defaults to the host IP address.
Change-Id: I36fe56c063ca921131ad98439bd452cb135916ac
Closes-Bug: 1328226

This change uses mod_version (shipped by default on everything we care
about) to set-up version-specific config within apache rather than
within devstack scripts.
Clean up the horizon and keystone config file generation to use the
internal apache matching.
Since I6478db385fda2fa1c75ced12d3e886b2e1152852 the apache matching in
'functions' is actually duplicated. just leave get_apache_version in
lib/apache as it is used for config-file name matching in there.
Change-Id: I6478db385fda2fa1c75ced12d3e886b2e1152852

Use the new ErrorLogFormat directive to make the Keystone logs
under Apache to look like the standard oslo log format.
Change-Id: Ie823abf2fa06b8ce22027c21bef455808a4a768e

Keystone's access log was going to httpd/access.log, which is the
common place for all access logging. This made it difficult to see
Keystone accesses apart from other access. Keystone's access log
will now be keystone_access.log
This makes the Keystone configuration similar to Horizon which uses
horizon_access.log.
Change-Id: I6e5ac121302b3d138758e6c49dffa9f05ad2fb85
Partial-Bug: #1359995 

Having Keystone's log level at debug caused a lot of uninteresting
Httpd-related log lines to be displayed which makes debugging more
difficult than it should be.
Rather than set the log level explicitly, Keystone will use the
Httpd server's setting, which defaults to warn.
Partial-Bug: #1359995
Change-Id: Ieef882944bafd98f7b27497a5276124b21c3e576

The normal extension for log files is .log. Log rotation should
work because the extension is as expected.
Change-Id: Ia5e42ea9d953e8395b86ab58cdef6c2d852efc94
Related-Bug: #1359995 

This change makes apache set process group name in the process name,
so when listing the processes the user can easily identify what's
running in each apache process. This is specially useful to debug
memory consumption or when a process consumming too much CPU.
Change-Id: I9787980052f451f452d3b8e5e51385ad5aa01e6a

I think since probably fdf1cffbd5
devstack on RHEL6 fails and it comes down to:
---
2014年07月17日 05:05:49.235 | +++ openstack role create admin -f value -c id
2014年07月17日 05:05:49.615 | ERROR: cliff.app Service Unavailable (HTTP 503)
[Thu Jul 17 15:05:46 2014] [error] [client 10.0.2.15]
 (13)Permission denied: mod_wsgi (pid=30125): Unable to connect to WSGI
 daemon process 'keystone-public' on
 '/etc/httpd/logs/wsgi.30098.0.1.sock' after multiple attempts.
---
The apache user doesn't have permissons to this directory. Adding
WSGISocketPath to /var/run solves it
Change-Id: If4b74019b6bd389b576fc981154bb1b3aa471c9b

Provide a template for running keystone as a mod_wsgi process and enable
it from configuration.
Based on: https://review.openstack.org/#/c/36474/
Also-by: zhang-hare <zhuadl@cn.ibm.com>
Implements: blueprint devstack-setup-apache-keystone
Change-Id: Icc9d7ddfa4a488c08816ff4ae0b53c0134a1016b