openstack/openstack-ansible-haproxy_server - openstack-ansible-haproxy_server - OpenDev: Free Software Needs Free Tools

At the moment for some reason we're not taking into account default
variables haproxy_rise/haproxy_fall but instead trying to count
based on amount of backends. This makes quite little sense to
depend amount of backend rechecks on amount of backends overall,
so we're chaning behaviour to pre-defined variables that already exist.
Change-Id: I1e53a997f6f443718ea2c6bdfbe8a0b98c44896d

Haproxy config check(/usr/sbin/haproxy -c -f /etc/haproxy/haproxy.cfg)
returns 3 warnings:
1. keyword 'forceclose' is deprecated in favor of 'httpclose', and will
not be supported by future versions.
2. backend 'galera-back' : 'option tcplog' directive is ignored in
backends.
3. 'http-request' rules ignored for backend 'galera-back' as they
require HTTP mode.
This change fixes 1. and 2.
Fixing 3. will be a bit more tricky as it's a part of
`openstack_haproxy_stick_table` defined in
/opt/openstack-ansible/inventory/group_vars/haproxy/haproxy.yml
Change-Id: Idaa4b5580039857435f90416924dee26a702deba

Right now we don't ensure haproxy conf validity and if it's incorrect
role will fail on attempt to reload haproxy. However it's really worth
adding validation step and do not proceed if configuration is wrong
Change-Id: I54717d4f7230b8d8dff2d293592831cc88c51d24

Currently there is no way of disabling SSL connection for stats frontend
as it implies more global variable. However, for some systems consuming
self-signed root certificate might be not an option and disabling
SSL verification tricky. Thus, we introduce new variable that allows to
nicely control if SSL should be served for stats frontend or not.
Change-Id: Ic4bc4393ec89469876e9e95b12bb9c4069972713

Add `haproxy_ssl_letsencrypt_domains` variable, which
contains a list (defaults to `external_lb_vip_address`)
for `--domains` certbot option.
Change-Id: I2ebfff9eeb5279a3964b8578a6e66aa132d763f5

This line was introduced by Ib4f33185202b694b9611cc5fd6323c30a1c8d489
for multi-os support, but should since be covered by the
distribution_major_version line above, introduced at a later date.
Change-Id: I23a8e7aaa3858bce47dcf7610acf1ee58d9e1fc1

In some user scenarious (like implementing DNS RR) it might be useful to
bind on 0.0.0.0 but at the same time do not conflict with other services
that are binded to the same ports. For that, we can specify a specific
interface, on which haproxy will be binded to 0.0.0.0.
In netstat it would be represented like `0.0.0.0%br-mgmt:5000`.
With that we also allow to fully override `vip_binds` if assumtions
that role make are not valid for some reason.
Change-Id: Ic4c58ef53abc5f454b6fbebbd87292a932d173ae

Right now we assume, that ca-cert is always present. Though, it might
not be the case for user-provided certs or let's encrypt, as they
are already in ca-certificates.
Change-Id: I101f82c5e378596e76a160aacb34a9e1e7e0c123

Based upon usual recommendations from:
https://hynek.me/articles/hardening-your-web-servers-ssl-ciphers/
Change-Id: I6e549ab3ffcacebe04e188cbf34d8707fb0fe05d

A new variable "haproxy_stick_table" is added which allows a custom
stick-table to be supplied that is used as the default stick-table
for all haproxy back-ends.
In addition, the variable service.haproxy_stick_table can be defined
for each service to allow a unique stick-table to be supplied for
a particular service.
The old default stick-table definition is removed as there was no
use case defined for it in this role before. An example is added
to defaults/main.yml to show how the custom stick-table can be used
to rate-limit requests that generate 4xx responses which commonly
occur during vulnerability scanning or credential stuffing attacks.
There are many other uses for stick-tables, consult the HAProxy
documentation for details.
Change-Id: I50daba08c10f071157d6450ea2fa97df448f99ec

Instead of hardcoding specific supported tunable options, we
just pass key as an option to haproxy config.
This change might break deployments during upgrades, since format of
values in variable has changed, but appropriate release note was written
We also increase maxrewrite by default, as otherwise usage of CSP leads
to 500 error.
Change-Id: I949960420ed5dbd6d58f0de7dae0ac629a85b7fc
Related-Bug: https://github.com/haproxy/haproxy/issues/1597
Needed-By: https://review.opendev.org/c/openstack/openstack-ansible-os_horizon/+/844815 

With sphinx release of 5.0.0, they changed default for language variable
to 'en' from None. With that current None valuable is not valid and should
not be used.
Change-Id: Ibea41d4f472fa7b375d9d04e94ad621696f565b1

Change-Id: I786353945ca45d16d91678d65610a97798bf98f0

Change-Id: Idea36d3b3fd8cdeb04e76f9b3ec7da24eac99b83

ipaddr filter has been moved from netcommon to utils collection [1]
Based on that we must add that collection to requirements.
[1] db4920ebf6
Depends-On: https://review.opendev.org/c/openstack/openstack-ansible/+/831525
Change-Id: Ib3f982f445cdd944738f897abd0d312e70645bf2

Enable TLS on internal communication has 2 parts
* Enabling TLS on the internal VIPs for haproxy frontends
* Enabling TLS on the service backends
Haproxy has support for enabling TLS on frontends and backends,
but doing so would cause downtime.
In the case of upgrading frontends, enabling TLS would prevent
openstack services from working until their config is changed
from http to https, as they do not follow redirects.
In the case of backends haproxy would mark each backend as down
because if could not initiate a HTTPS connection to the backend
until the backend is updated.
This patch fixes this and allows haproxy to accept both HTTP and
HTTPS on the same well known port for each service. It also
allows for both HTTP and HTTPS backends.
Support for HTTP and HTTPS on the frontend is enabled by setting
haproxy_tcp_upgrade_frontend: true
Support for HTTP and HTTPS on the backend is enabled by setting
haproxy_tcp_upgrade_backend: true
This is a temporary patch and will be removed once instances have
been upgraded to HTTPS for internal communications in a future
release of OSA.
Change-Id: I4279005d5b4e6133cf85ba43379b51149c838f17

Use a first_found lookup instead of a with_first_found loop so that
the 'paths' parameter can be used.
This ensures that only vars from the role are included, and not vars
from a parent calling role. This can happen when a parent role has
a higher priority vars file available for inclusion than the role
it calls.
Change-Id: I65564c23ff0003a575af984c709c1ae365292f35

This adds TLS v1.3 support to the HAProxy role by default, along
with a new variable to manage cipher suites.
The old variable for TLS v1.2 and below ciphers is renamed for
consistency, but is still supported as a default where overridden
by deployments.
Depends-On: https://review.opendev.org/c/openstack/openstack-ansible/+/823943
Change-Id: Iaf9709ac5f5ac8db281a9ec7278cef274186ba15

Change-Id: Iae5c21ee0d604fb015593337815840981ab10ef9

Change-Id: I290188edc92108960fd4287689ad283eff3503e1

If haproxy_backend_ca set to true, default to using system CA
so you dont have to specify the exact CA to use.
Change-Id: I536c32a0b152a2b754787e07574472ecfaebd7e7

This is required if a server only speaks HTTPS
Change-Id: Ib99eed929dfded2bbf11bc1a54c4184edafe8452

Option httplog directive is ignored in backends
http://cbonte.github.io/haproxy-dconv/2.0/configuration.html#4-option%20httplog
Change-Id: I792f22b8e20b3ca34be78a9c87ca0f1050f9225c

We have announced and documented haproxy_frontend_raw variable
while in fact introduced haproxy_raw. Since announced variable name
better reflects its purpose and it was announced,
we rename it to haproxy_frontend_raw in template generation.
Change-Id: I7ba9588b599f42dddad395df1a2e18ccfe6b3fe3

This could be achieved using the
haproxy_ssl_letsencrypt_setup_extra_params variable, but this
makes it a bit neater.
Change-Id: Iee2d5a10e1762b23fcb3f3140950c76a754743b7

Change-Id: I74da6c27a7d0992a45103657cd0800dab8143c4f

With releasing PKI role we broke Let's Encrypt option because of
changing directories where certs should be located
and not reflecting these changes for let's encrypt. At the same time
we should not generate self-signed cert when let's encrypt path is used.
Depends-On: https://review.opendev.org/c/openstack/openstack-ansible/+/811742
Closes-Bug: #1938961
Change-Id: I1a6701b171782528373bc1d0a39e70e6d1ef20ab

We have introduced variables to control CA/certificates regeneration
however never used them anywhere.
This patch fixes that, so that haproxy_pki_regen_cert and
haproxy_pki_regen_ca are respected now.
Change-Id: Id6d5395d5976ec9393a55be7fe9a946cf9ce745e

HAProxy added native Prometheus support from v2.0. This can be
enabled using the existing stats endpoint via an additional
/metrics path.
Change-Id: If9528969c7915db06138c0746dc419d8302f0e7c

In order to remove service currently we need to satisfy one of
conditions:
- haproxy_backend_nodes and haproxy_backup_nodes are empty and defined
but must be defined
- haproxy_service_enabled is False
- state is absent
There's big issue with logic regarding haproxy_backend_nodes and
haproxy_backup_nodes since they both should be defined and empy,
but in case haproxy_backup_nodes is not defined but haproxy_backend_nodes
is empty we should consider this as condition for removal as well.
But this will make it too complicated.
This change suggest consider rely only on haproxy_service_enabled and
state keys of haproxy_service_configs, as it's sufficient to
drop service based on these 2 options.
Change-Id: Ib37445ad852bcbd8d44d9eda9293565a4e52262b

We're providing an option to have an IP address per VIP
address. Currently it's used only for creating self-signed
SSLs signed with internal CA per each VIP. With follow-up
patches that will also allow to provide user certificates
per VIP, making possible to cover internal and external
endpoints with different non-wildcard certs.
Change-Id: I0a9eb7689eb42b50daf5c94c874bb7429b271efe

In some use cases you may want to define your own stick-table and
rules, this can be done using the backend_arguments variables.
As you can have only one stick-table per backend or frontend
the default stick-table needs to be disabled.
I am also not convinved the default stick-table is used for anything,
it just logs requests and never uses the logs, i think it could be
removed.
Depends-On: https://review.opendev.org/c/openstack/openstack-ansible/+/797819
Change-Id: I54307c00673ababb277257f2bb0e456e3e011ac4

Change-Id: I568273d0ef1d5ee391a42981e66cc9895b9d71b6

The external PKI role can generate a self signed CA and Intermediate
certificate, and then create a server certificate for haproxy if
no defaults are overridden.
The new openstack_pki_* settings allow an external self signed CA
to be used, but still create valid haproxy server certificates from
that external CA in an openstack-ansible deployment.
The original beheviour providing user supplied certificates in the
haproxy_user_ssl_* variables will still work, disabling the generation
of certificates but using the external PKI role to just install the
supplied certs and keys.
Depends-On: https://review.opendev.org/c/openstack/openstack-ansible/+/788031
Change-Id: I7482f55e991bacd9dccd2748c236dcd9d01124f3

This is needed for HAProxy 2.1 which is in Debian Bullseye
Change-Id: I912c6d810acc137c3b3e73dc40160d6376cb3884