openstack/openstack-ansible-haproxy_server - openstack-ansible-haproxy_server - OpenDev: Free Software Needs Free Tools

Certbot-auto is deprecated since 2020[1] and it is no longer available
under https://dl.eff.org/certbot-auto.
This change removes certbot-auto from haproxy_server role leaving
distro method as the only available option.
[1] https://community.letsencrypt.org/t/certbot-auto-deprecated-explanation-and-solutions/139821
Change-Id: Ibe0f13fc7308359d337fb382cb72998befb90d84

With current behavior, when haproxy role is imported multiple times in
the same playbook(by setup-openstack.yml as an example), variable
`_haproxy_service_configs_simplified` never gets purged so ansible just
keeps appending services this list.
To avoid this situation, `_haproxy_service_configs_simplified` has to be
explicitly defined as a blank list at the begining.
Change-Id: If62ec18842609957f09e0161a524fea88910ce9e

This patch allows haproxy role to create security.txt file.
Change-Id: Ided790a5a89a2298b3b758d4484b25091b92945b

In [1] new, simplified haproxy_service_configs format was introduced.
Temporary conversion from old vormat was added but it doesn't cover map files.
This change adds format conversion also for map files feature.
[1] https://review.opendev.org/c/openstack/openstack-ansible-haproxy_server/+/871188
Change-Id: If9c57bb61d3ae8d50f69780fe54a26ac0d67656a

At the moment PKI and haproxy do listen for the same notify, which results in
haproxy trying to generate certs in inappropriate places. This patch starts
leveraging `pki_handler_cert_installed` variable that enables us to trigger
haproxy certificate assemble only when required and expected.
Co-Authored-By: Damian Dąbrowski <damian@dabrowski.cloud>
Depends-On: https://review.opendev.org/c/openstack/ansible-role-pki/+/875757
Change-Id: I66f648e5c3104f71d6601a493b09f8cdcc3332fc

This change allows specific playbooks to configure their haproxy
service(s) separately by running the role and using tasks_from to
execute just the service template installation code path.
Change-Id: I88ce0eb92784b3d3a0d1a952e95a8eb1fa376e77
Co-Authored-By: Damian Dąbrowski <damian@dabrowski.cloud>

For historical reasons the ``haproxy_service_configs`` variable was
a list of nested mappings with only single valid key for the top
level mapping.
There have been no use-cases for extra keys, so this patch simplifies
the code by removing one level of nesting.
Change-Id: I50c17b7020a459ab8a88b004cc8828cac857f1c9

HAProxy supports the use of map files for selecting backends, or
a number of other functions. See [1] and [2].
This patch adds the key `maps` for each service definition allowing
fragments of a complete map to be defined across all the services,
with each service contributing some elements to the overall map file.
The service enabled/disabled and state flags are observed to add and
remove entries from the map file, and individual map entries can also
be marked as present/absent to make inclusion conditional.
[1] https://www.haproxy.com/blog/introduction-to-haproxy-maps/
[2] https://www.haproxy.com/documentation/hapee/latest/configuration/map-files/syntax/
Change-Id: I755c18a4d33ee69c42d68a50daa63614a2b2feb7

Currently default_backend for a service is always set to the
haproxy_service_name for a service, but this might not be what is
required for some configurations.
This patch allows haproxy_default_backend to be configured for
a service to customise the default_backend setting.
Depends-On: https://review.opendev.org/c/openstack/openstack-ansible-rabbitmq_server/+/876436
Change-Id: I9e2be37cb27a33350577a93f23b69e560493b320

Currently the role will run against all target hosts, and it is
possible that the calling playbook runs with a serial: setting
to control how many hosts are targetted simultaneously.
However, this is not sufficient to guarantee that each potential
haproxy server requests a LetsEncrypt certificate sequentially.
It is only possible for the loadbalancer to direct the challenge
from the ACME server to one certbot instance at a time, so this
patch enforces serialisation of the initial certificate generation
regardless of the number of target hosts and setting of serial:
outside this role.
Change-Id: If8ae64bc01510d3570fa4c554463bd6121b21f86

We haven't specified tags for let's encrypt task which resulted in task
not being executed when using them.
Change-Id: I294e962bdb796190d1e7a2555708fbfaa8384a0a
Co-Authored-By: Damian Dąbrowski <damian@dabrowski.cloud>

In change [1] we have added functionality to accept both HTTP and
HTTPS during an upgrade.
However it's only limited to internal VIP. I see no reason not to
implement this also for external VIP. Some people may find it useful.
[1] https://review.opendev.org/c/openstack/openstack-ansible-haproxy_server/+/829899
Change-Id: I672016b75d4b514d87dbb47119ff549bbc4e923e

Change-Id: I300206a79fcb9e809c1ae714f492583fb9d4e363

Currently this must be configured on a per-frontend basis through
service.haproxy_frontend_raw. This patch adds a new role default
variable haproxy_frontend_extra_raw which will be combined with all
per service raw config lines.
Change-Id: I506d46d64df93bbb9e6d1ebfa3d3caa44c80fdd5

haproxy_service_config.yml is not a valid place for selinux fix.
It should be moved to haproxy_post_install.yml.
Change-Id: Ice55e1cd9fdbac6e564c7f084dc1a020940a0da8

In case of using dns-01 challange deployers might want
to avoid using
standalone flag.
Change-Id: I3c6cfd7779e9ec9322e655cdda5bb6866bf695ca
Closes-Bug: #2006938 

The ternary options appear to be getting evaluated whether they
are used or not, so item['interface'] is always accessed.
This patch aims to check for the key's presence before performing
ternary operations, or use Ansible variables to postpone evaluation
until absolutely necessary.
Change-Id: Ib1462c04d1a0820a37998f989e2ed16566f71f54

With tox release of 4.0, some parameters were deprecated and are ignored now
which causes tox failures. One of the most spread issues we have is using
`whitelist_externals` isntead of `allowlist_externals`
Change-Id: I73cad1846dd3fbcbf9e3317227c472d769d1e7b6

Add file to the reno documentation build to show release notes for
stable/zed.
Use pbr instruction to increment the minor version number
automatically so that master versions are higher than the versions on
stable/zed.
Sem-Ver: feature
Change-Id: I819c1252ed66a169de60dcd5f8e88e4bc94c22ab

At the moment for some reason we're not taking into account default
variables haproxy_rise/haproxy_fall but instead trying to count
based on amount of backends. This makes quite little sense to
depend amount of backend rechecks on amount of backends overall,
so we're chaning behaviour to pre-defined variables that already exist.
Change-Id: I1e53a997f6f443718ea2c6bdfbe8a0b98c44896d

Haproxy config check(/usr/sbin/haproxy -c -f /etc/haproxy/haproxy.cfg)
returns 3 warnings:
1. keyword 'forceclose' is deprecated in favor of 'httpclose', and will
not be supported by future versions.
2. backend 'galera-back' : 'option tcplog' directive is ignored in
backends.
3. 'http-request' rules ignored for backend 'galera-back' as they
require HTTP mode.
This change fixes 1. and 2.
Fixing 3. will be a bit more tricky as it's a part of
`openstack_haproxy_stick_table` defined in
/opt/openstack-ansible/inventory/group_vars/haproxy/haproxy.yml
Change-Id: Idaa4b5580039857435f90416924dee26a702deba

Right now we don't ensure haproxy conf validity and if it's incorrect
role will fail on attempt to reload haproxy. However it's really worth
adding validation step and do not proceed if configuration is wrong
Change-Id: I54717d4f7230b8d8dff2d293592831cc88c51d24

Currently there is no way of disabling SSL connection for stats frontend
as it implies more global variable. However, for some systems consuming
self-signed root certificate might be not an option and disabling
SSL verification tricky. Thus, we introduce new variable that allows to
nicely control if SSL should be served for stats frontend or not.
Change-Id: Ic4bc4393ec89469876e9e95b12bb9c4069972713

Add `haproxy_ssl_letsencrypt_domains` variable, which
contains a list (defaults to `external_lb_vip_address`)
for `--domains` certbot option.
Change-Id: I2ebfff9eeb5279a3964b8578a6e66aa132d763f5

This line was introduced by Ib4f33185202b694b9611cc5fd6323c30a1c8d489
for multi-os support, but should since be covered by the
distribution_major_version line above, introduced at a later date.
Change-Id: I23a8e7aaa3858bce47dcf7610acf1ee58d9e1fc1

In some user scenarious (like implementing DNS RR) it might be useful to
bind on 0.0.0.0 but at the same time do not conflict with other services
that are binded to the same ports. For that, we can specify a specific
interface, on which haproxy will be binded to 0.0.0.0.
In netstat it would be represented like `0.0.0.0%br-mgmt:5000`.
With that we also allow to fully override `vip_binds` if assumtions
that role make are not valid for some reason.
Change-Id: Ic4c58ef53abc5f454b6fbebbd87292a932d173ae

Right now we assume, that ca-cert is always present. Though, it might
not be the case for user-provided certs or let's encrypt, as they
are already in ca-certificates.
Change-Id: I101f82c5e378596e76a160aacb34a9e1e7e0c123

Based upon usual recommendations from:
https://hynek.me/articles/hardening-your-web-servers-ssl-ciphers/
Change-Id: I6e549ab3ffcacebe04e188cbf34d8707fb0fe05d

A new variable "haproxy_stick_table" is added which allows a custom
stick-table to be supplied that is used as the default stick-table
for all haproxy back-ends.
In addition, the variable service.haproxy_stick_table can be defined
for each service to allow a unique stick-table to be supplied for
a particular service.
The old default stick-table definition is removed as there was no
use case defined for it in this role before. An example is added
to defaults/main.yml to show how the custom stick-table can be used
to rate-limit requests that generate 4xx responses which commonly
occur during vulnerability scanning or credential stuffing attacks.
There are many other uses for stick-tables, consult the HAProxy
documentation for details.
Change-Id: I50daba08c10f071157d6450ea2fa97df448f99ec

Instead of hardcoding specific supported tunable options, we
just pass key as an option to haproxy config.
This change might break deployments during upgrades, since format of
values in variable has changed, but appropriate release note was written
We also increase maxrewrite by default, as otherwise usage of CSP leads
to 500 error.
Change-Id: I949960420ed5dbd6d58f0de7dae0ac629a85b7fc
Related-Bug: https://github.com/haproxy/haproxy/issues/1597
Needed-By: https://review.opendev.org/c/openstack/openstack-ansible-os_horizon/+/844815 

With sphinx release of 5.0.0, they changed default for language variable
to 'en' from None. With that current None valuable is not valid and should
not be used.
Change-Id: Ibea41d4f472fa7b375d9d04e94ad621696f565b1

Change-Id: I786353945ca45d16d91678d65610a97798bf98f0

Change-Id: Idea36d3b3fd8cdeb04e76f9b3ec7da24eac99b83