openstack/openstack-ansible-haproxy_server - openstack-ansible-haproxy_server - OpenDev: Free Software Needs Free Tools

Depends-On: https://review.opendev.org/c/openstack/openstack-ansible-os_keystone/+/924474
Change-Id: I3d551c11a4b8fb25890f9db64ec6ec15be3a5240

Update the zed release notes configuration to build from
unmaintained/zed.
Change-Id: Id370b35484be8fe1ee5f51561cb7e1246640f936

In cases when internal and external haproxy frontends should use
different, pre-generated certificates, it's not possible to define them
with haproxy_user_ssl_cert because it accepts only one certificate.
In this case, certificates can be placed manually in pki/ directory.
Unfortunately, with current logic, certificates creation with PKI role
is disabled only when haproxy_user_ssl_cert is defined.
Possibility of explicitly disabling certificates generation will be
really useful.
Depends-On: https://review.opendev.org/c/openstack/openstack-ansible-os_nova/+/915320/
Change-Id: I4eed4d797160b885d5b7187e6106e6ee0073722f

``haproxy_static_files`` can be used to copy static files to target
hosts. These files may contain sensitive content that should not be
logged.
This patch disables logging for this task.
Change-Id: I8f1c01385d7aca8f17cc3f49aafcf2b7269fa554

We do allow to supply haproxy_backend_nodes as list of mappings rather
the regular list, which supports `ip_addr`, `name` and `backend_port` keys.
However, we do verify hostvars[host_name] and try to set ip_addr regardless
if this needed or not.
During hostvars[host_name] request Ansible tries to fetch all host variables
and resolve some of them, which not always can be possible or preffered
in some scenarios.
Good example of that would be Mozilla SOPS [1] encrypted variables for
specific host or group, which can not be decrypted by some operators.
In the meanwhile they can be eligible to configure haproxy frontend/backend
for this service. So we should have a way to avoid asking for specific
hostvars when it's not needed, and backend_nodes are already contain
all required information.
[1] https://docs.ansible.com/ansible/latest/collections/community/sops/docsite/guide.html
Change-Id: I17a7f2421cd31b37bbda4f9c85971b1825e54891

At the moment service templates are hardly readable, partially due to
complex logic, but incosistent presence of indetnation makes things
way worse, as there's no way to know if you're under some cycle
or condition for sure.
This patch aims to make indents correct which should improve template
readability overall.
Change-Id: Ie60ca87c044281104fbc8334d7254ac351d3d912

Update the victoria release notes configuration to build from
unmaintained/victoria.
Change-Id: I8420d1a72ebc16cc943c5f9aa683188e44460c83

With [1] a regression was introduced, where incorrect permissions were
applied to a bind mount corrupting access to /dev/log globally on hosts
where haproxy was running.
Default permissions are 0666 for /dev/log when it's managed by journald.
[1] https://review.opendev.org/c/openstack/openstack-ansible-haproxy_server/+/888143
Closes-Bug: #2055178
Change-Id: Ib8b9e4dea0ecd5d35f0e872dfaa0f2ec837a98f8

In order for http-check to work, option httpchk must be loaded first. Otherwise
regular L4 check will be issued and all `http-check` will be simply ignored.
Closes-Bug: #2046223
Depends-On: https://review.opendev.org/c/openstack/openstack-ansible/+/903488
Change-Id: Ie9ed322ab9c4a04d42cab4456567ac5d1f5c966b

Neither `vip_interface` nor `vip_address` are defined or available in
the context they're being used.
Thus we need to refer to available variables in order to render base config
properly
Current version fail with "AnsibleUndefinedVariable: 'vip_interface' is undefined"
on "Drop base haproxy config" task.
This fix the issue that was introduced with [1] and backported back to Zed
[1] https://review.opendev.org/q/Ib8be6b7fc3dada9d20905b0f07d90ddce0335605
Change-Id: I4e52378d8c5b3eaa6863ecaf0d04554d082e3dc0

Change-Id: Iba1f4a284beaba8d2d7f020ca7ad2d78d6360161

Rather then applying tag for each task inside the haproxy_service_config
file, it's better to apply it to include. Also, this closes the bug,
when role fails due to fact being undefined,
since setting fact was not covered by the tag.
Change-Id: I533070196dda5387a910f613cdd037fa36880cdb

Some environments use a dedicated PKI for monitoring and metric collection.
This change allows to configure the serving certificate for stats independently
by setting `haproxy_stats_ssl_cert_path`, the default is to use the same cert.
Also client certificate authentication for stats can now be enabled by defining
a CA cert via `haproxy_stats_ssl_client_cert_ca`.
Change-Id: Ib8be6b7fc3dada9d20905b0f07d90ddce0335605

When rotating certificates for HAProxy it's quite neat to have
tags that will allow to run specifically certificate rotation without
any extra steps.
Change-Id: If1b6d6e46a4b2941198b0f57c858d415fbbdc8d1

The 'netcat' package is no longer installable directly.
 Package netcat is a virtual package provided by:
 netcat-openbsd 1.219-1
 netcat-traditional 1.10-47
 You should explicitly select one to install.
 E: Package 'netcat' has no installation candidate
Change-Id: Ic708a7fd2223d1ba40ccacbd2b6863187fad0da9

Change-Id: I7647f067ba33fb0329f6e5e7d40b641fd45cb062

This patch implements extra variables/keys that can be used to
enable HTTP/2 protocol for frontends and backends.
With that patch does not add HTTP/2 support for any redirect frontends
since they can not be configured to use TLS and this it will
cause such redirect backends to be HTTP/2 only, which might break old
clients.
With that regular frontends, that are not terminating TLS can be
configured to be HTTP/2 only as well as TCP backends.
Change-Id: Ib14f031f3c61f31bf7aaf345a3ba635ca5fb9ff8

Since latest ansible handlers are not triggered inside the same
handlers flush, which means that triggering mysql restart
the way we did does not work anymore. So instead of
notifying inside handlers, we add listen key to tasks
that are triggered by these newly produced notifications.
This could be due to the bug [1], but ansible-core version that has
backport included still shows inconsistent behaviour
[1] https://github.com/ansible/ansible/issues/80880
Change-Id: I0d97e0b90a8d18a7b69e880e4effa851238d51d1

With update of ansible-lint to version >=6.0.0 a lot of new
linters were added, that enabled by default. In order to comply
with linter rules we're applying changes to the role.
With that we also update metdata to reflect current state.
Change-Id: I8c316dd62ac22ccd9578bb0199ab8f25c0104f9a

The security.txt RFC specifies a Content-Type of text/plain and
charset of utf-8 [1]. This adjusts the defaults so line breaks are
rendered correctly in a browser.
[1] https://datatracker.ietf.org/doc/html/rfc9116#section-3
Change-Id: I39c2dab5108a815ef966bab0d708d6300eb1a4d1

It's now possible to set ssl cert path in case you want to bind to
specific hostname via ``haproxy_bind`` and want to share a common
certificate. set ``haproxy_ssl_path`` to override per service.
Change-Id: Ib517f52c0edbc4ac8d0df2a2ae078c9138141aae

Add the possibility to have multiple backend services running on
differents ports.
Change-Id: I1748bfc15bdf879f78aa06c385af7b6c45bde7ff

With `haproxy_bind_*_lb_vip_address` set, use `*_lb_vip_address` for SAN
instead.
Change-Id: I33fc820be583bfaf7f9bee5233f0e0b99805144a

`extra_lb_tls_vip_addresses` is list of additional internal VIP
addresses, which gets parsed into `haproxy_tls_vip_binds` without
`interface` attribute.
Change-Id: I184021b65d6f3f28526c9fa09bea90a2baef77b2

This change fixes service-redirect.j2 template that was not working so
far, mainly by replacing:
- 'vip_bind' with 'vip_addres'
- 'item' with 'service'
Additionally, I removed `haproxy_tcp_upgrade_backend` support because
it's not really needed after haproxy separated service config was
implemented.
I also changed variable name `haproxy_tcp_upgrade_frontend` to
`haproxy_accept_both_protocols` to better describe what exactly it does.
Release note is not needed as ``haproxy_tcp_upgrade_frontend` was not
working properly before.
Depends-On: https://review.opendev.org/c/openstack/openstack-ansible/+/884445
Change-Id: Iba9156c5b909f7b18599638db4471bab12794f0e

Certbot-auto is deprecated since 2020[1] and it is no longer available
under https://dl.eff.org/certbot-auto.
This change removes certbot-auto from haproxy_server role leaving
distro method as the only available option.
[1] https://community.letsencrypt.org/t/certbot-auto-deprecated-explanation-and-solutions/139821
Change-Id: Ibe0f13fc7308359d337fb382cb72998befb90d84

With current behavior, when haproxy role is imported multiple times in
the same playbook(by setup-openstack.yml as an example), variable
`_haproxy_service_configs_simplified` never gets purged so ansible just
keeps appending services this list.
To avoid this situation, `_haproxy_service_configs_simplified` has to be
explicitly defined as a blank list at the begining.
Change-Id: If62ec18842609957f09e0161a524fea88910ce9e

This patch allows haproxy role to create security.txt file.
Change-Id: Ided790a5a89a2298b3b758d4484b25091b92945b

In [1] new, simplified haproxy_service_configs format was introduced.
Temporary conversion from old vormat was added but it doesn't cover map files.
This change adds format conversion also for map files feature.
[1] https://review.opendev.org/c/openstack/openstack-ansible-haproxy_server/+/871188
Change-Id: If9c57bb61d3ae8d50f69780fe54a26ac0d67656a

For certificates from widely trusted CA there is no need to provide
a specific CA file for an ssl backend, but the code fails with
undefined variable if only haproxy_backend_ssl is enabled.
A workaround is to set `haproxy_backend_ca: false` but this patch
allows haproxy_backend_ssl to be used on it's own.
Change-Id: I7c87317433acb4ed73070a2252240737b22dccfc

At the moment PKI and haproxy do listen for the same notify, which results in
haproxy trying to generate certs in inappropriate places. This patch starts
leveraging `pki_handler_cert_installed` variable that enables us to trigger
haproxy certificate assemble only when required and expected.
Co-Authored-By: Damian Dąbrowski <damian@dabrowski.cloud>
Depends-On: https://review.opendev.org/c/openstack/ansible-role-pki/+/875757
Change-Id: I66f648e5c3104f71d6601a493b09f8cdcc3332fc

This change allows specific playbooks to configure their haproxy
service(s) separately by running the role and using tasks_from to
execute just the service template installation code path.
Change-Id: I88ce0eb92784b3d3a0d1a952e95a8eb1fa376e77
Co-Authored-By: Damian Dąbrowski <damian@dabrowski.cloud>

For historical reasons the ``haproxy_service_configs`` variable was
a list of nested mappings with only single valid key for the top
level mapping.
There have been no use-cases for extra keys, so this patch simplifies
the code by removing one level of nesting.
Change-Id: I50c17b7020a459ab8a88b004cc8828cac857f1c9

HAProxy supports the use of map files for selecting backends, or
a number of other functions. See [1] and [2].
This patch adds the key `maps` for each service definition allowing
fragments of a complete map to be defined across all the services,
with each service contributing some elements to the overall map file.
The service enabled/disabled and state flags are observed to add and
remove entries from the map file, and individual map entries can also
be marked as present/absent to make inclusion conditional.
[1] https://www.haproxy.com/blog/introduction-to-haproxy-maps/
[2] https://www.haproxy.com/documentation/hapee/latest/configuration/map-files/syntax/
Change-Id: I755c18a4d33ee69c42d68a50daa63614a2b2feb7