openstack/ironic-python-agent - ironic-python-agent - OpenDev: Free Software Needs Free Tools

The idea is to retreive USB devices informations via 'lshw' and
return the list to ironic in order to be able to create introspection
rules based on USB devices.
Change-Id: I39d60cb467614fca7a7f701dbe576154213580a5

This reverts commit 33f01fa3c2.
There are a few issues with the patch - see my comments there.
The most pressing and the reasons to revert are:
1) It breaks deployments when the vmedia is present but does not
 have a network_data.json (the case for Metal3).
2) It assumes the presence of Glean which may not be the case.
Neither Julia nor myself have time to thoroughly fix the issue,
leaving a revert as the only option to unblock Metal3.
Change-Id: I3f1a18a4910308699ca8f88d8e814c5efa78baee
Closes-Bug: #2045255 

This change improves the regex to match an exact entry name, and to also
match with the the entry type from a set of recognised types.
The boot entry details start from the recognised type onwards.
This can be used by a step which deletes all entries of type 'HW' and
UsbClass.
Related-Bug: #2041901
Change-Id: I5d879f724efc2919b541fd3fef0f931df67ff9c7

In some cases the output of the multipath can differ
and we would return a wrong parent device.
Closes-Bug: 2043992
Change-Id: I848d7df798cc736bd5a55eed8fa46110caea1dc3

This commit:
 - fixes some "multipathd error handling improvement"
 release notes
 - fixes a related comment in the code
Related launchpad issue https://bugs.launchpad.net/ironic-python-agent/+bug/2031092
Change-Id: Ie3ba0601fa117b053cb8db6284e47249ca9c9134
Signed-off-by: Adam Rozman <adam.rozman@est.tech>

When performing DHCP-less deployments, the agent can start and
discover more than one configuration drive present on a host.
For example, a host was previously deployed using Ironic, and
is now being re-deployed again.
If Glean was present in the ramdisk, the glean-early.sh would end
mounting the folder based upon label.
If cloud-init, somehow is still in the ramdisk, the other folder
could somehow get mounted.
This patch, which is intended to be backportable, causes the agent
to unmount any configuration drive folders, mount the most likely
candidate based upon device type, partition, and overall state of
the machine, and then utilize that configuration, if present,
to re-configure and reload networking.
Thus allowing dhcp-less re-deployments to be fixed without
forcing any breaking changes.
It should also be noted that this fix was generated in concert
with an additional tempest test case, because this overall failure
case needed to be reproduced to ensure we had a workable non-breaking
path forward.
Closes-Bug: 2032377
Change-Id: I9a3b3dbb9ca98771ce2decf893eba7a4c1890eee

A step will be developed to delete all EFI entries of type HD. As part
of this get_boot_record will need to parse more of the output of
`efibootmgr -v`.
This change asserts the existing behaviour of get_boot_record, and the
test can evolve with the changes in get_boot_record.
Related-Bug: #2041901
Change-Id: I0c5ac4adc1044c528c27a4eaf580c619ceef47e0

Image caching was never fully supported in Ironic or IPA; this is vestigal
code leftover from a partial implementation.
Even if we implemetented it today, we'd likely use a completely different
methodology.
Change-Id: Id4ab7b3c4f106b209585dbd090cdcb229b1daa73

IPA now includes information about numa node id when collecting
information about PCI devices.
Closes-bug: #1622940
Co-Authored-By: Jay Faulkner <jay@jvf.cc>
Change-Id: I70b0cb3eff66d67bb8168982acbbf335de0599cd

This commit:
 - Adds the ability to ignore inconsequential OS error caused
 by starting the multipathd service when an instance of the
 service is already running.
Related launchpad issue https://bugs.launchpad.net/ironic-python-agent/+bug/2031092
Change-Id: Iebf486915bfdc2546451e6b38a450b4c241e43a8

Used helper function, `parse_device_tags`
from ironic_lib instead of the
shlex module for their identical
functionality. Updated
mock_execute.side_effect for lsblk
compatibility in utils.execute.
Closes-Bug: #2037572
Change-Id: I6600e054f9644c67ab003f0e0f6c380b5c217223

HTTP is a fun protocol.
Size is basically optional. And clients implicitly trust the server
and socket has transferred all the bytes. Which *really* means you
should always checksum.
But... previously we didn't checksum as part of retrying.
So if anything happened with python-requests, or lower level
library code or the system itself causing bytes to be lost off the
buffer, creating an incomplete transfer situation, then we wouldn't
know until the checksum.
So now, we checksum and re-trigger the download if there is a
failure of the checksum.
This involved a minor shift in the download logic, and resulted in
a needful minor fix to an image checksum test as it would loop for
90 seconds as well.
Closes-Bug: 2038934
Change-Id: I543a60555a2621b49dd7b6564bd0654a46db2e9a

This feature was proposed in https://bugs.launchpad.net/ironic-python-agent/+bug/2021947
Change-Id: I9dbfc1402240beb75b6736214753fd86dccae676

Change ordering to ensure mock tests work correctly.
Closes-Bug: #2037690
Change-Id: Ie9b884e58e4677a47e57c3ad39cadd65db8eec75

Add file to the reno documentation build to show release notes for
stable/2023.2.
Use pbr instruction to increment the minor version number
automatically so that master versions are higher than the versions on
stable/2023.2.
Sem-Ver: feature
Change-Id: I8150eb8f35a444ef5a2bc7a648ec301e5094e52d

Follow-up from service steps addition change to add a deploy steps
alias for the Nvidia Mellanox network device firmware update clean
steps. This allows deploy time firmware updates to be codified as
part of a deployment with custom steps.
Change-Id: I9d80447dee7cfde4d3f8d81d9d39e738916b7824

Initial code patches for service steps have merged in
ironic, and it is now time to add support into the
agent which allows service steps to be raised to
the service.
Updates the default hardware manager version to 1.2,
which has *rarely* been incremented due to oversight.
Change-Id: Iabd2c6c551389ec3c24e94b71245b1250345f7a7

Changes the default lookup timeout to be 600 seconds which
reduces the risk of lookup failing as a write operation
to the backing database is performed upon lookup thanks to
generation of an agent token.
Overall, this is fairly harmless since by default ramdisks
restart the agent if they were not able to successfully
start.
Change-Id: I35c64c0b4f9b3b607df1bc0c4c2a852aa3595cbd

When an underlying block device (or driver) only supports 4KB IO,
this can cause some issues with aspects like using an ISO9660 filesystem
which can only support a maximum of 2KB IO.
The agent will now attempt to mount the filesystem *before* deleting the
supplied file, and should that fail it will mount the configuration drive
file from the ramdisk utilizing a loopback, and then extract the contents
of the ramdisk into a newly created VFAT filesystem which supports 4KB
block IO.
Closes-Bug: #2028002
Change-Id: I336acb8e8eb5a02dde2f5e24c258e23797d200ee

Instead of the old dusty cs8 one.
Depends-On: I56a0473ecbff8ab8fc143954d3c493037765cdf1
Change-Id: I7bf9cbff9d10299c1a6b9b19fddd8124c1b185ba

If the node is locked, a lookup cannot be performed when an agent
token needs to be generated, which tends to error like this:
 ironic_python_agent.ironic_api_client [-] Failed looking up node
 with addresses '00:6f:bb:34:b3:4d,00:6f:bb:34:b3:4b' at
 https://172.22.0.2:6385. Error 409: Node
 c25e451b-d2fb-4168-b690-f15bc8365520 is locked by host 172.22.0.2,
 please retry after the current operation is completed..
 Check if inspection has completed.
Problem is, if we keep pounding on the door, we can actually worsen
the situation, and previously we would just just let tenacity
retry.
We will now hold for 30 seconds before proceeding, so we have
hopefully allowed the operation to complete.
Also fixes the error logging to help human's sanity.
Change-Id: I97d3e27e2adb731794a7746737d3788c6e7977a0

Rebuilding an instance on a RAIDed ESPs will fail due to sgdisk
running against an non-clean disk and bailing out. Check if there
is a RAIDed ESP already and skip creation if it exists.
Change-Id: I13617ae77515a9d34bc4bb3caf9fae73d5e4e578

Change-Id: I458d88bf14b55253179488cb771ae42e7b8c84d7

Python 2 is EOL. No environment should be defaulting to it. Our CI
environments certainly aren't.
Change-Id: Ib2e4304fc6c95c853570f48690a2d2a4aeeabdbe

In 2023.2 cycle testing runtime, project started adding python 3.10
[1] https://governance.openstack.org/tc/reference/runtimes/2023.2.html
Change-Id: I6e3eb0c9dec4c48e1bf1c7c53b8c177775ec91eb

When troubleshooting download issues, which may present
as checksum validation failures, it is difficult to understand
if the *entire* file was downloaded due to the way HTTP works.
In that, a download may start with a successful result code,
and the content is streamed out until the socket is closed.
But with HTTP there is no way to know if that socket closed
prematurely and the original server size is *also* an optional
field, so just log the size we got to so we don't drive the
humans [more-]insane.
Also now logs the (optional) content-length field if
supplied by the server.
Change-Id: Id71b167f4e330d54b9afddf95f1a2ef9e40398bf

Change-Id: I9a10e543d3256ceaa78c6fbdb01fc0d88c0ee6e6

Bandit 1.7.5 released with a timeout check for all requests and
urllib calls.
Fixed those.
In the process, then exposed a bandit b310 issue, which was already
covered by the code, but explicitly marked it as such.
Also, enables bandit checks to be voting for CI..
Change-Id: If0e87790191f5f3648366d571e1d85dd7393a548

Also fixes my use of set_override, as it is not on the actual
config object. You'd think I'd remember that, since I've done
that before...
Change-Id: I4b578c4319354001cbbd3b3856af96b30fd25555

Add a follow-up documentation for
"update NVIDIA NIC firmware images and settings by ironic-python-agent"
Icfaffd7c58c3c73c3fa28cfc2a6c954d2c93c16e
Change-Id: I481cdd622f360cbba3312c6f3d4af45383bb7e1b