openstack/ironic-python-agent - ironic-python-agent - OpenDev: Free Software Needs Free Tools

Adds support for bootable containers to be deployed by the agent.
Related: https://review.opendev.org/c/openstack/ironic/+/937897
Change-Id: I66cb37d117d2afc335f015fb1fc31bdbd5c3cee5

Prevents the UnboundLocalError in erase_devices_express clean step.
Closes-Bug: #2095499
Change-Id: I01ce5005a62638ff960d2a75f225f882b2d56973

Debug messages from modprobe failing to load ipmi drivers can
be confusing and they do not add anything since they're
not really errors.
This patch silence the message in the logs.
Change-Id: I7452bc9e56148e3d423be92f384ff9aeffbe88d7

Related-Change: #939500
Change-Id: Id54f5bedd5c79c587bc4484914a8eb492e018010

Fail without retries when Errno 28 - "No space left
on device" error is encountered.
Closes-Bug: #2094854
Change-Id: Ie84b422916ddc02f2474164fe3da083324ef4824

Use just md<index> as the default volume name if a volume name is not defined.
The original change (https://review.opendev.org/c/openstack/ironic-python-agent/+/853182)
introduced an error:
mdadm: Value "/dev/md0" cannot be set as name. Reason: Not POSIX compatible.\n
This change fixes it.
Closes-Bug: #2073406
Change-Id: Ic8bd473801fcb92fc814f6ad4e1d6dc316783bf3

ironic-lib is being retired; this change imports any used code from
ironic-lib and updates references.
This contains some changes to how we throw exceptions; aligning
ironic-lib code with IPA practice to have all exceptions be a RESTError.
This also allows us to remove code around serializing ironic-lib
exceptions.
Change-Id: I137340ce6820c68d8e0f1a32668151bba7b1ddd7

Extended the function that expose BMC MAC address in inventory data
for an IPv6 only interface.
Previously, if no IPv4 address was configured, no mac address was exposed.
Change-Id: I93e49d308cfd63be1c09749ced4428a87a3daff9

To support a safer take-over from the provisioning to the tenant network
for hardware that cannot be powered off, this change introduces a new
command system.lockdown. When invoked, it stops the API, the heartbeater
and disables all network interfaces (if possible).
Partial-Bug: #2077432
Change-Id: I211fc64a46226127b0d82ab458029b3c702b3f74

This is largely required for the future lockdown command but can also be
used before the normal shutdown, especially in the sync command which is
currently used before an out-of-band shutdown command is issued.
In addition to a plain sync, the new command also tells the kernel to
drop its cached and issues a low-level sync command to each block
device.
Partial-Bug: #2077432
Change-Id: I3fc87b20bc5387a466b24ebc19b9982e4e368d20

We are phasing out use of ironic-lib, and as such are removing the
metrics module from it. However, due to it's requirement of having
a statsd instance on the same subnet as the agent and there being no
support for prometheus exporting of metrics from IPA, these metrics are
no longer valuable (in the agent).
We are vendoring the module for the deprecation in order to facilitate
its removal from ironic-lib.
Change-Id: Ie50e078bc3f78d65cfa53680dc4116d1119ce155

It's clear the docstrings intended to indicate ProtectedDeviceError, but
instead indicate ProtectedDeviceFound.
This clears up the documentation to assist deployers trying to implement
hardware managers.
Change-Id: Iea534f58aeec60b3862099c1d89be829654a54c1

* The dd and is_http_url code is trivial, inline it.
* Migrate mounted (cannot be used in Ironic since it requires root).
* Remove the leftovers of run_as_root.
Change-Id: Ic6b117e34ccc7f55ebac5f808d2765305c4b317f

We traditionally have not done any translations for IPA, but some of the
code carried over from ironic_lib -- as well as a single hardware
manager -- were using the i18n wrappers. This removes them from
everywhere for consistency.
Change-Id: I85a607d7cfb50d605cc62ac8c5e3937802b4d2af

- Removes unused methods from disk_utils
- Unifies all image writing via populate_image
Change-Id: I860744458e52a45a4cb2882e4a59e0db9fbcb93d

``logical_sectors`` and ``physical_sectors`` sizes are now captured for
each hardware info ``disks`` entry, and also logged for ``lsblk`` calls.
This will be increasingly useful as storage devices with 4096 byte
sector sizes become more common.
Change-Id: I80b6b137f6e3071d9b8a4c1abe14416249aed9ac

I have a case where a user provided the checksum URL with SHA256
checksums, while Metal3 defaulted os_hash_algo to "md5". We're going
to change the Metal3 defaults in the next API version, but for now let
us issue a clear warning in such case.
Closes-Bug: #2085331
Change-Id: Ie4e62a378dc4a2089944f4302df3a8671b7c960f

This replaces the copied-in version of format_inspector with the shared
version located in oslo.utils.
Change-Id: I62b3876f4507f28c8582cd4ba80e9f7c4e0b71ef

The netifaces library was abandoned and archived. Replace it by psutil
which is already part of the requirements.
Closes-Bug: #2071596
Change-Id: Ibca206ec2af1374199d0c0cfad897dded1298733

... because it was removed in Python 3.12 [1].
[1] https://docs.python.org/3/whatsnew/3.12.html#ensurepip
Change-Id: I2d27da0db92c1eaae9f45a0905e3eb905a939571

Change-Id: I6af5e6d2c4781c24345d456cec4d77c364ae2da5

Check for IPMI device files before the use of the `'ipmitool lan.*'`
command, avoiding unnecessary calls on non-IPMI systems.
Closes-Bug: #2076367
Change-Id: Ib800717701e6f2828df55a0da0e999fc014c12e1

`gethostbyname` only supports IPv4 lookup. In IPv6-only setups, that
does not work. Hence, `gethostbyname` is replaced with `getaddrinfo`
which supports both address families.
Change-Id: I46f79ef0992b2e6650be9772776c7223e981fc17

When IPA gets a non-raw image, it performs an on-the-fly conversion
using qemu-img convert, as well as running qemu-img frequently to get
basic information about the image before validating it.
Now, we ensure that before any qemu-img calls are made, that we have
inspected the image for safety and pass through the detected format.
If given a disk_format=raw image and image streaming is enabled
(default), we retain the existing behavior of not inspecting it in
any way and streaming it bit-perfect to the device. In this case, we
never use qemu-based tools on the image at all.
If given a disk_format=raw image and image streaming is disabled, this
change fixes a bug where the image may have been converted if it was not
actually raw in the first place. We now stream these bit-perfect to the
device.
Adds two config options:
- [DEFAULT]/disable_deep_image_inspection, which can be set to "True" in
 order to disable all security features. Do not do this.
- [DEFAULT]/permitted_image_formats, default raw,qcow2, for image types
 IPA should accept.
Both of these configuration options are wired up to be set by the lookup
data returned by Ironic at lookup time.
This uses a image format inspection module imported from Nova; this
inspector will eventually live in oslo.utils, at which point we'll
migrate our usage of the inspector to it.
Closes-Bug: #2071740
Change-Id: I5254b80717cb5a7f9084e3eff32a00b968f987b7

Closes-Bug: #1639340
This commit adds the relevant changes to the get_cpu function, keeping it backwards compatible with the old method.
Change-Id: I3c3a792e88e9a041236eca7283ebfdf1026910d8

Fixed spelling where appropriate, added ignore where appropriate
Change-Id: I07f203d311484321e0dfcbdf02083784693f4b96

Fixes an issue where we could call evaluate_hardware_support multiple
times each run. Now, instead, we cache the values and use the cache
where needed.
Adds unit test coverage for get_managers and the new method.
Fixes issue where we were caching hardware managers between unit tests.
Also includes fixes for codespell CI:
- skip build files in repo
- fix spelling issues introduced to repo
Closes-bug: 2066308
Change-Id: Iebc5b6d2440bfc9f23daa322493379bbe69e84d0

Per https://review.opendev.org/c/openstack/ironic/+/918082 and
contributor recollection, we believe this has been resolved and can
thus be removed.
Change-Id: Icbf0f095cabf52a7b642cd4a6ddfbd62cc77964e

If this seems like deja vu, that is because it is. We had this
very same issue with the original CoreOS ramdisk. Since we don't
control the whole OS of the ramdisk, it only made sense to teach
the agent to umount the folder.
The folder is referenced already, and the agent does have safeguards
in place, but unfortunately this issue led to a rebuild breaking where
cloud-init, glean, and the agent were all trying do the right thing
as they thought, and there were just multiple /mnt/config folders
present in the OS. These are separate issues we also need to try and
remedy.
What happens is when the device is locked via a mount, the partition
table is never updated to the running OS as the mount creates a lock.
So the agent ends up thinking, in the case of a rebuild, that everything
including creating a configuration drive on that device has been
successful, but when you reboot, there is no partition table entry
for the new partition as the change was not successfully written.
This state prevented the workload from rebooting properly.
This change eliminates that possibility moving forward by attempting
to ensure that the cloud configuration folder is no longer mounted.
Change-Id: I4399dd0934361003cca9ff95a7e3e3ae9bba3dab

A sector size of 512 was assumed and hardcoded, causing dd to fail when
it tried to write in chunks smaller than the sector size for disks with
4096 bytes sectors. The size of GPT in sectors also depends on sector size.
Change-Id: Ide5318eb503d728cff3221c26bebbd1c214f6995

This commit introduces the following changes:
 - New optional `all_serial_and_wwn` argument for the block device
 listing logic. The new argument makes it possible to
 collect wwn and serial number information from both
 lsblk and udevadm at the same time
 - Both the short and the long serials are collected
 from udeavadm without prioritization when the new argument
 has teh value True
 - The new feature is automatically enabled during block device listing
 as part of the root disk selecetion
 - New options are added to the lsblk command when used in the block
 device discovery process, previously lsblk was not looking
 for wwn numbers and now it does
Closes-Bug: #2061437
Change-Id: I438a686d948cd929311e2f418bb02fb771805148
Signed-off-by: Adam Rozman <adam.rozman@est.tech>

Adds a deploy step ``clean_uefi_nvram`` to remove unrequired extra UEFI
NVRAM boot entries. By default any entry matching ``HD`` as the root
device, or with a ``shim`` or ``grub`` efi file in the path will be
deleted, ensuring that disk based boot entries are removed before the
new entry is created for the written image. The ``match_patterns``
parameter allows a list of regular expressions to be passed, where a
case insensitive search in the device path will result in that entry
being deleted.
Closes-Bug: #2041901
Change-Id: I3559dc800fcdfb0322286eba30ce47041419b0c6