openstack/ironic-python-agent - ironic-python-agent - OpenDev: Free Software Needs Free Tools

Initial code patches for service steps have merged in
ironic, and it is now time to add support into the
agent which allows service steps to be raised to
the service.
Updates the default hardware manager version to 1.2,
which has *rarely* been incremented due to oversight.
Change-Id: Iabd2c6c551389ec3c24e94b71245b1250345f7a7

In 2023.2 cycle testing runtime, project started adding python 3.10
[1] https://governance.openstack.org/tc/reference/runtimes/2023.2.html
Change-Id: I6e3eb0c9dec4c48e1bf1c7c53b8c177775ec91eb

Binary LLDP data is bloating inventory causing us to disable its collection
by default. For other similar low-level information, such as PCI devices
or DMI data, we already use inspection collectors instead. Now that the
inventory format is shared with out-of-band inspection, having LLDP
there makes even less sense.
This change adds a new collector ``lldp`` to replace the now-deprecated
inventory field.
Change-Id: I56be06a7d1db28407e1128c198c12bea0809d3a3

It has been removed from Zed and we're now testing with Python 3.9 in
tinyipa and Python 3.8 in CentOS Stream 9
Change-Id: I028121d593b910e585f44e464c7fcb3e635420e8

This reverts commit c4d2851a13.
Reason for revert: 3.6 is not broken yet, except that this change
breaks it. Let us revert until we fully switch to Stream 9.
Change-Id: Ia2870135a90128f744afb9c45524ab003878843f

[1] https://governance.openstack.org/tc/reference/runtimes/zed.html
Change-Id: Ifeec26e57dd2b2908da4f161032b37edf99cb8b8

Pair nodes dynamically via a distributed coordination backend for
network burn-in. The algorithm uses a group to pair nodes: after
acquiring a lock, a first node joins the group, releases the lock,
waits for a second node, then they both leave, and release the lock
for the next pair.
Story: #2007523
Task: #42796
Change-Id: I572093b144bc90a49cd76929c7e8685ed45d9f6e

We have updated the yoga testing runtime to keep the
py36 testing.
- https://review.opendev.org/c/openstack/governance/+/820195
Unit tests job template is also updated to keep python
3.6 as a voting job. So with the py3.6 and py3.9 testing as voting
job template, we are keeping python 3.6, 3.7, 3.8, and 3.8 as
tested versions in the Yoga cycle.
- https://review.opendev.org/c/openstack/openstack-zuul-jobs/+/820286
This commit re-add the python 3.6/3.7 versions in setup.cfg classifier.
Change-Id: I0f03a7f5bb2aa07c2ec2aab1a8ebfddc0c70ca87

Yoga testing runtime has been updated with py38 and py39
as voting and removed the py36 testing. Unit tests update are
handled by the job template change in openstack-zuul-job and you
can see the updated jobs running in gate.
- https://review.opendev.org/c/openstack/openstack-zuul-jobs/+/818609
this commit updates the tested py versions in setup.cfg classifier.
[1] https://governance.openstack.org/tc/reference/runtimes/yoga.html
Change-Id: I47cd32e5690b55b983827c650efba4fd99e7be55

Change-Id: I2f0e581575112d6c7ba0d211661cab3e0b6caca6

Setuptools v54.1.0 introduces a warning that the use of dash-separated
options in 'setup.cfg' will not be supported in a future version [1].
Get ahead of the issue by replacing the dashes with underscores. Without
this, we see 'UserWarning' messages like the following on new enough
versions of setuptools:
 UserWarning: Usage of dash-separated 'description-file' will not be
 supported in future versions. Please use the underscore name
 'description_file' instead
[1] https://github.com/pypa/setuptools/commit/a2e9ae4cb
Change-Id: I5b71e5ef287a9557ed60bd23cd97dfe74571b457

Adds a new poll extension to provide get_hardware_info and get_node_info
interfaces.
get_hardware_info will be used for node validation by ironic deploy
drivers.
get_node_info will be used for sending lookup data to IPA.
standalone mode is assumed as debug only, but it's not the case
considering the poll mode will be introduced, slightly updates the
description, also prevents the mdns lookup when standalone is true.
Story: 1526486
Task: 28724
Change-Id: I5ad772a18cc4584585c5a7b6fb127547cece1998

Now that we are running the Victoria tests that include a
voting py38, we can now add the Python 3.8 metadata to the
package information to reflect that support.
Change-Id: I7631fe14a812624e9d8d0a1946ffb5b0f6dec893
Signed-off-by: Sean McGinnis <sean.mcginnis@gmail.com>

Mostly adaptation of cleaning methods.
Co-Authored-By: Dmitry Tantsur <dtantsur@redhat.com>
Change-Id: Ife0502391bbece46d619a20a825dfdb191d5c2b4
Story: 2006963
Task: 37791

Make a few cleanups:
- Remove python 2.7 stanza from setup.py
- Add requires on python >= 3.6 to setup.cfg so that pypi and pip
 know about the requirement
- Remove obsolete sections from setup.cfg:
 * Wheel is not needed for python 3 only repo
 * Some other sections are obsolete
- Update classifiers
- Update requirements, no need for python_version anymore
- Use sphinx-build
Change-Id: I8abd764d97a6711d3e6347ec3a7b8967f1512006

OpenStack is dropping the py2.7 support in ussuri cycle.
ironic-python-agent is ready with python 3 and ok to drop
the python 2.7 support.
Complete discussion & schedule can be found in
- http://lists.openstack.org/pipermail/openstack-discuss/2019-October/010142.html
- https://etherpad.openstack.org/p/drop-python2-support
Ussuri Communtiy-wide goal - https://review.opendev.org/#/c/691178/
Change-Id: Ifd0e0b99bb82a7d7e82d6c14309468196f5734fc

Change-Id: I64e66682c1e54f6edc260a22f46f5f6df8e85af1
Story: 2005896
Task: 33756

This is a mechanically generated patch to ensure unit testing is in place
for all of the Tested Runtimes for Train.
See the Train python3-updates goal document for details:
https://governance.openstack.org/tc/goals/train/python3-updates.html
Change-Id: I409d9e3db05eaf11ccaaaef8784d693f40687328
Story: #2005924
Task: #34213 

Change-Id: I7bad0c5799be2cc550efee20507e882ed055cc9a

This patch adds support for rescue mode with DHCP tenant networks in
CoreOS. Applying network config from a configdrive is not yet supported
but will be in a future patch.
Co-Authored-By: Jay Faulkner <jay@jvf.cc>
Co-Authored-By: Taku Izumi <izumi.taku@jp.fujitsu.com>
Co-Authored-By: Annie Lezil <annie.lezil@gmail.com>
Co-Authored-By: Aparna <aparnavtce@gmail.com>
Co-Authored-By: Shivanand Tendulker <stendulker@gmail.com>
Change-Id: I7898ff22800dedba73d7fbfb3801378867abe183
Partial-Bug: 1526449

Correct instances of http to https and fix any remaining links
that were broken due to the document migration.
Change-Id: I492f7972839e63efdde469a1e16840d9be6d8b31

Change-Id: I43890d380b3cb6b19cfb84bf5d9dcad4f19c0062

Implements an optional collector 'collect_dmidecode_info' for
collecting detailed info of processor, memory and BIOS and
adds it to the inspection data.
Collects BIOS, processor, memory info and adds under the 'dmi' key.
Change-Id: Ida9b418ca579b15de70d17886631f3b21697987e
Closes-Bug: #1635057 

This change removes the now unused "warnerrors" setting,
which is replaced by "warning-is-error" in sphinx
releases >= 1.5 [1].
[1] http://lists.openstack.org/pipermail/openstack-dev/
2017-March/113085.html
Change-Id: I9bf18ff72f36dfd3496b9672604e8bb98999b133

Implement the optional collector for fetching the NUMA topology
details.
Collects RAM, CPU Cores, thread siblings and NICS data for
each NUMA node and stored under "numa_topology" key.
Closes-bug: #1635253
Co-Authored-By: Jaganathan Palanisamy <jpalanis@redhat.com>
Change-Id: I5a546c009d95f39b7af4d89cf785be8acb8ebc67
Signed-off-by: karthik s <ksundara@redhat.com>

This patch adds a new hardware manager, which will disable the embedded
LLDP agent on Intel CNA network cards in order to allow the gathering of
LLDP data during the inspection process.
Change-Id: I572756ac6a7bf67a7f446738ba9d145e1c1bdc48
Closes-Bug: #1623659 

The old generate_sample.sh is broken already as it refers to
non-existing openstack/common path.
Let's use oslo-config-generator as many other OpenStack projects do.
Also, where applicable, option descriptions are updated with the
corresponding kernel parameters to set those options durig pxe boot.
Change-Id: Id4a0df30ea573d52f3b359f357fe8f4a29751939

This patch add Mellanox Manager to support Mellanox
InfiniBand NICs.
It adds client_id to the NetworkInterface for the
InfiniBand network interface.
The Mellanox Manager provides it own implementation of
get_interface_info. The mlnx get_interface_info generate
InfiniBand MAC and client-id from the InfiniBand network
interface address.
Closes-Bug: #1532534
Change-Id: I4e7f7649a1bdeaa3ee99b2748037b0f37fea486c

Change-Id: I2c6d8b2aefd1a64ca75162d054309aef9917d2b8

The log extension is responsible for retrieving logs from the system,
if journalctl is present the logs will come from it, otherwise we
fallback to getting the logs from the /var/log directory + dmesg logs.
In the coreos ramdisk, we need to bind mount /run/log in the container
so the IPA service can have access to the journal.
For the tinyIPA ramdisk, the logs from IPA are now being redirected to
/var/logs/ironic-python-agent.log instead of only going to the default
stdout.
Inspector now shares the same method of collecting logs, extending its
capabilities for non-systemd systems.
Partial-Bug: #1587143
Change-Id: Ie507e2e5c58cffa255bbfb2fa5ffb95cb98ed8c4

Adds a new collector, which gathers list of PCI devices.
Each entry is a dictionary containing 2 keys:
- vendor-id
- product-id
Such information can then be used by the inspector to distinguish
appropriate PCI devices.
Change-Id: Id7521d66410e7d408d7eada692b6123e769ce084
Partial-Bug: #1580893 

Putting a version in setup.cfg requires updating it after every release.
pbr is smart enough to calculate the next version without our help.
Change-Id: I33d610adc8860fe92275096192db06000fb2d6bb

Update the pre-versioning version number to the next value that can
include new features to tell pbr that the version in setup.cfg and the
recent 1.0.0 tag do not conflict.
Change-Id: I1c9c8e4b8a821fb3a2727a49d6f0f45e43d41218

This is a port of downstream inspector ramdisk plugins we found helpful.
* logs - sends journald logs with inspection data.
* extra-hardware - uses hardware-detect utility to collect bigger
 hardware inventory and to run benchmarks.
Change-Id: If05402606c45185d618279eef46e68c51209f82b

Change-Id: Ie7d171e891f57fc7f8b92de1761ff7ef6ec4aaa9

Adds a new module ironic_python_agent.inspector and new entry point
for extensions, which will allow vendor-specific inspection.
Inspection is run on service start up just before the lookup.
Due to this early start, and due to the fact we don't even know
MAC address of nodes on inspection (to say nothing about IP addresses),
exception handling is a bit different from other agent features:
we try hard not to error out until we send at least something to inspector.
Change-Id: I00932463d41819fd0a050782e2c88eddf6fc08c6

This will ensure that any new commits to IPA won't pass a docs job if
they introduce improperly formatted doc strings. This will enable us to
ensure our generated documentation remains of a high quality.
I will be following this up with a change to project-config to have IPA
run the docs job on every change.
Change-Id: I90236225623298a7070db1ff22acb44dddb1c520

This adds and configures a sphinx doc builder. This is based on the code
in Ironic to build documentation.
The goal is for this to contain development documentation relevant to
hacking on the agent, including how to make custom HardwareManagers.
Change-Id: Ib6cecb9959f4ad5c71440fb37827b996582a9545

This will add support for in band cleaning operations to IPA and
replace the decom API that was unused.
Adds API support for get_clean_steps, which returns a list of
supported clean steps for the node, execute_clean_step, to execute
one of the steps returned by get_clean_steps.
Adds versioning and naming for hardware managers, so if a new hardware
manager version is deployed in the middle of cleaning/zapping, the
cleaning/zapping will be restarted to avoid incompatabilities.
blueprint implement-cleaning-states
blueprint inband-raid-configuration
blueprint implement-zaping-states
Depends-On: Ia2500ed5afb72058b4c5e8f41307169381cbce48
Change-Id: I750b80b9bf98b3ddc5643bb4c14a67d2052239af

Initially this extension supports installing a bootloader so the user
image can boot from the local disk.
Change-Id: Ia588aafc240b55119c02f1254addc0cf796f88c5

This extension allows IPA to be used with the PXE/iSCSI methodology of
deployment in Ironic.
Change-Id: I32ec9fa74182c0d03c7ef1b698b1d0c0e3007773

This changes the email in setup.cfg from an internal Rackspace
email to the openstack-dev list, and the author to OpenStack.
Change-Id: Iebd652c157ba639c72898b7554a23fe9bed77c22

Move extensions under an ironic_python_agent.extensions module. This
change also moves the @async_command() decorator into the base
extension module.
Change-Id: I4021fcc33a30f3460a31bca44a4bf776cd53d488

It makes possible of running data-driven flow which's the list of
the other existent extensions to be called.
Change-Id: Ib73ea4da92f291c872b7ae51e46ecc7fdd45ee16

Allow multiple extensions to be loaded by switching to ExtensionManager
from stevedore. Remove any reference to modes.
Change-Id: Ic160478625226b4dd17bd68b3d37f3b05823e519