cloud-init devel net-convert crash when --debug is enabled

Bug Description

Since 22.2 enabling "--debug" for "cloud-init devel net-convert" will make cloud-init crash.

Probably linked to 3e5938c6ae22b9f158f1404f41e3e43738cadff0 and the use of safe dumper.

Stack trace shows:
Traceback (most recent call last):
File "/xyz/git/cloud-init/bin/cloud-init", line 8, in <module>
sys.exit(main())
File "/xyz/git/cloud-init/lib/python3.8/site-packages/cloudinit/cmd/main.py", line 1059, in main
retval = util.log_time(
File "/xyz/git/cloud-init/lib/python3.8/site-packages/cloudinit/util.py", line 2637, in log_time
ret = func(*args, **kwargs)
File "/xyz/git/cloud-init/lib/python3.8/site-packages/cloudinit/cmd/devel/net_convert.py", line 136, in handle_args
"\n".join(["", "Internal State", safeyaml.dumps(ns, noalias=True), ""])
File "/xyz/git/cloud-init/lib/python3.8/site-packages/cloudinit/safeyaml.py", line 161, in dumps
return yaml.dump(
File "/xyz/git/cloud-init/lib/python3.8/site-packages/yaml/__init__.py", line 253, in dump
return dump_all([data], stream, Dumper=Dumper, **kwds)
File "/xyz/git/cloud-init/lib/python3.8/site-packages/yaml/__init__.py", line 241, in dump_all
dumper.represent(data)
File "/xyz/git/cloud-init/lib/python3.8/site-packages/yaml/representer.py", line 27, in represent
node = self.represent_data(data)
File "/xyz/git/cloud-init/lib/python3.8/site-packages/yaml/representer.py", line 58, in represent_data
node = self.yaml_representers[None](self, data)
File "/xyz/git/cloud-init/lib/python3.8/site-packages/yaml/representer.py", line 231, in represent_undefined
raise RepresenterError("cannot represent an object", data)
yaml.representer.RepresenterError: ('cannot represent an object', <cloudinit.net.network_state.NetworkState object at 0x7fa7979ec340>)

i tried to replace to dumper with the unsafe version and it was working again

Thank you for this bug and making cloud-init better and for the additional commit reference as a guess for when regression was introduced.
You are correct that I reintroduced SafeDumper default and that's what broke the cloud-init devel net-convert command.

Because the original commit cf30836645473c62599e838ab48b2d31677fa584 had inadvertently switched to using yaml.Dumper when dropping the direct yaml.safe_dump() call we deemed it reasonable to switch that dumper back to the original SafeDumper as all runtime callsites provide only simple types and not python objects.

Turns out, the command `cloud-init devel net-convert --debug` does attempt to stuff in a python object as you noted NetworkState.

I have reviewed all runtime call-sites to cloudinit.safeyaml.dumps and all of the objects that cloud-init currently provides are dictionaries of simple dicts with the exception of this devel --debug command.

Here is an upstream PR to fix this one callsite and add a unit test so we recognize this type of failure in the future. https://github.com/canonical/cloud-init/pull/1484

Thank you for the quick answer! I tried with your patch, and it looks good. Thx!

(quick test for info)
$ git logo -1
610d1138 (HEAD, chad/net-convert) net-convert: use yaml.dump for debugging python NetworkState obj
[May 30 09:34AM] 0m0s
~/git/cloud-init ((HEAD detached at chad/net-convert)) :)
$ cloud-init devel net-convert --debug -p/tmp/network-config -k yaml -D centos && echo ok
(...)
ok

Thanks for the test here Benjamin upstream commit landed with this fix. It will be included in cloud-init upstream version 22.3.

https://github.com/canonical/cloud-init/commit/153a7ea25e0d15f707a37655c06cf7155114ea11

This bug is believed to be fixed in cloud-init in version 22.3. If this is still a problem for you, please make a comment and set the state back to New

Thank you.

Tracked in Github Issues as https://github.com/canonical/cloud-init/issues/3979