cloud-config: cloud_dir setting not honored globally by cloud-init

Bug Description

Changes to /etc/cloud/cloud.cfg system_info: paths: cloud_dir setting is not honored globally in cloud-init. Some paths continue to hardcode /var/lib/cloud paths to certain operations.

Affects cloud-init version 22.2

[Test Plan]

cat > 95-custom-cloud-dir.cfg <<EOF
system_info:
paths:
cloud_dir: /new-cloud-dir
EOF
lxc launch ubuntu-daily:kinetic dev-k
lxc file push 95-custom-cloud-dir.cfg dev-k/etc/cloud/cloud.cfg.d/
lx exec dev-x -- rm -rf /var/lib/cloud
lxc exec dev-k -- cloud-init clean --logs --reboot
lxc exec dev-k -- cloud-init status --wait --long

# Expect no logs referencing use of /var/lib/cloud
lxc exec dev-k -- grep /var/lib /var/log/cloud-init.log
2022年06月01日 23:18:07,234 - atomic_helper.py[DEBUG]: Atomically writing to file /var/lib/cloud/data/status.json (via temporary file /var/lib/cloud/data/tmpqwldn_op) - w: [644] 486 bytes/chars
2022年06月01日 23:18:08,642 - atomic_helper.py[DEBUG]: Atomically writing to file /var/lib/cloud/data/status.json (via temporary file /var/lib/cloud/data/tmpy7wm60kf) - w: [644] 512 bytes/chars
2022年06月01日 23:18:13,156 - atomic_helper.py[DEBUG]: Atomically writing to file /var/lib/cloud/data/status.json (via temporary file /var/lib/cloud/data/tmpvw2y443q) - w: [644] 540 bytes/chars
2022年06月01日 23:18:13,996 - atomic_helper.py[DEBUG]: Atomically writing to file /var/lib/cloud/data/status.json (via temporary file /var/lib/cloud/data/tmpdzbb4lu7) - w: [644] 567 bytes/chars
2022年06月01日 23:18:13,996 - atomic_helper.py[DEBUG]: Atomically writing to file /var/lib/cloud/data/result.json (via temporary file /var/lib/cloud/data/tmpecr955s_) - w: [644] 64 bytes/chars
2022年06月01日 23:18:13,996 - util.py[DEBUG]: Creating symbolic link from '/run/cloud-init/result.json' => '../../var/lib/cloud/data/result.json'

# Expect to find no files in /var/lib/cloud
lxc exec dev-x find /var/lib/cloud
/var/lib/cloud
/var/lib/cloud/data
/var/lib/cloud/data/status.json

[ Possible hardcoded paths to resolve]
- cloudinit/util.py:fetch_ssl_details
- cloudinit/sources/DataSourceBigstep.py
- cloudinit/sources/DataSourceAzure.py
- cloudinit/cmd/main.py:status_wrapper
- cloudinit/cmd/devel/logs.py
- cloudinit/apport.py:USER_DATA_FILE
- cloudinit/config/cc_snap.py:ASSERTIONS_FILE

See https://github.com/canonical/cloud-init/pull/1523

This bug is believed to be fixed in cloud-init in version 22.3. If this is still a problem for you, please make a comment and set the state back to New

Thank you.

Tracked in Github Issues as https://github.com/canonical/cloud-init/issues/3981