mkinitcpio adds distro-specific system, probing pacman binaries, and xfs_repair #24

stellarpower commented 2019年10月14日 11:21:40 +02:00 (Migrated from github.com)

Hi, been up a long time so not sure what more to say, below is the commit comment.
I'm not a bash novice but it's not exactly my forté, and on top of that without being too familiar with the development process of KaOS, nor arch, nor PGKBUILDS, etc., there may want to be a number of reviews made before pulling this in fully :)

---

• This commit adds a basic distribution-specific loading feature
  for mkinitcpio configs, and an implementation that can use pacman
  to add the binaries for a specfic package to the ramdisk.

• The motivation for this was to add support for repairing the XFS
  root or usr filesystem in case of a no-boot.

• Both the general design and implementaion as well as the KaOS-specific
  code and motivating case and will probably need a good reviewing and
  possible modificaion as I did this in one very long night.

• If the system drops into initramfs because of a dirty filesystem,
  the fsck suite of programs is added, by default, from the upstream
  arch implementation, that determines the root filesystem type and includes
  these as necessary. However, XFS is not the usual target with linux and
  the fsck.xfs utility cannot do much by itself - it refers the user to
  the xfsprogs command xfs_repair.

  • The motivation of this commit was to enable the addition of this binary
    into the ramdisk, so that the filesystem can be repaired during the
    boot process, rather than needing to chroot via e.g. a live CD

• If KaOS moves to Illumos, the game will change significantly, but
  whilst using a linux kernel, the OS has a significant number of differences
  to most typical linuxen found today. So, I decided to aim for a way to
  add extra utilities to the ramdisk to help repairs and debugging, both
  for developers and end users, without doing too much manual work.

  • Based on the rolling-release model and the atomic level of KaOS
    packages, I wrote this to extract the binary executables from a
    given named package, and include these in the mkinitcpio image

    • The files are queried from installed pacman packages, and
      can be run against an arbitrary callback to filter them, based
      on directory location or filetype (links or not), for example.

    • Thus, if a package is added, that needs to be incorporated into
      the ramdisk, a line can simply be added in
      /usr/lib/initcpio/distros/kaos

    • This supporting pacman script may be of (some) use to other
      distros too, hence the aim to separate it form the motivation
      to write it:

  • the new hook distro was added, and this uses the output from
    lsb_release (added dependency) to obtain the current OS' ID.
    It then looks in the distros folder to find a file with this name,
    in lower-case

  • The distros folder isn't a .d-style folder: one file is sourced
    from it, and it is not expected to run everything in there; the
    OS' script may have helper scripts not to be executed directly.
    The idea also was to help tracing the OS' heritage: for example,
    mint coulde use functions from ubuntu which could use functions
    from debian, etc., so that upstream changes are incorporated,
    even if some are not executed, overwritten, or avoided.

  • this hook is added to the default configuration for any build,
    in /etc/mkinitcpio.conf; if the distribution-specific file is
    found, it is sourced, otherwise the hook is a no-op and the build
    continues harmlessly.

  • Note: the build process warns that a command failed, yet the
    process seems to function and the ramdisk works for me

  • TODO:

    • Add support for recursively getting a package and its dependencies?
      This could quickly explode the number of files included for, in
      many cases, no good reason
    • Work with shell scripts. The busybox provided shouldn't have a bash
      inside it, so it would be mecessary to see what sort of scripts can be
      included and what can't. I fear parsing the shebang may be too simple
      to work, and dependencies would need to be extracted somehow
    • Test for other filesystems - I don't have the partitions to do
      this now
    • Define a better way of telling which binaries form the package are
      'important', i.e. required for the core system, and which are extras,
      not likely to be included. At the moment, the config for this search
      is a const in the file, adn importance is just taken as 'does it live
      in /{s,}bin?'

Hi, been up a *long* time so not sure what more to say, below is the commit comment. I'm not a bash novice but it's not exactly my forté, and on top of that without being too familiar with the development process of KaOS, nor arch, nor PGKBUILDS, etc., there may want to be a number of reviews made before pulling this in fully :) * * * - This commit adds a basic distribution-specific loading feature for mkinitcpio configs, and an implementation that can use pacman to add the binaries for a specfic package to the ramdisk. - The motivation for this was to add support for repairing the XFS root or usr filesystem in case of a no-boot. - Both the general design and implementaion as well as the KaOS-specific code and motivating case and will probably need a good reviewing and possible modificaion as I did this in one very long night. - If the system drops into initramfs because of a dirty filesystem, the `fsck` suite of programs is added, by default, from the upstream arch implementation, that determines the root filesystem type and includes these as necessary. However, XFS is not the usual target with linux and the `fsck.xfs` utility cannot do much by itself - it refers the user to the `xfsprogs` command `xfs_repair`. - The motivation of this commit was to enable the addition of this binary into the ramdisk, so that the filesystem can be repaired during the boot process, rather than needing to chroot via e.g. a live CD - If KaOS moves to Illumos, the game will change significantly, but whilst using a linux kernel, the OS has a significant number of differences to most typical linuxen found today. So, I decided to aim for a way to add extra utilities to the ramdisk to help repairs and debugging, both for developers and end users, without doing too much manual work. - Based on the rolling-release model and the atomic level of KaOS packages, I wrote this to extract the binary executables from a given named package, and include these in the mkinitcpio image - The files are queried from *installed* pacman packages, and can be run against an arbitrary callback to filter them, based on directory location or filetype (links or not), for example. - Thus, if a package is added, that needs to be incorporated into the ramdisk, a line can simply be added in `/usr/lib/initcpio/distros/kaos` - This supporting pacman script may be of (some) use to other distros too, hence the aim to separate it form the motivation to write it: - the new hook `distro` was added, and this uses the output from lsb_release (added dependency) to obtain the current OS' ID. It then looks in the distros folder to find a file with this name, in lower-case - The distros folder isn't a .d-style folder: one file is sourced from it, and it is not expected to run everything in there; the OS' script may have helper scripts not to be executed directly. The idea also was to help tracing the OS' heritage: for example, `mint` coulde use functions from `ubuntu` which could use functions from `debian`, etc., so that upstream changes are incorporated, even if some are not executed, overwritten, or avoided. - this hook is added to the default configuration for any build, in `/etc/mkinitcpio.conf`; if the distribution-specific file is found, it is sourced, otherwise the hook is a no-op and the build continues harmlessly. - Note: the build process warns that a command failed, yet the process seems to function and the ramdisk works for me - TODO: - Add support for recursively getting a package and its dependencies? This could quickly explode the number of files included for, in many cases, no good reason - Work with shell scripts. The busybox provided shouldn't have a bash inside it, so it would be mecessary to see what sort of scripts can be included and what can't. I fear parsing the shebang may be too simple to work, and dependencies would need to be extracted somehow - Test for other filesystems - I don't have the partitions to do this now - Define a better way of telling which binaries form the package are 'important', i.e. required for the core system, and which are extras, not likely to be included. At the moment, the config for this search is a const in the file, adn importance is just taken as 'does it live in /{s,}bin?'

This pull request has changes conflicting with the target branch.