How to edit a list of generated files whose names contain spaces

Question

I've got a relatively small list of filenames generated from a pipeline based on find. The file names contain spaces and possibly punctuation but definitely no other non-printing characters or newlines.

For example,

Netherlands/Purge (GDPR) 2020年01月09日.txt
Netherlands/Purge (GDPR) 2020年01月27日.txt
Switzerland/New mailing 2020年01月27日.txt

I want to edit these files as a set (vi file1 file2 file3 rather than vi file1; vi file2; vi file3), partly so that I can easily jump forwards and backwards between them.

I've started with Using a generated list of filenames as argument list — with spaces , which has a standard find -print0 | xargs -0 mycommand solution. Unfortunately this does not work when mycommand is an editor because although xargs can assemble the set of files to edit, stdin is already taken up from the pipeline and I can't see a way to run an editor in-place. I can't use find -exec vi {} + because I'm using a pipeline to validate the set of filenames, and not just find itself.

My other option is to copy and paste, assembling the list of file names, surrounding them with quotes, and then prefixing the result with vi. For these three files it's trivial, but in the general case it's not an easily-reusable solution,

vi 'Netherlands/Purge (GDPR) 2020年01月09日.txt' 'Netherlands/Purge (GDPR) 2020年01月27日.txt' 'Switzerland/New mailing 2020年01月27日.txt'

Given a GNU/Linux platform with bash as my preferred shell (in case it matters), how can I edit a similarly generated list of files?

• pLumo: can you elaborate a bit more on the pipeline and that filename validation part ? Can't you do find | something | xargs vi with something like find -exec something -exec vi ?
• Chris Davies: @pLumo I'm hesitant to scar you with the entire pipeline, but it's along the lines of find -type f -mtime +14 -mtime -22 -iname '*.xml' | while IFS= read -f x; do xmlstarlet sel -T -t -v '//magicElement -n "$x" | grep -q magicValue && echo "$x"; done
• D. Ben Knoble: You could use the quickfix list as a workaround: vi -q (if yours supports it)
• Chris Davies: @D.BenKnoble I'm not familiar with that; would you like to post it as an answer?

Answer 1

Since you are in Bash,

#!/bin/bash
readarray -d '' -t files < <(find path -type f -print0)
vi -- "${files[@]}"

Replace find path -type f -print0 with your actual pipeline.

Although your files have no newlines, the support for such filenames has been added by user glenn jackman.

To use tabs instead of buffers for vi, add the -p flag: vi -p ....

---

If the pipeline was not required, you could straighforwardly use the -exec option:

find path -type f -exec vi -- {} +

• Chris Davies: That's almost perfect. Unfortunately my filenames are generated from a pipeline not just from find. I've made that statement in the question stronger
• glenn jackman: @Quasímodo, or, without the loop readarray -t files < <(find ...)
• glenn jackman: I did update the readarray command for newline safety, as it was not an intrusive change.
• ilkkachu: You can also use read -d '' with find -print0, but really readarray is the way to populate an array

Answer 2

Unfortunately this does not work when mycommand is an editor because although xargs can assemble the set of files to edit, stdin is already taken up from the pipeline and I can't see a way to run an editor in-place.

That way is documented in the manual page for the GNU Findutils xargs:

 -o, --open-tty
 Reopen stdin as /dev/tty in the child process before executing
 the command. This is useful if you want xargs to run an inter‐
 active application.

So that you could use

find . -name 'pattern' -print0 | xargs -0o vim

However, it is a newer feature. I don't see it in an older system that has xargs 4.4.2; I see it on Ubuntu 18, which has xargs 4.7.0.

Now xargs may not have had the -o option ten years ago, but Bash process substitution existed ten years ago, and xargs has the -a option to read from a file instead of standard input.

So the problem is solvable without xargs -o like this:

xargs -0 -a <(find . -name 'pattern' -print0) vim

Because xargs is reading from (what it thinks is) a file that it received as an argument, it has left standard input alone.

• David Foerster: +1. Even if process substitution is unavailable, one may resort to intermediary files, named FIFOs, or stream redirection plus /dev/fd/*. Another alternative is to add a wrapper around vim that reopens the TTY on stdin: find [...] | xargs [...] sh -c 'exec vim "$@" < "`tty`"' _

Answer 3

From the comments I get something similar like this is your command:

find -type f -mtime +14 -mtime -22 -iname '*.xml' | while IFS= read -f x; do xmlstarlet sel -T -t -v '//magicElement' -n "$x" | grep -q magicValue && echo "$x"; done 

Instead of piping to a while - loop you could use -exec sh -c '...' to filter files:

find -type f -mtime +14 -mtime -22 -iname '*.xml' \
 -exec sh -c 'xmlstarlet sel -T -t -v "//magicElement" "1ドル" | grep -q magicValue' find-sh {} \; \
 -exec vi -- {} +

---

Try:

Consider three files:

.
├── a:<magicElement>magicValue</magicElement>
├── b:<magicElement>magicValue</magicElement>
└── c:<magicElement>someOtherValue</magicElement>
$ find . -type f \
 -exec sh -c 'xmlstarlet sel -T -t -v "//magicElement" "1ドル" | grep -q magicValue' find-sh {} \; \
 -exec echo vi -- {} +

Output:

vi -- ./a ./b

Answer 4

If your vi supports it (and, if your vi is vim, it does), you could use the quickfix list. This is a feature that stores file-names¹ in a navigable list. The important commands are :cnext and :cprev, the equivalent of :next and :prev for quickfix entries. Many others, like :cfile, :cfirst, :clast, and :copen, also exist.

So, the question becomes, how to load the files into the quickfix list? Here are some options:

1. Put the filenames into some kind of file, and use vi -q <file>: the quickfix list will be set based on the file. But if you try this where file contains, e.g.,

Netherlands/Purge (GDPR) 2020年01月09日.txt
Netherlands/Purge (GDPR) 2020年01月27日.txt
Switzerland/New mailing 2020年01月27日.txt

You will be disappointed. The default 'errorformat', which tells vi how to parse file-names out of the error messages, is set for C compilers. So you will need

vi --cmd 'set errorformat=%f' -q <file>

There are several ways to create <file>; one is pipeline ... >errors. But then you have to delete the file.

More interesting, if your shell supports it, is

vi --cmd 'set errorformat=%f' -q <(pipeline)

2. Use the :cexpr command with the system() function: load up vi and run the commands

set errorformat=%f
cexpr system('pipeline')

This is similar to the command line version, but involves an extra step and a more advanced command. This is more useful if you're already in vi when you need to set the quickfix list (though at this point I might just do

:args `pipeline`

assuming that didn't break on spaces and I didn't care about the current argument list).

Notes

1. Often, the file-names are combined with line or column numbers and sometimes even messages—such as from a compiler. :help quickfix for more.

2. If you do this sort of thing a lot, you may like this shell function:

vq () {
 if (($# > 0)); then
 vim -q <("$@" 2>&1)
 else
 printf '%s\n' 'Usage: vq cmd' '' 'Use {cmd} output as quickfix list'
 fi
}

You provide a single command (often grep or the like) to vq and it does the rest; but it only works if the commands output fits into the default 'errorformat'. Adjusting 'errorformat' after the quickfix list is loaded should work, though.

Answer 5

Here are two ugly hacks I have used for years for this problem. both require X.

find ... -print0| ...| xargs -0r gvim -f
find ... -print0| ...| xargs -0r xterm -e vim

It works, even over ssh.

• Chris Davies: Very clever, I'll try to remember this for future use