Dash gives error "sh: 1: cannot create : Directory nonexistent"

Question

I am running the following script in Ubuntu 22.04:

#!/bin/bash
logfile="/absolute/path/to/log.txt"
find /absolute/path/to/haystack -type d -name "needle" -execdir sh -c 'pwd > $logfile' \;

The hope is that it would print the full directory path of each folder named needle/ within the folder haystack/ to the file log.txt. Instead, it prints nothing but a long list of the error sh: 1: cannot create : Directory nonexistent to the standard output (and it writes nothing to log.txt).

This answer demonstrates that the error originates in Dash, not Bash. Since I'm using the Bash shebang, this means the error must stem from the use of -execdir sh

-execdir sh -c 'pwd > $logfile' \;

since in Ubuntu sh is symlinked to Dash. My first theory was that Bash isn't passing its variable $logfile to Dash. This answer states that variables are copied to subshells, which was my understanding, but perhaps that doesn't hold true across completely different shells. To test it, I changed the shebang to #!/bin/sh, which is Dash on this system. The result is the same, however: rows of the error sh: 1: cannot create : Directory nonexistent.

What causes this error to happen?

Any help is appreciated, but please note that I've posted because I want to understand the origin of the error, just as the question asks. I'm not interested in any answer that takes a different approach to listing needle/ directories, unless that answer is accompanied by an explanation of why this script fails.

• ilkkachu: A subshell is a specific thing that the shell creates in certain situations. But there's no subshell here. The shell running the script runs find, which then just runs an independent shell. And you haven't exported the variable, so neither find nor the child shell see it.
• glenn jackman: If you export the variable, it will be visible in the sh process.

Answer 1

In your script...

#!/bin/bash
logfile="/absolute/path/to/log.txt"
find /absolute/path/to/haystack -type d -name "needle" -execdir sh -c 'pwd > $logfile' \;

The variable logfile isn't set inside the -execdir script:

• Your script is written inside single quotes (sh -c 'pwd > $logfile'), so the variable isn't expanded by the parent shell.

• logfile is not an environment variable, so it is not available in the child shell.

If you use double quotes instead, it should work as expected:

find /absolute/path/to/haystack -type d -name "needle" -execdir sh -c "pwd > \"$logfile\" \;

---

That said, it's not clear why you're using -execdir here; you could accomplish the same thing using -printf, like this:

find /absolute/path/to/haystack -type d -name "needle" -printf '%h\n' > "$logfile"

• ilkkachu: And even with pwd, moving the redirection to apply to the whole find command (as you did here) would make the shell and exporting the var unnecessary.
• G-Man Says 'Reinstate Monica': (1) Downvoted for telling the user to use a variable without quoting it. (2a) The OP’s command looks like an example of crawling before walking; i.e., trying to get -execdir to work before moving on to the next step of a problem. (2b) With the -L option, the output from pwd could be different from the output from -printf.
• larsks: You could have just left a comment and I would have updated the answer (or I guess you could have edited it yourself). I don't think that was a particularly important issue here: it wasn't germane to the problem, and the variable was being explicitly set to a fixed path with no whitespace, so in context it was obviously not going to be an issue. I've updated the answer just the same.

Answer 2

1. As larsks said, because 'pwd > $logfile' is in single quotes, it gets passed to the secondary shell as a literal string:

   p w d > $ l o g f i l e
   i.e., 14 separate, ordinary characters. The fact that logfile is a variable in the outer shell doesn’t matter (because of the single quotes).

   The secondary shell will then try to expand $logfile and will replace it with nothing (because it isn’t defined in that shell), and so it will try to execute pwd > (with no filename for the redirect). That’s what’s causing the error you are getting.

2. As others have said, the secondary shell doesn’t see the logfile variable because it isn’t exported. A quick fix to your commands is

   logfile="/absolute/path/to/log.txt"
   export logfile
   find /absolute/path/to/haystack -type d -name "needle" -execdir sh -c 'pwd > "$logfile"' \;
   

   Note that you should always quote shell variables unless you have a good reason not to, and you’re sure you know what you’re doing.

The following paragraph is included in an attempt to be complete, but it will confuse you:

3. Another approach that will work until it doesn’t is:
logfile="/absolute/path/to/log.txt"
find /...stack -type d -name "needle" -execdir sh -c "pwd > '$logfile'" \;
This will pass
p w d > ' / a b s o l u t e / p a t h / t o / l o g . t x t '
to the secondary shell (because pwd > '$logfile' is enclosed in double quotes). It then relies on the secondary shell to quote the pathname.

Don’t do this — it will fail if the pathname contains apostrophe(s).

4. As ilkkachu said, since all the output is going to a single file (specified by an absolute pathname), you don’t need to redirect output on every command execution. You could do

   logfile="/absolute/path/to/log.txt"
   find /absolute/path/to/haystack -type d -name "needle" -execdir sh -c 'pwd' \; > "$logfile"
   

   (applying the redirection once, to the entire find command).

   But, if you’re not redirecting output on every command execution, you don’t necessarily need to explicitly invoke the shell. Try

   logfile="/absolute/path/to/log.txt"
   find /absolute/path/to/haystack -type d -name "needle" -execdir pwd \; > "$logfile"
   

   However, this will cause find to execute /bin/pwd (or /usr/bin/pwd or whatever), which might produce different results from sh -c 'pwd', since pwd is a shell builtin command.

• Borea Deitz: Not confusing at all. Very well explained, and just what I needed to understand.