Are compiled shell scripts better for performance?

Question

After some googling, I found a way to compile BASH scripts to binary executables (using shc).

I know that shell is an interpreted language, but what does this compiler do? Will it improve the performance of my script in any way?

• Archemar: if your shell script do a lot of I/O (checksum, awk , grep and friends, walking directories) a compiled version isn't going to improve it significantly.
• waltinator: "Performance" and "bash script" are incompatible concepts. If you need "performance", rewrite your program in a compiled language, C, C++, Fortran, ....
• John Carrell: The OP didn't say they were "interested" in performance. They just wanted to know if a performance boost was the result of such "compilation."
• JBentley: @waltinator You're conflating optimal performance with increased performance. Just because a particular method can't achieve the former doesn't mean it isn't worth aiming for the latter. Rewriting a bash script in C++ is unlikely to be a sensible proposition for most use cases; whereas a quick and simple step that can increase performance by a smaller amount might be.

Answer 1

To answer the question in your title, compiled shell scripts could be better for performance — if the result of the compilation represented the result of the interpretation, without having to re-interpret the commands in the script over and over. See for instance ksh93's shcomp, zsh's zcompile, or the in-development Bunster for bash.

However, shc doesn’t compile scripts in this way. It’s not really a compiler, it’s a script "encryption" tool with various protection techniques of dubious effectiveness. When you compile a script with shc, the result is a binary whose contents aren’t immediately readable; when it runs, it decrypts its contents, and runs the tool the script was intended for with the decrypted script, making the original script easy to retrieve (it’s passed in its entirety on the interpreter’s command line, with extra spacing in an attempt to make it harder to find). So the overall performance will always be worse: on top of the time taken to run the original script, there’s the time taken to set the environment up and decrypt the script.

• Stephen Kitt: Comeau Computing used to market CCsh, but they’re out of business, and I’m not aware of any other shell compiler.
• Shadur-don't-feed-the-AI: To be honest, if performance is that big an issue you probably shouldn't be looking at shell scripts in the first place.
• Captain Man: @Shadur In addition, many times shell scripts are calling compiled programs that do the heavy lifting, so it's not too much of a boost anyways. E.g., if you have a script that calls awk, sed, and grep, those are all compiled.
• Peter Cordes: @CaptainMan: If you were programming in a compiled language, you'd read the data yourself and use a regex library instead of fork+exec of multiple separate processes, each of which have to pay the startup overhead for a new process, and for dynamic linking (which is a significant part of the total cost for running grep on a short file). Also, for large amounts of data, avoiding piping the data between processes, costing some overall memory bandwidth, and synchronization between cores in the kernel. You wouldn't expect a shell-script compiler to do that, hence Shadur's point: avoid entirely
• john doe: So essentially the only valid purpose of shc was for people who write insecure code and want to hide their hard coded passwords and vulnerabilities, or script kiddies who are trying to hide obvious malware with the lowest amount of effort possible?

Answer 2

After some googling, I found a way to compile BASH scripts to binary executables (using shc).

It's quite unfortunate that that shc contraption is still featured in google search results, even after it has been utterly debunked all these years: shc is not a compiler, and it does not prevent the source code of the script from being looked at and "stolen".

If anything, shc is even stupider than it has to be, because, after unmangling the script source, it's just passing it as an argument to bash -c, which means that it's visible in /proc/<pid>/cmdline to any user, not just the one running the script. That also runs into the Linux's length limit for a single command line argument (128k bytes). But to make things even more ridiculous, the first part of that argument is filled up with white spaces, so it doesn't appear in ps ;-)

Will it improve the performance of my script in any way?

Yes, your script may not work at all, which means that it will terminate sooner.

• Stephen Kitt: Its current maintainer has been engaging in a lot of "marketing", including creating a Wikipedia page for it which has somehow survived...
• java-addict301: this is the best answer. Very helpful info to know to steer clear of shc.
• Stephen Kitt: The padding with spaces is useless since ps can easily be told to show the full command-line length, and will do that automatically when piped e.g. to grep.
• Stéphane Chazelas: @StephenKitt, to be fair, at the time shc was written, that was not usually the case. On many systems, argument list of processes can only read by root (and you need a setuid root ps that only displays a few dozen bytes of the arg list), and until relatively recently you couldn't get more than 4KiB from the command line on Linux.
• Stephen Kitt: @StéphaneChazelas fair point, since 4.2 five years ago which is quite recent (and somewhat ironically, in a similar timeframe to shc’s revival and all the fuss around it).

Answer 3

In general, there is no way to compile a shell script, because new source text can be introduced by several method at run time, which has therefore bypassed the compiler phase. That new source would be unable to interact with the compiled-in functions or variables.

Two methods of creating runtime source would be:

Source a side file that may have been created or modified since the original script was compiled.

Construct at runtime an arbitrary command in a string, and exec it.