NAME

env_parallel - export environment to GNU parallel

SYNOPSIS

env_parallel [--record-env|--session|--end-session] [options for GNU Parallel]

DESCRIPTION

env_parallel is a shell function that exports the current environment to GNU parallel.

If the shell function is not loaded, a dummy script will be run instead that explains how to install the function.

env_parallel is 100 ms slower at startup than pure GNU parallel, and takes up to 30% longer to start a job (typically 15 ms).

Due to the problem with environment space (see below) you are recommended only to transfer the environment that you need.

To help you do that, you can mark names that should not be transferred. This can be done with either --session or --record-env.

# Record the "clean" environment (this only needs to be run once)
env_parallel--record-env
# Optionally edit ~/.parallel/ignored_vars (only needed once)
# Define whatever you want to use
myfunc(){myaliasandfunctions$myvarwork.1ドル.;}
aliasmyalias='echo Aliases'
myvar='and variables'
# Use --env _ to only transfer the names not in the "empty" environment
env_parallel--env_-Slocalhostmyfunc:::Hooray

Or:

# Do --record-env into $PARALLEL_IGNORED_NAMES
env_parallel--session
# Define whatever you want to use
myfunc(){myaliasandfunctions$myvarwork.1ドル.;}
aliasmyalias='echo Aliases'
myvar='and variables'
# env_parallel will not export names in $PARALLEL_IGNORED_NAMES
env_parallel-Slocalhostmyfunc:::Hooray
# Optionally
env_parallel--end-session

In csh --session is not supported:

# Record the "clean" environment - this only needs to be run once
env_parallel--record-env
# Optionally edit ~/.parallel/ignored_vars - only needed once
# Define whatever you want to use
aliasmyalias'echo Aliases $myvar \!*.'
setmyvar='and variables'
# Use --env _ to only transfer the names not in the "empty" environment
env_parallel--env_-Slocalhostmyalias:::work

Environment space

By default env_parallel will export all environment variables, arrays, aliases, functions and shell options (see details for the individual shells below).

But this only works if the size of the current environment is smaller than the maximal length of a command and smaller than half of the max if running remotely. E.g. The max size of Bash's command is 128 KB, so env_parallel will fail if 'set | wc -c' is bigger than 128 KB. Technically the limit is in execve(1) which IPC::open3 uses.

Bash completion functions are well-known for taking up well over 128 KB of environment space and the primary reason for causing env_parallel to fail.

Instead you can use --env to specify which variables, arrays, aliases and functions to export as this will only export those with the given name. Or follow the recommended usage in shown in DESCRIPTION.

OPTIONS

Same as GNU parallel in addition to these:

  • --end-session

Undo last --session

  • --record-env

Record all names currently defined to be ignored every time running env_parallel in the future.

  • --session

Ignore all names currently defined. Aliases, variables, arrays, and functions currently defined will not be transferred.

But names defined after running parallel --session will be transferred.

This is only valid in the running shell, and can be undone with parallel --end-session.

You can run multiple --session inside each other:

env_parallel--session
var=not
# var is transferred
env_parallel-Slocalhost'echo var is $var':::ignored
env_parallel--session
# var is not transferred
env_parallel-Slocalhost'echo var is $var':::ignored
env_parallel--end-session
# var is transferred again
env_parallel-Slocalhost'echo var is $var':::ignored

SUPPORTED SHELLS

Ash

Installation

Put this in $HOME/.profile:

.env_parallel.ash

E.g. by doing:

echo'. env_parallel.ash'>>$HOME/.profile

Supported use

--env is supported to export only the variable, or alias with the given name. Multiple --envs can be given.

--session is supported.

  • aliases

aliasmyecho='echo aliases'
env_parallelmyecho:::work
env_parallel-Sservermyecho:::work
env_parallel--envmyechomyecho:::work
env_parallel--envmyecho-Sservermyecho:::work
aliasmultiline='echo multiline
 echo aliases'
env_parallelmultiline:::work
env_parallel-Sservermultiline:::work
env_parallel--envmultilinemultiline:::work
env_parallel--envmultiline-Sservermultiline:::work

  • functions

ashcannotlistdefinedfunctions-thusisnotsupported.

  • variables

myvar=variables
env_parallelecho'$myvar':::work
env_parallel-Sserverecho'$myvar':::work
env_parallel--envmyvarecho'$myvar':::work
env_parallel--envmyvar-Sserverecho'$myvar':::work

  • arrays

Arrays are not supported by Ash.

Bash

Installation

Put this in $HOME/.bashrc:

.env_parallel.bash

E.g. by doing:

echo'. env_parallel.bash'>>$HOME/.bashrc

Supported use

--env is supported to export only the variable, alias, function, or array with the given name. Multiple --envs can be given.

--session is supported.

  • aliases

aliasmyecho='echo aliases'
env_parallelmyecho:::work
env_parallel-Sservermyecho:::work
env_parallel--envmyechomyecho:::work
env_parallel--envmyecho-Sservermyecho:::work
aliasmultiline='echo multiline
 echo aliases'
env_parallel'multiline {};
 echo but only when followed by a newline':::work
env_parallel-Sserver'multiline {};
 echo but only when followed by a newline':::work
env_parallel--envmultiline'multiline {};
 echo but only when followed by a newline':::work
env_parallel--envmultiline-Sserver'multiline {};
 echo but only when followed by a newline':::work

  • functions

myfunc(){echofunctions$*;}
env_parallelmyfunc:::work
env_parallel-Sservermyfunc:::work
env_parallel--envmyfuncmyfunc:::work
env_parallel--envmyfunc-Sservermyfunc:::work

  • variables

myvar=variables
env_parallelecho'$myvar':::work
env_parallel-Sserverecho'$myvar':::work
env_parallel--envmyvarecho'$myvar':::work
env_parallel--envmyvar-Sserverecho'$myvar':::work

  • arrays

myarray=(arrayswork,too)
env_parallel-kecho'${myarray[{}]}':::012
env_parallel-k-Sserverecho'${myarray[{}]}':::012
env_parallel-k--envmyarrayecho'${myarray[{}]}':::012
env_parallel-k--envmyarray-Sserver\
echo'${myarray[{}]}':::012

BUGS

Due to a bug in Bash, aliases containing newlines must be followed by a newline in the command. Some systems are not affected by this bug, but will print a warning anyway.

csh

env_parallel for csh breaks $PARALLEL, so do not use $PARALLEL.

Installation

Put this in $HOME/.cshrc:

source`whichenv_parallel.csh`

E.g. by doing:

echo'source `which env_parallel.csh`'>>$HOME/.cshrc

Supported use

--env is supported to export only the variable, alias, or array with the given name. Multiple --envs can be given.

  • aliases

aliasmyecho'echo aliases'
env_parallelmyecho:::work
env_parallel-Sservermyecho:::work
env_parallel--envmyechomyecho:::work
env_parallel--envmyecho-Sservermyecho:::work

  • functions

Not supported by csh.

  • variables

setmyvar=variables
env_parallelecho'$myvar':::work
env_parallel-Sserverecho'$myvar':::work
env_parallel--envmyvarecho'$myvar':::work
env_parallel--envmyvar-Sserverecho'$myvar':::work

  • arrays with no special chars

setmyarray=(arrayswork,too)
env_parallel-kecho\$'{myarray[{}]}':::123
env_parallel-k-Sserverecho\$'{myarray[{}]}':::123
env_parallel-k--envmyarrayecho\$'{myarray[{}]}':::123
env_parallel-k--envmyarray-Sserver\
echo\$'{myarray[{}]}':::123

Dash

Installation

Put this in $HOME/.profile:

.env_parallel.dash

E.g. by doing:

echo'. env_parallel.dash'>>$HOME/.profile

Supported use

--env is supported to export only the variable, or alias with the given name. Multiple --envs can be given.

--session is supported.

  • aliases

aliasmyecho='echo aliases'
env_parallelmyecho:::work
env_parallel-Sservermyecho:::work
env_parallel--envmyechomyecho:::work
env_parallel--envmyecho-Sservermyecho:::work
aliasmultiline='echo multiline
 echo aliases'
env_parallelmultiline:::work
env_parallel-Sservermultiline:::work
env_parallel--envmultilinemultiline:::work
env_parallel--envmultiline-Sservermultiline:::work

  • functions

dashcannotlistdefinedfunctions-thusisnotsupported.

  • variables

myvar=variables
env_parallelecho'$myvar':::work
env_parallel-Sserverecho'$myvar':::work
env_parallel--envmyvarecho'$myvar':::work
env_parallel--envmyvar-Sserverecho'$myvar':::work

  • arrays

dashdoesnotsupportarrays.

fish

Installation

Put this in $HOME/.config/fish/config.fish:

source(whichenv_parallel.fish)

E.g. by doing:

echo'source (which env_parallel.fish)'\
>>$HOME/.config/fish/config.fish

Supported use

--env is supported to export only the variable, alias, function, or array with the given name. Multiple --envs can be given.

--session is supported.

  • aliases

aliasmyecho'echo aliases'
env_parallelmyecho:::work
env_parallel-Sservermyecho:::work
env_parallel--envmyechomyecho:::work
env_parallel--envmyecho-Sservermyecho:::work

  • functions

functionmyfunc
echofunctions$argv
end
env_parallelmyfunc:::work
env_parallel-Sservermyfunc:::work
env_parallel--envmyfuncmyfunc:::work
env_parallel--envmyfunc-Sservermyfunc:::work

  • variables

setmyvarvariables
env_parallelecho'$myvar':::work
env_parallel-Sserverecho'$myvar':::work
env_parallel--envmyvarecho'$myvar':::work
env_parallel--envmyvar-Sserverecho'$myvar':::work

  • arrays

Arrays normally work, but fail intermitently.

setmyarrayarrayswork,too
env_parallel-kecho'$myarray[{}]':::123
env_parallel-k-Sserverecho'$myarray[{}]':::123
env_parallel-k--envmyarrayecho'$myarray[{}]':::123
env_parallel-k--envmyarray-Sserver\
echo'$myarray[{}]':::123

ksh

Installation

Put this in $HOME/.kshrc:

sourceenv_parallel.ksh

E.g. by doing:

echo'source env_parallel.ksh'>>$HOME/.kshrc

Supported use

--env is supported to export only the variable, alias, function, or array with the given name. Multiple --envs can be given.

--session is supported.

  • aliases

aliasmyecho='echo aliases'
env_parallelmyecho:::work
env_parallel-Sservermyecho:::work
env_parallel--envmyechomyecho:::work
env_parallel--envmyecho-Sservermyecho:::work
aliasmultiline='echo multiline
 echo aliases'
env_parallelmultiline:::work
env_parallel-Sservermultiline:::work
env_parallel--envmultilinemultiline:::work
env_parallel--envmultiline-Sservermultiline:::work

  • functions

myfunc(){echofunctions$*;}
env_parallelmyfunc:::work
env_parallel-Sservermyfunc:::work
env_parallel--envmyfuncmyfunc:::work
env_parallel--envmyfunc-Sservermyfunc:::work

  • variables

myvar=variables
env_parallelecho'$myvar':::work
env_parallel-Sserverecho'$myvar':::work
env_parallel--envmyvarecho'$myvar':::work
env_parallel--envmyvar-Sserverecho'$myvar':::work

  • arrays

myarray=(arrayswork,too)
env_parallel-kecho'${myarray[{}]}':::012
env_parallel-k-Sserverecho'${myarray[{}]}':::012
env_parallel-k--envmyarrayecho'${myarray[{}]}':::012
env_parallel-k--envmyarray-Sserver\
echo'${myarray[{}]}':::012

mksh

Installation

Put this in $HOME/.mkshrc:

sourceenv_parallel.mksh

E.g. by doing:

echo'source env_parallel.mksh'>>$HOME/.mkshrc

Supported use

--env is supported to export only the variable, alias, function, or array with the given name. Multiple --envs can be given.

--session is supported.

  • aliases

aliasmyecho='echo aliases'
env_parallelmyecho:::work
env_parallel-Sservermyecho:::work
env_parallel--envmyechomyecho:::work
env_parallel--envmyecho-Sservermyecho:::work
aliasmultiline='echo multiline
 echo aliases'
env_parallelmultiline:::work
env_parallel-Sservermultiline:::work
env_parallel--envmultilinemultiline:::work
env_parallel--envmultiline-Sservermultiline:::work

  • functions

myfunc(){echofunctions$*;}
env_parallelmyfunc:::work
env_parallel-Sservermyfunc:::work
env_parallel--envmyfuncmyfunc:::work
env_parallel--envmyfunc-Sservermyfunc:::work

  • variables

myvar=variables
env_parallelecho'$myvar':::work
env_parallel-Sserverecho'$myvar':::work
env_parallel--envmyvarecho'$myvar':::work
env_parallel--envmyvar-Sserverecho'$myvar':::work

  • arrays

myarray=(arrayswork,too)
env_parallel-kecho'${myarray[{}]}':::012
env_parallel-k-Sserverecho'${myarray[{}]}':::012
env_parallel-k--envmyarrayecho'${myarray[{}]}':::012
env_parallel-k--envmyarray-Sserver\
echo'${myarray[{}]}':::012

pdksh

Installation

Put this in $HOME/.profile:

sourceenv_parallel.pdksh

E.g. by doing:

echo'source env_parallel.pdksh'>>$HOME/.profile

Supported use

--env is supported to export only the variable, alias, function, or array with the given name. Multiple --envs can be given.

--session is supported.

  • aliases

aliasmyecho="echo aliases";
env_parallelmyecho:::work;
env_parallel-Sservermyecho:::work;
env_parallel--envmyechomyecho:::work;
env_parallel--envmyecho-Sservermyecho:::work

  • functions

myfunc(){echofunctions$*;};
env_parallelmyfunc:::work;
env_parallel-Sservermyfunc:::work;
env_parallel--envmyfuncmyfunc:::work;
env_parallel--envmyfunc-Sservermyfunc:::work

  • variables

myvar=variables;
env_parallelecho"\$myvar":::work;
env_parallel-Sserverecho"\$myvar":::work;
env_parallel--envmyvarecho"\$myvar":::work;
env_parallel--envmyvar-Sserverecho"\$myvar":::work

  • arrays

myarray=(arrayswork,too);
env_parallel-kecho"\${myarray[{}]}":::012;
env_parallel-k-Sserverecho"\${myarray[{}]}":::012;
env_parallel-k--envmyarrayecho"\${myarray[{}]}":::012;
env_parallel-k--envmyarray-Sserver\
echo"\${myarray[{}]}":::012

sh

Installation

Put this in $HOME/.profile:

.env_parallel.sh

E.g. by doing:

echo'. env_parallel.sh'>>$HOME/.profile

Supported use

--env is supported to export only the variable, or alias with the given name. Multiple --envs can be given.

--session is supported.

  • aliases

shdoesnotsupportaliases.

  • functions

myfunc(){echofunctions$*;}
env_parallelmyfunc:::work
env_parallel-Sservermyfunc:::work
env_parallel--envmyfuncmyfunc:::work
env_parallel--envmyfunc-Sservermyfunc:::work

  • variables

myvar=variables
env_parallelecho'$myvar':::work
env_parallel-Sserverecho'$myvar':::work
env_parallel--envmyvarecho'$myvar':::work
env_parallel--envmyvar-Sserverecho'$myvar':::work

  • arrays

shdoesnotsupportarrays.

tcsh

env_parallel for tcsh breaks $PARALLEL, so do not use $PARALLEL.

Installation

Put this in $HOME/.tcshrc:

source`whichenv_parallel.tcsh`

E.g. by doing:

echo'source `which env_parallel.tcsh`'>>$HOME/.tcshrc

Supported use

--env is supported to export only the variable, alias, or array with the given name. Multiple --envs can be given.

  • aliases

aliasmyecho'echo aliases'
env_parallelmyecho:::work
env_parallel-Sservermyecho:::work
env_parallel--envmyechomyecho:::work
env_parallel--envmyecho-Sservermyecho:::work

  • functions

Not supported by tcsh.

  • variables

setmyvar=variables
env_parallelecho'$myvar':::work
env_parallel-Sserverecho'$myvar':::work
env_parallel--envmyvarecho'$myvar':::work
env_parallel--envmyvar-Sserverecho'$myvar':::work

  • arrays with no special chars

setmyarray=(arrayswork,too)
env_parallel-kecho\$'{myarray[{}]}':::123
env_parallel-k-Sserverecho\$'{myarray[{}]}':::123
env_parallel-k--envmyarrayecho\$'{myarray[{}]}':::123
env_parallel-k--envmyarray-Sserver\
echo\$'{myarray[{}]}':::123

Zsh

Installation

Put this in $HOME/.zshrc:

.env_parallel.zsh

E.g. by doing:

echo'. env_parallel.zsh'>>$HOME/.zshenv

Supported use

--env is supported to export only the variable, alias, function, or array with the given name. Multiple --envs can be given.

--session is supported.

  • aliases

aliasmyecho='echo aliases'
env_parallelmyecho:::work
env_parallel-Sservermyecho:::work
env_parallel--envmyechomyecho:::work
env_parallel--envmyecho-Sservermyecho:::work
aliasmultiline='echo multiline
 echo aliases'
env_parallelmultiline:::work
env_parallel-Sservermultiline:::work
env_parallel--envmultilinemultiline:::work
env_parallel--envmultiline-Sservermultiline:::work

  • functions

myfunc(){echofunctions$*;}
env_parallelmyfunc:::work
env_parallel-Sservermyfunc:::work
env_parallel--envmyfuncmyfunc:::work
env_parallel--envmyfunc-Sservermyfunc:::work

  • variables

myvar=variables
env_parallelecho'$myvar':::work
env_parallel-Sserverecho'$myvar':::work
env_parallel--envmyvarecho'$myvar':::work
env_parallel--envmyvar-Sserverecho'$myvar':::work

  • arrays

myarray=(arrayswork,too)
env_parallel-kecho'${myarray[{}]}':::123
env_parallel-k-Sserverecho'${myarray[{}]}':::123
env_parallel-k--envmyarrayecho'${myarray[{}]}':::123
env_parallel-k--envmyarray-Sserver\
echo'${myarray[{}]}':::123

EXIT STATUS

Same as GNU parallel.

AUTHOR

When using GNU env_parallel for a publication please cite:

  1. Tange (2018): GNU Parallel 2018, March 2018, ISBN 9781387509881, DOI: 10.5281/zenodo.1146014.

This helps funding further development; and it won't cost you a cent. If you pay 10000 EUR you should feel free to use GNU Parallel without citing.

Copyright (C) 2007年10月18日 Ole Tange, http://ole.tange.dk

Copyright (C) 2008-2010 Ole Tange, http://ole.tange.dk

Copyright (C) 2010-2025 Ole Tange, http://ole.tange.dk and Free Software Foundation, Inc.

LICENSE

This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 3 of the License, or at your option any later version.

This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.

You should have received a copy of the GNU General Public License along with this program. If not, see <http://www.gnu.org/licenses/>.

Documentation license I

Permission is granted to copy, distribute and/or modify this documentation under the terms of the GNU Free Documentation License, Version 1.3 or any later version published by the Free Software Foundation; with no Invariant Sections, with no Front-Cover Texts, and with no Back-Cover Texts. A copy of the license is included in the file LICENSES/GFDL-1.3-or-later.txt.

Documentation license II

You are free:

  • to Share

to copy, distribute and transmit the work

  • to Remix

to adapt the work

Under the following conditions:

  • Attribution

You must attribute the work in the manner specified by the author or licensor (but not in any way that suggests that they endorse you or your use of the work).

  • Share Alike

If you alter, transform, or build upon this work, you may distribute the resulting work only under the same, similar or a compatible license.

With the understanding that:

  • Waiver

Any of the above conditions can be waived if you get permission from the copyright holder.

  • Public Domain

Where the work or any of its elements is in the public domain under applicable law, that status is in no way affected by the license.

  • Other Rights

In no way are any of the following rights affected by the license:

  • Your fair dealing or fair use rights, or other applicable copyright exceptions and limitations;

  • The author's moral rights;

  • Rights other persons may have either in the work itself or in how the work is used, such as publicity or privacy rights.

  • Notice

For any reuse or distribution, you must make clear to others the license terms of this work.

A copy of the full license is included in the file as LICENCES/CC-BY-SA-4.0.txt

DEPENDENCIES

env_parallel uses GNU parallel.

SEE ALSO

parallel(1), ash(1), bash(1), csh(1), dash(1), fish(1), ksh(1), pdksh(1) tcsh(1), zsh(1).