How can a bash function return multiple values?

Question

What is the best practice for return many values from a bash function?

Example1:

Function-script:

function mysqlquery {
 local dbserver='localhost'
 local dbuser='user'
 local dbpass='pass'
 local db='mydb'
 mysql -h "$dbserver" -u "$dbuser" -p "$dbpass" --skip-column-names --raw -e "$*" "$db"
 if [ $? -ne 0 ]; then
 return 1
 fi
}

Source-script:

for XY in $(mysqlquery "select XY from ABC where DEF = 123" 2>/dev/null);do
 dosomethingwith $XY
done
if mysqlquery "select XY from ABC where DEF = 123" 2>/dev/null; then
 echo true
fi

Example2:

Function-script:

function mysqlquery {
 local dbserver='localhost'
 local dbuser='user'
 local dbpass='pass'
 local db='mydb'
 result=$(mysql -h "$dbserver" -u "$dbuser" -p "$dbpass" -e "$*" "$db" 2>/dev/null)
 if [ $? -ne 0 -o -z "$result" ]; then
 return 1
 fi
}

Source-script:

result=$(mysqlquery "select XY from ABC where DEF = 123" 2>/dev/null)
for XY in $result;do
 dosomethingwith $XY
done
if mysqlquery "select XY from ABC where DEF = 123" 2>/dev/null; then
 echo true
fi

Or are there more approaches to returning multiple pieces of information (much more then a single int value)?

Answer 1

Yes, bash's return can only return numbers, and only integers between 0 and 255.

For a shell that can return anything (lists of things), you can look at es:

$ es -c "fn f {return (a 'b c' d \$*)}; printf '%s\n' <={f x y}"
a
b c
d
x
y

Now, in Korn-like shells like bash, you can always return the data in a pre-agreed variable. And that variable can be in any type supported by the shell.

For bash, that can be scalar, sparse arrays (associative arrays with keys restricted to positive integers) or associative arrays with non-empty keys (neither key nor values can contain NUL characters).

See also zsh with normal arrays and associative arrays without those restrictions.

The equivalent of the f es function above could be done with:

f() {
 reply=(a 'b c' d "$@")
}
f
printf '%s\n' "${reply[@]}"

Now, mysql queries generally return tables, that is two-dimensional arrays. The only shell that I know that has multi-dimensional arrays is ksh93 (like bash it doesn't support NUL characters in its variables though).

ksh also supports compound variables that would be handy to return tables with their headers.

It also supports passing variables by reference.

So, there, you can do:

function f {
 typeset -n var=1ドル
 var=(
 (foo bar baz)
 (1 2 3)
 }
}
f reply
printf '%s\n' "${reply[0][1]}" "${reply[1][2]}"

Or:

function f {
 typeset -n var=1ドル
 var=(
 (firstname=John lastname=Smith)
 (firstname=Alice lastname=Doe)
 )
}
f reply
printf '%s\n' "${reply[0].lastname}"

Now, to take the output of mysql and store that in some variables, we need to parse that output which is text with columns of the table separated by TAB characters and rows separated by NL and some encoding for the values to allow them to contain both NL and TAB.

Without --raw, mysql would output a NL as \n, a TAB as \t, a backslash as \\ and a NUL as 0円.

ksh93 also has read -C that can read text formatted as a variable definition (not very different from using eval though), so you can do:

function mysql_to_narray {
 awk -F '\t' -v q="'" '
 function quote(s) {
 gsub(/\\n/, "\n", s)
 gsub(/\\t/, "\t", s)
 gsub(/\\\\/, "\\", s)
 gsub(q, q "\\" q q, s)
 return q s q
 }
 BEGIN{print "("}
 {
 print "("
 for (i = 1; i <= NF; i++)
 print " " quote($i)
 print ")"
 }
 END {print ")"}'
}
function query {
 typeset -n var=1ドル
 typeset db=2ドル
 shift 2
 typeset -i n=0
 typeset IFS=' '
 typeset credentials=/path/to/file.my # not password on the command line!
 set -o pipefail
 mysql --defaults-extra-file="$credentials" --batch \
 --skip-column-names -e "$*" "$db" |
 mysql_to_narray |
 read -C var
}

To be used as

query myvar mydb 'select * from mytable' || exit
printf '%s\n' "${myvar[0][0]}"...

Or for a compound variable:

function mysql_to_array_of_compounds {
 awk -F '\t' -v q="'" '
 function quote(s) {
 gsub(/\\n/, "\n", s)
 gsub(/\\t/, "\t", s)
 gsub(/\\\\/, "\\", s)
 gsub(q, q "\\" q q, s)
 return q s q
 }
 BEGIN{print "("}
 NR == 1 {
 for (i = 1; i<= NF; i++) header[i] = $i
 next
 }
 {
 print "("
 for (i = 1; i <= NF; i++)
 print " " header[i] "=" quote($i)
 print ")"
 }
 END {print ")"}'
}
function query {
 typeset -n var=1ドル
 typeset db=2ドル
 shift 2
 typeset -i n=0
 typeset IFS=' '
 typeset credentials=/path/to/file.my # not password on the command line!
 set -o pipefail
 mysql --defaults-extra-file="$credentials" --batch \
 -e "$*" "$db" |
 mysql_to_array_of_compounds |
 read -C var
}

To be used as:

query myvar mydb 'select "First Name" as firstname, 
 "Last Name" as lastname from mytable' || exit
printf '%s\n' "${myvar[0].firstname"

Note that the header names (firstname, lastname above) have to be valid shell identifiers.

In bash or zsh or yash (though beware array indices start at 1 in zsh and yash and only zsh can store NUL characters), you could always return one array per column, by having awk generate the code to define them:

query() {
 typeset db="1ドル"
 shift
 typeset IFS=' '
 typeset credentials=/path/to/file.my # not password on the command line!
 set -o pipefail
 typeset output
 output=$(
 mysql --defaults-extra-file="$credentials" --batch \
 -e "$*" "$db" |
 awk -F '\t' -v q="'" '
 function quote(s) {
 gsub(/\\n/, "\n", s)
 gsub(/\\t/, "\t", s)
 gsub(/\\\\/, "\\", s)
 gsub(q, q "\\" q q, s)
 return q s q
 }
 NR == 1 {
 for (n = 1; n<= NF; n++) column[n] = $n "=("
 next
 }
 {
 for (i = 1; i < n; i++)
 column[i] = column[i] " " quote($i)
 }
 END {
 for (i = 1; i < n; i++)
 print column[i] ") "
 }'
 ) || return
 eval "$output"
}

To be used as:

query mydb 'select "First Name" as firstname, 
 "Last Name" as lastname from mytable' || exit
printf '%s\n' "${firstname[1]}"

Add a set -o localoptions with zsh or local - with bash4.4+ before the set -o pipefail for the setting of that option to be local to the function like with the ksh93 approach.

Note that in all the above, we're not converting back the 0円s to real NULs as bash or ksh93 would choke on them. You may want to do it if using zsh to be able to work with BLOBs but note that the gsub(/\0円/, "0円", s) would not work with all awk implementations.

In any case, here, I'd use more advanced languages than a shell like perl or python to do this kind of thing.

• BaseZen: If you are really strange like me and want to return a complex type but insist on a functional style because that's "cleaner" (ha hah), you can use stdout and subshells instead: echo some_values to_parse later in the function and use: retval=$(func_call) in the caller, and then parse retval with parameter expansion or whatever further ugly code. Just be sure that any output from all commands in func_call is redirected away from stdout! (to logging system, etc.)
• jwd630: For those like me wondering about es apparently see wryun.github.io/es-shell

Answer 2

Well, it depends on what kind of an output format you want/need. The easiest way is probably to just print the output from the function, that way the function behaves like any other command. Another way would be to set some variable (possibly an associative array) from inside the function. That has the benefit that different items are separated cleanly, but you may need to hard code some variables.

The function in your first example implements the former: whatever the mysql client prints from the function goes to the standard output of the function. Given that the data already comes as a stream of bytes, keeping it as such is fine.

Though here, the question becomes what to do with the output. for x in $(somecmd) ... isn't good, since the output of somecmd is split to words and processed for filename globs. It's usually better to use while read ..., see How can I read a file (data stream, variable) line-by-line (and/or field-by-field)?

To read the output from mysql line by line, you could do

mysql -h "$dbserver" etc. etc. | while read -r line ; do
 dosomethingwith "$line"
done

or with a function

mysqlquery() {
 ...
 mysql -h "$dbserver" etc. etc. 2>/dev/null
}
mysqlquery | while read -r line ; do ...

Note that you don't need the if [ $? -ne 0 ]; then return 1: the return value from the function is the same as that of the last command. Not that it's easy to look into the return value if you're using the function to feed a pipe.