Bash: increment a variable in a while loop?

Question

I have a bash script that for each file in a set, greps each line in the file for a string. Then it splits the line on commas, converts the 7th element to a float, and increments a running total by that value.

It looks like this:

for filename in data*.CSV; do
 echo $filename
 ACTUAL_COST=0
 grep '040302010' $filename | while read -r line ; do
 IFS=',' read -a array <<< "$line"
 ACTUAL_COST=$(echo "$ACTUAL_COST + ${array[7]}" | bc)
 echo $ACTUAL_COST
 done
 echo $ACTUAL_COST
done

But the problem I'm having is that this produces output like this:

53.4
72.2
109.1
0

The last value is always 0. After Googling a bit, I think this is because the while loop is executing in a subshell, and so the outer variable isn't changed.

I understand that I probably need to execute the inner loop in a function.

• Stéphane Chazelas: Related: Why is using a shell loop to process text considered bad practice?

Answer 1

That's not how you do shell scripting. You're running several commands in sequence for each line of the files!

Here you want something like:

awk -F, '/040302010/ {actual_cost += 7ドル}
 ENDFILE {print FILENAME ":", +actual_cost; actual_cost=0}
 ' data*.CSV

(assuming GNU awk).

That's one command in total for all your files.

Answer 2

To avoid the subshell you can use the following:

while read -r line
do
 your_stuff
done < <(grep '040302010' $filename')

That way you are able to fill in the result(s) into the variable.

Answer 3

It might be handy to introduce another command substitution, with the core loop logic defined in a function:

sum_cost() {
 sum=0
 while read -r line ; do
 IFS=',' read -a array <<< "$line"
 sum=$(echo "$sum + ${array[7]}" | bc)
 done
 echo $sum
}
for filename in data*.CSV; do
 echo $filename
 ACTUAL_COST=$(grep '040302010' $filename | sum_cost)
 echo $ACTUAL_COST
done