Show sum of file sizes in directory listing

Question

The Windows dir directory listing command has a line at the end showing the total amount of space taken up by the files listed. For example, dir *.exe shows all the .exe files in the current directory, their sizes, and the sum total of their sizes. I'd love to have similar functionality with my dir alias in bash, but I'm not sure exactly how to go about it.

Currently, I have alias dir='ls -FaGl' in my .bash_profile, showing

drwxr-x---+ 24 mattdmo 4096 Mar 14 16:35 ./
drwxr-x--x. 256 root 12288 Apr 8 21:29 ../
-rw------- 1 mattdmo 13795 Apr 4 17:52 .bash_history
-rw-r--r-- 1 mattdmo 18 May 10 2012 .bash_logout
-rw-r--r-- 1 mattdmo 395 Dec 9 17:33 .bash_profile
-rw-r--r-- 1 mattdmo 176 May 10 2012 .bash_profile~
-rw-r--r-- 1 mattdmo 411 Dec 9 17:33 .bashrc
-rw-r--r-- 1 mattdmo 124 May 10 2012 .bashrc~
drwx------ 2 mattdmo 4096 Mar 24 20:03 bin/
drwxrwxr-x 2 mattdmo 4096 Mar 11 16:29 download/

for example. Taking the answers from this question:

dir | awk '{ total += 4ドル }; END { print total }'

which gives me the total, but doesn't print the directory listing itself. Is there a way to alter this into a one-liner or shell script so I can pass any ls arguments I want to dir and get a full listing plus sum total? For example, I'd like to run dir -R *.jpg *.tif to get the listing and total size of those file types in all subdirectories. Ideally, it would be great if I could get the size of each subdirectory, but this isn't essential.

• bagavadhar: Why does't ls -lh help you ? It prints total sum in top. You can also run du -sh *.exe to get disk space usage information in human readable form.
• MattDMo: @ashwin I don't know what the 'total' ls -lh is printing, but it's not always related to what the awk scripts below calculate, or what I add up by hand. Sometimes it's close to the number of KB of files in the directory, but it doesn't seem to take the allocated sizes of subdirectories into effect. I'd be grateful if you could point me toward an explanation of what exactly that number is...
• bagavadhar: see if my answer below works for you
• aequalsb: ls -lh does not show the total of size of a dir calculating it's contents
• tuga: One liner: du -ach *.exe

Answer 1

There's already a UNIX command for this: du

Just do:

du -bch 

As per convention you can add one or more file or directory paths at the end of the command. -h is an extension to convert the size into a human-friendly format, -b gives you the file size instead of disk usage, and -c gives a total at the end.

• MikeB: Yes, du works fine. You can use the -c option (same as --total) to get a total at the end of the list.
• Stéphane Chazelas: Note that du gives the disk usage, not the sum of file sizes.
• lmat - Reinstate Monica: du -h doesn't sum the sizes of the files passed to it. du -h *.so shows the size of each file, but not the sum. I think what you're wanting here is du -hc *.so (or even du -hc *.so | tail -1). But of course, he wants the directory listing, too.
• Arnauld VM: ! -a means --all. Consider rather using --apparent-size
• ericek111: The -b switch is not in POSIX. macOS doesn't understand it.

Answer 2

You can use du -h -c directory|tail -1

This will generate a single line with memory usage.

• Stéphane Chazelas: du -hs directory if you only want the total.
• kmiklas: @StéphaneChazelas this should be the accepted answer
• Pablo A: @StéphaneChazelas Seems a good way if you want the total of a subset: du -hc /var/tmp/flatpak-cache-* | tail -1`

Answer 3

The following function does most of what you're asking for:

dir () { ls -FaGl "${@}" | awk '{ total += 4ドル; print }; END { print total }'; }

... but it won't give you what you're asking for from dir -R *.jpg *.tif, because that's not how ls -R works. You might want to play around with the find utility for that.

• Jon V: This is true, if you're looking for essentially the contents size of each file, NOT the size the file consumes on disk. This distinction is more pronounced for very small files. On my distro's each file is allocated space on disk in 4 KB chunks (so a 300 byte file still uses 4K on a disk, as reported by the du command). Given that's what the OP was looking for "how much space each file takes up", then du is the way to do it.
• dessert: dir is already the name of a popular GNU coreutil, I'd rather not name a function like that.
• Dmitry Shevkoplyas: Please fix so it works on both Linux and OS-S and your "-a" include "." and "..", which is not good : ) Here's the fixed command: dir () { ls -FAl "${@}" | awk '{ total += 5ドル; print }; END { print "total:"total }'; }
• MattDMo: @DmitryShevkoplyas running Cygwin bash on Windows, ls -a reports the size of . and .. as 0 bytes, so it doesn't affect the total. However, your point is valid on Linux and OSX, so I've changed my function definition of dir accordingly. Thanks!
• Marcelo Idemax: du -ch **/*.ext|tail -1 works pretty fine

Answer 4

with perl:

perl -le 'map { $sum += -s } @ARGV; print $sum' -- *.pdf

Size of all non-hidden PDF files in current directory.

Answer 5

Simply print the current line that you are summing the total of:

dir | awk '{ print; total += 4ドル }; END { print "total size: ",total }'

Answer 6

For counting files in a directory using a mask, I tend to follow this approach:

For bytes

du -ac --bytes | grep "zip$" | awk '{ print; total += 1ドル }; END { print "total lobsters: ", total, " Bytes" }'

For Kibibytes

du -ac --bytes | grep "zip$" | awk '{ print; total += 1ドル }; END { print "total lobsters: ", total/1024, " KiB" }'

For Mebibytes

du -ac --bytes | grep "zip$" | awk '{ print; total += 1ドル }; END { print "total lobsters: " total/1024/1024 " MiB" }'

You get the idea.

The breakdown is simple:

• du - disk usage

• -a - all files
• -c - total bytes
• --bytes - print output in bytes [in newer versions of bash, not sure if this applies anymore]

• grep - global regular expression print [prints output matching patterns]

• "zip$" - pattern to match. 'zip' is the string, and the '$' denotes end of string/line/etc - in this case, match lines that END with 'zip'. Conversely, placing '^' at the start of the string indicates that the pattern will be at the start of the string [ie: "^start" will match lines beginning with the word 'start'] - With this knowledge, wrapping a string in ^ and $ respectively, will match lines that start/end with the pattern used. "^hello people$" will match strings saying 'hello people'. "^hello(.*)people$" will match strings saying 'hello french people' and 'hello coding people', but not 'hello coding people with no lives'

• awk - a scripting language programmed by Aho, Weinberger, and Kernighan. Not a very original name, but a very powerful language that is excellent for text processing and data extraction.

• { print; total += 1ドル }
  • print - print the line currently being iterated over
  • total += 1ドル - initialize variable total if it isn't already, and add the first block separated by the field separator, in this case a space character. This can be changed by the -F flag.
  • ; - line / statement terminator. you can put multiple awk statements on a single line using this, similar to terminating linux command line statements. Otherwise you could have them in a multi line thing still surrounded by { ... }
  • END - this effectively means that awk will execute the actions specified before it exits.
• { print "total lobsters: " total " Bytes" }
  • print "total lobsters: " - first part of string being output
  • total - the variable containing the total sum of lines iterated over
  • " Bytes" - final part of printed string, tacked on to the end of the prior two statements
  • obviously, these three statements are encapsulated in { } like the first part.

So, stepping through an example in a case where we want to count the total number of zip files in a directory:

du -ac --bytes

836544 ./wp-content/themes/astra.1.8.1.zip
934364 ./wp-content/themes/astra.2.0.1.zip
400033 ./wp-content/uploads/2019/09/premium-addons-for-elementor-3.2.9-WJdFQT1mLd3GA81lQEAo.zip
117351218 ./wp-content/uploads/backwpup-fc5928-temp/2019-05-30_00-47-01_TX6FSKC601.zip
1192275 ./wp-content/plugins/essential-addons-elementor-master.zip
170 ./wp-content/plugins/gravityforms/images/doctypes/icon_zip.gif
1969651 ./wp-content/plugins/acf.zip
4284 ./wp-content/plugins/types/application/controllers/api/handler/import_from_zip_file.php

Two components of output, col 1 represented by the numeric values: 836544, 934364... etc, and col 2 being the path of the file.

However, since there are two lines that do not match what we want - icon_zip.gif and import_from_zip_file.php - we want to exclude these. Since du does not provide a way to filter recursively by extension (that I know of), we filter using grep

grep "zip$"

This effectively has the output from du piped to it, and filters the lines that end in zip, eliminating the two records we don't want:

836544 ./wp-content/themes/astra.1.8.1.zip
934364 ./wp-content/themes/astra.2.0.1.zip
400033 ./wp-content/uploads/2019/09/premium-addons-for-elementor-3.2.9-WJdFQT1mLd3GA81lQEAo.zip
117351218 ./wp-content/uploads/backwpup-fc5928-temp/2019-05-30_00-47-01_TX6FSKC601.zip
1192275 ./wp-content/plugins/essential-addons-elementor-master.zip
1969651 ./wp-content/plugins/acf.zip

Then awk parses each line, with the numerics in col 1 being stored in 1ドル

We get this:

836544 ./wp-content/themes/astra.1.8.1.zip
934364 ./wp-content/themes/astra.2.0.1.zip
400033 ./wp-content/uploads/2019/09/premium-addons-for-elementor-3.2.9-WJdFQT1mLd3GA81lQEAo.zip
117351218 ./wp-content/uploads/backwpup-fc5928-temp/2019-05-30_00-47-01_TX6FSKC601.zip
1192275 ./wp-content/plugins/essential-addons-elementor-master.zip
1969651 ./wp-content/plugins/acf.zip
total lobsters: 128.339 MiB

• MattDMo: This is a very thorough answer and explanation. I now have some if/else if statements in the awk part of the function that check the size of total and prints it with B, KB, MB, or GB, as appropriate. Thanks!

Answer 7

Adding the following to .bash_profile or .bashrc works for me.

dir ()
{
find . -iname "$@" -exec ls -lh {} \;
find . -iname "$@" -print0|xargs -r0 du -csh|tail -n 1;
}

Now when i do a dir *.mp3 it does recursively and prints total at the end. You can control how much depth you want by adding a maxdepth parameter to the find. I know running find twice is not a very effiecnt idea. But i couldnt think of a better way. Atleast it gets the job done.

Answer 8

Using du and a awk statement like the one mentioned above will provide what you are looking for.

Example: du /home/abc/Downloads/*.jpg | awk '{ print; total += 1ドル }; END { print "total size: ",total }'

This will list all files in folder Downloads of user abc ending in .jpg and prints the sum of all these files at the end of the listing.

Answer 9

To get both, dir output and size calculation, without using any of the other proposed options, you can use tee(1) and process substitution...

dir | tee >( awk '{ total += 4ドル }; END { print total }' )

• Stéphane Chazelas: No need for tee, just dir | awk '{print; total += 4ドル}; END {print total}'

Answer 10

tee will solve the problem of output disappearing on screen when it is piped to another command like awk.

So the command:

ls -FaGl | printf "%'d\n" $(awk '{SUM+=4ドル}END{print SUM}')

which only prints:

63,519,676,015

Is replaced with the command:

ls -FaGl | tee /dev/stderr | printf "%'d\n" $(awk '{SUM+=4ドル}END{print SUM}')

and now the full file listing appears with total:

total 62031069
drwxrwxrwx 1 rick 20480 Oct 9 15:47 ./
drwxrwxrwx 1 rick 12288 Jul 20 2020 ../
drwxrwxrwx 1 rick 0 Oct 15 2017 Captures/
-rwxrwxrwx 1 rick 504 Jun 29 2020 desktop.ini*
drwxrwxrwx 1 rick 4096 Nov 18 2017 Mass Effect Andromeda/
-rwxrwxrwx 1 rick 210355992 Nov 8 2019 Screencapture 2019年11月08日 at 13.07.14.mp4*
-rwxrwxrwx 1 rick 127445089 Nov 8 2019 Screencapture 2019年11月08日 at 13.43.55.mp4*
-rwxrwxrwx 1 rick 997439911 Nov 11 2019 simplescreenrecorder-2019年11月11日_21.42.51.mkv*
 ( Long listing snipped... )
-rwxrwxrwx 1 rick 1546689758 Sep 6 22:35 simplescreenrecorder-2021年09月06日_21.18.29*
-rwxrwxrwx 1 rick 422607080 Sep 18 19:13 simplescreenrecorder-2021年09月18日_18.57.00*
63,519,676,015

TL;DR

Insert | tee /dev/stderr into your pipeline.

Total in human readable format

In my own ~/.bashrc is this function:

$ grep 'BytesToHuman(' -A20 ~/.bashrc
function BytesToHuman() {
 # https://unix.stackexchange.com/questions/44040/a-standard-tool-to-convert-a-byte-count-into-human-kib-mib-etc-like-du-ls1/259254#259254
 read StdIn
 if ! [[ $StdIn =~ ^-?[0-9]+$ ]] ; then
 echo "$StdIn" # Simply pass back what was passed to us
 exit 1 # Floats or strings not allowed. Only integers.
 fi
 b=${StdIn:-0}; d=''; s=0; S=(Bytes {K,M,G,T,E,P,Y,Z}iB)
 while ((b > 1024)); do
 d="$(printf ".%02d" $((b % 1024 * 100 / 1024)))"
 b=$((b / 1024))
 let s++
 done
 echo "$b$d ${S[$s]}"
 exit 0 # Success!
} # BytesToHuman ()

So simply add | BytesToHuman to the end of the pipeline. Also remove printf builtin that was used previously:

ls -FaGl | tee /dev/stderr | awk '{SUM+=4ドル}END{print SUM}' | BytesToHuman

This will now display the total as:

59.15 GiB

If you would prefer to see 63.51 GB then the BytesToHuman() function needs to be changed from:

b=${StdIn:-0}; d=''; s=0; S=(Bytes {K,M,G,T,E,P,Y,Z}iB)
while ((b > 1024)); do
 d="$(printf ".%02d" $((b % 1024 * 100 / 1024)))"
 b=$((b / 1024))

To:

b=${StdIn:-0}; d=''; s=0; S=(Bytes {K,M,G,T,E,P,Y,Z}B)
while ((b > 1000)); do
 d="$(printf ".%02d" $((b % 1000 * 100 / 1000)))"
 b=$((b / 1000))

Answer 11

du path_to_your_files/*.jpg | awk '{ total += 1ドル }; END { print total }'

• MattDMo: No. First, just giving a command is not an answer. Second, if you'd bothered to read the whole question, the other answers, and my comments, you'd have seen this is NOT what I want.
• Sridhar Sarnobat: To be fair to the poster, this is the first search result for the Google search linux sum human readable sizes and is what I'm looking for.

Answer 12

Just for the sake of clarity and completeness, here is my current iteration of the dir() function, as well as answers to some common comments (including the most highly-voted answer here).

function dir() {
 ls -FAGl --color=always "${@}" | awk '{
 print;
 total += 4ドル
 }; END {
 if (total < 1024)
 print "\t\ttotal: ",total;
 else if (total < (1024 * 1024))
 print "\t\ttotal: ",total/1024,"KB";
 else if (total < (1024 * 1024 * 1024))
 print "\t\ttotal: ",total/(1024*1024),"MB";
 else if (total < (1024 * 1024 * 1024 * 1024))
 print "\t\ttotal: ",total/(1024*1024*1024),"GB";
 }'
}

Here's how it works:

-F appends an indicator (one of */=>@|) to entries

-A does not list the implied . and .. entities

-G omits group names (in systems that include them by default)

-l prints a long listing, which is what we need in order to get file sizes

--color=always should be self-explanatory

"${@}" passes any other arguments to ls

The awk script, which I cobbled together from several sources, including the accepted answer here, first prints a newline, then adds up all the numbers in column 4. That number is then tested to see if it's smaller than 1KB, 1MB, 1GB, and 1TB, respectively. It then prints the total with appropriate unit, indented by 2 tabs (16 spaces) which approximately aligns with the size column on most systems. This can be customized to your own needs.

---

Why do you call it dir? That's a GNU coreutil!

I named the function dir because I grew up using DOS, and when I started playing around with Linux (kernel 1.2!) a friend suggested I alias dir as ls -al as I had gotten so used to typing it. It's just muscle memory now. Eventually I found out about /usr/bin/dir, but I much prefer having long directory listings, so I never bothered changing my alias, and now function. If you use the dir program and/or don't want to shadow it, feel free to name the function whatever you want.

Why don't you use du?

I do, when all I'm interested in is how much space a directory takes up. I wanted to have this functionality when I'm doing directory listings as well (which I do much more frequently), so this function was born.

Answer 13

du * | awk -v sum=0 '{print sum+=1ドル}' | tail -1

• MattDMo: This doesn't do what I was asking for.