Delete folders which not match a list

Question

I need practical example how get rid folders which are not in the list in Linux. So i do not need to compare its contents or md5sums, just compare folders names.

For example, one folder has few folders inside

target_folder/
├── folder1
├── folder2
├── folder3
└── folder4

and my folders name list is txt file, includes folder1, folder2 and not folder3 and folder4.

How to remove folder3 and folder4 via bash script?

This has been answered on serverfault as

GLOBIGNORE=folder1:folder2
rm -r *
uset GLOBIGNORE

but my real task to delete bunch of folders. The txt list contains around 100 folders and target folder to clean is 200 folders.

Note that this should work both in Linux and FreeBSD.

EDIT: target_folder may contain folders with sub-folders and also files. No spaces and leading dots and names are not similar: foo.com bar.org emptydir file.txt simplefile. But all these items should be deleted except those names in the list.

First answer is more obvious and simple. Second one more advanced and flexible, it allows you to delete based on item type as well.

• Cyrus: Take a look at bash's environment variable $GLOBIGNORE.
• terdon: In what UNIX? Or do you mean Linux? The versions of the various tools are different in different operating systems.
• Demontager: @terdon, I mentioned that i need bash script it does not matter where it installed Unix or Linux.
• terdon: It matters a lot. Tools like find which could be very useful here behave differently on different *nix operating systems. Please always include your OS.
• Demontager: @terdon you right, i meant bash behavior, not tools like find.

Answer 1

Assuming your file names do not contain any of :\[*?, you could still use GLOBIGNORE. Just format your list of directories accordingly. For example:

$ cat names.txt
folder1
folder3

That is very easy to transform into a colon separated list:

$ paste -s -d : names.txt
folder1:folder3

So, you can now set that as the value of GLOBIGNORE:

GLOBIGNORE=$(paste -s -d : ../names.txt)

And proceed to delete them normally:

rm -r -- *

I just tested this on Linux with 300 directories and it worked perfectly.

• Demontager: i was thinking about how to convert list in row with colons, thanks for this great example. I tested on running server all done as expected.

Answer 2

How about this?

find ./target_folder/ \
 -mindepth 1 \
 -maxdepth 1 \
 -type d \
 -not -name 'anything[0-9]*' \
 -exec rm -rf {} \;

Let me explain:

• -mindepth 1 makes sure you don't match ./target_folder/
• -maxdepth 1 makes sure you don't match any subfolders.
• -type d tells find to only match directories, and not files.
• -not -name 'anything[0-9]*' excludes from the pattern anything that matches the pattern. Pretty obvious, that one.
• -exec is to be used with care, especially when it involves rm. You should always test this using echo before going ahead and rm -rfing stuff this way. find can be tricky to
• Don't forget the trailing \; when using exec.

You can read the find manual page for more info. It's an extremely handy tool.

Answer 3

A simple approach is to list all files and ignore the ones that are in the list file. Note that I arrange for the variable ignore_list to start and end with a newline, so that I can check if $x is included in a simple way (just checking that $ignore_list contains $x doesn't work, since it would also match files whose name is a substring of an element of the ignore list).

newline='
'
ignore_list="$newline$(cat list.txt)$newline"
cd target_folder
for x in * .[!.]* ..?*; do
 case "$x" in *"$newline"*) continue;; esac # sanity check: if the file name contains a newline, leave it alone
 case "$ignore_list" in
 *"$newline$x$newline"*) echo "Skipping $x";;
 *) rm -rf -- "$x";
 esac
done

• Demontager: Interesting approach as it does not involve conversion, but i tested and it deletes all folders and ignores my list. Of course before testing adjusted variables inside script.
• Gilles 'SO- stop being evil': @Demontager I just noticed a bug in my code: I used */ to match only directories in the target folder (and symbolic links to directories), but I forgot to strip it off. I've edited my answer.
• Demontager: Perfect, it works as expected now. it will be in my scripts collections. I only can't get this variable d=${d%/} its assignment. I checked by echoing it and it has folder names list which are ignored.
• Demontager: Also wanted to ask how to implement any file type here, i tested with folders and files and it deletes only folders.
• Gilles 'SO- stop being evil': @Demontager You asked for folders in your question, so I implemented this restriction. If you want to consider all files in target_folder and not just directories, change for d in */; do to for d in *; do. Note that dot files are ignored; if you want to remove them as well, change to for d in * .[!.]* ..?*; do

Answer 4

Another solution that matches begin and end of the folder names:

IFS=$'\n'
folders="""folder1
folder2"""
for folder in *; do
 found=0
 if [ -d "$folder" ]; then
 for i in $folders; do
 if [ $(echo "$folder" | grep -c -E "^$i\$") -eq 1 ]; then
 found=1
 fi
 done
 if [ $found -ne 1 ]; then
 echo "$folder found and deleted."
 rm -rf "$folder"
 fi
 fi
done

• Stéphane Chazelas: Note that """"foo"""" is not special, it's just the concatenation of "" with "foo" and "", so the same as "foo".
• Stéphane Chazelas: If your usage of grep -E is so that you can specify the list of folders as extended regular expressions, then you should rather do: if ! printf '%s\n' "$folder" | grep -qxEe "$folders"; then rm -rf -- "$folder"; fi. Though that won't work properly for file names that contain newline characters.

Answer 5

You can do it in bash like

for folder in /target_folder/*/
do
 folder=${folder%/}
 if ! grep -qx "${folder##*/}" folders_list.txt
 then
 rm -rf "$folder"
 fi
done

• Chris Davies: You can find /target_folder -maxdepth 1 -type d | while IFS= read folder; do ... done if the folder names contain spaces or start with a dot.
• Demontager: @Costas I tested your approach it deletes all folders. Also in my particular task folders are not named like folder1,2,3 they have not similar names. No spaces and dots before names, but they have dots inside like: bar.org foo.com
• Costas: @Demontager Suppose the problem in your folder_list.txt file. Check it by grep "some_folder_name" folders_list.txt
• Demontager: @Costas Double checked, my list.txt contains folder1 folder2 test.sh list.txt now if i run it does nothing. Have a look on this simplest-image-hosting.net/jpeg-0-122mz
• Costas: @Demontager I have check script againe and found logical mistake: reverse grep (with -v option) everytime is true because other lines in folder_list.txt is accepted to pattern. Repaired.

Answer 6

With zsh (note that GLOBIGNORE is bash-specific, neither bash nor zsh are installed by default on FreeBSD, but both are available):

(cd /target_folder || exit
all_dirs=(*(/))
to_exclude=(${(f)"$(<names.txt))"})
rm -rf -- ${all_dirs:|to_exclude})

The zsh-specific features:

• *(/). (/) is a glob qualifier that means only select the files of type directory.
• $(<file): expands to the content (minus the trailing newline characters) of file (comes from ksh, bash also has it but does fork a process to read the content of file there).
• ${(f)...}. That's a parameter expansion flag that splits the "$(...)" expansion on line feeds. With bash, you'd need to do something like: IFS=$'\n'; set -f; to_exclude=($(<names.txt)) instead.
• ${A:|B} expands to the elements of the $A array that are not also in the $B array.

Answer 7

On a GNU or FreeBSD system, you can do:

cd /target_folder &&
 printf '%s0円' * | grep --null -vxFf names.txt | xargs -r0 rm -rf