How to exclude some files not matching certain extensions with grep?

Question

I want to output all lines containing the word OK recursively from a directory. But there are a few extensions which I need to exclude from the result:

*~
*.map
*.js except *.debug.js

I tried:

grep -r --exclude={*~,*.map} "OK" /some/dir

Except that I don't know how to remove from the result all those non-debug .js files.

Answer 1

I would just pass that through a second grep to remove them:

grep -r --exclude={\*~,\*.map} "OK" bar/ | grep -vP '(?<!debug)\.js'

The -v reverses the match, printing lines that don't match the pattern and the -P enables Perl Compatible Regular Expressions which let us use negative lookbehinds. This particular regex, will match .js that is not prececeded by debug which means (since we are inverting the matches) that only those .js files will be printed.

However, as @QuestionOverflow pointed out int the comments, that could have the unintended side effect of filtering out lines that contain OK and js since the grep -v is applied to the entire output, not only the file name. To avoid that, just add a colon (that's what grep uses to separate file names from file contents):

grep -r --exclude={*~,*.map} "OK" bar/ | grep -vP '(?<!debug).js:'

That will still fail if your input line contains foo.js: or if your file name contains :. So, to be sure, use a different approach:

grep -Tr --exclude={*~,*.map} "OK" bar/ | grep -vP '(?<!debug).js\t'

The -T causes grep to print a tab between the file name and the file contents. So, if we simply add a \t to the end of the regex, it will only match against file names, and not the contents of the line.

Still, using find might make more sense regardless.

• Question Overflow: Would I be inadvertently excluding lines in those files that I want, but containing both OK and .js on the same line?
• terdon: @QuestionOverflow ah, yes indeed, good catch. See updated answer.
• Question Overflow: Fantastic answer. Have to accept yours since I ask specifically for grep. Thanks.
• terdon: @QuestionOverflow you're very welcome. In general though, find is probably better for this kind of thing. Getting the right grep can be tricky as you pointed out :).
• Ian D. Allen: Your solutions fail if one has the failglob option set in the shell: bash: no match: --exclude=*~ You need to quote your GLOB pattern arguments to --exclude to hide them from shell expansion, e.g. --exclude={\*~,\*.map}

Answer 2

I'd use find to locate the files and pipe the result through xargs:

$ find . -type f \! -name "*~" \
 \! -name "*.map" \
 \! \( -name "*.js" -and \! -name "*.debug.js" \) \
 -print0 | xargs -0 grep "OK"

This searches for every file not matching "*~", "*.map" or "*.js but not *.debug.js".

Using find you can easily search for rather complex rules and this approach saves you from accidentally removing false positives as could happen with double grep.

• Question Overflow: Nice answer too :)
• terdon: Yes, this is probably the best way, +1. You could also use -exec grep OK {} + instead of xargs and avoid an extra program.
• terdon: @IDAllen no, note that I suggested -exec + not -exec \;, that will run as few commands as possible, much like xargs.

Answer 3

With zsh you can do:

setopt extendedglob
grep OK some/dir/**/^(*~|*.map|(^*debug).js)

Provided of course the argument list isn't too long, in which case you can always do:

printf '%s0円' some/dir/**/^(*~|*.map|(^*debug).js) | xargs -0 grep OK

• don_crissti: Also, you could make the last one zsh-only: autoload zargs and zargs some/dir/**/^(*~|*.map|(^*debug).js) -- grep OK

Answer 4

You can use ripgrep. By default it ignores hidden files and respects your .gitignore file.

You can specify the inclusion or exclusion rules by using the following parameters:

^{-g/--glob GLOB Include or exclude files and directories for searching that match the given glob.}

^{-t/--type TYPE Only search files matching TYPE. Multiple type flags may be provided.}

^{-T/--type-not TYPE Do not search files matching TYPE.}

^{Use the --type-list flag to list all available types.}

Here are few simple examples:

rg -Tjs "OK" # Excludes *.js, *.jsx, *.vue files.
rg -tpy "OK" # Includes Python files.
rg --type-add 'map:*.map' -tmap PATTERN # Excludes *.map files.
rg -g '!*.js' -g '*.debug.js' PATTERN # Excludes *.js apart of *.debug.js.

Here is the complete solution to exclude *.~, *.map, *.js, but not *.debug.js:

rg -g '*.*' -g '!*.~' -g '!*.map' -g '!*.js' -g '*.debug.js' "OK"

---

Testing:

$ touch file.~ file.map file.js file.debug.js file.txt file.md
$ rg --files
file.debug.js
file.js
file.map
file.md
file.txt
$ rg -g '*.*' -g '!*.~' -g '!*.map' -g '!*.js' -g '*.debug.js' --files
file.debug.js
file.md
file.txt

Answer 5

If you don't mind seeing the output slightly out of order (if you do, you can sort it):

grep -r --exclude={*~,*.map,*.js} "OK" /some/dir **/*.debug.js

This requires that your shell supports ** for recursive globbing: zsh does out of the box, bash does after you run shopt -s globstar, ksh93 does after you run set -o globstar.

Without ** support in the shell, you can use two grep commands:

grep -r --exclude={*~,*.map,*.js} "OK" /some/dir
grep -r --include=*.debug.js "OK" /some/dir