Zig-based CLI to detect duplicate files

Zig 100%

Find a file

Roberto Gambuzzi a2db97562e adds --exclude-dir cli option ( #5 ) adds --exclude-dir cli option Reviewed-on: hgrsd/duplik#5 Reviewed-by: Daniël <hgrsd@noreply.codeberg.org>		2026年05月31日 20:02:42 +02:00
.forgejo/workflows	tests ( #1 )	2026年05月30日 20:44:24 +02:00
src	adds --exclude-dir cli option ( #5 )	2026年05月31日 20:02:42 +02:00
.gitignore	ignore swp	2026年05月28日 10:04:01 +01:00
build.zig	show files	2026年05月25日 16:23:33 +01:00
build.zig.zon	initial	2026年05月25日 15:11:14 +01:00
LICENSE	MIT license	2026年05月30日 21:22:34 +01:00
README.md	adds --exclude-dir cli option ( #5 )	2026年05月31日 20:02:42 +02:00

README.md

Duplik

Duplik is a small tool to detect duplicate files. Duplicate detection is done based on Blake 3 hashes of files combined with their sizes, ignoring empty files. Duplicates will be output as JSON. This tool is only tested with *Nix based systems

duplik:main* λ duplik -D 4 . | jq
[
 {
 "shortHash": "a7239b44140bb930",
 "bytes": 432136,
 "paths": [
 ".zig-cache/o/33bab1f3dd40ff9a1f36a1cd419b7e77/duplik",
 "zig-out/bin/duplik"
 ]
 },
 {
 "shortHash": "efe85429b396b1bf",
 "bytes": 41,
 "paths": [
 ".git/refs/heads/max-depth",
 ".git/refs/remotes/origin/max-depth"
 ]
 }
]

Installation

Install Zig (0.16 is supported currently)
Clone this repo and cd into the repo root.
Run zig build -Doptimize=ReleaseFast to compile the executable
Copy zig-out/bin/duplik to somewhere on your PATH (e.g., ~/.local/bin).

Usage

usage: duplik [options] <directory path>
options:
 -D/--max-depth <n> sets maximum recursion depth; default: 0
 -A/--absolute-paths outputs the paths as absolute paths; default: False
 -F/--exclude-dir exclude some dirs (exact match) from the recursion paths walk; can be specified multiple times

Recipes

Output files bigger than 1MB, exclude the one with the shortest path

./duplik -D 99 <directory path> | jq -r '[.[] | select(.bytes >= 1048576) | .paths | sort_by(length) | .[1:]] | flatten | .[]'