std.tar.extract fails on macOS due to case-sensitivity #35549

Zig Version

0.17.0-dev.607+456b2ec07

Steps to Reproduce, Observed Behavior, and Expected Behavior

I was trying to download MPICH with Zig, and they ship a tarball at their downloads page.

Reproduce & Observe

Running zig fetch --save https://www.mpich.org/static/downloads/5.0.1/mpich-5.0.1.tar.gz yields:

error: unable to unpack tarball
 note: unable to create file 'man/man3/MPI_Offset.3': PathAlreadyExists
 note: unable to create file 'man/man3/MPI_AINT.3': PathAlreadyExists
 note: unable to create file 'man/man3/MPI_Count.3': PathAlreadyExists

Manually inspecting the tarball via tar -tf mpich-5.0.1.tar.gz | grep -i "mpi_offset" yields:

mpich-5.0.1/man/man3/MPI_OFFSET.3
mpich-5.0.1/man/man3/MPI_Offset.3

So there are two files with different capitalizations, and since apfs is case-insensitive, it fails to unpack (file already exists on the createDirAndFile call (tar.zig:622)).

Expected behavior

Using the builtin macos tar utility, the file decompresses silently with the "MPI_Offet.3" file, so likely the last entry in the archive overwrites earlier ones. In fact, tar is perfectly happy overwriting whole directories.
It would be nice if zig fetch worked just like wget + tar does on mac, so I propose adding an exclusive: bool = false field to the ExtactOptions struct and have Fetch.zig set that to true.
Open question: what is the default behavior of tar on Linux?

If that seems like a good course of action I can make a PR.

AFAIK tar on linux will overwrite file with a file, but not a directory with a file or vice versa.

Missing a man page might not be a deal breaker,
but missing a source file is -- Zig's explicitness helps detect the issue earlier.

Consider the following:
A Zig app developed on a case-sensitive FS,
that has a ./src/Modules.zig and ./src/modules.zig

Overwriting one with the other on a case-insensitive FS will simply result in compilation errors due to missing/mismatched type/field declarations.

The paths field already exists to solve this problem.

If both files are included by the paths field, then the error is correct.

If one of the files is excluded, then the tar extraction code could be enhanced to apply this filter earlier and avoid the problem because there is no ambiguity.

If this is a "pristine tarball" fetch (i.e. no build.zig.zon file) then this is why it's planned to support the paths field on the "other side" of the fetch URL.

If tar specification says that subsequent files are supposed to overwrite previous ones, then that can be implemented. However I do not believe that is the case. I don't think there even is a specification. Therefore dealing with ambiguity with an error is appropriate.

the spec is https://www.gnu.org/software/tar/manual/html_chapter/Tar-Internals.html
also https://pubs.opengroup.org/onlinepubs/9799919799/utilities/pax.html#tag_20_94_13_06
core format is ISO/IEC 9945-1:1990 / IEEE Std 1003.1-1990 §10.1 for Archive/Interchange File Format
transcribed in the page above, which also documents gnu's extensions to it

also https://www.gnu.org/software/tar/manual/html_section/Portability.html

https://cgit.git.savannah.gnu.org/cgit/tar.git/tree/src/extract.c?h=v1.35#n1248
https://cgit.git.savannah.gnu.org/cgit/tar.git/tree/src/extract.c?h=v1.35#n1175
implementation depends on cli options

 -k, --keep-old-files
 Don't replace existing files when extracting.
 --overwrite
 Overwrite existing files when extracting.
 -U, --unlink-first
 Remove each file prior to extracting over it.

Totally makes sense, would definitely want to error out on anything in the source's paths colliding, and thanks for checking that there really isn't a standard.

This is a pristine tarball, so yeah I would love to have the option to add a .paths field to my dependency in build.zig.zon, as that would take care of the error. Is there prior discussion on this? How would that layering go if it the tarball wasn't pristine? I'd expect just the intersection of the two.

Alternative option: add the ability to give pristine tarballs their own manifest, right in your project's build.zig.zon. Could look something like:

.dependencies=.{.mpich=.{.url="https://www.mpich.org/static/downloads/5.0.1/mpich-5.0.1.tar.gz",.hash="hash of just the stuff in paths",.manifest=.{// Must follow build.zig.zon specs, so:.name=.mpich,.version="5.0.1",.fingerprint=0xffffffff,// Reserved for bare so we good.minimum_zig_version="0.0.0",// maybe someone ships a zig tarball w/o zon?.dependencies=.{}// could only be usefull if zig tarball w/o zon, which makes no sense..paths=.{"src/","modules/",...// Everything except man in my case}}}}

For a normal package (with a build.zig.zon), this could either be an error, or just completely overwrite the package's. If we go with the latter, then obviously having dependencies and minimum_zig_version make more sense. Could be a cool mechanic to force an upstream package to use a fork of a dependency or something (at the risk of being kinda redundant).