Return of the "Archive URL generated archives changed (but their content remains identical)" #2861

eschwartz commented 2026年07月12日 17:38:31 +02:00

Comment

See #1366

On July 8, Gentoo packaged https://codeberg.org/joostkremers/visual-fill-column/archive/2.7.1.tar.gz with the following integrity data:

DIST visual-fill-column-2.7.1.tar.gz 500293
BLAKE2B aaa29472163b62e86fa0cfe7988f308571a7cbc5e39207c56865353d2766c1f7b055319167b3768dc66747a953879317eb40a628a0a056bc5cd9b195b3d1589e
SHA512 cc359b50666a162ea53dc3705931cf46ee9353f25a39f1a85e10b90b221e95aab022dad2b1ab48f4d509d5e598dcc45e8867190577e903b55c82387d3f8368ec

Today, its b2sum is c3cec402b1eeb49819b421a777b69e632c7407c7cec3874fb5a043f3b9501f101b5e8e359ecb4b2f0f245ec41f4657e33b6aa22b4ff36a4748bc8f6d53f8a61a

If gzip -d is used to compare it with our mirror cache at http://distfiles.gentoo.org/distfiles/c6/visual-fill-column-2.7.1.tar.gz the resulting un-gzipped files are both b2sum: 8a93c55d6b15047d7cea1b49f5b71882210b2a17f3420b4546df64f64a640e8abc27be1406eaebdef5f0e1c87d7958b8f5ef5acc670b48b494fdb5e313c6f2e2

Has something broken with the intended fix?

### Comment See https://codeberg.org/Codeberg/Community/issues/1366 On July 8, Gentoo packaged https://codeberg.org/joostkremers/visual-fill-column/archive/2.7.1.tar.gz with the following integrity data: DIST visual-fill-column-2.7.1.tar.gz 500293 BLAKE2B aaa29472163b62e86fa0cfe7988f308571a7cbc5e39207c56865353d2766c1f7b055319167b3768dc66747a953879317eb40a628a0a056bc5cd9b195b3d1589e SHA512 cc359b50666a162ea53dc3705931cf46ee9353f25a39f1a85e10b90b221e95aab022dad2b1ab48f4d509d5e598dcc45e8867190577e903b55c82387d3f8368ec Today, its b2sum is c3cec402b1eeb49819b421a777b69e632c7407c7cec3874fb5a043f3b9501f101b5e8e359ecb4b2f0f245ec41f4657e33b6aa22b4ff36a4748bc8f6d53f8a61a If gzip -d is used to compare it with our mirror cache at http://distfiles.gentoo.org/distfiles/c6/visual-fill-column-2.7.1.tar.gz the resulting un-gzipped files are both b2sum: 8a93c55d6b15047d7cea1b49f5b71882210b2a17f3420b4546df64f64a640e8abc27be1406eaebdef5f0e1c87d7958b8f5ef5acc670b48b494fdb5e313c6f2e2 Has something broken with the intended fix?

eschwartz commented 2026年07月12日 17:41:20 +02:00

Originally reported by a https://codeberg.org/gentoo/guru user for the x11-misc/ly-1.4.1 overlay package, which has no mirroring, making this much worse. :) Should presumably extend to all archives everywhere though.

Originally reported by a https://codeberg.org/gentoo/guru user for the x11-misc/ly-1.4.1 overlay package, which has no mirroring, making this much worse. :) Should presumably extend to all archives everywhere though.

Gusted commented 2026年07月12日 17:44:26 +02:00

This is correct. Checksums has changed due to a change in gzip implementation.

This is correct. Checksums has changed due to a change in gzip implementation.

eschwartz commented 2026年07月12日 17:53:29 +02:00

Can you please fix it to be stable? :) This was supposed to be solved years ago, and no other forge seems to have been hit by it changing again?

Can you please fix it to be stable? :) This was supposed to be solved years ago, and no other forge seems to have been hit by it changing again?

Gusted commented 2026年07月12日 18:08:13 +02:00

First, sorry to point this out. But we don't particularly consider the checksum of a on-deman archive generation to be stable (mostly because the underlying tooling doesn't give this guarantee). This was only considered stable by chance and we do understand there's a need for some checksum of such files.

That said, it now points to gzip -cn, which to my understanding is also what Github now uses and is stable for the forseeable future (but again no guarentee about this by gzip to my knowledg).

We kinda missed the chance to revert it back to gzip -cn (after git 2.38 changed it), it was reported very late that this has changed and by then a lot of archives were already generated with git archive.

First, sorry to point this out. But we don't particularly consider the checksum of a on-deman archive generation to be stable (mostly because the underlying tooling doesn't give this guarantee). This was only considered stable by chance and we do understand there's a need for some checksum of such files. That said, it now points to `gzip -cn`, which to my understanding is also what Github now uses and is stable for the forseeable future (but again no guarentee about this by gzip to my knowledg). We kinda missed the chance to revert it back to `gzip -cn` (after git 2.38 changed it), it was reported very late that this has changed and by then a lot of archives were already generated with `git archive`.

eschwartz commented 2026年07月13日 00:59:05 +02:00

@Gusted wrote in #2861 (comment):

That said, it now points to gzip -cn, which to my understanding is also what Github now uses and is stable for the forseeable future (but again no guarentee about this by gzip to my knowledg).

Sorry, just trying to better understand the situation for myself. Are you saying that codeberg now changed to always use gzip -cn (and therefore this is a one-time change that will never happen again)?

@Gusted wrote in https://codeberg.org/Codeberg/Community/issues/2861#issuecomment-19077632: > That said, it now points to `gzip -cn`, which to my understanding is also what Github now uses and is stable for the forseeable future (but again no guarentee about this by gzip to my knowledg). Sorry, just trying to better understand the situation for myself. Are you saying that codeberg now changed to always use `gzip -cn` (and therefore this is a one-time change that will never happen again)?

Gusted commented 2026年07月13日 01:01:47 +02:00

@eschwartz wrote in #2861 (comment):

Are you saying that codeberg now changed to always use gzip -cn

Yes.

@eschwartz wrote in #2861 (comment):

(and therefore this is a one-time change that will never happen again)?

I hope so. If gzip -cn changes the output, then this happens again of breaking the checksum stability, but that should be rather unlikely.

@eschwartz wrote in https://codeberg.org/Codeberg/Community/issues/2861#issuecomment-19098398: > Are you saying that codeberg now changed to always use `gzip -cn` Yes. @eschwartz wrote in https://codeberg.org/Codeberg/Community/issues/2861#issuecomment-19098398: > (and therefore this is a one-time change that will never happen again)? I hope so. If gzip -cn changes the output, then this happens again of breaking the checksum stability, but that should be rather unlikely.

eschwartz commented 2026年07月13日 03:01:50 +02:00

Ah, thanks. That was the missing link for me.

Out of curiosity, do you have a link handy for where this change was made?

Ah, thanks. That was the missing link for me. Out of curiosity, do you have a link handy for where this change was made?

Gusted commented 2026年07月13日 05:44:39 +02:00

Codeberg-Infrastructure/scripted-configuration@db0baa8e28
Codeberg-Infrastructure/build-deploy-forgejo@3da100a756

https://codeberg.org/Codeberg-Infrastructure/scripted-configuration/commit/db0baa8e287a820f8d8c1bff7b0c964d0380ac76 https://codeberg.org/Codeberg-Infrastructure/build-deploy-forgejo/commit/3da100a75688a684053181d8496f095a9333ca94

WhyNotHugo commented 2026年07月13日 18:00:20 +02:00

But we don't particularly consider the checksum of a on-deman archive generation to be stable (mostly because the underlying tooling doesn't give this guarantee). This was only considered stable by chance and we do understand there's a need for some checksum of such files.

FWIW, pretty much any downstream consumer validates the checksum of releases, changes are problematic to pretty much all distros.

Please consider treating them as stable in future.

> But we don't particularly consider the checksum of a on-deman archive generation to be stable (mostly because the underlying tooling doesn't give this guarantee). This was only considered stable by chance and we do understand there's a need for some checksum of such files. FWIW, pretty much any downstream consumer validates the checksum of releases, changes are problematic to pretty much all distros. Please consider treating them as stable in future.

eschwartz commented 2026年07月13日 18:20:05 +02:00

@Gusted, thanks for the link.

I'm happy to hear that this is a one-time change "in the interest of future stability", and I can definitely live with a one-time change. In fact, "make a one-time change so that it will be stable going forward" is the kind of tactic that I would recommend as the second-best outcome (after "have the good luck that it was always stable from the beginning"). So from my perspective, all is well.¹

I'm also somehow not terribly surprised that zlib-ng was involved somehow. :)

@Gusted, thanks for the link. I'm happy to hear that this is a one-time change "in the interest of future stability", and I can definitely live with a one-time change. In fact, "make a one-time change so that it will be stable going forward" is the kind of tactic that I would recommend as the second-best outcome (after "have the good luck that it was always stable from the beginning"). So from my perspective, all is well.[^1] I'm also somehow not terribly surprised that zlib-ng was involved somehow. :) [^1]: in the sense that we have something reasonable to point to in order to explain why hashes are changing, rather than "dunno, maybe everything was hacked"