Repository cache of "global" artifact repositories, such as Maven Central, Pypi, NPM etc. #2146

jornfranke commented 2025年09月24日 19:24:15 +02:00

Comment

Recently the stewards of popular open infrastructure, such as Maven Central, Pypi, NPM published a letter on the lack of funding of their infrastructure which could endanger their operation. Additionally, almost all of them are US-hosted and controlled.

One could think that Codeberg.org could provide additional European-based and hosted artifact repository "caches" or "mirrors" of Maven Central, Pypi, NPM. I propose not to copy all the artifacts, but if a user of the Codeberg.org access through the repository "cache"/"mirror" a package it fetches it from Maven Central/Pypi/NPM in the background and keeps the version cached.

Note: I do not talk about the functionality in Codeberg to publish own artifacts in a registry this for sure should be used as well.

Of course one needs to decide which solution to deploy for this as to my knowledge this is not supported by Forgejo. Pulp can do this for some package types, but maybe there are others. Probably a European CDN needs to be chosen (but might not be needed initially).

One should also of course consider the risk, e.g. similar to what other infrastructure provider have as a risk, the Codeberg repository "cache"/"mirror" could be heavily used. Here it could be worth to check with:

• European Commission - They currently work on the Open Source strategy 2026-2030
• NLNet
• German Sovereign Tech Agency

### Comment Recently the stewards of popular open infrastructure, such as Maven Central, Pypi, NPM published [a letter on the lack of funding of their infrastructure which could endanger their operation](https://openssf.org/blog/2025/09/23/open-infrastructure-is-not-free-a-joint-statement-on-sustainable-stewardship/). Additionally, almost all of them are US-hosted and controlled. One could think that Codeberg.org could provide additional European-based and hosted artifact repository "caches" or "mirrors" of Maven Central, Pypi, NPM. I propose not to copy all the artifacts, but if a user of the Codeberg.org access through the repository "cache"/"mirror" a package it fetches it from Maven Central/Pypi/NPM in the background and keeps the version cached. Note: I do not talk about the functionality in Codeberg to publish own artifacts [in a registry](https://forgejo.org/docs/latest/user/packages/ ) this for sure should be used as well. Of course one needs to decide which solution to deploy for this as to my knowledge this is not supported by Forgejo. [Pulp](https://pulpproject.org/) can do this for some package types, but maybe there are others. Probably a European CDN needs to be chosen (but might not be needed initially). One should also of course consider the risk, e.g. similar to what other infrastructure provider have as a risk, the Codeberg repository "cache"/"mirror" could be heavily used. Here it could be worth to check with: * European Commission - They currently work on the [Open Source strategy](https://commission.europa.eu/about/departments-and-executive-agencies/digital-services/open-source-software-strategy_en) 2026-2030 * [NLNet](https://nlnet.nl/) * [German Sovereign Tech Agency](https://www.sovereign.tech/)

Gusted commented 2025年09月24日 22:49:10 +02:00

I'm not particularly sure this is in the scope of Codeberg and Codeberg's infrastructure is not set up in such a way to be used as a large-scale CDN. It would be hard to monitor that this mirror is actually used for free and open source development and not used by a (large) commercial entity that wants to avoid their responsibility of paying what they are using.

I'm not particularly sure this is in the scope of Codeberg and Codeberg's infrastructure is not set up in such a way to be used as a large-scale CDN. It would be hard to monitor that this mirror is actually used for free and open source development and not used by a (large) commercial entity that wants to avoid their responsibility of paying what they are using.

jornfranke commented 2025年09月25日 21:16:43 +02:00

Yes, I mentioned it as a possible risks. However, it is good to have your opinion on it.
Hence, I also added possible partners (e.g. European Commission, German Sovereign Tech Agency etc.). Maybe they can contribute to running it or run it with Codeberg.org contribution.

Yes, I mentioned it as a possible risks. However, it is good to have your opinion on it. Hence, I also added possible partners (e.g. European Commission, German Sovereign Tech Agency etc.). Maybe they can contribute to running it or run it with Codeberg.org contribution.

devnewton commented 2025年10月14日 18:30:06 +02:00

Artipie can be used for this.

This kind of service could be:

• free from job running inside codeberg ci ;
• charged for public access.

[Artipie](https://github.com/artipie/artipie) can be used for this. This kind of service could be: - free from job running inside codeberg ci ; - charged for public access.

jornfranke commented 2025年10月14日 19:56:45 +02:00

Thanks, good to know. Another alternative could be Pulp

Thanks, good to know. Another alternative could be [Pulp](https://pulpproject.org/)

jornfranke commented 2025年10月14日 20:07:10 +02:00

Btw. I like the idea that one can think about a charging model for public access. Maybe open source developers that have projects on Codeberg could get it for free for their local environment, but operating model is for sure something to look at.

Btw. I like the idea that one can think about a charging model for public access. Maybe open source developers that have projects on Codeberg could get it for free for their local environment, but operating model is for sure something to look at.

cstamas commented 2025年10月17日 20:59:33 +02:00

Latest Maven extension (obviously works for Maven only; usable in CI jobs for real caches not "storing local repository" thing) is Mimir https://maveniverse.eu/docs/mimir/

But I agree wholeheartedly, EU repositories are needed badly.

Latest Maven extension (obviously works for Maven only; usable in CI jobs for real caches not "storing local repository" thing) is Mimir https://maveniverse.eu/docs/mimir/ But I agree wholeheartedly, EU repositories are needed badly.