sacd is a huge repository (10GiB) of all of my scratch projects and metadata that goes with them, which i have locally. For the projects I want public, I would like to have them here and have some kind of CI job take care of mirroring them to scratch.
Now it would be insane to just copy paste them here. Git isn't made for blobs, and there would be so much asset duplication. I will need to unzip the sb3s. That way we can work with the assets and code separately. Plus, it's unnecessarily huge and codeberg would not like that
Note
I tried doing this with a python script but I did it in a pretty stupid way so it was unreasonably slow.
What I think would be a good idea:
- sort the sb3 files together. note we will need the metadata later
- how about sprite3 files? will i make them into sb3s manually? how crude. i will consider those later
- write it in rust for more speed - even if we rely on
gitfor git operations, unzipping is still expensive. While python might leverage a C api for this, there may be memory considerations and stuff (i'm not 100% convinced on this but there may be other performance concerns) - unzip the file
- all the assets of all the sb3s will be shared in a directory.
- the
project.jsonfiles will be renamed to{x}.jsonwhere{x}is the identifier of the project. this could be the name or the project id, i really dont care - there will be a giant toml config file which will contain all of the metadata about each project - the title, the instructions/notes, whether for it to be shared (default true).
- a lockfile should be given so that we dont build project we do not need to. this would probably be dependent on a git commit hash so that we can compare to that in the repo. because we cant request the metadata from scratch itself. im not 100% sure on this
- i will need a builder and deconstructor
- i will need some kind of way to automatically upload this to scratch, with the aforementioned lockfile in mind
sacd is a huge repository (10GiB) of all of my scratch projects and metadata that goes with them, which i have locally. For the projects I want public, I would like to have them here and have some kind of CI job take care of mirroring them to scratch.
Now it would be insane to just copy paste them here. Git isn't made for blobs, and there would be so much asset duplication. I will need to unzip the sb3s. That way we can work with the assets and code separately. Plus, it's unnecessarily huge and codeberg would not like that
> [!NOTE]
> I tried doing this with a [python script](https://codeberg.org/scratch-api/sb2git) but I did it in a pretty stupid way so it was unreasonably slow.
What I think would be a good idea:
- [ ] sort the sb3 files together. note we will need the metadata later
- [ ] how about sprite3 files? will i make them into sb3s manually? how crude. i will consider those later
- [ ] write it in rust for more speed - even if we rely on `git` for git operations, unzipping is still expensive. While python might leverage a C api for this, there may be memory considerations and stuff (i'm not 100% convinced on this but there may be other performance concerns)
- [ ] unzip the file
- [ ] all the assets of all the sb3s will be shared in a directory.
- [ ] the `project.json` files will be renamed to `{x}.json` where `{x}` is the identifier of the project. this could be the name or the project id, i really dont care
- [ ] there will be a giant toml config file which will contain all of the metadata about each project - the title, the instructions/notes, whether for it to be shared (default true).
- [ ] a lockfile should be given so that we dont build project we do not need to. this would probably be dependent on a git commit hash so that we can compare to that in the repo. because we cant request the metadata from scratch itself. im not 100% sure on this
- [ ] i will need a builder and deconstructor
- [ ] i will need some kind of way to automatically upload this to scratch, with the aforementioned lockfile in mind