feat(dev): add migrate-repos tool to handle local > forgejo mass upload #9
migrate-repos-helper-script into main And obviously the CLI flags don't work once I push it.. sigh... will try to fix later on
@yoctozepto this should be ready when you are
21f8871503
to 9b2893b636
9b2893b636
to e0509a131e
e0509a131e
to 8b71aed54b
@ -1,3 +1,13 @@
{
This file should not be changed in this PR.
@ -1,9 +1,16 @@
services:
hound:
image:codeberg.org/codeberg-search/hound:latest
restart:unless-stopped
For dev, it is recommended to leave the default because we want to know if the service experiences an outage due to our actions.
Which is precisely what this will let us do - get to the logs of the service that stops - otherwise, to my knowledge, those logs can't be retrieved?
They can the same. You can get logs for stopped containers (services in here).
I will not change this honestly - don't see a big deal in keeping it and if the service keeps crashing, it will let us know on it's own.
Not quite, you might miss some failures unless you observe the process.
Fine, fixed
@ -0,0 +1 @@
migrate-repos
You can make this more specific by ignoring ./migrate-repos instead.
@ -0,0 +1,15 @@
# migrate-repos
`migrate-repos` is a small helper script that lets you pass in a folder and push that whole folder to a Gitea/Forgejo instance.
s/script/tool
@ -0,0 +3,4 @@
go 1.20
require (
code.gitea.io/sdk/gitea v0.16.0
Why not forgejo?
Either I am dumb or the sdk for forgejo isn't nowhere to be found?
No idea, asked on matrix.
It seems it does not exist.
@ -0,0 +32,4 @@
}
funclocalGitRepos(dirstring)(gitRepos[]string){
ifstrings.HasPrefix(dir,"~/")||strings.HasPrefix(dir,"$HOME/"){
This should have been handled for you via the shell and file-related modules and should not be necessary to handle yourself.
Care to give an example of that? AFAIK, from my internet search, this is the correct method (maybe one doesn't have to check for $HOME, but for ~/ we do have to, surely?)
https://www.gnu.org/software/bash/manual/html_node/Tilde-Expansion.html
We don't need to do this. I mean, many tools out there will fail when given a tilde that has not been processed by the shell.
Fixed, tested it and it works fine.
@ -0,0 +37,4 @@
dir=filepath.Join(dirname,dir[2:])
}
filepath.WalkDir(dir,func(pathstring,dfs.DirEntry,errerror)error{
This is going to be expensive - it would be easier and faster to assume a certain structure to this, e.g., that the tool is given a directory with org/repo structure. This way you can work on a single org a time (first try to create the org, then all the repos) and you don't need to filter for anything else.
doesn't feel expensive and gives quite a lot of flexibility - the bottlenecks of this tool definitely aren't here
@ -0,0 +60,4 @@
}
ifstrings.Contains(err.Error(),"org does not exist"){
orgOpt:=gitea.CreateOrgOption{Name:org}
_,_,err:=c.CreateOrg(orgOpt)
You don't seem to retry the repo creation in this case.
Good catch, done
@ -0,0 +98,4 @@
Password:token,
}
pushOptions:=git.PushOptions{RemoteName:remoteCfg.Config().Name,Auth:auth}
I wonder if we really need to. I mean, we can access the file storage directly - we can write to it oob. More so, we can already point gitea to the existing repos. In case of OpenDev, it's Gerrit that managed the repos and Gitea is only a frontend that shows them so there is no "pushing" to Gitea taking place (afaiu).
🤷 I am not against alternative/additional implementation options, however this will work against any git server endpoint and that seems worth keeping
@ -0,0 +163,4 @@
os.Exit(0)
}
funcinit(){
As mentioned on the chat, using cobra would make CLI parsing much nicer. Though this is good enough for this simple utility.
I looked into Cobra yesterday, but honestly couldn't figure it out. Once I discovered flag.VisitAll() it was quite smooth sailing even for validating that we have all the flags that we need.
Seemingly, .gitignore either did not work as expected or you somehow manually added the binary. Please have a look.
@yoctozepto I've just now reverted the .gitignore change - ./migrate-repos doesn't work, migrate-repos work.
@fourstepper Maybe I misremembered and it always looks from the root of the repo. It's good enough, thanks for reverting.
Many thanks.
Breaking change that won't be backward compatible
Something is not working
Documentation changes
Improve existing functionality
New functionality
This is security issue
Issue or pull request related to testing
The priority is critical
The priority is high
The priority is low
The priority is medium
Issue has been confirmed
This issue or pull request already exists
Invalid issue
This issue won't be fixed
Somebody has started to work on this but abandoned work
Something is blocking this issue or pull request
Feedback is required to reproduce issue or to continue work
No due date set.
No dependencies set.
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?