13
13
Fork
You've already forked code-search
0

feat(dev): add migrate-repos tool to handle local > forgejo mass upload #9

Merged
yoctozepto merged 8 commits from migrate-repos-helper-script into main 2023年09月17日 15:54:07 +02:00
No description provided.
Author
Owner
Copy link

And obviously the CLI flags don't work once I push it.. sigh... will try to fix later on

And obviously the CLI flags don't work once I push it.. _sigh_... will try to fix later on
fourstepper changed title from (削除) feat(dev): add migrate-repos script to handle local > forgejo mass upload (削除ここまで) to WIP: feat(dev): add migrate-repos script to handle local > forgejo mass upload 2023年09月13日 21:27:24 +02:00
fourstepper changed title from (削除) WIP: feat(dev): add migrate-repos script to handle local > forgejo mass upload (削除ここまで) to feat(dev): add migrate-repos script to handle local > forgejo mass upload 2023年09月15日 23:24:22 +02:00
Author
Owner
Copy link

@yoctozepto this should be ready when you are

@yoctozepto this should be ready when you are
fourstepper force-pushed migrate-repos-helper-script from 21f8871503 to 9b2893b636 2023年09月15日 23:24:48 +02:00 Compare
fourstepper force-pushed migrate-repos-helper-script from 9b2893b636 to e0509a131e 2023年09月16日 09:58:01 +02:00 Compare
fourstepper force-pushed migrate-repos-helper-script from e0509a131e to 8b71aed54b 2023年09月16日 10:17:48 +02:00 Compare
@ -1,3 +1,13 @@
{

This file should not be changed in this PR.

This file should not be changed in this PR.
fourstepper marked this conversation as resolved
@ -1,9 +1,16 @@
services:
hound:
image:codeberg.org/codeberg-search/hound:latest
restart:unless-stopped

For dev, it is recommended to leave the default because we want to know if the service experiences an outage due to our actions.

For dev, it is recommended to leave the default because we want to know if the service experiences an outage due to our actions.
Author
Owner
Copy link

Which is precisely what this will let us do - get to the logs of the service that stops - otherwise, to my knowledge, those logs can't be retrieved?

Which is precisely what this will let us do - get to the logs of the service that stops - otherwise, to my knowledge, those logs can't be retrieved?

They can the same. You can get logs for stopped containers (services in here).

They can the same. You can get logs for stopped containers (services in here).
Author
Owner
Copy link

I will not change this honestly - don't see a big deal in keeping it and if the service keeps crashing, it will let us know on it's own.

I will not change this honestly - don't see a big deal in keeping it and if the service keeps crashing, it will let us know on it's own.

Not quite, you might miss some failures unless you observe the process.

Not quite, you might miss some failures unless you observe the process.
Author
Owner
Copy link

Fine, fixed

Fine, fixed
fourstepper marked this conversation as resolved
@ -0,0 +1 @@
migrate-repos

You can make this more specific by ignoring ./migrate-repos instead.

You can make this more specific by ignoring `./migrate-repos` instead.
fourstepper marked this conversation as resolved
@ -0,0 +1,15 @@
# migrate-repos
`migrate-repos` is a small helper script that lets you pass in a folder and push that whole folder to a Gitea/Forgejo instance.

s/script/tool

s/script/tool
fourstepper marked this conversation as resolved
@ -0,0 +3,4 @@
go 1.20
require (
code.gitea.io/sdk/gitea v0.16.0

Why not forgejo?

Why not forgejo?
Author
Owner
Copy link

Either I am dumb or the sdk for forgejo isn't nowhere to be found?

Either I am dumb or the sdk for forgejo isn't nowhere to be found?

No idea, asked on matrix.

No idea, asked on matrix.

It seems it does not exist.

It seems it does not exist.
yoctozepto marked this conversation as resolved
@ -0,0 +32,4 @@
}
funclocalGitRepos(dirstring)(gitRepos[]string){
ifstrings.HasPrefix(dir,"~/")||strings.HasPrefix(dir,"$HOME/"){

This should have been handled for you via the shell and file-related modules and should not be necessary to handle yourself.

This should have been handled for you via the shell and file-related modules and should not be necessary to handle yourself.
Author
Owner
Copy link

Care to give an example of that? AFAIK, from my internet search, this is the correct method (maybe one doesn't have to check for $HOME, but for ~/ we do have to, surely?)

Care to give an example of that? AFAIK, from my internet search, this is the correct method (maybe one doesn't have to check for $HOME, but for ~/ we do have to, surely?)

https://www.gnu.org/software/bash/manual/html_node/Tilde-Expansion.html

We don't need to do this. I mean, many tools out there will fail when given a tilde that has not been processed by the shell.

https://www.gnu.org/software/bash/manual/html_node/Tilde-Expansion.html We don't need to do this. I mean, many tools out there will fail when given a tilde that has not been processed by the shell.
Author
Owner
Copy link

Fixed, tested it and it works fine.

Fixed, tested it and it works fine.
fourstepper marked this conversation as resolved
@ -0,0 +37,4 @@
dir=filepath.Join(dirname,dir[2:])
}
filepath.WalkDir(dir,func(pathstring,dfs.DirEntry,errerror)error{

This is going to be expensive - it would be easier and faster to assume a certain structure to this, e.g., that the tool is given a directory with org/repo structure. This way you can work on a single org a time (first try to create the org, then all the repos) and you don't need to filter for anything else.

This is going to be expensive - it would be easier and faster to assume a certain structure to this, e.g., that the tool is given a directory with `org/repo` structure. This way you can work on a single org a time (first try to create the org, then all the repos) and you don't need to filter for anything else.
Author
Owner
Copy link

doesn't feel expensive and gives quite a lot of flexibility - the bottlenecks of this tool definitely aren't here

doesn't feel expensive and gives quite a lot of flexibility - the bottlenecks of this tool definitely aren't here
fourstepper marked this conversation as resolved
@ -0,0 +60,4 @@
}
ifstrings.Contains(err.Error(),"org does not exist"){
orgOpt:=gitea.CreateOrgOption{Name:org}
_,_,err:=c.CreateOrg(orgOpt)

You don't seem to retry the repo creation in this case.

You don't seem to retry the repo creation in this case.
Author
Owner
Copy link

Good catch, done

Good catch, done
fourstepper marked this conversation as resolved
@ -0,0 +98,4 @@
Password:token,
}
pushOptions:=git.PushOptions{RemoteName:remoteCfg.Config().Name,Auth:auth}

I wonder if we really need to. I mean, we can access the file storage directly - we can write to it oob. More so, we can already point gitea to the existing repos. In case of OpenDev, it's Gerrit that managed the repos and Gitea is only a frontend that shows them so there is no "pushing" to Gitea taking place (afaiu).

I wonder if we really need to. I mean, we can access the file storage directly - we can write to it oob. More so, we can already point gitea to the existing repos. In case of OpenDev, it's Gerrit that managed the repos and Gitea is only a frontend that shows them so there is no "pushing" to Gitea taking place (afaiu).
Author
Owner
Copy link

🤷 I am not against alternative/additional implementation options, however this will work against any git server endpoint and that seems worth keeping

🤷 I am not against alternative/additional implementation options, however this will work against any git server endpoint and that seems worth keeping
fourstepper marked this conversation as resolved
@ -0,0 +163,4 @@
os.Exit(0)
}
funcinit(){

As mentioned on the chat, using cobra would make CLI parsing much nicer. Though this is good enough for this simple utility.

As mentioned on the chat, using cobra would make CLI parsing much nicer. Though this is good enough for this simple utility.
Author
Owner
Copy link

I looked into Cobra yesterday, but honestly couldn't figure it out. Once I discovered flag.VisitAll() it was quite smooth sailing even for validating that we have all the flags that we need.

I looked into Cobra yesterday, but honestly couldn't figure it out. Once I discovered `flag.VisitAll()` it was quite smooth sailing even for validating that we have all the flags that we need.
fourstepper marked this conversation as resolved

Seemingly, .gitignore either did not work as expected or you somehow manually added the binary. Please have a look.

Seemingly, .gitignore either did not work as expected or you somehow manually added the binary. Please have a look.
Author
Owner
Copy link

@yoctozepto I've just now reverted the .gitignore change - ./migrate-repos doesn't work, migrate-repos work.

@yoctozepto I've just now reverted the .gitignore change - ./migrate-repos doesn't work, migrate-repos work.

@fourstepper Maybe I misremembered and it always looks from the root of the repo. It's good enough, thanks for reverting.

@fourstepper Maybe I misremembered and it always looks from the root of the repo. It's good enough, thanks for reverting.
yoctozepto changed title from (削除) feat(dev): add migrate-repos script to handle local > forgejo mass upload (削除ここまで) to feat(dev): add migrate-repos tool to handle local > forgejo mass upload 2023年09月17日 15:52:25 +02:00

Many thanks.

Many thanks.
yoctozepto deleted branch migrate-repos-helper-script 2023年09月17日 15:54:07 +02:00
Sign in to join this conversation.
No reviewers
Labels
Clear labels
Kind/Breaking

Breaking change that won't be backward compatible
Kind/Bug

Something is not working
Kind/Documentation

Documentation changes
Kind/Enhancement

Improve existing functionality
Kind/Feature

New functionality
Kind/Security

This is security issue
Kind/Testing

Issue or pull request related to testing
Priority
Critical

The priority is critical
Priority
High

The priority is high
Priority
Low

The priority is low
Priority
Medium

The priority is medium
Reviewed
Confirmed

Issue has been confirmed
Reviewed
Duplicate

This issue or pull request already exists
Reviewed
Invalid

Invalid issue
Reviewed
Won't Fix

This issue won't be fixed
Status
Abandoned

Somebody has started to work on this but abandoned work
Status
Blocked

Something is blocking this issue or pull request
Status
Need More Info

Feedback is required to reproduce issue or to continue work
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
2 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
Codeberg-Infrastructure/code-search!9
Reference in a new issue
Codeberg-Infrastructure/code-search
No description provided.
Delete branch "migrate-repos-helper-script"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?