Codeberg/Community
60
387
Fork
You've already forked Community
12

Various Forgejo API issues and peculiarities #1601

Open
opened 2024年07月03日 18:15:21 +02:00 by mkorje · 1 comment

Comment

Hi,

I've been writing some Python code to interact with the Forgejo API, and have found some things about the API that don't seem quite right. I have made a repository on next.forgejo.org - 8.0.0-dev-1520-c290a40063+gitea-1.22.0 - (and I am thus using the API docs at https://next.forgejo.org/api/swagger) to further test what I've found and document it.

I didn't want to clog up the forgejo repo's issues with this, and I wasn't sure where exactly to post this. So I decided to post it here in the community repo.

Repository milestones

A repository can have multiple milestones with the same title.

Getting all of a repository's opened milestones

The request

 curl -X 'GET' \
 'https://next.forgejo.org/api/v1/repos/mkorje/test/milestones' \
 -H 'accept: application/json'

returns the response body

 [
 {
 "id": 20,
 "title": "New Milestone 41.45",
 "description": "",
 "state": "open",
 "open_issues": 0,
 "closed_issues": 0,
 "created_at": "2024年07月03日T12:57:56Z",
 "updated_at": "2024年07月03日T12:57:56Z",
 "closed_at": null,
 "due_on": null
 },
 {
 "id": 21,
 "title": "New Milestone 41.45",
 "description": "",
 "state": "open",
 "open_issues": 0,
 "closed_issues": 0,
 "created_at": "2024年07月03日T12:58:05Z",
 "updated_at": "2024年07月03日T12:58:05Z",
 "closed_at": null,
 "due_on": null
 }
 ]

Allowing this seems fairly bizarre, and makes querying them with the API odd as well. As per the API, you can query a milestone by id or by title.

Getting a specific milestone

The request

 curl -X 'GET' \
 'https://next.forgejo.org/api/v1/repos/mkorje/test/milestones/New%20Milestone%2041.45' \
 -H 'accept: application/json'

returns the response body

 {
 "id": 20,
 "title": "New Milestone 41.45",
 "description": "",
 "state": "open",
 "open_issues": 0,
 "closed_issues": 0,
 "created_at": "2024年07月03日T12:57:56Z",
 "updated_at": "2024年07月03日T12:57:56Z",
 "closed_at": null,
 "due_on": null
 }

This only returns one of the milestones, the one with the lower id, and makes the existence of the other milestone difficult to ascertain (I would have to parse through the entire list of milestones in a repository and check whether each has a title that matches, so I can then get its id). Further, I find it difficult to see any situation where one would want or need two milestones that have identical titles; on the web UI, the two milestones are practically indistinguishable.

It seems that milestones are stored primarily by id, and so the title is irrelevant. I believe it would make sense to enforce that the title of a milestone is unique, and so is checked when creating a milestone. This would resolve the issue of searching for milestones by name with the API described above. Perhaps it would then also make sense to remove the ability to get/delete/update a milestone by its title, so that only the id can be used. I understand that it wouldn't make sense to forgo the id entirely as the title can contain arbitrary (?) text, which would make URLs a pain (going from https://next.forgejo.org/mkorje/test/milestone/20 to https://next.forgejo.org/mkorje/test/milestone/New%20Milestone%2041.45 would definitely suck), and I'm sure other things like sorting etc.

Repository labels

A repository can have multiple labels with the same name. This is basically the same as with the milestones.

Getting all of a repository's labels

The request

 curl -X 'GET' \
 'https://next.forgejo.org/api/v1/repos/mkorje/test/labels' \
 -H 'accept: application/json'

returns the response body

 [
 {
 "id": 1310,
 "name": "label a /a",
 "exclusive": false,
 "is_archived": false,
 "color": "70c24a",
 "description": "",
 "url": "https://next.forgejo.org/api/v1/repos/mkorje/test/labels/1310"
 },
 {
 "id": 1311,
 "name": "label a /a",
 "exclusive": false,
 "is_archived": false,
 "color": "70c24a",
 "description": "",
 "url": "https://next.forgejo.org/api/v1/repos/mkorje/test/labels/1311"
 }
 ]

And while not documented at https://next.forgejo.org/api/swagger, you can in fact search for labels by their name, not just by their id which the documentation suggests. A request to get/delete/update a specific label using the API requires the id of the label according to the documentation, and has type integer($int64).

Getting a specific label

The request

 curl -X 'GET' \
 'https://next.forgejo.org/api/v1/repos/mkorje/test/labels/label%20a%20%2Fa' \
 -H 'accept: application/json'

returns the response body

 {
 "id": 1310,
 "name": "label a /a",
 "exclusive": false,
 "is_archived": false,
 "color": "70c24a",
 "description": "",
 "url": "https://next.forgejo.org/api/v1/repos/mkorje/test/labels/1310"
 }

As with the milestones, this only returns one of the labels, the one with the lower id. Again, I find it difficult to see any situation where one would want or need two labels that have identical names; on the web UI, the two labels are practically indistinguishable (yes you could change their colour, but that only adds to bizarreness imo). So I think it would make sense to enforce that the name of a label is unique, and so is checked when creating a label. Perhaps it would then also make sense to remove the ability to get/delete/update a label by its name, so that only the id can be used (that is, functions how it should according to the API documentation currently). And then replace this by providing an interface to search for a label, like GET /repos/issues/search.

Weird permissions

This is issue #1713. Without a token on a public instance, you can make some GET requests in each of the categories with a successful response 200. For example (if the specific org/repo is public), GET /orgs/{org} and GET /orgs in organisation, GET /repos/{owner}/{repo} and GET /repos/search in repository. However, when using a scoped access token without read access to one of these categories, the API returns a 403 error with message:

 token does not have at least one of required scope(s): ...

This doesn't make much sense, as authorisation isn't required to access these endpoints!

The above is also closely related to issue #3988. The read:user scope is required to get information on the current user from the token, which is far too broad.

General observations

I've noticed that the API does not distinguish between an endpoint being valid and the resource itself not existing, and the endpoint just not existing. Both return a 404 page not found error.

I've also found it annoying trying to figure out what permissions an access token has. Presumably GET /users/{username}/tokens in user would be what I need, as it includes a token's scope as one of its key. However, these endpoints are only accessible using username/password authentication. And even if they were available, you would need the read:user scope on the token to access it.

In some parts of the documentation, the json object body parameter signals that some keys are required when making the request. But this is inconsistent. Further, there are some keys, like is_archived for labels, that clearly have a default value of false. But there is no indication of this in the documentation.

Creating a label

The request

 curl -X 'POST' \
 'https://next.forgejo.org/api/v1/repos/mkorje/test/labels' \
 -H 'accept: application/json' \
 -H 'Authorization: token <omitted>' \
 -H 'Content-Type: application/json' \
 -d '{
 "color": "#00aabb",
 "name": "string"
 }'

returns the response body

 {
 "id": 1312,
 "name": "string",
 "exclusive": false,
 "is_archived": false,
 "color": "00aabb",
 "description": "",
 "url": "https://next.forgejo.org/api/v1/repos/mkorje/test/labels/1312"
 }

In the above example, the colour is stored without the hashtag by the API. There's probably a reason for this, but I'd expect it to be consistent with the example suggested in the documentation of example: #00aabb. However, running the request again, but removing the hashtag from the colour in the request, also works. So it seems perhaps that this is just the documentation being a bit unclear, when the backend in fact deals with both of these situations. On the web UI, when editing the colour of a label, it sends the PATCH request to update the label with the hashtag in the colour. So I feel as though the backend should also share this consistency and store the colour with a hashtag.

The end

I will probably edit this to add more things (examples, details, etc.) as I continue to mess about with the API. I've also most definitely made statements that are entirely wrong, commented on things that are intended to be the way they are, or have completely misinterpreted how others expect these things to work. Please do reply if you agree, disagree, or have other comments to add.

I am interested in getting involved and contributing, possibly going about fixing things like this related to the API. Though my knowledge of Go and the Forgejo codebase is sparse, and so it may take a while as I learn and figure things out.

Here is a list of all the API related open issues I could find in the forgejo/forgejo repo
### Comment Hi, I've been writing some Python code to interact with the Forgejo API, and have found some things about the API that don't seem quite right. I have made a repository on [next.forgejo.org](https://next.forgejo.org/mkorje/test) - 8.0.0-dev-1520-c290a40063+gitea-1.22.0 - (and I am thus using the API docs at <https://next.forgejo.org/api/swagger>) to further test what I've found and document it. I didn't want to clog up the forgejo repo's issues with this, and I wasn't sure where exactly to post this. So I decided to post it here in the community repo. ## Repository milestones A repository can have multiple milestones with the same title. <details> <summary>Getting all of a repository's opened milestones</summary> The request ```shell curl -X 'GET' \ 'https://next.forgejo.org/api/v1/repos/mkorje/test/milestones' \ -H 'accept: application/json' ``` returns the response body ```json [ { "id": 20, "title": "New Milestone 41.45", "description": "", "state": "open", "open_issues": 0, "closed_issues": 0, "created_at": "2024年07月03日T12:57:56Z", "updated_at": "2024年07月03日T12:57:56Z", "closed_at": null, "due_on": null }, { "id": 21, "title": "New Milestone 41.45", "description": "", "state": "open", "open_issues": 0, "closed_issues": 0, "created_at": "2024年07月03日T12:58:05Z", "updated_at": "2024年07月03日T12:58:05Z", "closed_at": null, "due_on": null } ] ``` </details> Allowing this seems fairly bizarre, and makes querying them with the API odd as well. As per the API, you can query a milestone by `id` or by `title`. <details> <summary>Getting a specific milestone</summary> The request ```shell curl -X 'GET' \ 'https://next.forgejo.org/api/v1/repos/mkorje/test/milestones/New%20Milestone%2041.45' \ -H 'accept: application/json' ``` returns the response body ```json { "id": 20, "title": "New Milestone 41.45", "description": "", "state": "open", "open_issues": 0, "closed_issues": 0, "created_at": "2024年07月03日T12:57:56Z", "updated_at": "2024年07月03日T12:57:56Z", "closed_at": null, "due_on": null } ``` </details> This only returns one of the milestones, the one with the lower `id`, and makes the existence of the other milestone difficult to ascertain (I would have to parse through the entire list of milestones in a repository and check whether each has a `title` that matches, so I can then get its `id`). Further, I find it difficult to see any situation where one would want or need two milestones that have identical `title`s; on the [web UI](https://next.forgejo.org/mkorje/test/milestones), the two milestones are practically indistinguishable. It seems that milestones are stored primarily by `id`, and so the `title` is irrelevant. I believe it would make sense to enforce that the title of a milestone is unique, and so is checked when creating a milestone. This would resolve the issue of searching for milestones by name with the API described above. Perhaps it would then also make sense to remove the ability to get/delete/update a milestone by its `title`, so that only the `id` can be used. I understand that it wouldn't make sense to forgo the `id` entirely as the `title` can contain arbitrary (?) text, which would make URLs a pain (going from <https://next.forgejo.org/mkorje/test/milestone/20> to <https://next.forgejo.org/mkorje/test/milestone/New%20Milestone%2041.45> would definitely suck), and I'm sure other things like sorting etc. ## Repository labels A repository can have multiple labels with the same `name`. This is basically the same as with the milestones. <details> <summary>Getting all of a repository's labels</summary> The request ```shell curl -X 'GET' \ 'https://next.forgejo.org/api/v1/repos/mkorje/test/labels' \ -H 'accept: application/json' ``` returns the response body ```json [ { "id": 1310, "name": "label a /a", "exclusive": false, "is_archived": false, "color": "70c24a", "description": "", "url": "https://next.forgejo.org/api/v1/repos/mkorje/test/labels/1310" }, { "id": 1311, "name": "label a /a", "exclusive": false, "is_archived": false, "color": "70c24a", "description": "", "url": "https://next.forgejo.org/api/v1/repos/mkorje/test/labels/1311" } ] ``` </details> And while not documented at <https://next.forgejo.org/api/swagger>, you can in fact search for labels by their `name`, not just by their `id` which the documentation suggests. A request to get/delete/update a specific label using the API requires the `id` of the label according to the documentation, and has type `integer($int64)`. <details> <summary>Getting a specific label</summary> The request ```shell curl -X 'GET' \ 'https://next.forgejo.org/api/v1/repos/mkorje/test/labels/label%20a%20%2Fa' \ -H 'accept: application/json' ``` returns the response body ```json { "id": 1310, "name": "label a /a", "exclusive": false, "is_archived": false, "color": "70c24a", "description": "", "url": "https://next.forgejo.org/api/v1/repos/mkorje/test/labels/1310" } ``` </details> As with the milestones, this only returns one of the labels, the one with the lower `id`. Again, I find it difficult to see any situation where one would want or need two labels that have identical `names`; on the [web UI](https://next.forgejo.org/mkorje/test/labels), the two labels are practically indistinguishable (yes you could change their colour, but that only adds to bizarreness imo). So I think it would make sense to enforce that the `name` of a label is unique, and so is checked when creating a label. Perhaps it would then also make sense to remove the ability to get/delete/update a label by its `name`, so that only the `id` can be used (that is, functions how it should according to the API documentation currently). And then replace this by providing an interface to search for a label, like `GET /repos/issues/search`. ## Weird permissions This is issue [#1713](https://codeberg.org/forgejo/forgejo/issues/1713). Without a token on a public instance, you can make _some_ GET requests in each of the categories with a successful response 200. For example (if the specific org/repo is public), `GET /orgs/{org}` and `GET /orgs` in organisation, `GET /repos/{owner}/{repo}` and `GET /repos/search` in repository. However, when using a scoped access token without read access to one of these categories, the API returns a 403 error with message: ``` token does not have at least one of required scope(s): ... ``` This doesn't make much sense, as authorisation isn't required to access these endpoints! The above is also closely related to issue [#3988](https://codeberg.org/forgejo/forgejo/issues/3988). The `read:user` scope is required to get information on the current user from the token, which is far too broad. ## General observations I've noticed that the API does not distinguish between an endpoint being valid and the resource itself not existing, and the endpoint just not existing. Both return a `404 page not found` error. I've also found it annoying trying to figure out what permissions an access token has. Presumably `GET /users/{username}/tokens` in user would be what I need, as it includes a token's scope as one of its key. However, these endpoints are only accessible using username/password authentication. And even if they were available, you would need the `read:user` scope on the token to access it. In some parts of the documentation, the json object body parameter signals that some keys are required when making the request. But this is inconsistent. Further, there are some keys, like `is_archived` for labels, that clearly have a default value of `false`. But there is no indication of this in the documentation. <details> <summary>Creating a label</summary> The request ```shell curl -X 'POST' \ 'https://next.forgejo.org/api/v1/repos/mkorje/test/labels' \ -H 'accept: application/json' \ -H 'Authorization: token <omitted>' \ -H 'Content-Type: application/json' \ -d '{ "color": "#00aabb", "name": "string" }' ``` returns the response body ```json { "id": 1312, "name": "string", "exclusive": false, "is_archived": false, "color": "00aabb", "description": "", "url": "https://next.forgejo.org/api/v1/repos/mkorje/test/labels/1312" } ``` </details> In the above example, the colour is stored without the hashtag by the API. There's probably a reason for this, but I'd expect it to be consistent with the example suggested in the documentation of `example: #00aabb`. However, running the request again, but removing the hashtag from the colour in the request, also works. So it seems perhaps that this is just the documentation being a bit unclear, when the backend in fact deals with both of these situations. On the web UI, when editing the colour of a label, it sends the PATCH request to update the label _with_ the hashtag in the colour. So I feel as though the backend should also share this consistency and store the colour with a hashtag. ## The end I will probably edit this to add more things (examples, details, etc.) as I continue to mess about with the API. I've also most definitely made statements that are entirely wrong, commented on things that are intended to be the way they are, or have completely misinterpreted how others expect these things to work. Please do reply if you agree, disagree, or have other comments to add. I am interested in getting involved and contributing, possibly going about fixing things like this related to the API. Though my knowledge of Go and the Forgejo codebase is sparse, and so it may take a while as I learn and figure things out. <details> <summary> Here is a list of all the API related open issues I could find in the [forgejo/forgejo](https://codeberg.org/forgejo/forgejo) repo </summary> - <https://codeberg.org/forgejo/forgejo/issues/4190> - <https://codeberg.org/forgejo/forgejo/issues/4173> - <https://codeberg.org/forgejo/forgejo/issues/4167> - <https://codeberg.org/forgejo/forgejo/issues/4133> - <https://codeberg.org/forgejo/forgejo/issues/4126> - <https://codeberg.org/forgejo/forgejo/issues/4108> - <https://codeberg.org/forgejo/forgejo/issues/4025> - <https://codeberg.org/forgejo/forgejo/issues/4005> - <https://codeberg.org/forgejo/forgejo/issues/3988> - <https://codeberg.org/forgejo/forgejo/issues/3699> - <https://codeberg.org/forgejo/forgejo/issues/3643> - <https://codeberg.org/forgejo/forgejo/issues/3624> - <https://codeberg.org/forgejo/forgejo/issues/3577> - <https://codeberg.org/forgejo/forgejo/issues/3459> - <https://codeberg.org/forgejo/forgejo/issues/3395> - <https://codeberg.org/forgejo/forgejo/issues/2692> - <https://codeberg.org/forgejo/forgejo/issues/2516> - <https://codeberg.org/forgejo/forgejo/issues/2426> - <https://codeberg.org/forgejo/forgejo/issues/2415> - <https://codeberg.org/forgejo/forgejo/issues/2424> - <https://codeberg.org/forgejo/forgejo/issues/2336> - <https://codeberg.org/forgejo/forgejo/issues/2332> - <https://codeberg.org/forgejo/forgejo/issues/2202> - <https://codeberg.org/forgejo/forgejo/issues/2125> - <https://codeberg.org/forgejo/forgejo/issues/2109> - <https://codeberg.org/forgejo/forgejo/issues/1998> - <https://codeberg.org/forgejo/forgejo/issues/1887> - <https://codeberg.org/forgejo/forgejo/issues/1867> - <https://codeberg.org/forgejo/forgejo/issues/1713> - <https://codeberg.org/forgejo/forgejo/issues/1712> - <https://codeberg.org/forgejo/forgejo/issues/1603> - <https://codeberg.org/forgejo/forgejo/issues/1107> - <https://codeberg.org/forgejo/forgejo/issues/1030> - <https://codeberg.org/forgejo/forgejo/issues/933> - <https://codeberg.org/forgejo/forgejo/issues/813> - <https://codeberg.org/forgejo/forgejo/issues/506> - <https://codeberg.org/forgejo/forgejo/issues/412> </details>
Owner
Copy link

Thank you very much for your detailed feedback. It might take a while to evaluate and address it, but it is much appreciated.

Thank you very much for your detailed feedback. It might take a while to evaluate and address it, but it is much appreciated.
Sign in to join this conversation.
No Branch/Tag specified
main
No results found.
Labels
Clear labels
accessibility
Reduces accessibility and is thus a "bug" for certain user groups on Codeberg.
bug
Something is not working the way it should. Does not concern outages.
bug
infrastructure
 Errors evidently caused by infrastructure malfunctions or outages
Codeberg
This issue involves Codeberg's downstream modifications and settings and/or Codeberg's structures.
contributions welcome
Please join the discussion and consider contributing a PR!
docs
No bug, but an improvement to the docs or UI description will help
duplicate
This issue or pull request already exists
enhancement
New feature
infrastructure
Involves changes to the server setups, use `bug/infrastructure` for infrastructure-related user errors.
legal
An issue directly involving legal compliance
licence / ToS
involving questions about the ToS, especially licencing compliance
please chill
we are volunteers
 Please consider editing your posts and remember that there is a human on the other side. We get that you are frustrated, but it's harder for us to help you this way.
public relations
Things related to Codeberg's external communication
question
More information is needed
question
user support
 This issue contains a clearly stated problem. However, it is not clear whether we have to fix anything on Codeberg's end, but we're helping them fix it and/or find the cause.
s/Forgejo
Related to Forgejo. Please also check Forgejo's issue tracker.
s/Forgejo/migration
Migration related issues in Forgejo
s/Pages
Issues related to the Codeberg Pages feature
s/Weblate
Issue is related to the Weblate instance at https://translate.codeberg.org
s/Woodpecker
Woodpecker CI related issue
security
involves improvements to the sites security
service
Add a new service to the Codeberg ecosystem (instead of implementing into Forgejo)
upstream
An open issue or pull request to an upstream repository to fix this issue (partially or completely) exists (i.e. Forgejo, Weblate, etc.)
wontfix
Codeberg's current set of contributors are not planning to spend time on delegating this issue.
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
2 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
Codeberg/Community#1601
Reference in a new issue
Codeberg/Community
No description provided.
Delete branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?