forgejo/discussions
49
43

Scaling out Forgejo release notes - postmortem #197

Open
opened 2024年07月24日 09:02:40 +02:00 by earl-warren · 7 comments

Forgejo release notes do not fit the popular trend that consists on enforcing conventional commits so that tools can collect them because:

  • The unit of change in Forgejo is the pull request, not the commit. It is not uncommon for a pull request to contain multiple commits that are related but independent from each other. Squashing them into a single commit is not an option. There are diverging opinions on the matter and enforcing a merge policy where pull requests are always squashed into a single commit leads to a nicely flat commit history. But Forgejo made a different choice.
  • A single commit or pull request may require multiple release note lines. When upgrading a dependency by modifying one line in the go.mod file leads to user facing changes, each of them needs to be described in the release notes. Forgejo relies on hundreds of other Free Software projects and although most of them are plumbing that the Forgejo user do not need to be aware of, others are very visible such as the web editor.
  • Release notes need manual editing. No matter how good and disciplined the author of a pull request is, there will be typos and unification problems that are best addressed when preparing the release, at the very last stage.

Discussions started early 2024 (here and there) to distribute the release notes workload to the author of each pull request rather than relying on a single person manually doing the same a few days before the release. As the number of contributors to Forgejo increased significantly early 2024, the quality of the v7.0.0 release notes was sub-standard and a solution had to be found.

The nuclear option would have been to enforce a merge policy that was a fit for the existing tooling. But even that would have not solved the case where a dependency upgrade needed multiple lines in the release notes so it was never considered to be a possibility.

With over a year of practice in publishing Forgejo releases and the associated release notes manually, the method to do that was well defined and stable. It was never really documented but looking at the discussions and pull requests it was not difficult to reverse engineer the workflow. There were some blurry technical points that were never solved because a human brain took a different path, such as figuring out the backport of a pull request based on the commits it contains. But the workflow itself was precise and it was not based on speculations, it was deduced from the making of over twenty releases.

This is the ideal situation when implementing a software: there is plenty of evidence that a human being did a number of things manually. The software just needs to do the same and automate as much as possible. There is no risk of over-engineering, as long as the developer does not get carried away and keeps it simple, stupid.

The release notes assistant was developed in that context, for the benefit of three type of users:

  • The author of a pull request, to show them how their pull request will look in the final release notes so they can fine tune it
  • The reviewer of a pull request, to ask for changes based on the result rather than the source
  • The release manager, to collect them and allow manual editing in a single file before the release

A number of tools and actions in Forgejo, including some rather advanced are written in shell. But the complexity of the release notes assistant was above the threshold that would allow for long term maintenance. And the reasons are mainly:

  • the logic to figure out which pull request is backported from which
  • stable branches and how their history evolve in parallel with the development branch

It was decided to implement it in Go instead and has 90%+ test coverage. But the test environment which spawns a Forgejo instance with fixtures that mimic the structure of the Forgejo repository is derived from the existing end-to-end test framework and in shell.

The workflow added in the Forgejo repository is all the author and reviewers need to agree on a good enough release note.

The release manager still needs to use the tool manually and can, for instance, direct it to update the milestone, which was done for the Forgejo v7.0.6 release. And once the release is published, it could be added to the description of the release itself. Which would indirectly solve the rather annoying burden of maintaining a file in the repository with the same information.

In conclusion, although the release-notes-assistant was written to be generic and has nothing Forgejo specific, there is little chance it is actually used anywhere else. People won't discover it by chance and pushing it in this direction is not something anyone is ready to do at this point. Having a robust tool for publishing Forgejo and related projects is great. Maintaining a tool used by a variety of users with a whole range of needs is an entirely different proposition.

Note: Release notes lines written by the author of the pull request are good enough for point releases that need no explanation. But they should come with a companion blog post to provide context for major releases

Forgejo release notes do not fit the popular trend that consists on enforcing [conventional commits](https://www.conventionalcommits.org/) so that [tools](https://github.com/orhun/git-cliff) can collect them because: * **The unit of change in Forgejo is the pull request, not the commit.** It is not uncommon for a pull request to contain multiple commits that are related but independent from each other. Squashing them into a single commit is not an option. There are diverging opinions on the matter and enforcing a merge policy where pull requests are always squashed into a single commit leads to a nicely flat commit history. But Forgejo made a different choice. * **A single commit or pull request may require multiple release note lines.** When upgrading a dependency by modifying one line in the `go.mod` file leads to user facing changes, each of them needs to be described in the release notes. Forgejo relies on hundreds of other Free Software projects and although most of them are plumbing that the Forgejo user do not need to be aware of, others are very visible such as the web editor. * **Release notes need manual editing.** No matter how good and disciplined the author of a pull request is, there will be typos and unification problems that are best addressed when preparing the release, at the very last stage. Discussions started early 2024 ([here](https://codeberg.org/forgejo/discussions/issues/159) and [there](https://codeberg.org/forgejo/discussions/issues/155)) to distribute the release notes workload to the author of each pull request rather than relying on a single person manually doing the same a few days before the release. As the number of contributors to Forgejo increased significantly early 2024, the quality of the [v7.0.0 release notes](https://codeberg.org/forgejo/forgejo/src/branch/forgejo/RELEASE-NOTES.md#7-0-0) was sub-standard and a solution had to be found. The nuclear option would have been to enforce a merge policy that was a fit for the existing tooling. But even that would have not solved the case where a dependency upgrade needed multiple lines in the release notes so it was never considered to be a possibility. With over a year of practice in publishing Forgejo releases and the associated release notes manually, the method to do that was well defined and stable. It was never really documented but looking at the discussions and pull requests it was not difficult to reverse engineer the workflow. There were some blurry technical points that were never solved because a human brain took a different path, such as figuring out the backport of a pull request based on the commits it contains. But the workflow itself was precise and it was not based on speculations, it was deduced from the making of over twenty releases. This is the ideal situation when implementing a software: there is plenty of evidence that a human being did a number of things manually. The software just needs to do the same and automate as much as possible. There is no risk of over-engineering, as long as the developer does not get carried away and keeps it simple, stupid. The [release notes assistant](https://code.forgejo.org/forgejo/release-notes-assistant) was developed in that context, for the benefit of three type of users: * The author of a pull request, to show them how their pull request will look in the final release notes so they can fine tune it * The reviewer of a pull request, to ask for changes based on the result rather than the source * The release manager, to collect them and allow manual editing in a single file before the release A number of tools and actions in Forgejo, including some [rather advanced](https://codeberg.org/forgejo/tools/src/commit/d158f39d2572ad7388a9744efdabaec614e6f1ae/scripts/weekly-cherry-pick.sh) are written in shell. But the complexity of the release notes assistant was above the threshold that would allow for long term maintenance. And the reasons are mainly: * the logic to figure out which pull request is backported from which * stable branches and how their history evolve in parallel with the development branch It was decided to implement it in Go instead and has 90%+ test coverage. But the test environment which spawns a Forgejo instance with [fixtures that mimic the structure of the Forgejo repository](https://code.forgejo.org/forgejo/release-notes-assistant/src/commit/d7a2ee0fc6e43475b8790d69cd41392ac68c2703/tests.sh#L282-L371) is derived from the [existing end-to-end](https://code.forgejo.org/forgejo/end-to-end) test framework and in shell. The workflow [added in the Forgejo repository](https://codeberg.org/forgejo/forgejo/src/commit/bca3f857dc4d420712f6314cde9b742d0017059e/.forgejo/workflows/release-notes-assistant.yml) is all the author and reviewers need to agree on a good enough release note. The release manager still needs to use the tool manually and can, for instance, direct it to update the milestone, which was done for the [Forgejo v7.0.6 release](https://codeberg.org/forgejo/forgejo/src/commit/bca3f857dc4d420712f6314cde9b742d0017059e/.forgejo/workflows/release-notes-assistant.yml). And once the release is published, it could be added to the description of the release itself. Which would indirectly solve the rather annoying burden of maintaining a file in the repository with the same information. In conclusion, although the release-notes-assistant was written to be generic and has nothing Forgejo specific, there is little chance it is actually used anywhere else. People won't discover it by chance and pushing it in this direction is not something anyone is ready to do at this point. Having a robust tool for publishing Forgejo and related projects is great. Maintaining a tool used by a variety of users with a whole range of needs is an entirely different proposition. Note: Release notes lines written by the author of the pull request are good enough for point releases that need no explanation. But they should come with a companion blog post to provide context for major releases

@fnetX this is not directly of interest to where your focus is at the moment, I imagine. But I thought about you because this is a nice case of user research. Not in the usual sense, but in how this piece of software (the release notes assistant) retraces the step of a human being and does nothing else. It just automates what was done manually: the human no longer needs to do it and case be busy doing something else.

I'm not sure how it can be applied to User eXperience but maybe. If when you observe user going from A to B to C and you realize that the step B can be removed entirely and lead to the same result, this is an improvement, right?

@fnetX this is not directly of interest to where your focus is at the moment, I imagine. But I thought about you because this is a nice case of user research. Not in the usual sense, but in how this piece of software (the release notes assistant) retraces the step of a human being and does nothing else. It just automates what was done manually: the human no longer needs to do it and case be busy doing something else. I'm not sure how it can be applied to User eXperience but maybe. If when you observe user going from A to B to C and you realize that the step B can be removed entirely and lead to the same result, this is an improvement, right?

@jean-daricade feel free to copy/paste from here for the monthly update.

@jean-daricade feel free to copy/paste from here for the monthly update.
Owner
Copy link

@earl-warren I agree it would be interesting to see people working with the tool. However, I fear that I lack the time to do so right now. Also, the impact (improving one developers workflow) is probably less than:

observing users read the release notes.

I continued my idea in #198

@earl-warren I agree it would be interesting to see people working with the tool. However, I fear that I lack the time to do so right now. Also, the impact (improving one developers workflow) is probably less than: observing users read the release notes. I continued my idea in https://codeberg.org/forgejo/discussions/issues/198

I agree it would be interesting to see people working with the tool.

Sorry for being unclear, it was not what I had in mind. I opened a pull request to the user research repository to clarify that.

> I agree it would be interesting to see people working with the tool. Sorry for being unclear, it was not what I had in mind. I [opened a pull request to the user research repository](https://codeberg.org/forgejo/user-research/pulls/38) to clarify that.

I enjoyed contributing to the release-notes-assistant in the past weeks. And I will help as much as I can when Forgejo needs it. Should there be a warning: "not intended to be used outside of Forgejo" be in the README?

I enjoyed contributing to the release-notes-assistant in the past weeks. And I will help as much as I can when Forgejo needs it. Should there be a warning: "not intended to be used outside of Forgejo" be in the README?

Hello, I may be off-topic and I'm surely very late in the discussion, but have you considered Changie (https://changie.dev/)?

Hello, I may be off-topic and I'm surely very late in the discussion, but have you considered Changie (https://changie.dev/)?

Interesting. Added an issue to the release notes assistant that was developed about six months ago https://code.forgejo.org/forgejo/release-notes-assistant/issues/58

Interesting. Added an issue to the release notes assistant that was developed about six months ago https://code.forgejo.org/forgejo/release-notes-assistant/issues/58
Sign in to join this conversation.
No Branch/Tag specified
No results found.
No results found.
Labels
Clear labels
User research - Accessibility
Requires input about accessibility features, likely involves user testing.
User research - Blocked
Do not pick as-is! We are happy if you can help, but please coordinate with ongoing redesign in this area.
User research - Community
Community features, such as discovering other people's work or otherwise feeling welcome on a Forgejo instance.
User research - Config (instance)
Instance-wide configuration, authentication and other admin-only needs.
User research - Errors
How to deal with errors in the application and write helpful error messages.
User research - Filters
How filter and search is being worked with.
User research - Future backlog
The issue might be inspiring for future design work.
User research - Git workflow
AGit, fork-based and new Git workflow, PR creation etc
User research - Labels
Active research about Labels
User research - Moderation
Moderation Featuers for Admins are undergoing active User Research
User research - Needs input
Use this label to let the User Research team know their input is requested.
User research - Notifications/Dashboard
Research on how users should know what to do next.
User research - Rendering
Text rendering, markup languages etc
User research - Repo creation
Active research about the New Repo dialog.
User research - Repo units
The repo sections, disabling them and the "Add more" button.
User research - Security
User research - Settings (in-app)
How to structure in-app settings in the future?
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
4 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Reference
forgejo/discussions#197
Reference in a new issue
forgejo/discussions
No description provided.
Delete branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?