WIP: feat: Promote serializers to metacommands #263

pastor commented 2026年03月08日 16:16:05 +01:00

This PR promotes our serializers to meta-commands.

The new interface allows chaining serialization actions through the CLI arguments, for example:

$ blue ,serialize commands configuration ui

The PR is marked as WIP because there is a FIXME that I'm not sure how to address, which is that the procedure serialize-configuration needs to be run with the context stored in the .blue-store, this was done automatically when the procedure was a normal command, but now that it's a procedure invoked by a meta-command, the context needs to be injected.

@old, do you know how we can do that?

This PR promotes our serializers to meta-commands. The new interface allows chaining serialization actions through the CLI arguments, for example: ``` $ blue ,serialize commands configuration ui ``` The PR is marked as WIP because there is a FIXME that I'm not sure how to address, which is that the procedure [serialize-configuration](https://codeberg.org/lapislazuli/blue/src/promote-serializers-to-metacommands/blue/ui/entrypoints/serialize.scm#L153) needs to be run with the context stored in the `.blue-store`, this was done automatically when the procedure was a normal command, but now that it's a procedure invoked by a meta-command, the context needs to be injected. @old, do you know how we can do that?

old commented 2026年03月09日 00:30:32 +01:00

The serializer command could takes the first argument at the store. So you always need one. We can do the same for other variables as well (srcdir, builddir) and blueprint file.

What do you think? We could also act like the UI does.

The serializer command could takes the first argument at the store. So you always need one. We can do the same for other variables as well (srcdir, builddir) and blueprint file. What do you think? We could also act like the UI does.

pastor commented 2026年03月09日 18:56:32 +01:00

Hum, I don't understand what you mean.

Hum, I don't understand what you mean.

old commented 2026年03月09日 19:33:09 +01:00

We have commands that are ran in a context. This context is:

• User blueprint (inputs)
• Source-directory (transitive inputs from blueprint)
• Build-directory (outputs for users)
• The store-directory (outputs of blue)

For the autocomplete engine, We need the blueprint (optional) and a store directory for compiling the blueprint (faster invocations).

Right now, the autocomplete engine create a cache entry at ~/.cache/blue/path-to-blueprint/.blue-store.

For the serialization command, things are different. The users project needed to be loaded. I don't think it makes sens to call the serialization command outside of a project. But it's perfectly fine to do so for the autocomplete.

So the stories are different here and I would like to understand better why the serialization command has to be a meta-command. What is the problem this will solve in practice? That way, I think I would figure out a good solution like for the autocomplete engine.

We have commands that are ran in a context. This context is: - User blueprint (inputs) - Source-directory (transitive inputs from blueprint) - Build-directory (outputs for users) - The store-directory (outputs of blue) For the autocomplete engine, We need the blueprint (optional) and a store directory for compiling the blueprint (faster invocations). Right now, the autocomplete engine create a cache entry at `~/.cache/blue/path-to-blueprint/.blue-store`. For the serialization command, things are different. The users project **needed** to be loaded. I don't think it makes sens to call the serialization command outside of a project. But it's perfectly fine to do so for the autocomplete. So the stories are different here and I would like to understand better why the serialization command has to be a meta-command. What is the problem this will solve in practice? That way, I think I would figure out a good solution like for the autocomplete engine.

pastor commented 2026年03月09日 21:44:14 +01:00

@old wrote in #263 (comment):

So the stories are different here and I would like to understand better why the serialization command has to be a meta-command. What is the problem this will solve in practice? That way, I think I would figure out a good solution like for the autocomplete engine.

Well, we had a discussion about this in one of our meetings. The thing is that we want serialization to not break if the user introduces a syntax error in their blueprint because that would break external UIs, so we would like to have some kind of smoother fallback to the known information.

@old wrote in https://codeberg.org/lapislazuli/blue/pulls/263#issuecomment-11427964: > So the stories are different here and I would like to understand better why the serialization command has to be a meta-command. What is the problem this will solve in practice? That way, I think I would figure out a good solution like for the autocomplete engine. Well, we had a discussion about this in one of our meetings. The thing is that we want serialization to not break if the user introduces a syntax error in their blueprint because that would break external UIs, so we would like to have some kind of smoother fallback to the known information.

pastor commented 2026年03月09日 21:47:13 +01:00

Also external UI should work whether you are or not in a project. So the information about built-in commands should be serialized despite errors in the user blueprint or even if there is no blueprint.

Also external UI should work whether you are or not in a project. So the information about built-in commands should be serialized despite errors in the user blueprint or even if there is no blueprint.

old commented 2026年03月09日 22:06:32 +01:00

And why is separating stdout and stderr not an option here? The UI is already doing all of this. It will load an empty blueprint, use the store etc.

If the problem is just mangling of error and output, it is easy to fix this.

And why is separating stdout and stderr not an option here? The UI is already doing all of this. It will load an empty blueprint, use the store etc. If the problem is just mangling of error and output, it is easy to fix this.

pastor commented 2026年03月11日 08:41:31 +01:00

Do you propose that UIs ignore a non-zero exit status and still parse the output? It seems a bit unclean for when there could be a real fatal error that the UI want to notice and avoid executing. I guess we could have different exit codes.

Do you propose that UIs ignore a non-zero exit status and still parse the output? It seems a bit unclean for when there could be a real fatal error that the UI want to notice and avoid executing. I guess we could have different exit codes.

old commented 2026年03月11日 13:51:26 +01:00

The serialize command should not emit an error (status wise). In which case is this happening? Currently, if the blueprint failed to get loaded, an error is indeed returned, I think. But we could add an an option to tell the UI to fallback to the default blueprint and continue.

So, the serialize command should always be successful, but you could still separate stdout/stderr for the data and the logs. If the serialize command failed, then that's another story.

The serialize command should not emit an error (status wise). In which case is this happening? Currently, if the blueprint failed to get loaded, an error is indeed returned, I think. But we could add an an option to tell the UI to fallback to the default blueprint and continue. So, the serialize command should always be successful, but you could still separate stdout/stderr for the data and the logs. If the serialize command failed, then that's another story.

pastor commented 2026年03月13日 16:23:54 +01:00

@old wrote in #263 (comment):

The serialize command should not emit an error (status wise). In which case is this happening? Currently, if the blueprint failed to get loaded, an error is indeed returned, I think. But we could add an an option to tell the UI to fallback to the default blueprint and continue.

Well, that's the point. That we don't want to break UIs when the user makes a mistake in the blueprint, we should definitely fallback to something so the UIs do not choke.

So, the serialize command should always be successful, but you could still separate stdout/stderr for the data and the logs. If the serialize command failed, then that's another story.

Well, another point we discussed is that we want this to be fast. So if a metacomand can avoid loading some Guile modules to speed-up startup, that's also a nice bonus. Did you meant that since for serializing the data we need to load the configuration, we cannot save much startup time by making the serializers metacommands?

@old wrote in https://codeberg.org/lapislazuli/blue/pulls/263#issuecomment-11514874: > The serialize command should not emit an error (status wise). In which case is this happening? Currently, if the blueprint failed to get loaded, an error is indeed returned, I think. But we could add an an option to tell the UI to fallback to the default blueprint and continue. Well, that's the point. That we don't want to break UIs when the user makes a mistake in the blueprint, we should definitely fallback to something so the UIs do not choke. > So, the serialize command should always be successful, but you could still separate stdout/stderr for the data and the logs. If the serialize command failed, then that's another story. Well, another point we discussed is that we want this to be fast. So if a metacomand can avoid loading some Guile modules to speed-up startup, that's also a nice bonus. Did you meant that since for serializing the data we need to load the configuration, we cannot save much startup time by making the serializers metacommands?

This pull request has changes conflicting with the target branch.