I think this is a good idea, will be better than prefixing everything with _, although it might hurt discoverability of where completions are coming from a bit. Maybe adding a README file that points to the default completions will be nice.
Do you have a proposal for what to do with the already deprecated completions starting with _? We can remove the prefix when moving them, but maybe a distinction between 3rd party completion fallbacks (like for cobra) and stuff that are implemented here should stay? This does introduce another check for file, but it seems we already do a lot of those.
I guess #1329 (comment) is also related

I also kind of like the idea. Leaning towards a separate completions-dist or the like rather than completions/default, because the latter could cause problems with completion for command named default.

I suppose the same consideration applies to the helpers dir, i.e. we'd have helpers-dist?

BTW I prefer the name dist over default, because default makes it sound as if we'd load something from there by default (with high precedence) while the reality is kind of the other way around (we load from there ~last).

@scop This subtopic is actually the one I think is related to #1399. My (current) conclusion is that the root cause of the problem with the UAPI spec for image-based Linux is /etc/bash_completion.d/000_bash_completion_compat.bash installed by us.

Background: First of all, we actually haven't officially deprecated API v2 (which is provided through 000_bash_completion_compat.bash) because external completions should use API v2 for now to work with different versions of bash-completion in the market. Even after declaring the switch to API v3 for external completions, we don't have a plan to remove API v2 as long as it doesn't affect maintainability. In this sense, 000_bash_completion_compat.bash isn't deprecated but is simply a file that should be loaded at startup. Other packages also want some settings to be loaded at the startup of bash-completion. For example, fzf wants to rewrite the default completion complete -D after bash-completion is loaded, so it puts a file in /etc/bash_completion.d/fzf utilizing the fact that /etc/bash_completion.d/* are loaded at startup, even though they are not API-v1 completions. So, I'd like to suggest introducing a new directory for (non-deprecated) eagerly-loaded files, such as /etc/bash-completion.d/startup{,-core} and move 000_bash_completion_compat.bash there.

PR 1399: The problem that #1399 tries to solve is actually the conflict of /etc/bash_completion.d/000_bash_completion_compat.bash within the image-based Linux (where /etc is externally mounted). However, the file will be moved to another directory under this subtopic (if accepted). Otherwise, we don't put any files in /etc/bash_completion.d, so there shouldn't be a problem with the image-based Linux.

External packages using /etc/bash_completion.d: Existing external packages that put files in /etc/bash_completion.d need to be updated regardless of #1399, by changing their installation location to /usr/share/bash-completion/completions with API v2 (or, if they want their file to be loaded at startup like fzf and _python-argcomplete, putting the file in /usr/share/bash-completion/startup, i.e., the directory suggested in this subtopic). I doubt the necessity of a new directory that essentially has the same purpose as /usr/share/bash-completion/completions, such as /usr/share/bash-completion/compat for API v1 introduced in #1339. Rather, it is strange to provide a new location for API-v1 completions when API-v1 completions are already broken (as we've already removed some functions of API v1, see #1220).

Hmm, I've been forgetting about the benefit of updating compatdir I mentioned in #1399. The location of external completion settings in the upstream may be changed to outside /etc altogether through compatdir, if the upstream uses the pkg-config variable compatdir.

Quote from #1399 (review) :

• In this context, if the project author sets up the installation of the completion script using autoconf or cmake following our README.md, if we change the compatdir in pkg-config, the installation location should automatically be adjusted even if the project is not actively maintained. In this sense, changing the default location of compatdir now still has some effects. A little concern is how widely our pkg-config settings are installed in the system in distributions.

After finding the UAPI spec, I'm not sure if the alternative location should be /usr/share/bash-completion/compat (as in the current #1399), /usr/share/bash_completion.d (as in the initial version of #1399), or /usr/etc/bash_completion.d (as suggested in the UAPI spec).

• If we had originally put files in /etc/bash_completion.d because we considered the files a part of the configuration at that time, it may be minimal and natural to change the compatdir as { => /usr}/etc/bash_completion.d to allow overriding the "default" completions. Overriding under the UAPI rule is actually useful to allow users to disable the distribution-provided setting (such as /usr/etc/bash_completion.d/fzf by putting an empty file in /etc/bash_completion.d (such as /etc/bash_completion.d/fzf).
  • However, the same mechanism for overriding may also be implemented for /usr/share/bash-completion/compat in the current #1399, so it's not a benefit only possible with /usr/etc/bash_completion.d. In addition, I think it is also useful for /usr/share/bash-completion/startup to have the same mechanism.

Makes sense to me to prefer cmdname.bash over cmdname for the completion file, and also recommend doing so in the docs for 3rd party completions. As long as we try to load both, possibility for a mixup still exists though (possibly loading task.bash that is intended for task when completing task.bash if task.bash.bash does not exist). Dropping loading non-.bash would be too big a change though, so we get to pick our poison here. Anyway we could make the preference stronger by documenting completion filenames without added .bash as deprecated.

Thank you for the comment. OK, let's consider this direction.

Dropping loading non-.bash would be too big a change though, so we get to pick our poison here.

I think we can drop the support for non-.bash for completions-dist or completions-core, where we have full control. For compatibility, I think we can continue to check non-.bash files (at least for a sufficient transition period).

Anyway we could make the preference stronger by documenting completion filenames without added .bash as deprecated.

OK. I agree with this.

Current suggestions are

• Rename our completions/* to completions-core/*.bash
• Check both <cmd>.bash and <cmd> in completions but only <cmd>.bash in completions
• Recommend <cmd>.bash and deprecate <cmd> in README

I prefer merging sub-directries (or filter out) at ./configure and make install instead of implementing something to bash-completion. I think user is not interested about deprecated or not.

I don't understand. We discuss the separation of the directory structure of the installation target but not the one in the source tree. The reason that I suggest introducing a separate directory is to avoid conflicts happening in the installation target. In that sense, the directory structures within the source tree don't need to be modified, but the directory tree in the target tree needs to be modified to solve the problem.

User does not need both of completions: for fallbacking and deprecated. So they does not conflict at file level. No?

Right. Fallback and deprecated completions wouldn't conflict with each other because both are maintained by us, but the original discussion should have been the conflict between the deprecated/fallback completions vs external completions. If files completions/<subdir>/* are merged (or filtered out) into the parent directory completions, the conflicts appear.

If you talk about merging the completion files into a single sub-directory completions/<single-subdir>, the conflict problem doesn't appear, but bash-completion doesn't support it currently, so we still need to implement it after discussion.

If we implement the completions-dist idea from above, would it not make this unnecessary? I.e. everything we ship is in completions-dist, and everything in it can be overridden by placing a suitably named file in completions, so there may not be any need to deprecate any of our files in a sense. (We may want to continue documenting in our files's comments where we know that a preferred upstream completion exists though.)

I think this is technically unnecessary, though it could be useful for maintenance purposes.

• Even though we keep old files, I think it could be useful to separate the deprecated files (whose behaviors shouldn't be updated) from the non-deprecated files (which should be updated following the upstream change). For example, when the upstream of a command started to offer a completion script by themselves in version 1.2.3, and when we deprecated the completion script in our repository for version 1.2.2, I think we should freeze the completion candidates generated by the completion setting for 1.2.2, rather than updating them for version >= 1.2.3 because the load of the deprecated file implies that the system has an old version (<= 1.2.2) of the command.
• These files are kept for compatibility with old versions of external commands that didn't ship their completions, so if we separate the deprecated completion files, the directory name could be completions-compat.
• Maybe we can separate the directories for deprecated completions in the repository tree, but merge them in the installation target.

though they are not strong opinions. How do you feel?