homepage

I have developed a new tool virt-sandbox-service using argparse. Currently the argparse has the ability to print help and usage. Being naturally lazy, I think it should be able to print out a man page also. This would help with development and make maintaining the different documents.

Are you aware of help2man? It can transmogrify the output of "your-program-that-uses-argparse --help" to a man page. Another solution is to use docutils’ rst2man to convert some documentation file to the man format.
On the other hand, this blog post contains code implementing a custom optparse formatter for man pages, which could probably be adapted for argparse easily, so it does not look hard to implement. (Note that the code has no licensing information.)
Steven, what do you think? Too specialized for the stdlib or small, useful new feature?

Forgot the link to the blog post: https://andialbrecht.wordpress.com/2009/03/17/creating-a-man-page-with-distutils-and-optparse/ 

I think adding a new formatter for man pages would be generally useful. Assuming someone provides a patch. ;-)

Alright :) I’ve contacted the author of the blog article to ask him if we can reuse his code.

Andi, the author of the blog post, will work on a patch.

Hi, 
I have been wanting this feature for quite a long time. IMHO, binaries and scripts should always include a man page. The Debian developers require that. However, man pages have a 'bizarre' format. 
Long talk, short time. I did implement something. I tested it on Python 2.7 since my project currently only supports Python 2.7. 
I think it should not be complicated to port to Python 3.X. 
I doubt if the correct place for formatting a man page should be in argparse.py itself. My solution is an extension of Andial's brecht solution that uses ofcourse argparse.
You can see it in action in https://github.com/pwman3/pwman3
I am also attaching the code here. 
I hope you will find this file useful. I would appreciate your comments too. 
Regards, 
Oz

> I have been wanting this feature for quite a long time
Me too.

@Raymond Hettinger, 
You don't have to wait anymore. I would be happy if you review the patch, and share your opinion.
I might have posted my patch too early. I am still doing changes on the upstream, in my github posted earlier.So, please test the one upstream.

Any news on this?
The code posted by Oz Tiram (I took the latest from his github repo) works quite well, even with Python 3. (I just tested it)
It would be great if argparse could include the formatter class, as well as the distutils command.

Oz Tiram wrote:
> I doubt if the correct place for formatting a man page should be in argparse.py itself. My solution is an extension of Andial's brecht solution
Thank you for that, Oz.
I see an easy seam to refactor this work into separate concerns.
The formatting of a man page from the parser for the command, can be coupled to the ‘argparse’ module since it is dependent only on an ArgumentParser plus some extra properties for populating the man page.
The building of one man page should not, IMO, be so tightly coupled to a Python distribution. There may be several different commands installed by one distribution. Each man page's fields should mostly come from the parser for the command, not from the distribution metadata. Instead, the build process should be told which parser to use for generating each of the man pages.
I propose to work on Oz's module and produce a man page formatter for exactly one ArgumentParser, as an enhancement for ‘argparse’.

I would like to work on a patch if nobody currently are working on it. I'll keep you updated

Hi all, I've created a PR for this, please help for code review.
I found that previous method from Oz had a problem, that man page and general help page will share a Formatter, that cause an unexpected behavior that help page will generate with troff format (Unix man page used this format).
I switch to another method that creates a Manpage class and a private _ManpageFormatter, we just need to put parser into this Manpage, the __str__ method will generate the man page that we want.
This approach prevents help page format affect by ManpageFormatter, and the user can happily switching formatter_class to RawDescriptionHelpForatter, RawTextHelpForatter and others, since the Manpage class is separate from HelpFormatter, and _ManpageFormatter will used the formatter provide from parser.
The attach file is a dummy argparser file, you can try it by this:
 ./python poc.py > poc.1 && man ./poc.1

How to use this feature? argparse is executed every time when the end user runs the program. But generating a man page is the action that should be executed at develop or build stage. The end user gets a generated man page, he doesn't need the option to generate a man page on a fly. The developer needs the ability to generate a man page rather than running the program for its main purpose. What should be the structure of the program that uses argparse for parsing arguments and for generating a man page? How generating a man page should be invoked?

> How to use this feature?
> argparse is executed every time when the end user runs the program.
> But generating a man page is the action that should be executed at
> develop or build stage.
> ...
> How generating a man page should be invoked?
For now, man page will only be generated by using `print_manpage` function, and argparse won't add a shortcut like help page (-h).
Developer can use `parser.add_manpage_section` to add custom manpage section, e.g. copyright, report bugs.
The only thing that will affect it that `add_manpage_section` will be adding at runtime, but user won't used it.
> The end user gets a generated man page, he doesn't need the option to generate a man page on a fly. The developer needs the ability to generate a man page rather than running the program for its main purpose.
Yes, this patch will only offer the ability to generate man page, but if developer want to build with `setup.py`, it still need to combine them.
> What should be the structure of the program that uses argparse for parsing arguments and for generating a man page?
It won't need additional change, just used current method and it will work.

But how to use print_manpage()? Can you provide a new template of a program that uses argparse?

Attachment is the poc of generating man page via `print_manpage`:
 $ ./python poc_2.py > output && man ./output

Also, `print_manpage` use the same infra as `print_help` and `print_usage`, so it can use the parameter `file` to output to different stream or file.

Your example just prints a man page. It doesn't do anything useful besides this.
I'm asking how argparse can do two incompatible things: provide a command-line interface for an end user and generate a man page in a build script.
Do you mean that the programmer should create a separate script for generating a man page and copy a part of the code from the main program? This workflow should be documented, with examples. And it is not applicable for simple one-file scripts.

Sorry that I didn't figure out what you said in the previous msg.
> provide a command-line interface for an end user
I add a parameter that developer can switch command line option for man page, if the option is on, user can do this:
 ./python foo.py --manpage
and open the manpage, but I think this isn't a good approach, since normal user will simply use `man foo.py`.
> generate a man page in a build script.
> Do you mean that the programmer should create a separate script for generating a man page and copy a part of the code from the main program? This workflow should be documented, with examples. And it is not applicable for simple one-file scripts.
I not sure if argparse need to provide a setuptools command, if need, the approach that Oz provide can be used, and developer only need to do like this in setup.py
 from argparser import build_manpage
 cmdclass={'build_manpage': build_manpage}
then at the command line, developer can build manpage as `./python setup.py build_manpage --output=foo.1 --parser=foo.parser`
If provide this is function too far for argparse, then as you said, a well-documented workflow should be provided.
----
Even if argparse provide this function, end-user still can't easily do the trick `man foo.py` or `man foo` if script point is provide in setup.py. If need to approach things like `man foo`, it may need to integrate with setuptools to put the man page file to the correct path (e.g. /usr/share/man/man1)

If you can import the module that defines the parser, and get at the generated parser, this should be trivial to integrate with the build system. Something like:
 PYTHONPATH=. python3 -c 'import mymodule; p=mymodule.make_parser(); p.print_manpage(file="mymodule.1")'
This operation could either be done automatically during build always, or it could just be done by the developers and the man page put under version control.

I think this should be something that is not included in argparse itself. I can imagine a scenario in which the manpage is accessible through `./python foo.py --manpage` but the manpage is not installed systemwide. This will be very confusing for users. Also, as Serhiy comments, argparse provides functionality that at this point is relevant for the enduser of the application. Generating a man page in a build script sounds like something that should be on its own or as a help function of the build system. The only thing argparse could do (maybe) is trying to read an already installed manpage and use that, but I still think that is not a good idea because it can generate conflicts and similar if multiple versions are installed or if the help of the package deviates from the manpage.

On 14-Jun-2018, Pablo Galindo Salgado wrote:
> I think this should be something that is not included in argparse
> itself. I can imagine a scenario in which the manpage is accessible
> through `./python foo.py --manpage` but the manpage is not installed
> systemwide.
This bug report is not asking that ‘argparse’ install a manual page,
and it is not asking for any new command-line option in programs. So I
don't know the relevance of that point.
> This will be very confusing for users.
What would cause that confusion? This is not something that would
change how ‘argparse’ parses its arguments, so I don't know what
you're referring to.
> Generating a man page in a build script sounds like something that
> should be on its own or as a help function of the build system.
This is asking that the ‘argparse’ library should have an API to
create a manual page, for use in the build system.
The reason to have it in the ‘argparse’ library is that the library
already knows how to build a single document (the ‘--help’ output)
from the collection of arguments, so this would be just another
rendering of that information. It makes sense to have it in the
‘argparse’ library as an API for other tools to use.

>What would cause that confusion? This is not something that would
>change how ‘argparse’ parses its arguments, so I don't know what
>you're referring to.
The (possible) confusion is the existence of a manpage only available though argparse (`./python foo.py --manpage`) and not systemwide. I am not sure how many people do something like `./python poc_2.py > output && man ./output` to **read** the manpage.
>This is asking that the ‘argparse’ library should have an API to
>create a manual page, for use in the build system.
At this point argparse is user facing in the sense that once configured, it provides functionality for the user of the command line application. My opinion is that it will be weird to have it provide also APIs for creating man pages (which is a developer utility). My humble opinion is that it if argparse starts to provide APIs for usage at install time is not "doing one thing and doing it well".
>The reason to have it in the ‘argparse’ library is that the library
>already knows how to build a single document (the ‘--help’ output)
>from the collection of arguments, so this would be just another
>rendering of that information. It makes sense to have it in the
>‘argparse’ library as an API for other tools to use
Argparse --help makes sense because people expect to use -h or --help with command line tools but IMHO opinion nobody expects --manpage to be the way to read the manpage. I do not know any command line utility that provides its manpage that way. If the argument is using --manpage is the way to generate one, then we are mixing user-facing options with developer ones.
Anyway, this is just my view of the problem, nothing more.

On Thu, 2018年06月14日 20:02 +0000, Pablo Galindo Salgado <report@bugs.python.org> wrote:
> The (possible) confusion is the existence of a manpage only available 
> though argparse (`./python foo.py --manpage`)
This report isn't asking for that. (I see only one person proposing such an interface, and even that person said it's not a good idea.) So please don't conflate that with the original bug report.
> I am not sure how many people do something like `./python poc_2.py > output 
> && man ./output` to **read** the manpage.
Right, I am not asking for anything like that; I'm not asking that ‘argparse’ grow a way to read a manual page,. I am asking only for a standard way to programmatically generate that manual page from the information ‘argparse’ already knows.
> >This is asking that the ‘argparse’ library should have an API to
> >create a manual page, for use in the build system.
> 
> At this point argparse is user facing in the sense that once configured, 
> it provides functionality for the user of the command line application. 
> My opinion is that it will be weird to have it provide also APIs for 
> creating man pages (which is a developer utility).
Creating an argument parser itself is already a developer activity, and we don't see that as weird that ‘argparse’ allows for that.
I'm arguing that the library already knows how to turn the argument collection into a user-facing document (the usage message), and a manual page is a different way of rendering that same internal data. So that's why ‘argparse’ is a consistent place to have that functionality.
> My humble opinion is 
> that it if argparse starts to provide APIs for usage at install time is 
> not "doing one thing and doing it well".
That API already exists: the specific ‘ArgumentParser’ is available to be imported for a program which defines one. So, the library already provides APIs for usage at install time (or any other run-time).
I am asking to make that API more useful for the distribution of programs.

Thanks Ben for expressing exactly what this ticket is about and why the code would be in argparse. (The author of argparse and Raymond a senior core dev also agree that adding a man page formatter to argparse would be useful.)

Couldn't such a tool exist outside the standard library. I'm thinking a function that you would import and wrap the parser object, similar to how argcomplete works (https://argcomplete.readthedocs.io/en/latest/index.html). The downside is that developers would have to opt-in for it to work (much like they currently have to opt-in to bash completion with things like argcomplete). But it would allow much more flexibility being outside the standard library.
I completely agree that it should be done in Python either way. help2man is very limited in its flexibility (it doesn't help that it's written in Perl), and there are fundamental limits to what you can do from parsing the --help output, vs. just generating correct troff from the source.
Installing the manpage is a separate concern. That would need to go in setuptools or distutils, if anywhere. But before you can worry about how to install it you need to be able to generate it in the first place.

On Thu, 2018年06月14日 23:46 +0000, Aaron Meurer <report@bugs.python.org> wrote:
> Couldn't such a tool exist outside the standard library.
I've tried writing such a tool. It would ideally re-use as much as feasible of the functionality that assembles the usage message. But that is hampered by the fact the usage message generation is not easily accessible from outside.
I am hoping that, in order to re-use that functionality, a common set of "take the argument collection as input, generate a document structure" functionality can be factored out for other use — initially, for generating a manual page.
So this bug report asks for that work to be done in the ‘argparse’ library.
> Installing the manpage is a separate concern.
Yes, I agree. That is not part of this bug report.

I see. I haven't dug much into the argoarse source, so I don't have a good feel for how feasible such a tool would be to write. Such refactoring would also be useful for generating HTML or RST for the help. I've previously used help2man and man2html to generate html help, but both tools are very limited in what they can do.

On Friday, June 15, 2018 1:52:41 AM CEST Ben Finney wrote:
> On Thu, 2018年06月14日 23:46 +0000, Aaron Meurer <report@bugs.python.org> wrote:
> > Couldn't such a tool exist outside the standard library.
> 
> I've tried writing such a tool. It would ideally re-use as much as feasible of
> the functionality that assembles the usage message.
FWIW, I followed you and andialbrecht's solution, and did some changes to the
code (now the code is in PyPi as argparse-manpage). Feel free to take what's
considered useful.
> So this bug report asks for that work to be done in the ‘argparse’ library.
Agreed.
> > Installing the manpage is a separate concern.
> 
> Yes, I agree. That is not part of this bug report.
I think installation is valid concern; so it should be tracked somewhere.

On 06/14/2018 07:15 PM, Ben Finney wrote:
> Ben Finney <ben+python@benfinney.id.au> added the comment:
>
> On Thu, 2018年06月14日 20:02 +0000, Pablo Galindo Salgado <report@bugs.python.org> wrote:
>> The (possible) confusion is the existence of a manpage only available
>> though argparse (`./python foo.py --manpage`)
> This report isn't asking for that. (I see only one person proposing such an interface, and even that person said it's not a good idea.) So please don't conflate that with the original bug report.
>
>> I am not sure how many people do something like `./python poc_2.py > output
>> && man ./output` to **read** the manpage.
> Right, I am not asking for anything like that; I'm not asking that ‘argparse’ grow a way to read a manual page,. I am asking only for a standard way to programmatically generate that manual page from the information ‘argparse’ already knows.
>
>>> This is asking that the ‘argparse’ library should have an API to
>>> create a manual page, for use in the build system.
>> At this point argparse is user facing in the sense that once configured,
>> it provides functionality for the user of the command line application.
>> My opinion is that it will be weird to have it provide also APIs for
>> creating man pages (which is a developer utility).
> Creating an argument parser itself is already a developer activity, and we don't see that as weird that ‘argparse’ allows for that.
>
> I'm arguing that the library already knows how to turn the argument collection into a user-facing document (the usage message), and a manual page is a different way of rendering that same internal data. So that's why ‘argparse’ is a consistent place to have that functionality.
>
>> My humble opinion is
>> that it if argparse starts to provide APIs for usage at install time is
>> not "doing one thing and doing it well".
> That API already exists: the specific ‘ArgumentParser’ is available to be imported for a program which defines one. So, the library already provides APIs for usage at install time (or any other run-time).
>
> I am asking to make that API more useful for the distribution of programs.
>
> ----------
>
> _______________________________________
> Python tracker <report@bugs.python.org>
> <https://bugs.python.org/issue14102>
> _______________________________________
Correct, the reason I would want this is to add something to a Makefile 
to automatically generate a man page, when you do a make install
manpages: foo.py
	./python foo.py --manpage > foo.1
install.manpages: manpages
	install -D -m 644 foo.1 ${DESTDIR}/usr/share/man/man1

As already pointed out, it can get the parser somehow, you don't need an explicit extra option --man. That's the approach suggesed in:
man-argparse and build_manpage.py attached here
You can use it a make file like this:
install:
 ./setup.py build_manpage 
 ./setup.py install
 install -D -m 644 foo.1 ${DESTDIR}/usr/share/man/man1

On Friday, June 15, 2018 11:54:04 AM CEST Daniel Walsh wrote:
> Correct, the reason I would want this is to add something to a Makefile 
> ...
> manpages: foo.py
> 	./python foo.py --manpage > foo.1
The /bin/argparse-manpage could help temporarily (if you don't mind
the additional build-dep).