The current pyplot wrappers all have an argspec of (*args, **kwargs), which means that any interactive tools that show the possible arguments will not be able to show anything very useful for the wrapped functions. Also, since the docstrings are generated dynamically, any static code analysis tools will not see them. I coded up what I think is an improvement, but since it has a lot of potential to break something and release time is near, I didn't commit my changes into svn. I tried playing around with git, to see if it is useful for communicating patches. You can see the files here: http://github.com/jkseppan/matplotlib/tree/boilerplate or view the commits at: http://github.com/jkseppan/matplotlib/commit/a94764aa78da1716ee21b0108c6a771af02a0ffc http://github.com/jkseppan/matplotlib/commit/fc4a62b810cd34f73156c6bccc77a1cb9f0049dc (the first one is a small bugfix, the second one is the big change). Git users can clone git://github.com/jkseppan/matplotlib.git which is a fork of Andrew Straw's repository; the boilerplate branch has the changes. Any git gurus out there: is this the "right" way for using git to discuss patches, or am I missing something? -- Since we only support Python >=2.4 now, we could get rid of the whole boilerplate code system and replace it with something more dynamic as envisioned in http://groups.google.com/group/comp.lang.python/browse_frm/thread/dcd63ec13096a0f6/1b14640f3a4ad3dc?#1b14640f3a4ad3dc but I don't see any way of keeping the wrapped function's argspec without doing something much like what we do now. -- Jouni K. Seppänen http://www.iki.fi/jks
Jouni K. Seppänen wrote: > I coded up what I think is an improvement, but since it has a lot of > potential to break something and release time is near, I didn't commit > my changes into svn. I tried playing around with git, to see if it is > useful for communicating patches. I think it's useful. > Any git gurus out there: is this the "right" way for using git to discuss > patches, or am I missing something? Yes, as much as there is one... My git mirror isn't automatically synced so often falls behind a bit - as it was when you did this (I have just now updated, though). Furthermore, I recently discovered that I collapsed a bunch of svn commits into a single big git commit (the git repo's idea of svn r7100 contains a lot more than the real svn r7100). So, the branch currently labeled "master" in my github repository should not become the cannonical git repository. (Of course, it will be very easy to label the correct branch -- when it gets made -- "master".) Anyhow, now that I've pushed up a more recent master, assuming you want to apply your work onto that, you could either rebase your commits onto the master -- thus ignoring the true historical state you developed them against -- or merge the master branch -- causing the history to be more complicated, as it's no longer linear. And now that we're talking about this, Eric, as the only MPL dev that I know who prefers Hg over git, have you seen http://github.com/blog/439-hg-git-mercurial-plugin and would you be willing to try it out? I'd be very interested to know if it could make a git central repo work seamlessly for both Hg and git clients.
>> And now that we're talking about this, Eric, as the only MPL dev that I >> know who prefers Hg over git, have you seen >> http://github.com/blog/439-hg-git-mercurial-plugin and would you be >> willing to try it out? I'd be very interested to know if it could make a >> git central repo work seamlessly for both Hg and git clients. > > I had not seen it until you mentioned it above. Now I downloaded it, > enabled it, and tried to make a clone of Jouni's repo. Hit an > exception. It's probably because I am using a newer version of hg than > what hg-git is being developed with. This will take a little time to > sort out. OK, I was able to clone successfully by switching to the last released version of hg. More testing later. Eric
Eric Firing wrote: > >>> And now that we're talking about this, Eric, as the only MPL dev that I >>> know who prefers Hg over git, have you seen >>> http://github.com/blog/439-hg-git-mercurial-plugin and would you be >>> willing to try it out? I'd be very interested to know if it could >>> make a >>> git central repo work seamlessly for both Hg and git clients. >> >> I had not seen it until you mentioned it above. Now I downloaded it, >> enabled it, and tried to make a clone of Jouni's repo. Hit an >> exception. It's probably because I am using a newer version of hg >> than what hg-git is being developed with. This will take a little >> time to sort out. > > OK, I was able to clone successfully by switching to the last released > version of hg. More testing later. Great. I would suggest not attempting to judge on a crazy git-hg-svn triangle -- git/svn itself is enough pain I wouldn't recommend it to anyone. Hopefully git/hg is better. (In other words, if there's a project you're interested that is exclusively git based, it may be best to play around with that than rather than using hg to interact with my git mirror of the MPL svn repo.) BTW, I have no reason to disagree with you that learning hg is easier than learning git. I haven't used Hg for anything complicated, but it was certainly no trouble for me to do basic stuff with. I'm glad to read about their bookmarks extension, which apparently makes hg able to branch more like git rather than copying the entire repository. -Andrew
Andrew Straw wrote: > Jouni K. Seppänen wrote: > >> I coded up what I think is an improvement, but since it has a lot of >> potential to break something and release time is near, I didn't commit >> my changes into svn. I tried playing around with git, to see if it is >> useful for communicating patches. > > I think it's useful. > >> Any git gurus out there: is this the "right" way for using git to discuss >> patches, or am I missing something? > > Yes, as much as there is one... My git mirror isn't automatically synced > so often falls behind a bit - as it was when you did this (I have just > now updated, though). Furthermore, I recently discovered that I > collapsed a bunch of svn commits into a single big git commit (the git > repo's idea of svn r7100 contains a lot more than the real svn r7100). > So, the branch currently labeled "master" in my github repository should > not become the cannonical git repository. (Of course, it will be very > easy to label the correct branch -- when it gets made -- "master".) > > Anyhow, now that I've pushed up a more recent master, assuming you want > to apply your work onto that, you could either rebase your commits onto > the master -- thus ignoring the true historical state you developed them > against -- or merge the master branch -- causing the history to be more > complicated, as it's no longer linear. > > And now that we're talking about this, Eric, as the only MPL dev that I > know who prefers Hg over git, have you seen > http://github.com/blog/439-hg-git-mercurial-plugin and would you be > willing to try it out? I'd be very interested to know if it could make a > git central repo work seamlessly for both Hg and git clients. I had not seen it until you mentioned it above. Now I downloaded it, enabled it, and tried to make a clone of Jouni's repo. Hit an exception. It's probably because I am using a newer version of hg than what hg-git is being developed with. This will take a little time to sort out. I'm moderately optimistic about hg-git despite hitting that pothole; I suspect hg-git interoperability is much easier than interoperability of either with svn. For the latter, I have been doing some testing of hgsubversion, but I am quite leery of trying to work routinely with a central svn and local dvcs. In any case, I can work with git if I have to (although after some experience with git I still think that hg is *much* easier to learn and use casually), and I really would like to see numpy and mpl both move completely to either git or hg. And it doesn't even have to be the same one for both. Eric
Andrew Straw <str...@as...> writes: > Anyhow, now that I've pushed up a more recent master, assuming you want > to apply your work onto that, you could either rebase your commits onto > the master -- thus ignoring the true historical state you developed them > against -- or merge the master branch -- causing the history to be more > complicated, as it's no longer linear. I opted for the merge, since all the git documentation I have read seems to frown on rebasing anything that has been published. Besides, dealing with non-linear history in this relatively simple case is probably good practice for us git newbies. By the way, would it have been possible for me to use git-svn on my own repository, update it from the official svn and then have everything work out right once you updated your repository? I suppose that should be possible if git-svn guarantees that every one of its users gets the exact same trees for every svn commit, but perhaps even then it would not show up as a proper merge in git. -- Jouni K. Seppänen http://www.iki.fi/jks
Jouni K. Seppänen wrote: > The current pyplot wrappers all have an argspec of (*args, **kwargs), > which means that any interactive tools that show the possible arguments > will not be able to show anything very useful for the wrapped functions. > Also, since the docstrings are generated dynamically, any static code > analysis tools will not see them. Jouni, This looks like a very good idea. It is in some ways a separate change, but it would be nice if the boilerplate were generated at build time, and/or generated as a separate file from which pyplot.py would import everything. I don't like having a single file that is partly hand-edited and partly machine-generated. > -- > > Since we only support Python >=2.4 now, we could get rid of the whole > boilerplate code system and replace it with something more dynamic as > envisioned in > > http://groups.google.com/group/comp.lang.python/browse_frm/thread/dcd63ec13096a0f6/1b14640f3a4ad3dc?#1b14640f3a4ad3dc > > but I don't see any way of keeping the wrapped function's argspec > without doing something much like what we do now. > Have you looked at the decorator module? http://pypi.python.org/pypi/decorator Eric
Eric Firing <ef...@ha...> writes: >> The current pyplot wrappers all have an argspec of (*args, **kwargs), >> which means that any interactive tools that show the possible arguments > > It is in some ways a separate change, but it would be nice if the > boilerplate were generated at build time, and/or generated as a separate > file from which pyplot.py would import everything. I don't like having > a single file that is partly hand-edited and partly machine-generated. You're right, it would be better to generate it at build time. One problem with this is that my version of boilerplate.py needs to import axes.py to access the methods of the Axes class, and axes.py in turn imports other things, making it impossible to run boilerplate.py without having compiled the compilable modules. That's why I munge sys.path before importing axes -- and there are lots of things that could go wrong with that. I guess a more correct approach would be to use something like the parser module to read axes.py without executing the code. >> Since we only support Python >=2.4 now, we could get rid of the whole >> boilerplate code system and replace it with something more dynamic as >> envisioned in >> >> http://groups.google.com/group/comp.lang.python/browse_frm/thread/dcd63ec13096a0f6/1b14640f3a4ad3dc?#1b14640f3a4ad3dc >> >> but I don't see any way of keeping the wrapped function's argspec >> without doing something much like what we do now. >> > > Have you looked at the decorator module? > http://pypi.python.org/pypi/decorator That looks like it could work -- the memoize example seems to be pretty close to our wrapping needs. I'll spend some time thinking about this later. Thanks for the link! -- Jouni K. Seppänen http://www.iki.fi/jks
On Mon, Jun 8, 2009 at 2:29 AM, Jouni K. Seppänen<jk...@ik...> wrote: >> Have you looked at the decorator module? >> http://pypi.python.org/pypi/decorator > > That looks like it could work -- the memoize example seems to be pretty > close to our wrapping needs. I'll spend some time thinking about this > later. Thanks for the link! > We use Michele's decorators.py extensively in ipython for writing testing wrappers, we carry a local copy we call decorators_msim and then add some utilities on top. Feel free to pillage as needed or just read for info: http://bazaar.launchpad.net/~ipython-dev/ipython/0.10/annotate/head%3A/IPython/testing/decorators.py Cheers, f
On Sun, Jun 7, 2009 at 8:05 PM, Eric Firing<ef...@ha...> wrote: > Jouni K. Seppänen wrote: >> The current pyplot wrappers all have an argspec of (*args, **kwargs), >> which means that any interactive tools that show the possible arguments >> will not be able to show anything very useful for the wrapped functions. >> Also, since the docstrings are generated dynamically, any static code >> analysis tools will not see them. > > Jouni, > > This looks like a very good idea. I agree -- the presence of *args and **kwargs in the pyplot docstrings, the interface most people use, is a major wart. I suggest committing this to HEAD when you are ready to go. Once that is done we can remove all the redundant pyplot docstrings from axes.py which will make Eric very happy. JDH
John Hunter <jd...@gm...> writes: > On Sun, Jun 7, 2009 at 8:05 PM, Eric Firing<ef...@ha...> wrote: >> This looks like a very good idea. > > I agree -- the presence of *args and **kwargs in the pyplot > docstrings, the interface most people use, is a major wart. I suggest > committing this to HEAD when you are ready to go. I would like to ask the opinion of other developers before checking it in. I now have two different implementations in two branches of my github repository (patches attached): http://github.com/jkseppan/matplotlib/tree/boilerplate http://github.com/jkseppan/matplotlib/tree/autoboiler The first one in an improved boilerplate.py as before, so it can be used to write a large part of pyplot.py so that the signatures and docstrings are all in there. I still haven't added any automation to call it while building matplotlib, but that's something of a separate problem. The second one is a version of pyplot.py that inspects the methods of Axes objects and creates corresponding functions automatically. It uses decorator.py, which I have included with the matplotlib source. I think I personally like the boilerplate branch better. While the code is a little hairy, you can see its output in pyplot.py, so if it ever needs to be modified, there is something to run "diff" on. The code in the autoboiler branch was more difficult to get right, and it creates the functions at runtime via "exec" (which is how decorator.py can set the signature of the functions it creates). In the autoboiler branch, importing pyplot seems to be very slightly slower. Here is the average time per example it took to run the examples in the pylab_examples subdirectory in the three branches using the template backend, in two different runs: master boilerplate autoboiler run 1 1.013 1.028 1.043 run 2 1.028 1.027 1.043 It looks like there is a difference, but it is as large as the difference between runs in the master branch; much more careful experimentation would be needed to be sure about the magnitude of the difference. -- Jouni K. Seppänen http://www.iki.fi/jks
Jouni K. Seppänen <jk...@ik...> writes: > I now have two different implementations in two branches of my > github repository (patches attached): > > http://github.com/jkseppan/matplotlib/tree/boilerplate > http://github.com/jkseppan/matplotlib/tree/autoboiler I finally committed the "boilerplate" variant. It seems to pass the tests in pylab_examples, but now might be a good time for everyone to take a look to see if I have broken anything. -- Jouni K. Seppänen http://www.iki.fi/jks
On Wed, Jul 22, 2009 at 8:15 AM, Jouni K. Seppänen<jk...@ik...> wrote: > I finally committed the "boilerplate" variant. It seems to pass the > tests in pylab_examples, but now might be a good time for everyone to > take a look to see if I have broken anything. There is one problem, but I am not sure what the workaround is yet. The doc strings in rest are broken, eg compare the pyplot Line2D properties with the Axes equivalent: http://matplotlib.sourceforge.net/api/pyplot_api.html#matplotlib.pyplot.loglog vs http://matplotlib.sourceforge.net/api/axes_api.html#matplotlib.axes.Axes.loglog Both use the artist.kwdocs inspector to autogenerate the tables. There is a rc param in doc/matplotlibrc which we use to force rest formatting docstring.hardcopy : True # set this when you want to generate hardcopy docstring which is False by default for src installs and True when we build the docs. Then in matplotlib.artist, we do def kwdoc(a): hardcopy = matplotlib.rcParams['docstring.hardcopy'] if hardcopy: return '\n'.join(ArtistInspector(a).pprint_setters_rest(leadingspace=2)) else: return '\n'.join(ArtistInspector(a).pprint_setters(leadingspace=2)) to format rest when making hardcopy. Then we we interpolate into the docstring at runtime:: # from Axes.loglog def loglog(self, *args, **kwargs): """ ...snip The remaining valid kwargs are :class:`~matplotlib.lines.Line2D` properties: %(Line2D)s **Example:** .. plot:: mpl_examples/pylab_examples/log_demo.py """ ...snip return l loglog.__doc__ = cbook.dedent(loglog.__doc__) % martist.kwdocd we get plain text when hardcopy is False and rest when it is True. Thus we have the best of both worlds from the interactive shell and on the website and PDF. It looks like you are interpolating the strings in in boilerplate, when they should be left to be interpolated *at runtime* as we do in the other modules. Ie, it appears you are doing runtime interpolating when you generate pyplot.py from boilerplate.py. In the old version, the interpolation of pyplot doc strings were deferred until runtime, eg from older pyplot.py:: # This function was autogenerated by boilerplate.py. Do not edit as # changes will be lost def loglog(*args, **kwargs): # allow callers to override the hold state by passing hold=True|False b = ishold() h = kwargs.pop('hold', None) if h is not None: hold(h) try: ret = gca().loglog(*args, **kwargs) draw_if_interactive() except: hold(b) raise hold(b) return ret if Axes.loglog.__doc__ is not None: loglog.__doc__ = dedent(Axes.loglog.__doc__) + """ Additional kwargs: hold = [True|False] overrides default hold state""" Is there a way to preserve this in the new boilerplate configuration? JDH
John Hunter <jd...@gm...> writes: > On Wed, Jul 22, 2009 at 8:15 AM, Jouni K. Seppänen<jk...@ik...> wrote: > >> I finally committed the "boilerplate" variant. It seems to pass the >> tests in pylab_examples, but now might be a good time for everyone to >> take a look to see if I have broken anything. > > There is one problem, but I am not sure what the workaround is yet. > The doc strings in rest are broken, eg compare the pyplot Line2D > properties with the Axes equivalent: I think I have fixed this, but I'm having some trouble building the docs myself -- could you check if the hardcopy docs come out right now? -- Jouni K. Seppänen http://www.iki.fi/jks