I have been working on updating the trunk to provide enthought.traits version 2.6b1. backend_driver.py is running without exceptions using the traited config package with the internal traits package. Issues: 1) there are lots of absolute package imports scattered throughout traits' code. I worked around this by adding a line to matplotlib/__init__.py: sys.path.append(os.path.split(__file__)[0]) This lets matplotlib access enthought.traits without modifying enthoughts code (anymore than Gael had already done by stripping the pkg_resources imports). 2) When I tried updating rc_traits.py to import matplotlib.enthought.traits instead of enthought.traits (which isnt on the PYTHONPATH), I discovered a problem: enthought.traits.trait_errors.TraitError: The 'parents_items' trait of a ViewElements instance must be a TraitListEvent, but a value of <matplotlib.enthought.traits.trait_handlers.TraitListEvent object at 0x850454c> was specified. So traits would be a behind-the-scenes package, for internal mpl use only. 3) We can not include traits-3 without either adding setuptools as an external dependency (which is already true for python-2.3 users) or monkey-patching distutils. traits-3 includes some pyrex code, which standard distutils does not recognize. I have not committed my work to svn yet. I wanted to get some feedback on points 1 and 2 first. Is it acceptable to use traits internally, but not expose it to the end user? I think the answer is yes, and that this is even a benefit. If we want to make it an external dependency we can strip the package out without impacting any user code. Darren
On Tue, Nov 06, 2007 at 09:00:23PM -0500, Darren Dale wrote: > I have not committed my work to svn yet. I wanted to get some feedback on > points 1 and 2 first. Is it acceptable to use traits internally, but not > expose it to the end user? I think the answer is yes, and that this is even a > benefit. If we want to make it an external dependency we can strip the > package out without impacting any user code. I agree that by itself this is a benefit. It might also be interesting to discuss this usecase on the enthought mailing list. If you tell these guys that this is an important usecase, that projects like ipython, matplotlib, would like to use traits as an internal dependency, but would still like to be able to use the benefit of traits when interfacing with other libraries, there might be a solution. Gaël
On Nov 7, 2007 1:52 AM, Gael Varoquaux <gae...@no...> wrote: > On Tue, Nov 06, 2007 at 09:00:23PM -0500, Darren Dale wrote: > > I have not committed my work to svn yet. I wanted to get some feedback on > > points 1 and 2 first. Is it acceptable to use traits internally, but not > > expose it to the end user? I think the answer is yes, and that this is even a > > benefit. If we want to make it an external dependency we can strip the > > package out without impacting any user code. > > I agree that by itself this is a benefit. It might also be interesting to > discuss this usecase on the enthought mailing list. If you tell these > guys that this is an important usecase, that projects like ipython, > matplotlib, would like to use traits as an internal dependency, but would > still like to be able to use the benefit of traits when interfacing with > other libraries, there might be a solution. I am not wild on the idea of an "internal dependency". Since this is the first step in providing traitified mpl properties, and users will presumably want to be able to set event handlers on these properties, etc, it seems best to me if the users are exposed to a proper installation of enthought traits. Is there a reason not to treat traits like we do dateutil and pytz: check at runtime if it is installed and if not install it as enthought.traits using Gael's tarball? If traits is already installed and the version is too old, complain and fail figuring if users have installed it before, they have the wherewithall to upgrade. This gets enthought.traits (preferably with traits.ui included) on more platforms. Ideally we would set up our install to play nicely with packagers like debian, so they can get dateutil, pytz and enthought.traits through their normal dependency handlers. We are simply trying to keep the barrier of installation as low as possible for the typical mpl user. JDH JDH
On Wednesday 07 November 2007 09:25:51 am John Hunter wrote: > On Nov 7, 2007 1:52 AM, Gael Varoquaux <gae...@no...> wrote: > > On Tue, Nov 06, 2007 at 09:00:23PM -0500, Darren Dale wrote: > > > I have not committed my work to svn yet. I wanted to get some feedback > > > on points 1 and 2 first. Is it acceptable to use traits internally, but > > > not expose it to the end user? I think the answer is yes, and that this > > > is even a benefit. If we want to make it an external dependency we can > > > strip the package out without impacting any user code. > > > > I agree that by itself this is a benefit. It might also be interesting to > > discuss this usecase on the enthought mailing list. If you tell these > > guys that this is an important usecase, that projects like ipython, > > matplotlib, would like to use traits as an internal dependency, but would > > still like to be able to use the benefit of traits when interfacing with > > other libraries, there might be a solution. > > I am not wild on the idea of an "internal dependency". Since this is > the first step in providing traitified mpl properties, and users will > presumably want to be able to set event handlers on these properties, > etc, it seems best to me if the users are exposed to a proper > installation of enthought traits. Is there a reason not to treat > traits like we do dateutil and pytz: check at runtime if it is > installed and if not install it as enthought.traits using Gael's > tarball? If traits is already installed and the version is too old, > complain and fail figuring if users have installed it before, they > have the wherewithall to upgrade. > > This gets enthought.traits (preferably with traits.ui included) on > more platforms. Ideally we would set up our install to play nicely > with packagers like debian, so they can get dateutil, pytz and > enthought.traits through their normal dependency handlers. We are > simply trying to keep the barrier of installation as low as possible > for the typical mpl user. I will try this approach tonight.
On Wed, Nov 07, 2007 at 08:25:51AM -0600, John Hunter wrote: > I am not wild on the idea of an "internal dependency". Since this is > the first step in providing traitified mpl properties, and users will > presumably want to be able to set event handlers on these properties, > etc, it seems best to me if the users are exposed to a proper > installation of enthought traits. I agree with that. This would clearly be a temporary solution waiting for better packaging of traits. > Is there a reason not to treat traits like we do dateutil and pytz: > check at runtime if it is installed and if not install it as > enthought.traits using Gael's tarball? This does not seem a terribly good solution. First of all, I assume you mean install time rather than run-time, as I can't see this happening at run-time for a question of permission. Package management is not such an easy task, as the endeavour of setup_tools has shown. Of course the solution for platforms with proper package managers, is to use this package manager, but when there isn't one, try to provide one, even a simple one, is a risky endeavour. If you are going to try to do this, I would try to do it through setuptools, maybe by including the relevant code from setuptools in MPL. Then there is the problem of compiling. Traits has a bit of compiled code, and not everybody has a compiler installed. So you have to provide binary packages for the major platforms. This also points towards eggs, as eggs are binary packages (eggs are nothing more than a zipped python package), and we are already building binary eggs for traits and Co. > This gets enthought.traits (preferably with traits.ui included) on > more platforms. Yes, the big and important work is to get traits and traitsUI packaged natively on more platforms. Ondrej Certik has started debian packages, but he does not have much time to work on them, and they are progressing slowly. Some people have offered packaging for Fedora, but the problem is the same. If you know people who are willing to do some packaging work, we need some help :->. > We are simply trying to keep the barrier of installation as low as > possible for the typical mpl user. This is very important, indeed. Can you tell us a bit more what you plan to use traits for exactly (what are the usecases) ? It might help me understand how to find a beautiful solution to this chicken and eggs problem. Gaël
And again, if MPL becomes "traitified", how will that effect users that need to roll executables for distribution? On Nov 7, 2007 9:39 AM, Gael Varoquaux <gae...@no...> wrot= e: > On Wed, Nov 07, 2007 at 08:25:51AM -0600, John Hunter wrote: > > I am not wild on the idea of an "internal dependency". Since this is > > the first step in providing traitified mpl properties, and users will > > presumably want to be able to set event handlers on these properties, > > etc, it seems best to me if the users are exposed to a proper > > installation of enthought traits. > > I agree with that. This would clearly be a temporary solution waiting for > better packaging of traits. > > > Is there a reason not to treat traits like we do dateutil and pytz: > > check at runtime if it is installed and if not install it as > > enthought.traits using Gael's tarball? > > This does not seem a terribly good solution. First of all, I assume you > mean install time rather than run-time, as I can't see this happening at > run-time for a question of permission. Package management is not such an > easy task, as the endeavour of setup_tools has shown. Of course the > solution for platforms with proper package managers, is to use this > package manager, but when there isn't one, try to provide one, even a > simple one, is a risky endeavour. If you are going to try to do this, I > would try to do it through setuptools, maybe by including the relevant > code from setuptools in MPL. > > Then there is the problem of compiling. Traits has a bit of compiled > code, and not everybody has a compiler installed. So you have to provide > binary packages for the major platforms. This also points towards eggs, > as eggs are binary packages (eggs are nothing more than a zipped python > package), and we are already building binary eggs for traits and Co. > > > This gets enthought.traits (preferably with traits.ui included) on > > more platforms. > > Yes, the big and important work is to get traits and traitsUI packaged > natively on more platforms. Ondrej Certik has started debian packages, > but he does not have much time to work on them, and they are progressing > slowly. Some people have offered packaging for Fedora, but the problem is > the same. If you know people who are willing to do some packaging work, > we need some help :->. > > > We are simply trying to keep the barrier of installation as low as > > possible for the typical mpl user. > > This is very important, indeed. > > Can you tell us a bit more what you plan to use traits for exactly (what > are the usecases) ? It might help me understand how to find a beautiful > solution to this chicken and eggs problem. > > Ga=EBl > > > ------------------------------------------------------------------------- > This SF.net email is sponsored by: Splunk Inc. > Still grepping through log files to find problems? Stop. > Now Search log events and configuration files using AJAX and a browser. > Download your FREE copy of Splunk now >> http://get.splunk.com/ > _______________________________________________ > Matplotlib-devel mailing list > Mat...@li... > https://lists.sourceforge.net/lists/listinfo/matplotlib-devel >
On Wednesday 07 November 2007 10:08:01 am william ratcliff wrote: > And again, if MPL becomes "traitified", how will that effect users > that need to roll executables for distribution? We are considering changes that would require testing before we actually ma= de=20 a release that would effect users. > On Nov 7, 2007 9:39 AM, Gael Varoquaux <gae...@no...>=20 wrote: > > On Wed, Nov 07, 2007 at 08:25:51AM -0600, John Hunter wrote: > > > I am not wild on the idea of an "internal dependency". Since this is > > > the first step in providing traitified mpl properties, and users will > > > presumably want to be able to set event handlers on these properties, > > > etc, it seems best to me if the users are exposed to a proper > > > installation of enthought traits. > > > > I agree with that. This would clearly be a temporary solution waiting f= or > > better packaging of traits. > > > > > Is there a reason not to treat traits like we do dateutil and pytz: > > > check at runtime if it is installed and if not install it as > > > enthought.traits using Gael's tarball? > > > > This does not seem a terribly good solution. First of all, I assume you > > mean install time rather than run-time, as I can't see this happening at > > run-time for a question of permission. Package management is not such an > > easy task, as the endeavour of setup_tools has shown. Of course the > > solution for platforms with proper package managers, is to use this > > package manager, but when there isn't one, try to provide one, even a > > simple one, is a risky endeavour. If you are going to try to do this, I > > would try to do it through setuptools, maybe by including the relevant > > code from setuptools in MPL. > > > > Then there is the problem of compiling. Traits has a bit of compiled > > code, and not everybody has a compiler installed. So you have to provide > > binary packages for the major platforms. This also points towards eggs, > > as eggs are binary packages (eggs are nothing more than a zipped python > > package), and we are already building binary eggs for traits and Co. > > > > > This gets enthought.traits (preferably with traits.ui included) on > > > more platforms. > > > > Yes, the big and important work is to get traits and traitsUI packaged > > natively on more platforms. Ondrej Certik has started debian packages, > > but he does not have much time to work on them, and they are progressing > > slowly. Some people have offered packaging for Fedora, but the problem = is > > the same. If you know people who are willing to do some packaging work, > > we need some help :->. > > > > > We are simply trying to keep the barrier of installation as low as > > > possible for the typical mpl user. > > > > This is very important, indeed. > > > > Can you tell us a bit more what you plan to use traits for exactly (what > > are the usecases) ? It might help me understand how to find a beautiful > > solution to this chicken and eggs problem. > > > > Ga=EBl > > > > > > -----------------------------------------------------------------------= =2D- > > This SF.net email is sponsored by: Splunk Inc. > > Still grepping through log files to find problems? Stop. > > Now Search log events and configuration files using AJAX and a browser. > > Download your FREE copy of Splunk now >> http://get.splunk.com/ > > _______________________________________________ > > Matplotlib-devel mailing list > > Mat...@li... > > https://lists.sourceforge.net/lists/listinfo/matplotlib-devel > > ------------------------------------------------------------------------- > This SF.net email is sponsored by: Splunk Inc. > Still grepping through log files to find problems? Stop. > Now Search log events and configuration files using AJAX and a browser. > Download your FREE copy of Splunk now >> http://get.splunk.com/ > _______________________________________________ > Matplotlib-devel mailing list > Mat...@li... > https://lists.sourceforge.net/lists/listinfo/matplotlib-devel =2D-=20 Darren S. Dale, Ph.D. Staff Scientist Cornell High Energy Synchrotron Source Cornell University 275 Wilson Lab Rt. 366 & Pine Tree Road Ithaca, NY 14853 dar...@co... office: (607) 255-3819 fax: (607) 255-9001 http://www.chess.cornell.edu
On Nov 7, 2007, at 7:25 AM, John Hunter wrote: > Is there a reason not to treat > traits like we do dateutil and pytz: check at runtime if it is > installed and if not install it as enthought.traits using Gael's > tarball? I don't see the check-then-tarball as a feature, but as a significant complication that I have to be aware of when I'm building MatPlotLib. I embed MatPlotLib inside a Macintosh application, so there is a significant amount of packaging and verification that I need to do to ensure a seamless installation for an end-user. I need to know more about Python culture and Python packages, I suppose, but for the moment I'm trying to keep things as straightforward as possible. So for what it's worth, my vote is for limited cleverness in the attempt to resolve dependencies. Let the compile fail. Tell them they need a "real" enthought traits package. Then they can use their package management to make it right.
On Nov 7, 2007 12:01 PM, Boyd Waters <bw...@nr...> wrote: > Tell them they need a "real" enthought traits package. Then they can > use their package management to make it right. This is of course assuming their package management has enthought.traits, which by and large, it won't. I am sympathetic to this view, but our recent experience has been that getting enthought.traits to install properly even for matplotlib developers took a fair amount of work and consultation with the enthought dev team. My concern is that 50% of new and existing users would simply not install or upgrade under these circumstances. The difficultly getting scipy installed followed it for years (that is why mpl never depended on it), and we is in danger of obtaining a similar "difficult to install so I won't upgrade" reputation. I would rather inconvenience a sophisticated developer like you (embedding mpl in an app, following the dev list) primarily because you can take it and work with it. For a newbie, having the install fail and being pointed to a setuptools process they don't understand that may also fail may be the difference between their using matplotlib and not. If you could set a simple flag or environment variable to turn off all the optional install "smarts" that should be enough, no?
On Nov 7, 2007, at 12:04 PM, John Hunter wrote: > On Nov 7, 2007 12:01 PM, Boyd Waters <bw...@nr...> wrote: > >> Tell them they need a "real" enthought traits package. Then they can >> use their package management to make it right. > > This is of course assuming their package management has > enthought.traits, which by and large, it won't. > > I am sympathetic to this view, but our recent experience has been that > getting enthought.traits to install properly even for matplotlib > developers took a fair amount of work and consultation with the > enthought dev team. Ouch, was it really that hard? I don't remember it being that hard, but I am almost certainly forgetting something about the traits tweaks that were needed to get MatPlotLib built... maybe I didn't upgrade MPL after running into MPL build errors with traits. Hmm. FWIW, here are the MacPorts for "standard" enthought traits that I cobbled up :
On Wednesday 07 November 2007 01:01:52 pm Boyd Waters wrote: > On Nov 7, 2007, at 7:25 AM, John Hunter wrote: > > Is there a reason not to treat > > traits like we do dateutil and pytz: check at runtime if it is > > installed and if not install it as enthought.traits using Gael's > > tarball? > > I don't see the check-then-tarball as a feature, but as a significant > complication that I have to be aware of when I'm building MatPlotLib. Does the way matplotlib handles pytz and datetime present significant complications as well? > So for what it's worth, my vote is for limited cleverness in the > attempt to resolve dependencies. Let the compile fail. It follows then that matplotlib should not provide AGG, pyparsing, pytz, datetime, subprocess ... We try to find a balance, checking at compile time (not runtime) for required dependencies and building what is needed. Who has time to spend an afternoon working through a series of compiler errors to track down the missing deps? > Tell them they need a "real" enthought traits package. Then they can > use their package management to make it right. But not all systems have a package manager, and not all package managers have support for traits. Darren
On Nov 7, 2007, at 12:08 PM, Darren Dale wrote: >> I don't see the check-then-tarball as a feature, but as a significant >> complication that I have to be aware of when I'm building MatPlotLib. > > Does the way matplotlib handles pytz and datetime present significant > complications as well? > >> So for what it's worth, my vote is for limited cleverness in the >> attempt to resolve dependencies. Let the compile fail. > > It follows then that matplotlib should not provide AGG, pyparsing, > pytz, > datetime, subprocess ... Well, OK, I think you are right: the included things do actually help very much. Sorry. I think it's a testament effectiveness of the MPL-included things that I am not always aware of the bits. I don't let a package manager mess with installing AGG anymore. I just use the MPL-included one for MPL. Because of version drift, mostly. So never mind: I like the MPL-provisioned things as "internal" dependencies, if it works the same as the current MPL-AGG datetime etc. - boyd Boyd Waters Scientific Programmer National Radio Astronomy Observatory Socorro, New Mexico http://www.aoc.nrao.edu/~bwaters
> But not all systems have a package manager, and not all package managers have > support for traits. Does *anything* or does *anyone* outside of enthought actually support traits now? I know Fernando has said he is sold on it for ipython, but I did not see it in the latest release. While I greatly appreciate the time and effort you have put into developing the first mpl use of traits, Darren, I remain unconvinced that now is the time--if at all--to tie mpl irrevocably to traits. Eric
On Wednesday 07 November 2007 02:19:25 pm Eric Firing wrote: > > But not all systems have a package manager, and not all package managers > > have support for traits. > > Does *anything* or does *anyone* outside of enthought actually support > traits now? I know Fernando has said he is sold on it for ipython, but > I did not see it in the latest release. I'm not sure. There was some talk about debs and rpms, but I dont know if or when traits would be supported by the package managers. > While I greatly appreciate the time and effort you have put into > developing the first mpl use of traits, Darren, I remain unconvinced > that now is the time--if at all--to tie mpl irrevocably to traits. I appreciate the acknowledgement, and your cautious approach toward changing mpl. I see some benefit in the new config package, even without traits. If we wanted, I think we could modify it so it doesnt depend on traits, but traits could be added in the future. The addition of traits would only happen after the transforms branch has had time to settle. At that time, if we feel like we still need convincing that this is the right thing to do, we can create a traits branch and see how things pan out. By the time we are ready to roll out the final product, maybe everybody will be converting to py3k and we'll have a whole new can of worms to consider. Darren
Darren Dale wrote: > Does the way matplotlib handles pytz and datetime present significant > complications as well? not too bad, but pytz and datetime are already pretty easy to install, self-contained packages. > But not all systems have a package manager, and not all package managers have > support for traits. I really don't think there is much choice here. Until (if) traits is widely available in an easy-to-install form (binaries for Windows and OS-X, "just works" with easy-install, rpms, and debs for Fedora, Ubuntu, Debian) MPL is going to have to keep a copy internally, just like it does for Agg, etc. The first step to the easy-to-install form is for: setup.py build to "just work" on all platforms that are setup right to build python extensions (have the right compiler, etc) -- is that the case yet? Once it is, it's not hard for various folks to build binaries for various platforms. As for bundling -- if it's installed and working, py2exe and py2app should just work -- if they don't, then a little tweaking may be needed, which we can make into a recipe for py2app, and a Wiki page for py2exe (is there anything like recipes for py2exe?) -Chris -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chr...@no...
On Wednesday 07 November 2007 02:31:47 pm Christopher Barker wrote: > Darren Dale wrote: > > Does the way matplotlib handles pytz and datetime present significant > > complications as well? > > not too bad, but pytz and datetime are already pretty easy to install, > self-contained packages. > > > But not all systems have a package manager, and not all package managers > > have support for traits. > > I really don't think there is much choice here. Until (if) traits is > widely available in an easy-to-install form (binaries for Windows and > OS-X, "just works" with easy-install, rpms, and debs for Fedora, Ubuntu, > Debian) MPL is going to have to keep a copy internally, just like it > does for Agg, etc. > > The first step to the easy-to-install form is for: > > setup.py build > > to "just work" on all platforms that are setup right to build python > extensions (have the right compiler, etc) -- is that the case yet? It is the case on the platform I have tested. We will test others soon.
Darren Dale wrote: > I have been working on updating the trunk to provide enthought.traits version > 2.6b1. backend_driver.py is running without exceptions using the traited > config package with the internal traits package. > > Issues: > > 1) there are lots of absolute package imports scattered throughout traits' > code. I worked around this by adding a line to matplotlib/__init__.py: > > sys.path.append(os.path.split(__file__)[0]) > > This lets matplotlib access enthought.traits without modifying enthoughts code > (anymore than Gael had already done by stripping the pkg_resources imports). > > 2) When I tried updating rc_traits.py to import matplotlib.enthought.traits > instead of enthought.traits (which isnt on the PYTHONPATH), I discovered a > problem: > > enthought.traits.trait_errors.TraitError: The 'parents_items' trait of a > ViewElements instance must be a TraitListEvent, but a value of > <matplotlib.enthought.traits.trait_handlers.TraitListEvent object at > 0x850454c> was specified. > > So traits would be a behind-the-scenes package, for internal mpl use only. Regarding (1) and (2), and asking out of ignorance: 1) If an internal version of traits is to be used, how hard would it be to patch it in such a way that it *could* be used externally? 2) Does Gael's version already get around the slow initialization problem? (I presume so--it was pkg-resources that was causing the trouble, wasn't it?) I think this is important. Some mpl applications involve running simple scripts many times, so startup time matters. I would not want to see traits as an external dependency if that brought with it the startup lag--apart from all the other questions associated with making it an external dependency in any of its present forms. > > 3) We can not include traits-3 without either adding setuptools as an external > dependency (which is already true for python-2.3 users) or monkey-patching > distutils. traits-3 includes some pyrex code, which standard distutils does > not recognize. Is it a viable alternative to add pyrex as an external dependency? Pyrex.Distutils makes it easy to use pyrex modules via otherwise standard setup.py scripts. And pyrex itself is pure python, very easy to install. Part of my motivation in bringing this up is that I have found pyrex extremely useful for quick speedups and for interfacing to larger chunks of C code. (And I have a strong personal bias towards C and pyrex for extension code in preference to C++ and any of the mechanisms used to provide bindings for C++ code. I find the former vastly more readable, understandable, and hence maintainable.) Pyrex (actually its variant, cython) is already essential to basemap, and I am hoping for increasing basemap integration, as well as judicious use of extension code to make the transforms branch lightening-fast. Eric
On Wed, Nov 07, 2007 at 10:34:18AM -1000, Eric Firing wrote: > 1) If an internal version of traits is to be used, how hard would it be > to patch it in such a way that it *could* be used externally? I think pretty hard, but you would have to ask enthought's dev. > 2) Does Gael's version already get around the slow initialization > problem? (I presume so--it was pkg-resources that was causing the > trouble, wasn't it?) As far as I can tell, yes. Setuptools is no longer a run time dependency, just a compile time dependency. It is imported nowwhere in the code base. > I think this is important. I think it is _very_ important. Gaël
On Wednesday 07 November 2007 03:34:18 pm Eric Firing wrote:
> Darren Dale wrote:
> > I have been working on updating the trunk to provide enthought.traits
> > version 2.6b1. backend_driver.py is running without exceptions using the
> > traited config package with the internal traits package.
> >
> > Issues:
> >
> > 1) there are lots of absolute package imports scattered throughout
> > traits' code. I worked around this by adding a line to
> > matplotlib/__init__.py:
> >
> > sys.path.append(os.path.split(__file__)[0])
> >
> > This lets matplotlib access enthought.traits without modifying enthoughts
> > code (anymore than Gael had already done by stripping the pkg_resources
> > imports).
> >
> > 2) When I tried updating rc_traits.py to import
> > matplotlib.enthought.traits instead of enthought.traits (which isnt on
> > the PYTHONPATH), I discovered a problem:
> >
> > enthought.traits.trait_errors.TraitError: The 'parents_items' trait of a
> > ViewElements instance must be a TraitListEvent, but a value of
> > <matplotlib.enthought.traits.trait_handlers.TraitListEvent object at
> > 0x850454c> was specified.
> >
> > So traits would be a behind-the-scenes package, for internal mpl use
> > only.
>
> Regarding (1) and (2), and asking out of ignorance:
>
> 1) If an internal version of traits is to be used, how hard would it be
> to patch it in such a way that it *could* be used externally?
I think this would be too significant an undertaking to seriously consider.
The easier part is replacing all the "from enthought" imports in the library
to "from matplotlib.enthought", and they are scattered throughout the code.
The harder part seems to be the error messages due to traits expecting
enthought.traits objects, not matplotlib.enthought.traits objects.
Johns suggestion that we try to deal with traits the same way we do pytz and
datetime should solve this, with a caveat:
> 2) Does Gael's version already get around the slow initialization
> problem? (I presume so--it was pkg-resources that was causing the
> trouble, wasn't it?) I think this is important. Some mpl applications
> involve running simple scripts many times, so startup time matters. I
> would not want to see traits as an external dependency if that brought
> with it the startup lag--apart from all the other questions associated
> with making it an external dependency in any of its present forms.
I think it does get us around the initialization problem, although I need to
run backend_driver once I get the installation worked out to be sure.
However, if the user already has traits installed on their system, it will
not be stripped of pkg_imports, so the work-around would only apply to the
mpl-shipped version of traits.
> > 3) We can not include traits-3 without either adding setuptools as an
> > external dependency (which is already true for python-2.3 users) or
> > monkey-patching distutils. traits-3 includes some pyrex code, which
> > standard distutils does not recognize.
>
> Is it a viable alternative to add pyrex as an external dependency?
> Pyrex.Distutils makes it easy to use pyrex modules via otherwise
> standard setup.py scripts. And pyrex itself is pure python, very easy
> to install.
I usually look to you and John for answers to questions like these. I think I
tried this yesterday, and ran into a problem. You have to import build_ext
from Pyrex.Distutils, and then pass "cmdclass = {'build_ext': build_ext}" to
setup(). The build failed, maybe because build_ext was being applied to
non-pyrex extension code, or maybe because I didn't know what I was doing.
> Part of my motivation in bringing this up is that I have found pyrex
> extremely useful for quick speedups and for interfacing to larger chunks
> of C code. (And I have a strong personal bias towards C and pyrex for
> extension code in preference to C++ and any of the mechanisms used to
> provide bindings for C++ code. I find the former vastly more readable,
> understandable, and hence maintainable.)
Would I be able to understand it? :)
> Pyrex (actually its variant,
> cython) is already essential to basemap, and I am hoping for increasing
> basemap integration, as well as judicious use of extension code to make
> the transforms branch lightening-fast.
On Wednesday 07 November 2007 3:54:36 pm Darren Dale wrote: > On Wednesday 07 November 2007 03:34:18 pm Eric Firing wrote: > > Darren Dale wrote: > > > I have been working on updating the trunk to provide enthought.traits > > > version 2.6b1. backend_driver.py is running without exceptions using > > > the traited config package with the internal traits package. [...] > > 2) Does Gael's version already get around the slow initialization > > problem? (I presume so--it was pkg-resources that was causing the > > trouble, wasn't it?) I think this is important. Some mpl applications > > involve running simple scripts many times, so startup time matters. I > > would not want to see traits as an external dependency if that brought > > with it the startup lag--apart from all the other questions associated > > with making it an external dependency in any of its present forms. > > I think it does get us around the initialization problem, although I need > to run backend_driver once I get the installation worked out to be sure. I removed the old enthought package from lib/matplotlib, and added the setuptools-less 2.6b1 package to lib/. It now installs like pytz or dateutil, directly into site-packages and only if it is not already available. The only modification I made to the enthought code was in the version files, which return '2.6b1-mpl' instead of ''. The sys.path workaround mentioned in the original post is no longer necessary and has been removed. Building does not seem to be a problem, just do the usual python setup.py build, etc. I ran backend_driver on my not-so-speedy home computer: with traited config: Backend Agg took 2.77 minutes to complete template ratio 1.748, template residual 1.187 Backend PS took 2.64 minutes to complete template ratio 1.666, template residual 1.055 Backend Template took 1.59 minutes to complete template ratio 1.000, template residual 0.000 Backend PDF took 3.35 minutes to complete template ratio 2.112, template residual 1.764 Backend SVG took 2.90 minutes to complete template ratio 1.826, template residual 1.309 and without: Backend Agg took 2.59 minutes to complete template ratio 1.898, template residual 1.226 Backend PS took 2.29 minutes to complete template ratio 1.675, template residual 0.921 Backend Template took 1.36 minutes to complete template ratio 1.000, template residual 0.000 Backend PDF took 2.92 minutes to complete template ratio 2.139, template residual 1.555 Backend SVG took 2.91 minutes to complete template ratio 2.129, template residual 1.541 changes in svn-4156. Darren
Darren Dale wrote: > Building does not seem to be a problem, just do the usual python setup.py > build, etc. I ran backend_driver on my not-so-speedy home computer: Darren, Well done! I blew away other versions of traits, updated mpl, built and installed, and traits appeared. No pain. The not quite as good news is that on my Lenovo T60 laptop, the Template takes 0.57 minutes with traited config versus 0.45 with old config. Not a huge difference, and roughly consistent with your timing, but still a penalty. I also see about 20% in simple_plot using Agg. It's probably tolerable. (The comparison was made by switching only the NEWCONFIG variable in __init__.py.) Eric > > with traited config: > > Backend Agg took 2.77 minutes to complete > template ratio 1.748, template residual 1.187 > Backend PS took 2.64 minutes to complete > template ratio 1.666, template residual 1.055 > Backend Template took 1.59 minutes to complete > template ratio 1.000, template residual 0.000 > Backend PDF took 3.35 minutes to complete > template ratio 2.112, template residual 1.764 > Backend SVG took 2.90 minutes to complete > template ratio 1.826, template residual 1.309 > > > and without: > > Backend Agg took 2.59 minutes to complete > template ratio 1.898, template residual 1.226 > Backend PS took 2.29 minutes to complete > template ratio 1.675, template residual 0.921 > Backend Template took 1.36 minutes to complete > template ratio 1.000, template residual 0.000 > Backend PDF took 2.92 minutes to complete > template ratio 2.139, template residual 1.555 > Backend SVG took 2.91 minutes to complete > template ratio 2.129, template residual 1.541 > > > changes in svn-4156. > > Darren
On Thursday 08 November 2007 03:16:14 am Eric Firing wrote: > Darren Dale wrote: > > Building does not seem to be a problem, just do the usual python setup.= py > > build, etc. I ran backend_driver on my not-so-speedy home computer: > > Darren, > > Well done! I blew away other versions of traits, updated mpl, built and > installed, and traits appeared. No pain. > > The not quite as good news is that on my Lenovo T60 laptop, the Template > takes 0.57 minutes with traited config versus 0.45 with old config. Not > a huge difference, and roughly consistent with your timing, but still a > penalty. I also see about 20% in simple_plot using Agg. It's probably > tolerable. (The comparison was made by switching only the NEWCONFIG > variable in __init__.py.) Yes, these results are not consistent with what Fernando reported after=20 working with Dave Peterson: # Using traits maqroll[mpl-traits-debug]> time ./simple_plot.py *** Using Traits!!! 1.844u 0.212s 0:02.13 96.2% =A0 =A0 0+0k 0+0io 0pf+0w maqroll[mpl-traits-debug]> time ./simple_plot.py *** Using Traits!!! 1.840u 0.216s 0:02.58 79.4% =A0 =A0 0+0k 0+0io 0pf+0w maqroll[mpl-traits-debug]> time ./simple_plot.py *** Using Traits!!! 1.836u 0.196s 0:02.12 95.2% =A0 =A0 0+0k 0+0io 0pf+0w # NOT Using traits maqroll[mpl-traits-debug]> time ./simple_plot.py 2.200u 0.280s 0:02.67 92.8% =A0 =A0 0+0k 0+0io 0pf+0w maqroll[mpl-traits-debug]> time ./simple_plot.py 2.248u 0.220s 0:02.74 89.7% =A0 =A0 0+0k 0+0io 0pf+0w maqroll[mpl-traits-debug]> time ./simple_plot.py 2.216u 0.244s 0:02.72 90.0% =A0 =A0 0+0k 0+0io 0pf+0w =46ernando, do you have a record of the changes you guys made at SciPy-2007? > > with traited config: > > > > Backend Agg took 2.77 minutes to complete > > template ratio 1.748, template residual 1.187 > > Backend PS took 2.64 minutes to complete > > template ratio 1.666, template residual 1.055 > > Backend Template took 1.59 minutes to complete > > template ratio 1.000, template residual 0.000 > > Backend PDF took 3.35 minutes to complete > > template ratio 2.112, template residual 1.764 > > Backend SVG took 2.90 minutes to complete > > template ratio 1.826, template residual 1.309 > > > > > > and without: > > > > Backend Agg took 2.59 minutes to complete > > template ratio 1.898, template residual 1.226 > > Backend PS took 2.29 minutes to complete > > template ratio 1.675, template residual 0.921 > > Backend Template took 1.36 minutes to complete > > template ratio 1.000, template residual 0.000 > > Backend PDF took 2.92 minutes to complete > > template ratio 2.139, template residual 1.555 > > Backend SVG took 2.91 minutes to complete > > template ratio 2.129, template residual 1.541
On Nov 8, 2007 7:10 AM, Darren Dale <dar...@co...> wrote: > Fernando, do you have a record of the changes you guys made at SciPy-2007? No, we just went through the codebase by hand and quickly removed anything that looked like it would do namespace packages. Once that was out of the way, the traits init was actually *shorter* than the non-traits one, since traits property-like checks are dispatched in a tight, highly optimized C core (ctraits.c) instead of via pure python. Sorry not to have that benchmark code available, at this point I think I'd have to redo it by hand. Cheers, f
Eric Firing wrote: > Darren Dale wrote: > > >> 3) We can not include traits-3 without either adding setuptools as an external >> dependency (which is already true for python-2.3 users) or monkey-patching >> distutils. traits-3 includes some pyrex code, which standard distutils does >> not recognize. >> > > Is it a viable alternative to add pyrex as an external dependency? > Pyrex.Distutils makes it easy to use pyrex modules via otherwise > standard setup.py scripts. And pyrex itself is pure python, very easy > to install. > We can also just package the .c file built by Pyrex whenever the developer changes the .pyx file. That way, there is no further infrastructure to include. On the other hand, as Darren suggested later, one can do what setuptools' build_ext command does without setuptools (if an extension source ends in .pyx, call pyrexc on it. If pyrexc is not available, skip that step. In both cases, then compile the .c file with the c compiler.) Darren, if you're still having trouble with this, I can dig up the old code I had that did this (I switched to setuptools for this, and other stuff, a long time ago).
On Sunday 11 November 2007 7:08:38 pm Andrew Straw wrote: > Eric Firing wrote: > > Darren Dale wrote: > >> 3) We can not include traits-3 without either adding setuptools as an > >> external dependency (which is already true for python-2.3 users) or > >> monkey-patching distutils. traits-3 includes some pyrex code, which > >> standard distutils does not recognize. > > > > Is it a viable alternative to add pyrex as an external dependency? > > Pyrex.Distutils makes it easy to use pyrex modules via otherwise > > standard setup.py scripts. And pyrex itself is pure python, very easy > > to install. > > We can also just package the .c file built by Pyrex whenever the > developer changes the .pyx file. That way, there is no further > infrastructure to include. On the other hand, as Darren suggested later, > one can do what setuptools' build_ext command does without setuptools > (if an extension source ends in .pyx, call pyrexc on it. If pyrexc is > not available, skip that step. In both cases, then compile the .c file > with the c compiler.) Darren, if you're still having trouble with this, > I can dig up the old code I had that did this (I switched to setuptools > for this, and other stuff, a long time ago). Thanks for the offer. For now, I think it is best that we use traits-2.6, which doesnt have any pyrex code. Once traits-3 stabilizes and is released, I'll try what you have suggested. Darren