matplotlib-devel Mailing List for matplotlib

On Sun, Jan 3, 2010 at 4:23 AM, David Cournapeau <cou...@gm...> wrote:
> On Sun, Jan 3, 2010 at 8:05 PM, Nathaniel Smith <nj...@po...> wrote:
>> What I do -- and documented for people in my lab to do -- is set up
>> one virtualenv in my user account, and use it as my default python. (I
>> 'activate' it from my login scripts.) The advantage of this is that
>> easy_install (or pip) just works, without any hassle about permissions
>> etc.
>
> It just works if you happen to be able to build everything from
> sources. That alone means you ignore the majority of users I intend to
> target.
>
> No other community (except maybe Ruby) push those isolated install
> solutions as a general deployment solutions. If it were such a great
> idea, other people would have picked up those solutions.
AFAICT, R works more-or-less identically (once I convinced it to use a
per-user library directory); install.packages() builds from source,
and doesn't automatically pull in and build random C library
dependencies.
I'm not advocating the 'every app in its own world' model that
virtualenv's designers had min mind, but virtualenv is very useful to
give each user their own world. Normally I only use a fraction of
virtualenv's power this way, but sometimes it's handy that they've
solved the more general problem -- I can easily move my environment
out of the way and rebuild if I've done something stupid, or
experiment with new python versions in isolation, or whatever. And
when you *do* have to reproduce some old environment -- if only to
test that the new improved environment gives the same results -- then
it's *really* handy.
>> This should be easier, but I think the basic approach is sound.
>> "Integration with the package system" is useless; the advantage of
>> distribution packages is that distributions can provide a single
>> coherent system with consistent version numbers across all packages,
>> etc., and the only way to "integrate" with that is to, well, get the
>> packages into the distribution.
>
> Another way is to provide our own repository for a few major
> distributions, with automatically built packages. This is how most
> open source providers work. Miguel de Icaza explains this well:
>
> http://tirania.org/blog/archive/2007/Jan-26.html
>
> I hope we will be able to reuse much of the opensuse build service
> infrastructure.
Sure, I'm aware of the opensuse build service, have built third-party
packages for my projects, etc. It's a good attempt, but also has a lot
of problems, and when talking about scientific software it's totally
useless to me :-). First, I don't have root on our compute cluster.
Second, even if I did I'd be very leery about installing third-party
packages because there is no guarantee that the version numbering will
be consistent between the third-party repo and the real distro repo --
suppose that the distro packages 0.1, then the third party packages
0.2, then the distro packages 0.3, will upgrades be seamless? What if
the third party screws up the version numbering at some point? Debian
has "epochs" to deal with this, but third-parties can't use them and
maintain compatibility. What if the person making the third party
packages is not an expert on these random distros that they don't even
use? Will bug reporting tools work properly? Distros are complicated.
Third, while we shouldn't advocate that people screw up backwards
compatibility, version skew is a real issue. If I need one version of
a package and my lab-mate needs another and we have submissions due
tomorrow, then filing bugs is a great idea but not a solution. Fourth,
even if we had expert maintainers taking care of all these third-party
packages and all my concerns were answered, I couldn't convince our
sysadmin of that; he's the one who'd have to clean up if something
went wrong we don't have a big budget for overtime.
Let's be honest -- scientists, on the whole, suck at IT
infrastructure, and small individual packages are not going to be very
expertly put together. IMHO any real solution should take this into
account, keep them sandboxed from the rest of the system, and focus on
providing the most friendly and seamless sandbox possible.
>> On another note, I hope toydist will provide a "source prepare" step,
>> that allows arbitrary code to be run on the source tree. (For, e.g.,
>> cython->C conversion, ad-hoc template languages, etc.) IME this is a
>> very common pain point with distutils; there is just no good way to do
>> it, and it has to be supported in the distribution utility in order to
>> get everything right. In particular:
>> -- Generated files should never be written to the source tree
>> itself, but only the build directory
>> -- Building from a source checkout should run the "source prepare"
>> step automatically
>> -- Building a source distribution should also run the "source
>> prepare" step, and stash the results in such a way that when later
>> building the source distribution, this step can be skipped. This is a
>> common requirement for user convenience, and necessary if you want to
>> avoid arbitrary code execution during builds.
>
> Build directories are hard to implement right. I don't think toydist
> will support this directly. IMO, those advanced builds warrant a real
> build tool - one main goal of toydist is to make integration with waf
> or scons much easier. Both waf and scons have the concept of a build
> directory, which should do everything you described.
Maybe I was unclear -- proper build directory handling is nice,
Cython/Pyrex's distutils integration get it wrong (not their fault,
distutils is just impossible to do anything sensible with, as you've
said), and I've never found build directories hard to implement
(perhaps I'm missing something). But what I'm really talking about is
having a "pre-build" step that integrates properly with the source and
binary packaging stages, and that's not something waf or scons have
any particular support for, AFAIK.
-- Nathaniel

>> 2) It was pointed out that there is a strange connection between the color
>> cycle and the lines.color rcParam. This connection looks to me like a bit
>> of legacy that can be dropped with little risk of pain in user land, since
>> the default would still be to have the initial color in the cycle (blue) be
>> the same as the default lines.color (blue). The proposed difference is that
>> setting lines.color would have no effect on the color_cycle, and vice-versa.
>> John, I think the present connection dates all the way back to your work
>> around svn r500 or so; please tell me if there is some compelling reason to
>> keep it. It appears to me that breaking the connection would make both the
>> code and the actual mpl behavior simpler and less surprising, with no loss
>> of useful functionality.
> 
> I think deprecating lines.color and making the first element of the
> cycle be the default color makes the most sense.
The only problem is that lines.color is the default for LineCollection 
and Line2D, both of which are fairly separate from Axes, so having them 
default to rcParams['axes.color_cycle'][0] seems a little odd.
Eric

On Sun, Jan 3, 2010 at 2:39 PM, Eric Firing <ef...@ha...> wrote:
> 1) Should the the color_cycle be in the axes group? Although it affects
> lines, it is defined only at the Axes level, and affects only lines drawn by
> plot.
>
> Alternative: since it affects only plot, should there be a new "plot" group
> to make that explicit, so it would be rcParams['plot.color_cycle']?
I think it is better to do at the Axes level, because even if plot is
the only command to do this now, I could imagine others commands
wanting to use it (fill_between, bar?). I have had people asking me
just for access to the list so they could use it for some purpose or
another, so having the list easily accessible is desirable.
> 2) It was pointed out that there is a strange connection between the color
> cycle and the lines.color rcParam. This connection looks to me like a bit
> of legacy that can be dropped with little risk of pain in user land, since
> the default would still be to have the initial color in the cycle (blue) be
> the same as the default lines.color (blue). The proposed difference is that
> setting lines.color would have no effect on the color_cycle, and vice-versa.
> John, I think the present connection dates all the way back to your work
> around svn r500 or so; please tell me if there is some compelling reason to
> keep it. It appears to me that breaking the connection would make both the
> code and the actual mpl behavior simpler and less surprising, with no loss
> of useful functionality.
I think deprecating lines.color and making the first element of the
cycle be the default color makes the most sense.
> 3) Would it make sense to add color_cycle to the Axes API, so that it can be
> set directly for a given axes as an alternative to going through the rc
> mechanism? Right now it can be set via a function, but must be set before
> Axes creation; this doesn't make sense to me. It would make more sense as
> an Axes property that could be set at any time, and would apply to
> subsequent calls to plot.
It might be best to make this a class level attribute, that way people
can tweak it globally or at the instance level, and have access to the
list *before* the Axes instance is created. The downside of this is it
falls outside of the usual properties pattern that people can access
via the normal introspection facilities.
The linestyles cycle question is as you say a good bit harder, and
would best be served by having some sort of configurable color scheme,
where one could set grayscale or something like that to get default
black/white/gray colors and styles for the cycle.
JDH

On Sun, Jan 3, 2010 at 2:41 PM, Gökhan Sever <gok...@gm...> wrote:
> You seemed like forgetting to check-in the qt4_editor_options.svg, because I
> get file not found error:
>
> I[2]: Cannot open file
> '.../matplotlib/lib/matplotlib/mpl-data/images/qt4_editor_options.svg',
> because: No such file or directory
Oops, just added. Thanks for the head's up.
JDH

John Hunter wrote:
> On Sat, Jan 2, 2010 at 12:34 AM, Eric Firing <ef...@ha...> wrote:
>> Jae-Joon Lee wrote:
>>> The error happens because of the *.rst files under doc/examples that
>>> are not in sync with examples/*.py.
>>> Removing that directory (doc/examples) will solve the problem (the
>>> directory will be repopulated when you run make.py again). Here is a
>>> related link.
>>>
>>> http://old.nabble.com/python-make.py-html-failure-tt26894350.html
>> Thank you for the quick response. That was it, exactly. There are
>> still some example script failures, but nothing that stops the build in
>> its tracks.
>>
>>> Maybe we need something like "python make.py clean"?
>> Yes, if the generated files can't all land in the "build" directory (as
>> would seem preferable), then the next-best thing would be to have
>> make.py able to do a thorough cleaning.
> 
> I just do svn-clean whenever I have a problem
> 
> http://svn.collab.net/repos/svn/trunk/contrib/client-side/svn-clean
I didn't know about that. It seems strange that svn has no such command 
built in. Mercurial has "hg purge", so I was expecting to find 
something similar in svn. It seems like a pretty basic operation.
Do you know why the plot directive (I assume that is the cause of the 
problem) cannot put all generated output in build? Or is this a more 
general Sphinx wart?
Eric

On Sun, Jan 3, 2010 at 12:35 PM, John Hunter <jd...@gm...> wrote:
> On Sun, Jan 3, 2010 at 9:28 AM, Darren Dale <dsd...@gm...> wrote:
>
> > Your patch file is backwards, it would revert your changes if applied
> > to the updated code. It doesn't matter, the patch is so small that the
> > changes can just be cut and pasted. Unfortunately, however, I will not
> > have time to review your contribution closely enough to consider
> > committing it for at least two weeks, maybe more (conference and long
> > hours at work). I'll look into it when I get a chance, if someone else
> > doesn't beat me to it.
>
> Hey Pierre, Darren,
>
> I took a stab at this. I put the helper code in a backends.qt4_editor
> package, and put the toolbar button right after the configure subplots
> button. Thanks for the patch and the license change Pierre, and sorry
> it took us so long to incorporate it. Let me know if any of the
> reorganizations are a problem for you.
>
> JDH
>
>
John,
You seemed like forgetting to check-in the qt4_editor_options.svg, because I
get file not found error:
I[2]: Cannot open file
'.../matplotlib/lib/matplotlib/mpl-data/images/qt4_editor_options.svg',
because: No such file or directory
Besides, thanks for including this improvement in the trunk. It was long
time waited :)
>
> ------------------------------------------------------------------------------
> This SF.Net email is sponsored by the Verizon Developer Community
> Take advantage of Verizon's best-in-class app development support
> A streamlined, 14 day to market process makes app distribution fast and
> easy
> Join now and get one step closer to millions of Verizon customers
> http://p.sf.net/sfu/verizon-dev2dev
> _______________________________________________
> Matplotlib-devel mailing list
> Mat...@li...
> https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
>
-- 
Gökhan

http://www.mail-archive.com/mat...@li.../msg14772.html
The above thread includes two reasonable requests:
1) add color cycle as an rcParam
2) add a line style cycle as well
The first of these is easier to implement, although it requires a bit 
more than the patch provided. Questions:
1) Should the the color_cycle be in the axes group? Although it affects 
lines, it is defined only at the Axes level, and affects only lines 
drawn by plot.
Alternative: since it affects only plot, should there be a new "plot" 
group to make that explicit, so it would be rcParams['plot.color_cycle']?
2) It was pointed out that there is a strange connection between the 
color cycle and the lines.color rcParam. This connection looks to me 
like a bit of legacy that can be dropped with little risk of pain in 
user land, since the default would still be to have the initial color in 
the cycle (blue) be the same as the default lines.color (blue). The 
proposed difference is that setting lines.color would have no effect on 
the color_cycle, and vice-versa. John, I think the present connection 
dates all the way back to your work around svn r500 or so; please tell 
me if there is some compelling reason to keep it. It appears to me that 
breaking the connection would make both the code and the actual mpl 
behavior simpler and less surprising, with no loss of useful functionality.
3) Would it make sense to add color_cycle to the Axes API, so that it 
can be set directly for a given axes as an alternative to going through 
the rc mechanism? Right now it can be set via a function, but must be 
set before Axes creation; this doesn't make sense to me. It would make 
more sense as an Axes property that could be set at any time, and would 
apply to subsequent calls to plot.
My sense is that as a matter of design strategy, whenever possible, one 
should be able to use the API, via properties, methods, or functions, to 
 locally set any options for which rcParams gives the global default 
values.
Eric

On Sat, Jan 2, 2010 at 12:34 AM, Eric Firing <ef...@ha...> wrote:
> Jae-Joon Lee wrote:
>> The error happens because of the *.rst files under doc/examples that
>> are not in sync with examples/*.py.
>> Removing that directory (doc/examples) will solve the problem (the
>> directory will be repopulated when you run make.py again). Here is a
>> related link.
>>
>> http://old.nabble.com/python-make.py-html-failure-tt26894350.html
>
> Thank you for the quick response. That was it, exactly. There are
> still some example script failures, but nothing that stops the build in
> its tracks.
>
>>
>> Maybe we need something like "python make.py clean"?
>
> Yes, if the generated files can't all land in the "build" directory (as
> would seem preferable), then the next-best thing would be to have
> make.py able to do a thorough cleaning.
I just do svn-clean whenever I have a problem
http://svn.collab.net/repos/svn/trunk/contrib/client-side/svn-clean
JDH

2010年1月3日 John Hunter <jd...@gm...>:
> On Sun, Jan 3, 2010 at 9:28 AM, Darren Dale <dsd...@gm...> wrote:
>
>> Your patch file is backwards, it would revert your changes if applied
>> to the updated code. It doesn't matter, the patch is so small that the
>> changes can just be cut and pasted. Unfortunately, however, I will not
>> have time to review your contribution closely enough to consider
>> committing it for at least two weeks, maybe more (conference and long
>> hours at work). I'll look into it when I get a chance, if someone else
>> doesn't beat me to it.
>
> Hey Pierre, Darren,
>
> I took a stab at this. I put the helper code in a backends.qt4_editor
> package, and put the toolbar button right after the configure subplots
> button. Thanks for the patch and the license change Pierre, and sorry
> it took us so long to incorporate it. Let me know if any of the
> reorganizations are a problem for you.
>
> JDH
>
That is perfect!
Thanks very much John.
Pierre

2010年1月3日 Darren Dale <dsd...@gm...>:
> On Sun, Jan 3, 2010 at 9:11 AM, Pierre Raybaut <co...@py...> wrote:
>> 2009年12月1日 Darren Dale <dsd...@gm...>:
>>> On Tue, Apr 28, 2009 at 8:18 AM, Pierre Raybaut <co...@py...> wrote:
>>>> Hi all,
>>>>
>>>> I would like to contribute to matplotlib with this enhancement for the
>>>> PyQt4 backend: the idea is to add a toolbar button to configure figure
>>>> options (axes, curves, ...).
>>>>
>>>> It's based on a tiny module called formlayout to generate PyQt4 form
>>>> dialog automatically.
>>>>
>>>> Some screenshots:
>>>> http://code.google.com/p/formlayout/
>>>>
>>>> So, if you're interested (all the following is GPL2):
>>>>
>>>> *matplotlib patch*
>>>
>>> Would you please submit an actual patch? I don't know exactly where
>>> you intend these changes to be placed.
>>>
>>>> In FigureManagerQT.__init__, added:
>>>> self.canvas.axes = self.canvas.figure.add_subplot(111)
>>>
>>> What is the purpose of this change? What if I didn't want such an axes
>>> on my canvas? What if I want to layout my own axes([.2,.2,.75,.75]) or
>>> add_subplot(311)? I don't think these changes can be accepted in the
>>> current form, they don't appear to integrate well with the standard
>>> behavior of the library.
>>>
>>> Darren
>>
>> Ok, I admit that it was very difficult to fix this -- it took me a lot
>> of seconds without knowing very well matplotlib... ;-)
>
> [...]
>
>> Here is a zip file containing all you need (patch for backend_qt4.py,
>> two scripts to be copied in backends/, and one .svg image to be copied
>> to mpl-data/images/)
>
> [...]
>
> Your patch file is backwards, it would revert your changes if applied
> to the updated code.
Sorry for this... I admit that I did this maybe too quickly.
> It doesn't matter, the patch is so small that the
> changes can just be cut and pasted. Unfortunately, however, I will not
> have time to review your contribution closely enough to consider
> committing it for at least two weeks, maybe more (conference and long
> hours at work). I'll look into it when I get a chance, if someone else
> doesn't beat me to it.
>
> Darren
No problem, this is already part of Spyder's matplotlib patch for six
months now, and I'm only using matplotlib through Spyder, so there is
absolutely no hurry for me. Since the beginning I was just trying to
be useful, to share something. I simply thought that this kind of
feature was missing in matplotlib: matplotlib produces good looking
figures but -compared to MATLAB- there is a serious lack of
interactivity (and performance -- but matplotlib is at worst as slow
as MATLAB, so it doesn't matter much).
Pierre

On Sun, Jan 3, 2010 at 9:28 AM, Darren Dale <dsd...@gm...> wrote:
> Your patch file is backwards, it would revert your changes if applied
> to the updated code. It doesn't matter, the patch is so small that the
> changes can just be cut and pasted. Unfortunately, however, I will not
> have time to review your contribution closely enough to consider
> committing it for at least two weeks, maybe more (conference and long
> hours at work). I'll look into it when I get a chance, if someone else
> doesn't beat me to it.
Hey Pierre, Darren,
I took a stab at this. I put the helper code in a backends.qt4_editor
package, and put the toolbar button right after the configure subplots
button. Thanks for the patch and the license change Pierre, and sorry
it took us so long to incorporate it. Let me know if any of the
reorganizations are a problem for you.
JDH

On Sun, Jan 3, 2010 at 9:11 AM, Pierre Raybaut <co...@py...> wrote:
> 2009年12月1日 Darren Dale <dsd...@gm...>:
>> On Tue, Apr 28, 2009 at 8:18 AM, Pierre Raybaut <co...@py...> wrote:
>>> Hi all,
>>>
>>> I would like to contribute to matplotlib with this enhancement for the
>>> PyQt4 backend: the idea is to add a toolbar button to configure figure
>>> options (axes, curves, ...).
>>>
>>> It's based on a tiny module called formlayout to generate PyQt4 form
>>> dialog automatically.
>>>
>>> Some screenshots:
>>> http://code.google.com/p/formlayout/
>>>
>>> So, if you're interested (all the following is GPL2):
>>>
>>> *matplotlib patch*
>>
>> Would you please submit an actual patch? I don't know exactly where
>> you intend these changes to be placed.
>>
>>> In FigureManagerQT.__init__, added:
>>> self.canvas.axes = self.canvas.figure.add_subplot(111)
>>
>> What is the purpose of this change? What if I didn't want such an axes
>> on my canvas? What if I want to layout my own axes([.2,.2,.75,.75]) or
>> add_subplot(311)? I don't think these changes can be accepted in the
>> current form, they don't appear to integrate well with the standard
>> behavior of the library.
>>
>> Darren
>
> Ok, I admit that it was very difficult to fix this -- it took me a lot
> of seconds without knowing very well matplotlib... ;-)
[...]
> Here is a zip file containing all you need (patch for backend_qt4.py,
> two scripts to be copied in backends/, and one .svg image to be copied
> to mpl-data/images/)
[...]
Your patch file is backwards, it would revert your changes if applied
to the updated code. It doesn't matter, the patch is so small that the
changes can just be cut and pasted. Unfortunately, however, I will not
have time to review your contribution closely enough to consider
committing it for at least two weeks, maybe more (conference and long
hours at work). I'll look into it when I get a chance, if someone else
doesn't beat me to it.
Darren

2009年12月1日 Darren Dale <dsd...@gm...>:
> On Tue, Apr 28, 2009 at 8:18 AM, Pierre Raybaut <co...@py...> wrote:
>> Hi all,
>>
>> I would like to contribute to matplotlib with this enhancement for the
>> PyQt4 backend: the idea is to add a toolbar button to configure figure
>> options (axes, curves, ...).
>>
>> It's based on a tiny module called formlayout to generate PyQt4 form
>> dialog automatically.
>>
>> Some screenshots:
>> http://code.google.com/p/formlayout/
>>
>> So, if you're interested (all the following is GPL2):
>>
>> *matplotlib patch*
>
> Would you please submit an actual patch? I don't know exactly where
> you intend these changes to be placed.
>
>> In FigureManagerQT.__init__, added:
>> self.canvas.axes = self.canvas.figure.add_subplot(111)
>
> What is the purpose of this change? What if I didn't want such an axes
> on my canvas? What if I want to layout my own axes([.2,.2,.75,.75]) or
> add_subplot(311)? I don't think these changes can be accepted in the
> current form, they don't appear to integrate well with the standard
> behavior of the library.
>
> Darren
Ok, I admit that it was very difficult to fix this -- it took me a lot
of seconds without knowing very well matplotlib... ;-)
Anyway, now, you can't say no any longer :-)
Here is a zip file containing all you need (patch for backend_qt4.py,
two scripts to be copied in backends/, and one .svg image to be copied
to mpl-data/images/) -- I can't make it clearer than this...
Cheers,
Pierre

On Sun, Jan 3, 2010 at 8:05 PM, Nathaniel Smith <nj...@po...> wrote:
> On Tue, Dec 29, 2009 at 6:34 AM, David Cournapeau <cou...@gm...> wrote:
>> Buildout, virtualenv all work by sandboxing from the system python:
>> each of them do not see each other, which may be useful for
>> development, but as a deployment solution to the casual user who may
>> not be familiar with python, it is useless. A scientist who installs
>> numpy, scipy, etc... to try things out want to have everything
>> available in one python interpreter, and does not want to jump to
>> different virtualenvs and whatnot to try different packages.
>
> What I do -- and documented for people in my lab to do -- is set up
> one virtualenv in my user account, and use it as my default python. (I
> 'activate' it from my login scripts.) The advantage of this is that
> easy_install (or pip) just works, without any hassle about permissions
> etc.
It just works if you happen to be able to build everything from
sources. That alone means you ignore the majority of users I intend to
target.
No other community (except maybe Ruby) push those isolated install
solutions as a general deployment solutions. If it were such a great
idea, other people would have picked up those solutions.
> This should be easier, but I think the basic approach is sound.
> "Integration with the package system" is useless; the advantage of
> distribution packages is that distributions can provide a single
> coherent system with consistent version numbers across all packages,
> etc., and the only way to "integrate" with that is to, well, get the
> packages into the distribution.
Another way is to provide our own repository for a few major
distributions, with automatically built packages. This is how most
open source providers work. Miguel de Icaza explains this well:
http://tirania.org/blog/archive/2007/Jan-26.html
I hope we will be able to reuse much of the opensuse build service
infrastructure.
>
> On another note, I hope toydist will provide a "source prepare" step,
> that allows arbitrary code to be run on the source tree. (For, e.g.,
> cython->C conversion, ad-hoc template languages, etc.) IME this is a
> very common pain point with distutils; there is just no good way to do
> it, and it has to be supported in the distribution utility in order to
> get everything right. In particular:
> -- Generated files should never be written to the source tree
> itself, but only the build directory
> -- Building from a source checkout should run the "source prepare"
> step automatically
> -- Building a source distribution should also run the "source
> prepare" step, and stash the results in such a way that when later
> building the source distribution, this step can be skipped. This is a
> common requirement for user convenience, and necessary if you want to
> avoid arbitrary code execution during builds.
Build directories are hard to implement right. I don't think toydist
will support this directly. IMO, those advanced builds warrant a real
build tool - one main goal of toydist is to make integration with waf
or scons much easier. Both waf and scons have the concept of a build
directory, which should do everything you described.
David

On Tue, Dec 29, 2009 at 6:34 AM, David Cournapeau <cou...@gm...> wrote:
> Buildout, virtualenv all work by sandboxing from the system python:
> each of them do not see each other, which may be useful for
> development, but as a deployment solution to the casual user who may
> not be familiar with python, it is useless. A scientist who installs
> numpy, scipy, etc... to try things out want to have everything
> available in one python interpreter, and does not want to jump to
> different virtualenvs and whatnot to try different packages.
What I do -- and documented for people in my lab to do -- is set up
one virtualenv in my user account, and use it as my default python. (I
'activate' it from my login scripts.) The advantage of this is that
easy_install (or pip) just works, without any hassle about permissions
etc. This should be easier, but I think the basic approach is sound.
"Integration with the package system" is useless; the advantage of
distribution packages is that distributions can provide a single
coherent system with consistent version numbers across all packages,
etc., and the only way to "integrate" with that is to, well, get the
packages into the distribution.
On another note, I hope toydist will provide a "source prepare" step,
that allows arbitrary code to be run on the source tree. (For, e.g.,
cython->C conversion, ad-hoc template languages, etc.) IME this is a
very common pain point with distutils; there is just no good way to do
it, and it has to be supported in the distribution utility in order to
get everything right. In particular:
 -- Generated files should never be written to the source tree
itself, but only the build directory
 -- Building from a source checkout should run the "source prepare"
step automatically
 -- Building a source distribution should also run the "source
prepare" step, and stash the results in such a way that when later
building the source distribution, this step can be skipped. This is a
common requirement for user convenience, and necessary if you want to
avoid arbitrary code execution during builds.
And if you just set up the distribution util so that the only place
you can specify arbitrary code execution is in the "source prepare"
step, then even people who know nothing about packaging will
automatically get all of the above right.
Cheers,
-- Nathaniel

On Sun, Jan 3, 2010 at 3:27 AM, Andrew Straw <str...@as...> wrote:
>>
> Typically, the dependencies only depend on the smallest subset of what
> they require (if they don't need lapack, they'd only depend on
> python-numpy-core in your example), but yes, if there's an unsatisfiable
> condition, then apt-get will raise an error and abort. In practice, this
> system seems to work quite well, IMO.
Yes, but:
 - debian dependency resolution is complex. I think many people don't
realize how complex the problem really is (AFAIK, any correct scheme
to resolve dependencies in debian requires an algorithm which is
NP-complete )
 - introducing a lot of variants significantly slow down the whole thing.
I think it worths thinking whether our problems warrant such a complexity.
>
> Anyhow, here's the full Debian documentation:
> http://www.debian.org/doc/debian-policy/ch-relationships.html
This is not the part I am afraid of. This is:
http://people.debian.org/~dburrows/model.pdf
cheers,
David

On Sun, Jan 03, 2010 at 03:05:54AM -0800, Nathaniel Smith wrote:
> What I do -- and documented for people in my lab to do -- is set up
> one virtualenv in my user account, and use it as my default python. (I
> 'activate' it from my login scripts.) The advantage of this is that
> easy_install (or pip) just works, without any hassle about permissions
> etc. This should be easier, but I think the basic approach is sound.
> "Integration with the package system" is useless; the advantage of
> distribution packages is that distributions can provide a single
> coherent system with consistent version numbers across all packages,
> etc., and the only way to "integrate" with that is to, well, get the
> packages into the distribution.
That works because either you use packages that don't have much hard-core
compiled dependencies, or these are already installed.
Think about installing VTK or ITK this way, even something simpler such
as umfpack. I think that you would loose most of your users. In my lab,
I do lose users on such packages actually.
Beside, what you are describing is possible without package isolation, it
is simply the use of a per-user local site-packages, which now semi
automatic in python2.6 using the '.local' directory. I do agree that, in
a research lab, this is a best practice.
Gaël