[Python-Dev] Status of packaging in 3.3

Thu Jun 21 23:04:37 CEST 2012

On 6/21/12 10:46 PM, Dag Sverre Seljebotn wrote:
...
>> I think we should, as you proposed, list a few projects w/ compilation
>> needs -- from the simplest to the more complex, then see how a standard
>> *description* could be used by any tool
>> It's not clear to me what you mean by description. Package metadata, 
> install information or description of what/how to build?
>> I hope you don't mean the latter, that would be insane...it would 
> effectively amount to creating a build tool that's both more elegant 
> and more powerful than any option that's currently already out there.
>> Assuming you mean the former, that's what David did to create Bento. 
> Reading and understanding Bento and the design decisions going into it 
> would be a better use of time than redoing a discussion, and would at 
> least be a very good starting point.

What I mean is : what would it take to use Bento (or another tool) as 
the compiler in a distutils-based project, without having to change the 
distutils metadata.
>> But anyway, some project types from simple to advanced:
>> - Simple library using Cython + NumPy C API
> - Wrappers around HPC codes like mpi4py, petsc4py
> - NumPy
> - SciPy (uses Fortran compilers too)
> - Library using code generation, Cython, NumPy C API, Fortran 90 
> code, some performance tuning with CPU characteristics (instruction 
> set, cache size, optimal loop structure) decided compile-time

I'd add:
- A Distutils project with a few ExtensionsThe other thing is, the folks 
in distutils2 and myself, have zero
>> knowledge about compilers. That's why we got very frustrated not to see
>> people with that knowledge come and help us in this area.
>> Here's the flip side: If you have zero knowledge about compilers, it's 
> going to be almost impossible to have a meaningful discussion about a 
> compilation PEP. It's very hard to discuss standards unless everybody 
> involved have the necessary prerequisite knowledge. You don't go 
> discussing details of the Linux kernel without some solid C experience 
> either.
Consider me as the end user that want to have his 2 C modules compiled 
in their Python project.
>> The necessary prerequisites in this case is not merely "knowledge of 
> compilers". To avoid repeating mistakes of the past, the prerequisites 
> for a meaningful discussion is years of hard-worn experience building 
> software in various languages, on different platforms, using different 
> build tools.
>> Look, these problems are really hard to deal with. Myself I have 
> experience with building 2-3 languages using 2-3 build tools on 2 
> platforms, and I consider myself a complete novice and usually decide 
> to trust David's instincts over trying to make up an opinion of my own 
> -- simply because I know he's got a lot more experience than I have.
>> Theoretically it is possible to separate and isolate concerns so that 
> one set of people discuss build integration and another set of people 
> discuss installation. Problem is that all the problems tangle -- in 
> particular when the starting point is distutils!
>> That's why *sometimes*, not always, design by committee is the wrong 
> approach, and one-man-shows is what brings technology forwards.

I am not saying this should be designed by a commitee, but rather - if 
such a tool can be made compatible with simple Distutils project, the 
guy behind this tool can probably help on a PEP with feedback from a 
larger audience than the sci community.
What bugs me is to say that we live in two separate worlds and cannot 
build common pieces. This is not True.
>>> So, I reiterate my proposal, and it could also be expressed like this:
>>>> 1/ David writes a PEP where he describes how Bento interact with a
>> project -- metadata, description files, etc
>> 2/ Someone from distutils2 completes the PEP by describing how setup.cfg
>> works wrt Extensions
>> 3/ we see if we can have a common standard even if it's a subset of
>> bento capabilities
>> bento isn't a build tool, it's a packaging tool, competing directly 
> with distutils2. It can deal with simple distutils-like builds using a 
> bundled build tool, and currently has integration with waf for 
> complicated builds; integration with other build systems will 
> presumably be added later as people need it (the main point is that 
> bento is designed for it).
I am not interested in Bento-the-tool. I am interested in what such a 
tool needs from a project to use it =>
"It can deal with simple distutils-like builds using a bundled build 
tool" => If I understand this correctly, does that mean that Bento can 
build a distutils project with the distutils Metadata ?
If this is the case it means that there a piece of function that 
translates Distutils metadata into something Bento deals with.
That's the part I am interested in for interoperability.
>> Dag
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> http://mail.python.org/mailman/options/python-dev/ziade.tarek%40gmail.com