matplotlib-devel Mailing List for matplotlib (Page 5)

I've pushed a fix to v1.2.x and master for this new problem 
(35ee2184111fb8f80027869d8ee309c7f4e5a467). Unfortunately, another 
rebase of your branches is in order in order to get this fix.
Mike
On 12/02/2012 12:23 PM, Thomas Kluyver wrote:
> On 2 December 2012 17:02, Damon McDougall <dam...@gm... 
> <mailto:dam...@gm...>> wrote:
>
> > Still failing even with the workaround. Here's proof:
> > https://github.com/matplotlib/matplotlib/pull/1549
>
> And looks like Thomas reported an issue too:
> https://github.com/matplotlib/matplotlib/issues/1548
>
>
> This is a different problem, though (unless it's a really bizarre 
> symptom of the other problem). Now it's an error in compiling matplotlib.
>
> Thomas

On Sun, Dec 2, 2012 at 11:00 AM, Damon McDougall
<dam...@gm...> wrote:
> On Sat, Dec 1, 2012 at 12:56 PM, Damon McDougall
> <dam...@gm...> wrote:
>> On Sat, Dec 1, 2012 at 12:28 PM, Michael Droettboom <md...@st...> wrote:
>>> Ok. I'll go ahead and put the workaround in our .travis.yml. Any
>>> active pull requests that we want to have good Travis results for will
>>> need to be rebased to get the workaround.
>>
>> That's better than having no testing at all for Python 3.x, so that
>> gets my +1. Thanks Mike.
>
> Still failing even with the workaround. Here's proof:
> https://github.com/matplotlib/matplotlib/pull/1549
And looks like Thomas reported an issue too:
https://github.com/matplotlib/matplotlib/issues/1548
>
>>
>>>
>>> Mike
>>>
>>> On 11/30/2012 06:20 PM, Damon McDougall wrote:
>>>> Forwarding to list again...
>>>>
>>>>
>>>> ---------- Forwarded message ----------
>>>> From: Nathaniel Smith <nj...@po...>
>>>> Date: Fri, Nov 30, 2012 at 5:13 PM
>>>> Subject: Re: [matplotlib-devel] Travis numpy build failures on Python 3.x
>>>> To: Damon McDougall <dam...@gm...>
>>>>
>>>>
>>>> On Fri, Nov 30, 2012 at 10:25 PM, Damon McDougall
>>>> <dam...@gm...> wrote:
>>>>> We seem to have inherited these recently. I am questioning whether it
>>>>> is something caused by us or not. Can anybody build numpy/mpl under
>>>>> Python 3.x on their own machine successfully?
>>>> Not your bug.
>>>>
>>>> Workaround:
>>>> https://github.com/travis-ci/travis-cookbooks/issues/48#issuecomment-10843018
>>>>
>>>> Also for context:
>>>> https://github.com/numpy/numpy/issues/2761
>>>> https://github.com/pypa/virtualenv/issues/359
>>>>
>>>> -n
>>>>
>>>>
>>>
>>>
>>> ------------------------------------------------------------------------------
>>> Keep yourself connected to Go Parallel:
>>> INSIGHTS What's next for parallel hardware, programming and related areas?
>>> Interviews and blogs by thought leaders keep you ahead of the curve.
>>> http://goparallel.sourceforge.net
>>> _______________________________________________
>>> Matplotlib-devel mailing list
>>> Mat...@li...
>>> https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
>>
>>
>>
>> --
>> Damon McDougall
>> http://www.damon-is-a-geek.com
>> Institute for Computational Engineering Sciences
>> 201 E. 24th St.
>> Stop C0200
>> The University of Texas at Austin
>> Austin, TX 78712-1229
>
>
>
> --
> Damon McDougall
> http://www.damon-is-a-geek.com
> Institute for Computational Engineering Sciences
> 201 E. 24th St.
> Stop C0200
> The University of Texas at Austin
> Austin, TX 78712-1229
-- 
Damon McDougall
http://www.damon-is-a-geek.com
Institute for Computational Engineering Sciences
201 E. 24th St.
Stop C0200
The University of Texas at Austin
Austin, TX 78712-1229

On Sat, Dec 1, 2012 at 12:56 PM, Damon McDougall
<dam...@gm...> wrote:
> On Sat, Dec 1, 2012 at 12:28 PM, Michael Droettboom <md...@st...> wrote:
>> Ok. I'll go ahead and put the workaround in our .travis.yml. Any
>> active pull requests that we want to have good Travis results for will
>> need to be rebased to get the workaround.
>
> That's better than having no testing at all for Python 3.x, so that
> gets my +1. Thanks Mike.
Still failing even with the workaround. Here's proof:
https://github.com/matplotlib/matplotlib/pull/1549
>
>>
>> Mike
>>
>> On 11/30/2012 06:20 PM, Damon McDougall wrote:
>>> Forwarding to list again...
>>>
>>>
>>> ---------- Forwarded message ----------
>>> From: Nathaniel Smith <nj...@po...>
>>> Date: Fri, Nov 30, 2012 at 5:13 PM
>>> Subject: Re: [matplotlib-devel] Travis numpy build failures on Python 3.x
>>> To: Damon McDougall <dam...@gm...>
>>>
>>>
>>> On Fri, Nov 30, 2012 at 10:25 PM, Damon McDougall
>>> <dam...@gm...> wrote:
>>>> We seem to have inherited these recently. I am questioning whether it
>>>> is something caused by us or not. Can anybody build numpy/mpl under
>>>> Python 3.x on their own machine successfully?
>>> Not your bug.
>>>
>>> Workaround:
>>> https://github.com/travis-ci/travis-cookbooks/issues/48#issuecomment-10843018
>>>
>>> Also for context:
>>> https://github.com/numpy/numpy/issues/2761
>>> https://github.com/pypa/virtualenv/issues/359
>>>
>>> -n
>>>
>>>
>>
>>
>> ------------------------------------------------------------------------------
>> Keep yourself connected to Go Parallel:
>> INSIGHTS What's next for parallel hardware, programming and related areas?
>> Interviews and blogs by thought leaders keep you ahead of the curve.
>> http://goparallel.sourceforge.net
>> _______________________________________________
>> Matplotlib-devel mailing list
>> Mat...@li...
>> https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
>
>
>
> --
> Damon McDougall
> http://www.damon-is-a-geek.com
> Institute for Computational Engineering Sciences
> 201 E. 24th St.
> Stop C0200
> The University of Texas at Austin
> Austin, TX 78712-1229
-- 
Damon McDougall
http://www.damon-is-a-geek.com
Institute for Computational Engineering Sciences
201 E. 24th St.
Stop C0200
The University of Texas at Austin
Austin, TX 78712-1229

For point of comparison, my branch now has a Cython and C++ version of 
the same thing.
Here's the Cython version:
https://github.com/mdboom/matplotlib/blob/no_cxx/src/_png.pyx
Here's the C++ version:
https://github.com/mdboom/matplotlib/blob/no_cxx/src/_png_wrap.cpp
Some interesting things to note:
The Cython version isn't that much shorter than the C++ version. It 
mostly consists of declarations. These declarations aren't exact 
matches to what one would find in the header file(s) because Cython 
doesn't support exact-width data types etc. The Cython documentation 
says "not to worry", but I do wonder how well this will work across 
different architectures etc. I'm not sure why some of the Python/C API 
calls I needed were not defined in Cython's include wrappers.
The Cython extension only builds with "-fpermissive" because I can't 
seem to get the casts and const coercions to work. Maybe there's a 
simple solution...
The exception handling in the png_core.cpp file will need to be updated 
because Cython only supports handling built-in C++ exceptions (and 
subclasses), and the Cython custom exception handler doesn't provide a 
way to get at the exception object that was thrown.
It seems that a lot of things pass through the Cython compiler, but then 
fail in the C compiler -- you then have to wade through the generated C 
code to figure out what's going wrong. This reminds me of the bad old 
days of C++ when the error messages generated would be dozens of lines 
long and rather inscrutable.
Once things compiled, due to my own mistake, calling the function 
segfaulted. Debugging that segfault in gdb required, again, wading 
through the generated code. Using gdb on hand-written code is *much* nicer.
So, it's probably clear that I'm not much of a fan of this approach, but 
I am trying to find something that the whole community around matplotlib 
finds easier and more accessible so that the C/C++ experts among use are 
not exclusively burdened to maintain this part of the code base. I 
would be interested to see what others think now that we have an 
apples-to-apples comparison.
Cheers,
Mike
On 12/01/2012 01:36 PM, Michael Droettboom wrote:
> For the PNG extension specifically, it was creating callbacks that can
> be called from C and the setjmp magic that libpng requires. I think
> it's possible to do it, but I was surprised at how non-obvious those
> pieces of Cython were. I was really hoping by creating this experiment
> that a Cython expert would step up and show the way ;)
>
> The Agg backend has more C++-specific challenges, particularly
> instantiating very complex template expressions -- but I haven't really
> followed that on through.
>
> Mike
>
> On 11/30/2012 09:13 AM, Jason Grout wrote:
>> On 11/29/12 10:59 AM, Michael Droettboom wrote:
>>> I've not had
>>> much luck with Cython for this kind of thing in the past, but I know it
>>> is popular.
>> I'm curious about what problems you've run into and how long it was. In
>> the past, Cython hasn't supported C++ very well, but the situation has
>> greatly improved recently. See
>> http://docs.cython.org/src/userguide/wrapping_CPlusPlus.html for some
>> details.
>>
>> Thanks,
>>
>> Jason
>>
>>
>> ------------------------------------------------------------------------------
>> Keep yourself connected to Go Parallel:
>> TUNE You got it built. Now make it sing. Tune shows you how.
>> http://goparallel.sourceforge.net
>> _______________________________________________
>> Matplotlib-devel mailing list
>> Mat...@li...
>> https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
>
> ------------------------------------------------------------------------------
> Keep yourself connected to Go Parallel:
> INSIGHTS What's next for parallel hardware, programming and related areas?
> Interviews and blogs by thought leaders keep you ahead of the curve.
> http://goparallel.sourceforge.net
> _______________________________________________
> Matplotlib-devel mailing list
> Mat...@li...
> https://lists.sourceforge.net/lists/listinfo/matplotlib-devel

On 12/1/12 12:36 PM, Michael Droettboom wrote:
> I was really hoping by creating this experiment
> that a Cython expert would step up and show the way ;)
There are a lot more Cython experts on the Cython mailing list ;).
Thanks,
Jason

Including the Cython-generated C in the tarballs and optionally the git 
repository as well can certainly be considered to reduce the need for 
Cython for developers and users alike. However, the Cython source 
should also be included in the repository for the inevitable times when 
it does need to be updated -- it shouldn't be off somewhere else.
The png, path, ft2font, backend_agg, gtkagg, tkagg, tri, and image 
modules all use CXX. The backend_agg, image and ft2font ones are 
particularly complex, but some of that complexity could be reduced by 
using Numpy arrays in place of the image buffer types that each of them 
contain (that code predates matplotlib's numpy requirement, so it's not 
terribly surprising that a more complex approach was taken).
Mike
On 12/01/2012 09:44 AM, Michiel de Hoon wrote:
> In my experience, Benjamin is right that the C code is rarely touched. 
> This is even more true for the Python/C glue code, at least from my 
> experience with the Mac OS X backend. Since the Python/C glue code is 
> modified only very rarely, there may not be a need for regenerating 
> the Python/C glue code by developers or users from a Cython source code.
>
> In addition, it is much easier to maintain the Python/C glue code than 
> to write it from scratch. Once you have the Python/C glue code, it's 
> relatively straightforward to modify it by looking at the existing 
> Python/C glue code.
>
> This argues against making the Cython source code a part of the 
> matplotlib codebase.
>
> At the same time, to minimize errors, we could use Cython to create 
> the initial Python/C glue code, and then add the generated code to the 
> matplotlib codebase. Then neither users nor developers have to install 
> Cython, we don't have to worry about inconsistencies (if any) between 
> different Cython versions, we don't have to worry about keeping the 
> Cython source code and the generated code in sync, and we will still 
> get a high-quality Cython-generated Python/C glue code.
>
> By the way, how many modules in matplotlib make use of CXX, and would 
> have to be converted?
>
> Best,
> -Michiel.
>
> --- On *Fri, 11/30/12, Benjamin Root /<ben...@ou...>/* wrote:
>
>
> From: Benjamin Root <ben...@ou...>
> Subject: Re: [matplotlib-devel] Experiments in removing/replacing
> PyCXX
> To: "Nathaniel Smith" <nj...@po...>
> Cc: "Michiel de Hoon" <mjl...@ya...>,
> "mat...@li..."
> <mat...@li...>, "Chris Barker - NOAA
> Federal" <chr...@no...>
> Date: Friday, November 30, 2012, 8:32 PM
>
>
>
> On Fri, Nov 30, 2012 at 6:44 PM, Nathaniel Smith <nj...@po...
> </mc/compose?to=nj...@po...>> wrote:
>
> On Fri, Nov 30, 2012 at 11:40 PM, Michiel de Hoon
> <mjl...@ya... </mc/compose?to=mjl...@ya...>> wrote:
> > One package (Pysam) that I use a lot relies on Cython, and
> requires users to install Cython before they can install Pysam
> itself. With Cython, is that always the case? Will all users
> need to install Cython? Or is it sufficient if only matplotlib
> developers install Cython?
>
> You can set things up so that end-users don't have to install
> cython.
> You just convert the .pyx files to regular .c files before
> distributing your package. Numpy itself uses cython, but end-users
> don't notice or care. (It's something more of a hassle for
> developers
> to do things this way, and cython is very easy to install, so
> I don't
> know if it's worth it. But it's certainly possible.)
>
>
> Since when has numpy used Cython? I specifically remember a
> rather involved discussion thread on numpy-discussion about the
> pros-and-cons of including cython. Now, SciPy on the other hand,
> does utilize Cython in some spots IIRC, but does it in a way that
> it isn't even required for the developers to have cython installed
> to build from source.
>
> I would not be against such an approach. Much of the C/C++ stuff
> is rarely touched. If we have some source cython that is used to
> generate C/C++ source code that is packaged in the same way as the
> current code is, I would have no problem with that. Given that
> matplotlib is such a fundamental tool in the ecosystem, I want to
> make sure that the decisions we make are ones that improves our
> packaging situation.
>
> Cheers!
> Ben Root
>
>
>
> ------------------------------------------------------------------------------
> Keep yourself connected to Go Parallel:
> INSIGHTS What's next for parallel hardware, programming and related areas?
> Interviews and blogs by thought leaders keep you ahead of the curve.
> http://goparallel.sourceforge.net
>
>
> _______________________________________________
> Matplotlib-devel mailing list
> Mat...@li...
> https://lists.sourceforge.net/lists/listinfo/matplotlib-devel

On Sat, Dec 1, 2012 at 12:28 PM, Michael Droettboom <md...@st...> wrote:
> Ok. I'll go ahead and put the workaround in our .travis.yml. Any
> active pull requests that we want to have good Travis results for will
> need to be rebased to get the workaround.
That's better than having no testing at all for Python 3.x, so that
gets my +1. Thanks Mike.
>
> Mike
>
> On 11/30/2012 06:20 PM, Damon McDougall wrote:
>> Forwarding to list again...
>>
>>
>> ---------- Forwarded message ----------
>> From: Nathaniel Smith <nj...@po...>
>> Date: Fri, Nov 30, 2012 at 5:13 PM
>> Subject: Re: [matplotlib-devel] Travis numpy build failures on Python 3.x
>> To: Damon McDougall <dam...@gm...>
>>
>>
>> On Fri, Nov 30, 2012 at 10:25 PM, Damon McDougall
>> <dam...@gm...> wrote:
>>> We seem to have inherited these recently. I am questioning whether it
>>> is something caused by us or not. Can anybody build numpy/mpl under
>>> Python 3.x on their own machine successfully?
>> Not your bug.
>>
>> Workaround:
>> https://github.com/travis-ci/travis-cookbooks/issues/48#issuecomment-10843018
>>
>> Also for context:
>> https://github.com/numpy/numpy/issues/2761
>> https://github.com/pypa/virtualenv/issues/359
>>
>> -n
>>
>>
>
>
> ------------------------------------------------------------------------------
> Keep yourself connected to Go Parallel:
> INSIGHTS What's next for parallel hardware, programming and related areas?
> Interviews and blogs by thought leaders keep you ahead of the curve.
> http://goparallel.sourceforge.net
> _______________________________________________
> Matplotlib-devel mailing list
> Mat...@li...
> https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
-- 
Damon McDougall
http://www.damon-is-a-geek.com
Institute for Computational Engineering Sciences
201 E. 24th St.
Stop C0200
The University of Texas at Austin
Austin, TX 78712-1229

For the PNG extension specifically, it was creating callbacks that can 
be called from C and the setjmp magic that libpng requires. I think 
it's possible to do it, but I was surprised at how non-obvious those 
pieces of Cython were. I was really hoping by creating this experiment 
that a Cython expert would step up and show the way ;)
The Agg backend has more C++-specific challenges, particularly 
instantiating very complex template expressions -- but I haven't really 
followed that on through.
Mike
On 11/30/2012 09:13 AM, Jason Grout wrote:
> On 11/29/12 10:59 AM, Michael Droettboom wrote:
>> I've not had
>> much luck with Cython for this kind of thing in the past, but I know it
>> is popular.
> I'm curious about what problems you've run into and how long it was. In
> the past, Cython hasn't supported C++ very well, but the situation has
> greatly improved recently. See
> http://docs.cython.org/src/userguide/wrapping_CPlusPlus.html for some
> details.
>
> Thanks,
>
> Jason
>
>
> ------------------------------------------------------------------------------
> Keep yourself connected to Go Parallel:
> TUNE You got it built. Now make it sing. Tune shows you how.
> http://goparallel.sourceforge.net
> _______________________________________________
> Matplotlib-devel mailing list
> Mat...@li...
> https://lists.sourceforge.net/lists/listinfo/matplotlib-devel

Ok. I'll go ahead and put the workaround in our .travis.yml. Any 
active pull requests that we want to have good Travis results for will 
need to be rebased to get the workaround.
Mike
On 11/30/2012 06:20 PM, Damon McDougall wrote:
> Forwarding to list again...
>
>
> ---------- Forwarded message ----------
> From: Nathaniel Smith <nj...@po...>
> Date: Fri, Nov 30, 2012 at 5:13 PM
> Subject: Re: [matplotlib-devel] Travis numpy build failures on Python 3.x
> To: Damon McDougall <dam...@gm...>
>
>
> On Fri, Nov 30, 2012 at 10:25 PM, Damon McDougall
> <dam...@gm...> wrote:
>> We seem to have inherited these recently. I am questioning whether it
>> is something caused by us or not. Can anybody build numpy/mpl under
>> Python 3.x on their own machine successfully?
> Not your bug.
>
> Workaround:
> https://github.com/travis-ci/travis-cookbooks/issues/48#issuecomment-10843018
>
> Also for context:
> https://github.com/numpy/numpy/issues/2761
> https://github.com/pypa/virtualenv/issues/359
>
> -n
>
>

Drat, re-sending on the list.
On 1 December 2012 16:40, Thomas Kluyver <th...@kl...> wrote:
> On 1 December 2012 14:44, Michiel de Hoon <mjl...@ya...> wrote:
>
>> At the same time, to minimize errors, we could use Cython to create the
>> initial Python/C glue code, and then add the generated code to the
>> matplotlib codebase. Then neither users nor developers have to install
>> Cython, we don't have to worry about inconsistencies (if any) between
>> different Cython versions, we don't have to worry about keeping the Cython
>> source code and the generated code in sync, and we will still get a
>> high-quality Cython-generated Python/C glue code.
>
>
> Having looked at some bits of Cython-generated C code, I wouldn't
> recommend that. I'm sure it's high quality in terms of compiling and
> running correctly, but it's definitely not designed to be read or
> maintained directly. Here's a sample from SciPy to illustrate:
>
>
> https://github.com/scipy/scipy/blob/master/scipy/stats/vonmises_cython.c#L2269
>
> For another reason, there have been cases where the Cython-generated C
> code was broken in some way, and it was fixed by regenerating with a newer
> version of Cython. I experienced this with pyzmq when testing with Python
> 3.3 for example - it completely failed to import until I installed a newer
> version of Cython and redid the conversion. If you don't keep the original
> Cython code, you don't have this option.
>
> Best wishes,
> Thomas
>

I'm +1 on Cython. I think its prevalence in the community gives us a larger potential contributor pool than CXX or hand-coded python C-API. I know using Cython would open up that part of the code base for me.
Ryan
On Dec 1, 2012, at 8:44, Michiel de Hoon <mjl...@ya...> wrote:
> 
> In my experience, Benjamin is right that the C code is rarely touched. This is even more true for the Python/C glue code, at least from my experience with the Mac OS X backend. Since the Python/C glue code is modified only very rarely, there may not be a need for regenerating the Python/C glue code by developers or users from a Cython source code.
> 
> In addition, it is much easier to maintain the Python/C glue code than to write it from scratch. Once you have the Python/C glue code, it's relatively straightforward to modify it by looking at the existing Python/C glue code.
> 
> This argues against making the Cython source code a part of the matplotlib codebase.
> 
> At the same time, to minimize errors, we could use Cython to create the initial Python/C glue code, and then add the generated code to the matplotlib codebase. Then neither users nor developers have to install Cython, we don't have to worry about inconsistencies (if any) between different Cython versions, we don't have to worry about keeping the Cython source code and the generated code in sync, and we will still get a high-quality Cython-generated Python/C glue code.
> 
> By the way, how many modules in matplotlib make use of CXX, and would have to be converted?
> 
> Best,
> -Michiel.
> 
> --- On Fri, 11/30/12, Benjamin Root <ben...@ou...> wrote:
> 
> From: Benjamin Root <ben...@ou...>
> Subject: Re: [matplotlib-devel] Experiments in removing/replacing PyCXX
> To: "Nathaniel Smith" <nj...@po...>
> Cc: "Michiel de Hoon" <mjl...@ya...>, "mat...@li..." <mat...@li...>, "Chris Barker - NOAA Federal" <chr...@no...>
> Date: Friday, November 30, 2012, 8:32 PM
> 
> 
> 
> On Fri, Nov 30, 2012 at 6:44 PM, Nathaniel Smith <nj...@po...> wrote:
> On Fri, Nov 30, 2012 at 11:40 PM, Michiel de Hoon <mjl...@ya...> wrote:
> > One package (Pysam) that I use a lot relies on Cython, and requires users to install Cython before they can install Pysam itself. With Cython, is that always the case? Will all users need to install Cython? Or is it sufficient if only matplotlib developers install Cython?
> 
> You can set things up so that end-users don't have to install cython.
> You just convert the .pyx files to regular .c files before
> distributing your package. Numpy itself uses cython, but end-users
> don't notice or care. (It's something more of a hassle for developers
> to do things this way, and cython is very easy to install, so I don't
> know if it's worth it. But it's certainly possible.)
> 
> 
> Since when has numpy used Cython? I specifically remember a rather involved discussion thread on numpy-discussion about the pros-and-cons of including cython. Now, SciPy on the other hand, does utilize Cython in some spots IIRC, but does it in a way that it isn't even required for the developers to have cython installed to build from source.
> 
> I would not be against such an approach. Much of the C/C++ stuff is rarely touched. If we have some source cython that is used to generate C/C++ source code that is packaged in the same way as the current code is, I would have no problem with that. Given that matplotlib is such a fundamental tool in the ecosystem, I want to make sure that the decisions we make are ones that improves our packaging situation.
> 
> Cheers!
> Ben Root
> ------------------------------------------------------------------------------
> Keep yourself connected to Go Parallel: 
> INSIGHTS What's next for parallel hardware, programming and related areas?
> Interviews and blogs by thought leaders keep you ahead of the curve.
> http://goparallel.sourceforge.net
> _______________________________________________
> Matplotlib-devel mailing list
> Mat...@li...
> https://lists.sourceforge.net/lists/listinfo/matplotlib-devel

In my experience, Benjamin is right that the C code is rarely touched. This is even more true for the Python/C glue code, at least from my experience with the Mac OS X backend. Since the Python/C glue code is modified only very rarely, there may not be a need for regenerating the Python/C glue code by developers or users from a Cython source code.
In addition, it is much easier to maintain the Python/C glue code than to write it from scratch. Once you have the Python/C glue code, it's relatively straightforward to modify it by looking at the existing Python/C glue code.
This argues against making the Cython source code a part of the matplotlib codebase.
At the same time, to minimize errors, we could use Cython to create the initial Python/C glue code, and then add the generated code to the matplotlib codebase. Then neither users nor developers have to install Cython, we don't have to worry about inconsistencies (if any) between different Cython versions, we don't have to worry about keeping the Cython source code and the generated code in sync, and we will still get a high-quality Cython-generated Python/C glue code.
By the way, how many modules in matplotlib make use of CXX, and would have to be converted?
Best,
-Michiel.
--- On Fri, 11/30/12, Benjamin Root <ben...@ou...> wrote:
From: Benjamin Root <ben...@ou...>
Subject: Re: [matplotlib-devel] Experiments in removing/replacing PyCXX
To: "Nathaniel Smith" <nj...@po...>
Cc: "Michiel de Hoon" <mjl...@ya...>, "mat...@li..." <mat...@li...>, "Chris Barker - NOAA Federal" <chr...@no...>
Date: Friday, November 30, 2012, 8:32 PM
On Fri, Nov 30, 2012 at 6:44 PM, Nathaniel Smith <nj...@po...> wrote:
On Fri, Nov 30, 2012 at 11:40 PM, Michiel de Hoon <mjl...@ya...> wrote:
> One package (Pysam) that I use a lot relies on Cython, and requires users to install Cython before they can install Pysam itself. With Cython, is that always the case? Will all users need to install Cython? Or is it sufficient if only matplotlib developers install Cython?
You can set things up so that end-users don't have to install cython.
You just convert the .pyx files to regular .c files before
distributing your package. Numpy itself uses cython, but end-users
don't notice or care. (It's something more of a hassle for developers
to do things this way, and cython is very easy to install, so I don't
know if it's worth it. But it's certainly possible.)
Since when has numpy used Cython? I specifically remember a rather involved discussion thread on numpy-discussion about the pros-and-cons of including cython. Now, SciPy on the other hand, does utilize Cython in some spots IIRC, but does it in a way that it isn't even required for the developers to have cython installed to build from source.
I would not be against such an approach. Much of the C/C++ stuff is rarely touched. If we have some source cython that is used to generate C/C++ source code that is packaged in the same way as the current code is, I would have no problem with that. Given that matplotlib is such a fundamental tool in the ecosystem, I want to make sure that the decisions we make are ones that improves our packaging situation.
Cheers!
Ben Root

On 12/01/2012 02:32 AM, Benjamin Root wrote:
> 
> 
> Since when has numpy used Cython? I specifically remember a rather
> involved discussion thread on numpy-discussion about the pros-and-cons
> of including cython. Now, SciPy on the other hand, does utilize Cython
> in some spots IIRC, but does it in a way that it isn't even required for
> the developers to have cython installed to build from source.
If you should choose cython please don't follow scipy too closely.
Up until rather recent git head they did not ship the cython sources in
their source tarballs which occasionally lead to inconsistent generated
files (e.g. in 0.10.1 interpnd.pyx) and causes trouble for distributors
(see e.g. debian bug 589731)
A better example to follow would be e.g. pyzmq which ships both the
cython and generated sources and has an easy to use cython setup.py
target to recythonize.

>> > One package (Pysam) that I use a lot relies on Cython, and requires
>> > users to install Cython before they can install Pysam itself. With Cython,
>> > is that always the case? Will all users need to install Cython? Or is it
>> > sufficient if only matplotlib developers install Cython?
>>
>> You can set things up so that end-users don't have to install cython.
>> You just convert the .pyx files to regular .c files before
>> distributing your package. Numpy itself uses cython, but end-users
>> don't notice or care. (It's something more of a hassle for developers
>> to do things this way, and cython is very easy to install, so I don't
>> know if it's worth it. But it's certainly possible.)
>>
>
> Since when has numpy used Cython? I specifically remember a rather involved
> discussion thread on numpy-discussion about the pros-and-cons of including
> cython. Now, SciPy on the other hand, does utilize Cython in some spots
> IIRC, but does it in a way that it isn't even required for the developers to
> have cython installed to build from source.
You just ship the c/c++ code for the developpers as well as for the
end users. This is what we do with scikit-learn. It requires the
developpers to make sure to compile the cython code, and commit both
files. It is also quite annoying for reviews to have the generated c++
code, so the cython code needs to be compile after the reviews.
The reason the scikit's developpers chose to use cython instead of
something else is to decrease the maintenance burden: more
contributors understand cython code than c/c++ code (or more
precisely, understand c++ code written by someone else). Hence, this
increases the bus number.
> I would not be against such an approach. Much of the C/C++ stuff is rarely
> touched. If we have some source cython that is used to generate C/C++
> source code that is packaged in the same way as the current code is, I would
> have no problem with that. Given that matplotlib is such a fundamental tool
> in the ecosystem, I want to make sure that the decisions we make are ones
> that improves our packaging situation.
>
> Cheers!
> Ben Root
>
> ------------------------------------------------------------------------------
> Keep yourself connected to Go Parallel:
> INSIGHTS What's next for parallel hardware, programming and related areas?
> Interviews and blogs by thought leaders keep you ahead of the curve.
> http://goparallel.sourceforge.net
> _______________________________________________
> Matplotlib-devel mailing list
> Mat...@li...
> https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
>

On Fri, Nov 30, 2012 at 6:44 PM, Nathaniel Smith <nj...@po...> wrote:
> On Fri, Nov 30, 2012 at 11:40 PM, Michiel de Hoon <mjl...@ya...>
> wrote:
> > One package (Pysam) that I use a lot relies on Cython, and requires
> users to install Cython before they can install Pysam itself. With Cython,
> is that always the case? Will all users need to install Cython? Or is it
> sufficient if only matplotlib developers install Cython?
>
> You can set things up so that end-users don't have to install cython.
> You just convert the .pyx files to regular .c files before
> distributing your package. Numpy itself uses cython, but end-users
> don't notice or care. (It's something more of a hassle for developers
> to do things this way, and cython is very easy to install, so I don't
> know if it's worth it. But it's certainly possible.)
>
>
Since when has numpy used Cython? I specifically remember a rather
involved discussion thread on numpy-discussion about the pros-and-cons of
including cython. Now, SciPy on the other hand, does utilize Cython in
some spots IIRC, but does it in a way that it isn't even required for the
developers to have cython installed to build from source.
I would not be against such an approach. Much of the C/C++ stuff is rarely
touched. If we have some source cython that is used to generate C/C++
source code that is packaged in the same way as the current code is, I
would have no problem with that. Given that matplotlib is such a
fundamental tool in the ecosystem, I want to make sure that the decisions
we make are ones that improves our packaging situation.
Cheers!
Ben Root