matplotlib-devel Mailing List for matplotlib (Page 2)

On 2014年06月04日 6:26 AM, Benjamin Root wrote:
> A theory...
>
> If I remember correctly, the nosttests was set up to execute in parallel
> using the default Multiprocessing settings, which is to have a process
> worker for each available CPU core. Perhaps this might be the crux of
> the issue with so many simultaneous tests running that the amount of
> memory used at the same time becomes too large. Or, am I thinking of the
> doc build system?
>
> Ben Root
Ben,
Top shows a single process. The VM is configured with 2 cores.
Eric

A theory...
If I remember correctly, the nosttests was set up to execute in parallel
using the default Multiprocessing settings, which is to have a process
worker for each available CPU core. Perhaps this might be the crux of the
issue with so many simultaneous tests running that the amount of memory
used at the same time becomes too large. Or, am I thinking of the doc build
system?
Ben Root
On Wed, Jun 4, 2014 at 12:20 PM, Nelle Varoquaux <nel...@gm...>
wrote:
> > Our standard test has gotten out of control. The most serious problem is
> > that running a full test suite now fails on a linux VM with 4 GB--it's
> out
> > of memory. Half-way through the set, it is already using more than 2 GB.
> > That's ridiculous. Running nosetests separately on each test module
> keeps
> > the max reported by top to 1.6 GB, and the max by report_memory to 0.5
> GB;
> > still quite a bit, but tolerable. (I don't know why there is this
> factor of
> > 3 between top and report_memory.) This scheme of running test modules
> one
> > at a time also speeds it up by a factor of 2; I don't understand why.
> >
> > The script I used for the module-at-a-time test is attached. It is a
> > modification of matplotlib.tests().
> >
> > Are there any nosetest experts out there with ideas about how to
> streamline
> > the standard test routine?
>
> This issue is probably worth mentionning on other mailing list of
> people using nosetest, and nosetests.
> I'm thinking of scikit-learn in particular, which also uses nosetest
> heavily. The scipy-users list might be a good place to exchange
> experience.
>
> N
>
> >
> > Eric
> >
> >
> ------------------------------------------------------------------------------
> > Time is money. Stop wasting it! Get your web API in 5 minutes.
> > www.restlet.com/download
> > http://p.sf.net/sfu/restlet
> > _______________________________________________
> > Matplotlib-devel mailing list
> > Mat...@li...
> > https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
> >
>
>
> ------------------------------------------------------------------------------
> Learn Graph Databases - Download FREE O'Reilly Book
> "Graph Databases" is the definitive new guide to graph databases and their
> applications. Written by three acclaimed leaders in the field,
> this first edition is now available. Download your free book today!
> http://p.sf.net/sfu/NeoTech
> _______________________________________________
> Matplotlib-devel mailing list
> Mat...@li...
> https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
>

> Our standard test has gotten out of control. The most serious problem is
> that running a full test suite now fails on a linux VM with 4 GB--it's out
> of memory. Half-way through the set, it is already using more than 2 GB.
> That's ridiculous. Running nosetests separately on each test module keeps
> the max reported by top to 1.6 GB, and the max by report_memory to 0.5 GB;
> still quite a bit, but tolerable. (I don't know why there is this factor of
> 3 between top and report_memory.) This scheme of running test modules one
> at a time also speeds it up by a factor of 2; I don't understand why.
>
> The script I used for the module-at-a-time test is attached. It is a
> modification of matplotlib.tests().
>
> Are there any nosetest experts out there with ideas about how to streamline
> the standard test routine?
This issue is probably worth mentionning on other mailing list of
people using nosetest, and nosetests.
I'm thinking of scikit-learn in particular, which also uses nosetest
heavily. The scipy-users list might be a good place to exchange
experience.
N
>
> Eric
>
> ------------------------------------------------------------------------------
> Time is money. Stop wasting it! Get your web API in 5 minutes.
> www.restlet.com/download
> http://p.sf.net/sfu/restlet
> _______________________________________________
> Matplotlib-devel mailing list
> Mat...@li...
> https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
>

Yes please. Last year's BoF was well-attended. I would expect nothing less
this year.
Ben
On Wed, Jun 4, 2014 at 10:39 AM, Damon McDougall <dam...@gm...>
wrote:
> Shall I go ahead and set up a MEP bof? Just got an email for a call
> for BoFs which reminded me to ask.
>
> On Mon, Jun 2, 2014 at 2:15 PM, Benjamin Root <ben...@ou...> wrote:
> > That is unfortunate that we can't have a summit before/after SciPy 2014.
> I
> > have also booked my flights and hotel, and the only time I would have to
> fit
> > a "summit" outside of SciPy 2014 would be Saturday, July 5th in the
> evening.
> >
> > I will be there, though, for the entire conference (including both sprint
> > days). Perhaps we can have a somewhat formalized Birds-of-a-feather
> session?
> > Maybe with a discussion panel and some short presentations on our visions
> > for future matplotlib development?
> >
> > Ben Root
> >
> >
> >
> > On Fri, May 30, 2014 at 9:35 AM, Michael Droettboom <md...@st...>
> wrote:
> >>
> >> Hello all,
> >>
> >> Sorry to be writing this at this late point, but I've been hoping I
> >> could find a way around it. I won't be able to attend an extra day at
> >> either end of Scipy year, both due to personal commitments and new
> >> funding constraints at NASA. I do plan to attend/host the matplotlib
> >> sprint again, however, which is not a bad opportunity to catch up on
> >> some of these issues.
> >>
> >> So, an extra developer summit day is still possible if someone else is
> >> able to organize it -- I just, unfortunately, won't be able to attend.
> >> We can still use the matplotlib donated funds to cover the cost of the
> >> extra hotel night (assuming the numbers of people wanting to do that is
> >> not too large) and meeting space (if the cost is not too high, though
> >> maybe locals like Damon have a connection for free and/or cheap space).
> >> For reimbursement, I would need a receipt for that hotel night (ideally
> >> with that one night broken out individually), which will then be
> >> submitted to numfocus, who will reimburse you directly.
> >>
> >> Sorry to be uncommunicative on this (and uncommunicative in general
> >> lately). I hope something can still work out at this late date!
> >>
> >> Mike
> >>
> >> On 02/27/2014 11:28 AM, Michael Droettboom wrote:
> >> > How many matplotlib developers are planning to attend SciPy this year?
> >> >
> >> > If we used some of our funds to support an extra hotel night, would
> any
> >> > of you be interested in spending an extra day for a "matplotlib
> >> > developer summit" to discuss matplotlib projects? This would be in
> >> > addition to the sprints, which I see probably being a larger group.
> Your
> >> > response isn't a committment at this point, I'm just trying to gauge
> how
> >> > much interest there might be.
> >> >
> >> > Mike
> >> >
> >>
> >>
> >> --
> >> Michael Droettboom
> >> Science Software Branch
> >> Space Telescope Science Institute
> >>
> >> http://www.droettboom.com
> >>
> >>
> >>
> >>
> ------------------------------------------------------------------------------
> >> Time is money. Stop wasting it! Get your web API in 5 minutes.
> >> www.restlet.com/download
> >> http://p.sf.net/sfu/restlet
> >> _______________________________________________
> >> Matplotlib-devel mailing list
> >> Mat...@li...
> >> https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
> >
> >
> >
> >
> ------------------------------------------------------------------------------
> > Learn Graph Databases - Download FREE O'Reilly Book
> > "Graph Databases" is the definitive new guide to graph databases and
> their
> > applications. Written by three acclaimed leaders in the field,
> > this first edition is now available. Download your free book today!
> > http://p.sf.net/sfu/NeoTech
> > _______________________________________________
> > Matplotlib-devel mailing list
> > Mat...@li...
> > https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
> >
>
>
>
> --
> Damon McDougall
> http://www.damon-is-a-geek.com
> Institute for Computational Engineering Sciences
> 201 E. 24th St.
> Stop C0200
> The University of Texas at Austin
> Austin, TX 78712-1229
>

Shall I go ahead and set up a MEP bof? Just got an email for a call
for BoFs which reminded me to ask.
On Mon, Jun 2, 2014 at 2:15 PM, Benjamin Root <ben...@ou...> wrote:
> That is unfortunate that we can't have a summit before/after SciPy 2014. I
> have also booked my flights and hotel, and the only time I would have to fit
> a "summit" outside of SciPy 2014 would be Saturday, July 5th in the evening.
>
> I will be there, though, for the entire conference (including both sprint
> days). Perhaps we can have a somewhat formalized Birds-of-a-feather session?
> Maybe with a discussion panel and some short presentations on our visions
> for future matplotlib development?
>
> Ben Root
>
>
>
> On Fri, May 30, 2014 at 9:35 AM, Michael Droettboom <md...@st...> wrote:
>>
>> Hello all,
>>
>> Sorry to be writing this at this late point, but I've been hoping I
>> could find a way around it. I won't be able to attend an extra day at
>> either end of Scipy year, both due to personal commitments and new
>> funding constraints at NASA. I do plan to attend/host the matplotlib
>> sprint again, however, which is not a bad opportunity to catch up on
>> some of these issues.
>>
>> So, an extra developer summit day is still possible if someone else is
>> able to organize it -- I just, unfortunately, won't be able to attend.
>> We can still use the matplotlib donated funds to cover the cost of the
>> extra hotel night (assuming the numbers of people wanting to do that is
>> not too large) and meeting space (if the cost is not too high, though
>> maybe locals like Damon have a connection for free and/or cheap space).
>> For reimbursement, I would need a receipt for that hotel night (ideally
>> with that one night broken out individually), which will then be
>> submitted to numfocus, who will reimburse you directly.
>>
>> Sorry to be uncommunicative on this (and uncommunicative in general
>> lately). I hope something can still work out at this late date!
>>
>> Mike
>>
>> On 02/27/2014 11:28 AM, Michael Droettboom wrote:
>> > How many matplotlib developers are planning to attend SciPy this year?
>> >
>> > If we used some of our funds to support an extra hotel night, would any
>> > of you be interested in spending an extra day for a "matplotlib
>> > developer summit" to discuss matplotlib projects? This would be in
>> > addition to the sprints, which I see probably being a larger group. Your
>> > response isn't a committment at this point, I'm just trying to gauge how
>> > much interest there might be.
>> >
>> > Mike
>> >
>>
>>
>> --
>> Michael Droettboom
>> Science Software Branch
>> Space Telescope Science Institute
>>
>> http://www.droettboom.com
>>
>>
>>
>> ------------------------------------------------------------------------------
>> Time is money. Stop wasting it! Get your web API in 5 minutes.
>> www.restlet.com/download
>> http://p.sf.net/sfu/restlet
>> _______________________________________________
>> Matplotlib-devel mailing list
>> Mat...@li...
>> https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
>
>
>
> ------------------------------------------------------------------------------
> Learn Graph Databases - Download FREE O'Reilly Book
> "Graph Databases" is the definitive new guide to graph databases and their
> applications. Written by three acclaimed leaders in the field,
> this first edition is now available. Download your free book today!
> http://p.sf.net/sfu/NeoTech
> _______________________________________________
> Matplotlib-devel mailing list
> Mat...@li...
> https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
>
-- 
Damon McDougall
http://www.damon-is-a-geek.com
Institute for Computational Engineering Sciences
201 E. 24th St.
Stop C0200
The University of Texas at Austin
Austin, TX 78712-1229

Hi,
On Tue, Jun 3, 2014 at 1:00 PM, Russell E. Owen <ro...@uw...> wrote:
> In article
> <CAL...@ma...>,
> Chris Barker <chr...@no...> wrote:
>
>> On Mon, Jun 2, 2014 at 5:28 PM, Matthew Brett
>> <mat...@gm...>
>> wrote:
>>
>> > > what is this going to do on OS-X 10.7 and 10.8 systems running homebrew
>> > or
>> > > macports pythons? It seems this list could get pretty long!
>> >
>> Yes, it could, but this list:
>> >
>> > so we would have to add all those if we wanted to support them...
>>
>>
>>
>> > https://www.adium.im/sparkle/?year=2014&week=22&graph=bar#osVersion
>> >
>> >
>> very interesting stats! I wonder how representative those are? Makes we
>> think we can drop 32 bit support, too. Maybe the newest 2.7 py.org binaries
>> could be 64 bit only. It would simplify things a bit.
>
> I hope you will not drop 32-bit support yet.. I still use it to
> distribute some Tkinter apps. All recent versions of ActiveState Tcl/Tk
> 8.5 have a nasty crashing bug that I have not found a workaround for,
> and old enough versions that don't have the bug need to run in 32-bit
> mode on Mavericks.
Do you need 32 bit support for the wheels or just for the MacPython
binaries? It's getting harder to build 32 / 64 bit universal
binaries these days...
Cheers,
Matthew

In article 
<CAL...@ma...>,
 Chris Barker <chr...@no...> wrote:
> On Mon, Jun 2, 2014 at 5:28 PM, Matthew Brett 
> <mat...@gm...>
> wrote:
> 
> > > what is this going to do on OS-X 10.7 and 10.8 systems running homebrew
> > or
> > > macports pythons? It seems this list could get pretty long!
> >
> Yes, it could, but this list:
> >
> > so we would have to add all those if we wanted to support them...
> 
> 
> 
> > https://www.adium.im/sparkle/?year=2014&week=22&graph=bar#osVersion
> >
> >
> very interesting stats! I wonder how representative those are? Makes we
> think we can drop 32 bit support, too. Maybe the newest 2.7 py.org binaries
> could be 64 bit only. It would simplify things a bit.
I hope you will not drop 32-bit support yet.. I still use it to 
distribute some Tkinter apps. All recent versions of ActiveState Tcl/Tk 
8.5 have a nasty crashing bug that I have not found a workaround for, 
and old enough versions that don't have the bug need to run in 32-bit 
mode on Mavericks.
-- Russell

On Mon, Jun 2, 2014 at 5:28 PM, Matthew Brett <mat...@gm...>
wrote:
> > what is this going to do on OS-X 10.7 and 10.8 systems running homebrew
> or
> > macports pythons? It seems this list could get pretty long!
>
Yes, it could, but this list:
>
> so we would have to add all those if we wanted to support them...
> https://www.adium.im/sparkle/?year=2014&week=22&graph=bar#osVersion
>
>
very interesting stats! I wonder how representative those are? Makes we
think we can drop 32 bit support, too. Maybe the newest 2.7 py.org binaries
could be 64 bit only. It would simplify things a bit.
> suggests that 10.9 is the majority, and that 10.8 and 10.7 are only
> about 14 percent combined. I guess a better case could be made for
> 10.6 (still 16 percent), but I wonder how many 10.6 hold-outs are
> updating their homebrew / macports numpies regularly.
>
not many -- it can be a really a pain to do so -- macports and homebrew
really expect you to have a recent compiler, which I think is difficult or
impossible to install on 10.6...
-Chris
-- 
Christopher Barker, Ph.D.
Oceanographer
Emergency Response Division
NOAA/NOS/OR&R (206) 526-6959 voice
7600 Sand Point Way NE (206) 526-6329 fax
Seattle, WA 98115 (206) 526-6317 main reception
Chr...@no...

Hi,
On Mon, Jun 2, 2014 at 5:14 PM, Chris Barker <chr...@no...> wrote:
> On Mon, Jun 2, 2014 at 11:48 AM, Matthew Brett <mat...@gm...>
> wrote:
>>
>> I want to rename the matplotlib wheel OSX wheel files on pypi so they
>> will also install into homebrew, macports and system python.
>
>
> +1
>
>>
>>
>> matplotlib-1.3.1-cp27-none-macosx_10_6_intel.macosx_10_9_intel.macosx_10_9_x86_64.whl
>
>
> what is this going to do on OS-X 10.7 and 10.8 systems running homebrew or
> macports pythons? It seems this list could get pretty long!
Yes, it could, but this list:
https://www.adium.im/sparkle/?year=2014&week=22&graph=bar#osVersion
suggests that 10.9 is the majority, and that 10.8 and 10.7 are only
about 14 percent combined. I guess a better case could be made for
10.6 (still 16 percent), but I wonder how many 10.6 hold-outs are
updating their homebrew / macports numpies regularly.
Cheers,
Matthew

On Mon, Jun 2, 2014 at 11:48 AM, Matthew Brett <mat...@gm...>
wrote:
> I want to rename the matplotlib wheel OSX wheel files on pypi so they
> will also install into homebrew, macports and system python.
>
+1
>
> matplotlib-1.3.1-cp27-none-macosx_10_6_intel.macosx_10_9_intel.macosx_10_9_x86_64.whl
>
what is this going to do on OS-X 10.7 and 10.8 systems running homebrew or
macports pythons? It seems this list could get pretty long!
-Chris
-- 
Christopher Barker, Ph.D.
Oceanographer
Emergency Response Division
NOAA/NOS/OR&R (206) 526-6959 voice
7600 Sand Point Way NE (206) 526-6329 fax
Seattle, WA 98115 (206) 526-6317 main reception
Chr...@no...

That is unfortunate that we can't have a summit before/after SciPy 2014. I
have also booked my flights and hotel, and the only time I would have to
fit a "summit" outside of SciPy 2014 would be Saturday, July 5th in the
evening.
I will be there, though, for the entire conference (including both sprint
days). Perhaps we can have a somewhat formalized Birds-of-a-feather
session? Maybe with a discussion panel and some short presentations on our
visions for future matplotlib development?
Ben Root
On Fri, May 30, 2014 at 9:35 AM, Michael Droettboom <md...@st...> wrote:
> Hello all,
>
> Sorry to be writing this at this late point, but I've been hoping I
> could find a way around it. I won't be able to attend an extra day at
> either end of Scipy year, both due to personal commitments and new
> funding constraints at NASA. I do plan to attend/host the matplotlib
> sprint again, however, which is not a bad opportunity to catch up on
> some of these issues.
>
> So, an extra developer summit day is still possible if someone else is
> able to organize it -- I just, unfortunately, won't be able to attend.
> We can still use the matplotlib donated funds to cover the cost of the
> extra hotel night (assuming the numbers of people wanting to do that is
> not too large) and meeting space (if the cost is not too high, though
> maybe locals like Damon have a connection for free and/or cheap space).
> For reimbursement, I would need a receipt for that hotel night (ideally
> with that one night broken out individually), which will then be
> submitted to numfocus, who will reimburse you directly.
>
> Sorry to be uncommunicative on this (and uncommunicative in general
> lately). I hope something can still work out at this late date!
>
> Mike
>
> On 02/27/2014 11:28 AM, Michael Droettboom wrote:
> > How many matplotlib developers are planning to attend SciPy this year?
> >
> > If we used some of our funds to support an extra hotel night, would any
> > of you be interested in spending an extra day for a "matplotlib
> > developer summit" to discuss matplotlib projects? This would be in
> > addition to the sprints, which I see probably being a larger group. Your
> > response isn't a committment at this point, I'm just trying to gauge how
> > much interest there might be.
> >
> > Mike
> >
>
>
> --
> Michael Droettboom
> Science Software Branch
> Space Telescope Science Institute
>
> http://www.droettboom.com
>
>
>
> ------------------------------------------------------------------------------
> Time is money. Stop wasting it! Get your web API in 5 minutes.
> www.restlet.com/download
> http://p.sf.net/sfu/restlet
> _______________________________________________
> Matplotlib-devel mailing list
> Mat...@li...
> https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
>

Hi,
Sorry for those of you on the numpy mailing list - this email will
seem a bit familiar.
I want to rename the matplotlib wheel OSX wheel files on pypi so they
will also install into homebrew, macports and system python.
I'm just doing this now for numpy and scipy, but I wanted to make sure
y'all had no objections for matplotlib.
The logic of the renaming is explained here:
https://github.com/MacPython/wiki/wiki/Spinning-wheels
Basically, by adding extra 'platform tags' to the wheel filename, we
can signal to pip that the wheel is compatible with these other
systems.
The page above explains why the wheels I built are compatible with the
other Pythons, and this test grid:
https://travis-ci.org/matthew-brett/scipy-stack-osx-testing/builds/26482436
confirms that the renamed wheels install and test OK on homebrew,
macports, system python.
So, I propose to rename the current matplotlib python 2.7 wheel from:
matplotlib-1.3.1-cp27-none-macosx_10_6_intel.whl
to
matplotlib-1.3.1-cp27-none-macosx_10_6_intel.macosx_10_9_intel.macosx_10_9_x86_64.whl
and so on for python 3.3, 3.4.
I don't think this has a downside, but I'm happy for any feedback,
Cheers,
Matthew

Our standard test has gotten out of control. The most serious problem 
is that running a full test suite now fails on a linux VM with 4 
GB--it's out of memory. Half-way through the set, it is already using 
more than 2 GB. That's ridiculous. Running nosetests separately on each 
test module keeps the max reported by top to 1.6 GB, and the max by 
report_memory to 0.5 GB; still quite a bit, but tolerable. (I don't 
know why there is this factor of 3 between top and report_memory.) This 
scheme of running test modules one at a time also speeds it up by a 
factor of 2; I don't understand why.
The script I used for the module-at-a-time test is attached. It is a 
modification of matplotlib.tests().
Are there any nosetest experts out there with ideas about how to 
streamline the standard test routine?
Eric