matplotlib-devel Mailing List for matplotlib

Hi all,
Since we're considering the possibility of making a matplotlib 2.0
release with a better default colormap, it occurred to me that it
might make sense to take this opportunity to improve other visual
defaults.
Defaults are important. Obviously for publication graphs you'll want
to end up tweaking every detail, but (a) not everyone does but we
still have to read their graphs, and (b) probably only 1% of the plots
I make are for publication; the rest are quick one-offs that I make
on-the-fly to help me understand my own data. For such plots it's
usually not worth spending much/any time tweaking layout details, I
just want something usable, quickly. And I think there's a fair amount
of low-hanging improvements possible.
Batching multiple visual changes like this together seems much better
than spreading them out over multiple releases. It keeps the messaging
super easy to understand: "matplotlib 2.0 is just like 1.x, your code
will still work, the only difference is that your plots will look
better by default". And grouping these changes together makes it
easier to provide for users who need to revert back to the old
defaults -- it's easy to provide simple binary choice between "before
2.0" versus "after 2.0", harder to keep track of a bunch of different
changes spread over multiple releases.
Some particular annoyances I often run into and that might be
candidates for changing:
- The default method of choosing axis limits is IME really, really
annoying, because of the way it tries to find "round number"
boundaries. It's a clever idea, but in practice I've almost never seen
this pick axis limits that are particularly meaningful for my data,
and frequently it picks particularly bad ones. For example, suppose
you want to plot the spectrum of a signal; because of FFT's preference
for power-of-two sizes works it's natural to end up with samples
ranging from 0 to 255. If you plot this, matplotlib will give you an
xlim of (0, 300), which looks pretty ridiculous. But even worse is the
way this method of choosing xlims can actually obscure data -- if the
extreme values in your data set happen to fall exactly on a "round
number", then this will be used as the axis limits, and you'll end up
with data plotted directly underneath the axis spine. I frequently
encounter this when making scatter plots of data in the 0-1 range --
the points located at exactly 0 and 1 are very important to see, but
are nearly invisible by default. A similar case I ran into recently
was when plotting autocorrelation functions for different signals. For
reference I wanted to include the theoretically ideal ACF for white
noise, which looks like this:
 plt.plot(np.arange(1000), [1] + [0] * 999)
Good luck reading that plot!
R's default rule for deciding axis limits is very simple: extend the
data range by 4% on each side; those are your limits. IME this rule --
while obviously not perfect -- always produces something readable and
unobjectionable.
- Axis tickmarks should point outwards rather than inwards: There's
really no advantage to making them point inwards, and pointing inwards
means they can obscure data. My favorite example of this is plotting a
histogram with 100 bins -- that's an obvious thing to do, right? Check
it out:
 plt.hist(np.random.RandomState(0).uniform(size=100000), bins=100)
This makes me do a double-take every few months until I remember
what's going on: "WTF why is the bar on the left showing a *stacked*
barplot...ohhhhh right those are just the ticks, which happen to be
exactly the same width as the bar." Very confusing.
Seaborn's built-in themes give you the options of (1) no axis ticks at
all, just a background grid (by default the white-on-light-grey grid
as popularized by ggplot2), (2) outwards pointing tickmarks. Either
option seems like a better default to me!
- Default line colors: The rgbcmyk color cycle for line plots doesn't
appear to be based on any real theory about visualization -- it's just
the corners of the RGB color cube, which is a highly perceptually
non-uniform space. The resulting lines aren't terribly high contrast
against the default white background, and the different colors have
varying luminance that makes some lines "pop out" more than others.
Seaborn's default is to use a nice isoluminant variant on matplotlib's default:
 http://web.stanford.edu/~mwaskom/software/seaborn/tutorial/aesthetics.html
ggplot2 uses isoluminant colors with maximally-separated hues, which
also works well. E.g.:
 http://www.cookbook-r.com/Graphs/Colors_%28ggplot2%29/ggplot2_scale_hue_colors_l45.png
- Line thickness: basically every time I make a line plot I wish the
lines were thicker. This is another thing that seaborn simply changes
unconditionally.
In general I guess we could do a lot worse than to simply adopt
seaborn's defaults as the matplotlib defaults :-) Their full list of
overrides can be seen here:
 https://github.com/mwaskom/seaborn/blob/master/seaborn/rcmod.py#L135
 https://github.com/mwaskom/seaborn/blob/master/seaborn/rcmod.py#L301
- Dash styles: a common recommendation for line plots is to
simultaneously vary both the color and the dash style of your lines,
because redundant cues are good and dash styles are more robust than
color in the face of greyscale printing etc. But every time I try to
follow this advice I find myself having to define new dashes from
scratch, because matplotlib's default dash styles ("-", "--", "-.",
":") have wildly varying weights; in particular I often find it hard
to even see the dots in the ":" and "-." styles. Here's someone with a
similar complaint:
 http://philbull.wordpress.com/2012/03/14/custom-dashdot-line-styles-in-matplotlib/
Just as very rough numbers, something along the lines of "--" = [7,
4], "-." = [7, 4, 3, 4], ":" = [2, 1.5] looks much better to me.
It might also make sense to consider baking the advice I mentioned
above into matplotlib directly, and having a non-trivial dash cycle
enabled by default. (So the first line plotted uses "-", second uses
"--" or similar, etc.) This would also have the advantage that if we
make the length of the color cycle and the dash cycle relatively
prime, then we'll dramatically increase the number of lines that can
be plotted on the same graph with distinct appearances. (I often run
into the annoying situation where I throw up a quick-and-dirty plot,
maybe with something like pandas's dataframe.plot(), and then discover
that I have multiple indistinguishable lines.)
Obviously one could quibble with my specific proposals here, but does
in general seem like a useful thing to do?
-n
-- 
Nathaniel J. Smith
Postdoctoral researcher - Informatics - University of Edinburgh
http://vorpus.org

On 2014年11月21日, 4:42 PM, Nathaniel Smith wrote:
> On Fri, Nov 21, 2014 at 5:46 PM, Darren Dale <dsd...@gm...> wrote:
>> On Fri, Nov 21, 2014 at 12:32 PM, Phil Elson <pel...@gm...> wrote:
>>>
>>> Please use this thread to discuss the best choice for a new default
>>> matplotlib colormap.
>>>
>>> This follows on from a discussion on the matplotlib-devel mailing list
>>> entitled "How to move beyond JET as the default matplotlib colormap".
>>
>>
>> I remember reading a (peer-reviewed, I think) article about how "jet" was a
>> very unfortunate choice of default. I can't find the exact article now, but
>> I did find some other useful ones:
>>
>> http://cresspahl.blogspot.com/2012/03/expanded-control-of-octaves-colormap.html
>> http://www.sandia.gov/~kmorel/documents/ColorMaps/
>> http://www.sandia.gov/~kmorel/documents/ColorMaps/ColorMapsExpanded.pdf
>
> Those are good articles. There's a lot of literature on the problems
> with "jet", and lots of links in the matplotlib issue [1]. For those
> trying to get up to speed quickly, MathWorks recently put together a
> nice review of the literature [2]. One particularly striking paper
> they cite studied a group of medical students and found that (a) they
> were used to/practiced at using jet, (b) when given a choice of
> colormaps they said that they preferred jet, (c) they nonetheless made
> more *medical diagnostic errors* when using jet than with better
> designed colormaps (Borkin et al, 2011).
>
> I won't suggest a specific colormap, but I do propose that whatever we
> chose satisfy the following criteria:
>
> - it should be a sequential colormap, because diverging colormaps are
> really misleading unless you know where the "center" of the data is,
> and for a default colormap we generally won't.
>
> - it should be perceptually uniform, i.e., human subjective judgements
> of how far apart nearby colors are should correspond as linearly as
> possible to the difference between the numerical values they
> represent, at least locally. There's lots of research on how to
> measure perceptual distance -- a colleague and I happen to have
> recently implemented a state-of-the-art model of this for another
> project, in case anyone wants to play with it [3], or just using
> good-old-L*a*b* is a reasonable quick-and-dirty approximation.
>
> - it should have a perceptually uniform luminance ramp, i.e. if you
> convert to greyscale it should still be uniform. This is useful both
> in practical terms (greyscale printers are still a thing!) and because
> luminance is a very strong and natural cue to magnitude.
>
> - it should also have some kind of variation in hue, because hue
> variation is a really helpful additional cue to perception, having two
> cues is better than one, and there's no reason not to do it.
>
> - the hue variation should be chosen to produce reasonable results
> even for viewers with the more common types of colorblindness. (Which
> rules out things like red-to-green.)
>
> And, for bonus points, it would be nice to choose a hue ramp that
> still works if you throw away the luminance variation, because then we
> could use the version with varying luminance for 2d plots, and the
> version with just hue variation for 3d plots. (In 3d plots you really
> want to reserve the luminance channel for lighting/shading, because
> your brain is *really* good at extracting 3d shape from luminance
> variation. If the 3d surface itself has massively varying luminance
> then this screws up the ability to see shape.)
>
> Do these seem like good requirements?
Goals, yes, though I wouldn't put much weight on the "bonus" criterion. 
 I would add that it should be aesthetically pleasing, or at least 
comfortable, to most people. Perfection might not be attainable, and 
some tradeoffs may be required. Is anyone set up to produce test images 
and/or metrics for judging existing colormaps, or newly designed ones, 
on all of these criteria?
Eric
>
> -n
>
> [1] https://github.com/matplotlib/matplotlib/issues/875
> [2] http://uk.mathworks.com/company/newsletters/articles/rainbow-color-map-critiques-an-overview-and-annotated-bibliography.html
> [3] https://github.com/njsmith/pycam02ucs ; install (or just run out
> of the source tree) and then use pycam02ucs.deltaEp_sRGB to compute
> the perceptual distance between two RGB colors. It's also possible to
> use the underlying color model stuff to do things like generate colors
> with evenly spaced luminance and hues, or draw 3d plots of the shape
> of the human color space. It's pretty fun to play with :-)
>

On Fri, Nov 21, 2014 at 5:46 PM, Darren Dale <dsd...@gm...> wrote:
> On Fri, Nov 21, 2014 at 12:32 PM, Phil Elson <pel...@gm...> wrote:
>>
>> Please use this thread to discuss the best choice for a new default
>> matplotlib colormap.
>>
>> This follows on from a discussion on the matplotlib-devel mailing list
>> entitled "How to move beyond JET as the default matplotlib colormap".
>
>
> I remember reading a (peer-reviewed, I think) article about how "jet" was a
> very unfortunate choice of default. I can't find the exact article now, but
> I did find some other useful ones:
>
> http://cresspahl.blogspot.com/2012/03/expanded-control-of-octaves-colormap.html
> http://www.sandia.gov/~kmorel/documents/ColorMaps/
> http://www.sandia.gov/~kmorel/documents/ColorMaps/ColorMapsExpanded.pdf
Those are good articles. There's a lot of literature on the problems
with "jet", and lots of links in the matplotlib issue [1]. For those
trying to get up to speed quickly, MathWorks recently put together a
nice review of the literature [2]. One particularly striking paper
they cite studied a group of medical students and found that (a) they
were used to/practiced at using jet, (b) when given a choice of
colormaps they said that they preferred jet, (c) they nonetheless made
more *medical diagnostic errors* when using jet than with better
designed colormaps (Borkin et al, 2011).
I won't suggest a specific colormap, but I do propose that whatever we
chose satisfy the following criteria:
- it should be a sequential colormap, because diverging colormaps are
really misleading unless you know where the "center" of the data is,
and for a default colormap we generally won't.
- it should be perceptually uniform, i.e., human subjective judgements
of how far apart nearby colors are should correspond as linearly as
possible to the difference between the numerical values they
represent, at least locally. There's lots of research on how to
measure perceptual distance -- a colleague and I happen to have
recently implemented a state-of-the-art model of this for another
project, in case anyone wants to play with it [3], or just using
good-old-L*a*b* is a reasonable quick-and-dirty approximation.
- it should have a perceptually uniform luminance ramp, i.e. if you
convert to greyscale it should still be uniform. This is useful both
in practical terms (greyscale printers are still a thing!) and because
luminance is a very strong and natural cue to magnitude.
- it should also have some kind of variation in hue, because hue
variation is a really helpful additional cue to perception, having two
cues is better than one, and there's no reason not to do it.
- the hue variation should be chosen to produce reasonable results
even for viewers with the more common types of colorblindness. (Which
rules out things like red-to-green.)
And, for bonus points, it would be nice to choose a hue ramp that
still works if you throw away the luminance variation, because then we
could use the version with varying luminance for 2d plots, and the
version with just hue variation for 3d plots. (In 3d plots you really
want to reserve the luminance channel for lighting/shading, because
your brain is *really* good at extracting 3d shape from luminance
variation. If the 3d surface itself has massively varying luminance
then this screws up the ability to see shape.)
Do these seem like good requirements?
-n
[1] https://github.com/matplotlib/matplotlib/issues/875
[2] http://uk.mathworks.com/company/newsletters/articles/rainbow-color-map-critiques-an-overview-and-annotated-bibliography.html
[3] https://github.com/njsmith/pycam02ucs ; install (or just run out
of the source tree) and then use pycam02ucs.deltaEp_sRGB to compute
the perceptual distance between two RGB colors. It's also possible to
use the underlying color model stuff to do things like generate colors
with evenly spaced luminance and hues, or draw 3d plots of the shape
of the human color space. It's pretty fun to play with :-)
-- 
Nathaniel J. Smith
Postdoctoral researcher - Informatics - University of Edinburgh
http://vorpus.org

I am a bit wary of doing a 2.0 _just_ to change the color map, but when
every I try to write out why, they don't sound convincing. We may end up
with a 3.0 within a year or so due to the possible plotting API/pyplot work
that is (hopefully) coming.
If we are going to do this, I think we should do the 1.4.3 release
(scheduled for Feb 1, RCs in mid January), then put one commit to change
the color map on top of that, tag 2.0 and then master turns into 2.1.x
(targeted for right after scipy?).
There is also the thought to get the major c++ refactor work tagged and
released sooner rather than later so maybe we want to do 1.4.3, 1.5.0 and
2.0 in Feb with 2.0 based off of 1.5 not 1.4.
On Fri Nov 21 2014 at 12:52:03 PM Benjamin Root <ben...@ou...> wrote:
> As a point of clarification, is this proposed 2.0 release different from
> the 1.5 release?
>
> On Fri, Nov 21, 2014 at 12:32 PM, Phil Elson <pel...@gm...> wrote:
>
>> Many of you will be aware of there has been an ongoing issue (#875,
>> http://goo.gl/xLZvuL) which recommends the removal of Jet as the default
>> colormap in matplotlib.
>>
>> The argument against Jet is compelling and I think that as a group who
>> care about high quality visualisation we should be seriously discussing how
>> matplotlib can move beyond Jet.
>>
>> There was recently an open letter to the climate science community
>> <http://www.climate-lab-book.ac.uk/2014/end-of-the-rainbow/> asking for
>> scientists to "pledge" against using rainbow like colormaps (such as Jet),
>> and there are similar initiatives in other scientific fields, as well as
>> there being a plethora of well researched literature on the subject.
>>
>> As such, it's time to agree on a solution on how matplotlib can reach the
>> end of the rainbow.
>>
>>
>> The two major hurdles, AFAICS, to replacing the three little characters
>> which control the default colormap of matplotlib are:
>>
>> * We haven't had a clear (decisive) discussion about what we should
>> replace Jet with.
>> * There are concerns about changing the default as it would change the
>> existing widespread behaviour.
>>
>> To address the first point I'll start a new mailinglist thread (entitled
>> "Matplotlib's new default colormap") where new default colormap suggestions
>> can be made. The thread should strictly avoid "+1" type comments, and
>> generally try to stick to reference-able/demonstrable fact, rather than
>> opinion. There *will* be a difference of opinion, however the final
>> decision has to come down to the project lead (sorry Mike) who I know will
>> do whatever is necessary to make the best choice for matplotlib.
>>
>> The second point is a reasonable response when we consider that
>> matplotlib as a project has no *clear* statement on backwards
>> compatibility. As a result, matplotlib is highly change averse between
>> minor releases (to use semantic versioning terms) and therefore changing
>> the default colormap is unpalatable in the v1.x release series. As a result
>> I'd like to propose that the next release of matplotlib be called 2.0, with
>> the *only* major backwards-incompatible change be the removal of Jet as
>> the default colormap.
>>
>> As a project matplotlib mustn't get caught up in the trap of shying away
>> from a major version release when the need arises, and in my opinion
>> helping our users to avoid using a misleading colormap is a worthy cause
>> for a v2.0.
>>
>> Please try to keep this thread on the "how", and not on the "what" of the
>> replacement default colormap, for which there is a dedicated thread.
>>
>> Thanks,
>>
>> Phil
>>
>> (#endrainbow)
>>
>>
>> ------------------------------------------------------------------------------
>> Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server
>> from Actuate! Instantly Supercharge Your Business Reports and Dashboards
>> with Interactivity, Sharing, Native Excel Exports, App Integration & more
>> Get technology previously reserved for billion-dollar corporations, FREE
>>
>> http://pubads.g.doubleclick.net/gampad/clk?id=157005751&iu=/4140/ostg.clktrk
>> _______________________________________________
>> Matplotlib-devel mailing list
>> Mat...@li...
>> https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
>>
>>
> ------------------------------------------------------------
> ------------------
> Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server
> from Actuate! Instantly Supercharge Your Business Reports and Dashboards
> with Interactivity, Sharing, Native Excel Exports, App Integration & more
> Get technology previously reserved for billion-dollar corporations, FREE
> http://pubads.g.doubleclick.net/gampad/clk?id=157005751&
> iu=/4140/ostg.clktrk_______________________________________________
> Matplotlib-devel mailing list
> Mat...@li...
> https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
>

As a point of clarification, is this proposed 2.0 release different from
the 1.5 release?
On Fri, Nov 21, 2014 at 12:32 PM, Phil Elson <pel...@gm...> wrote:
> Many of you will be aware of there has been an ongoing issue (#875,
> http://goo.gl/xLZvuL) which recommends the removal of Jet as the default
> colormap in matplotlib.
>
> The argument against Jet is compelling and I think that as a group who
> care about high quality visualisation we should be seriously discussing how
> matplotlib can move beyond Jet.
>
> There was recently an open letter to the climate science community
> <http://www.climate-lab-book.ac.uk/2014/end-of-the-rainbow/> asking for
> scientists to "pledge" against using rainbow like colormaps (such as Jet),
> and there are similar initiatives in other scientific fields, as well as
> there being a plethora of well researched literature on the subject.
>
> As such, it's time to agree on a solution on how matplotlib can reach the
> end of the rainbow.
>
>
> The two major hurdles, AFAICS, to replacing the three little characters
> which control the default colormap of matplotlib are:
>
> * We haven't had a clear (decisive) discussion about what we should
> replace Jet with.
> * There are concerns about changing the default as it would change the
> existing widespread behaviour.
>
> To address the first point I'll start a new mailinglist thread (entitled
> "Matplotlib's new default colormap") where new default colormap suggestions
> can be made. The thread should strictly avoid "+1" type comments, and
> generally try to stick to reference-able/demonstrable fact, rather than
> opinion. There *will* be a difference of opinion, however the final
> decision has to come down to the project lead (sorry Mike) who I know will
> do whatever is necessary to make the best choice for matplotlib.
>
> The second point is a reasonable response when we consider that matplotlib
> as a project has no *clear* statement on backwards compatibility. As a
> result, matplotlib is highly change averse between minor releases (to use
> semantic versioning terms) and therefore changing the default colormap is
> unpalatable in the v1.x release series. As a result I'd like to propose
> that the next release of matplotlib be called 2.0, with the *only* major
> backwards-incompatible change be the removal of Jet as the default colormap.
>
> As a project matplotlib mustn't get caught up in the trap of shying away
> from a major version release when the need arises, and in my opinion
> helping our users to avoid using a misleading colormap is a worthy cause
> for a v2.0.
>
> Please try to keep this thread on the "how", and not on the "what" of the
> replacement default colormap, for which there is a dedicated thread.
>
> Thanks,
>
> Phil
>
> (#endrainbow)
>
>
> ------------------------------------------------------------------------------
> Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server
> from Actuate! Instantly Supercharge Your Business Reports and Dashboards
> with Interactivity, Sharing, Native Excel Exports, App Integration & more
> Get technology previously reserved for billion-dollar corporations, FREE
>
> http://pubads.g.doubleclick.net/gampad/clk?id=157005751&iu=/4140/ostg.clktrk
> _______________________________________________
> Matplotlib-devel mailing list
> Mat...@li...
> https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
>
>

On Fri, Nov 21, 2014 at 12:32 PM, Phil Elson <pel...@gm...> wrote:
> Please use this thread to discuss the best choice for a new *default*
> matplotlib colormap.
>
> This follows on from a discussion on the matplotlib-devel mailing list
> entitled "How to move beyond JET as the default matplotlib colormap".
>
I remember reading a (peer-reviewed, I think) article about how "jet" was a
very unfortunate choice of default. I can't find the exact article now, but
I did find some other useful ones:
http://cresspahl.blogspot.com/2012/03/expanded-control-of-octaves-colormap.html
http://www.sandia.gov/~kmorel/documents/ColorMaps/
http://www.sandia.gov/~kmorel/documents/ColorMaps/ColorMapsExpanded.pdf
Darren

Please use this thread to discuss the best choice for a new *default*
matplotlib colormap.
This follows on from a discussion on the matplotlib-devel mailing list
entitled "How to move beyond JET as the default matplotlib colormap".
It is accepted that there can never be a *best* colormap for *all* data, so
some documentation on choosing an appropriate colormap for specific data
should always be sought. Nonetheless, matplotlib does need a default, and
it has been shown just how damaging the Jet (matplotlib's current default)
colormap really is, so we need to come up with a genuine alternative.
To keep this thread as useful as possible please avoid posting "+1" type
messages. If you'd like to suggest a colormap for consideration as
matplotlib's new *default* please try to keep to
reference-able/demonstrable fact.
Thanks,
Phil

Many of you will be aware of there has been an ongoing issue (#875,
http://goo.gl/xLZvuL) which recommends the removal of Jet as the default
colormap in matplotlib.
The argument against Jet is compelling and I think that as a group who care
about high quality visualisation we should be seriously discussing how
matplotlib can move beyond Jet.
There was recently an open letter to the climate science community
<http://www.climate-lab-book.ac.uk/2014/end-of-the-rainbow/> asking for
scientists to "pledge" against using rainbow like colormaps (such as Jet),
and there are similar initiatives in other scientific fields, as well as
there being a plethora of well researched literature on the subject.
As such, it's time to agree on a solution on how matplotlib can reach the
end of the rainbow.
The two major hurdles, AFAICS, to replacing the three little characters
which control the default colormap of matplotlib are:
 * We haven't had a clear (decisive) discussion about what we should
replace Jet with.
 * There are concerns about changing the default as it would change the
existing widespread behaviour.
To address the first point I'll start a new mailinglist thread (entitled
"Matplotlib's new default colormap") where new default colormap suggestions
can be made. The thread should strictly avoid "+1" type comments, and
generally try to stick to reference-able/demonstrable fact, rather than
opinion. There *will* be a difference of opinion, however the final
decision has to come down to the project lead (sorry Mike) who I know will
do whatever is necessary to make the best choice for matplotlib.
The second point is a reasonable response when we consider that matplotlib
as a project has no *clear* statement on backwards compatibility. As a
result, matplotlib is highly change averse between minor releases (to use
semantic versioning terms) and therefore changing the default colormap is
unpalatable in the v1.x release series. As a result I'd like to propose
that the next release of matplotlib be called 2.0, with the *only* major
backwards-incompatible change be the removal of Jet as the default colormap.
As a project matplotlib mustn't get caught up in the trap of shying away
from a major version release when the need arises, and in my opinion
helping our users to avoid using a misleading colormap is a worthy cause
for a v2.0.
Please try to keep this thread on the "how", and not on the "what" of the
replacement default colormap, for which there is a dedicated thread.
Thanks,
Phil
(#endrainbow)