-
-
Notifications
You must be signed in to change notification settings - Fork 8k
Colorbar axis zoom and pan #19515
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Colorbar axis zoom and pan #19515
Conversation
Fwiw the feature looks pretty natural to me, and it's certainly something I've been wanting to have before.
243f8a1
to
5a6c99b
Compare
Now that #20054 is in I think this is now working properly on horizontal/vertical colorbars.
I didn't find any tests of the Zoom/Pan tools, am I missing those somewhere, or do we not test those ones and only test the widgets separately?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
OK, so if I zoom, I change the color limits of the ScalarMappable, but if I set the limits of the axis, i.e. with set_xlim
it zooms in on part of the colorbar (at least after #20054) but leaves the mappable unchanged. This is inconsistent, and just to be clear the new behaviour in #20054 was requested for various reasons I dont' fully understand.
Further, I don't see how to expand the limits with this GUI API.
Overall I'm not 100% sold that the colorbar is the right GUI for this.
I wonder if slightly more obscure, but equally evocative mouse motions could be used. ie. perhaps shift-scroll-up/down to increase the range and ctrl-scroll-up/down to move the centre? Then its not confused with just setting the limits?
Anyway, I think this is an interesting idea, but I'm not sure if it should be exactly the same as zoom and pan from the toolbar.
I agree with @jklymak concerns. We have two spaces, the data space (a, b) and the normalized "color" range (0, 1), which is associated with the color values. A the colored area in a colormap represents this color range and naively, zooming-in into the color range would result in a subrange and this a fraction of the color-range to be used, while the norm (i.e. the data-to-color relation) is unchanged. This however is less useful because you don't get additional detail. Actually, I'm wondering if it is useful at all.
What people usually want is to manipulate the norm. Technically, you would like to move the ticks on the colorbar, but that's not something we can directly support as a mouse interaction.
So two questions:
- Is the naive color-range manipulation something that we consider supporting at all? If so, we cannot use zooming and panning tools for norm manipulation.
- If not, is the changed semantics of the tools for colorbars as proposed here bearable in terms of user expectation or is that too confusing?
Mouse scrolling
There are standard conventions for mouse scrolling
scroll-up/down: vertical scroll
shift+scroll-up/down: horizontal scroll
ctrl+scroll-up/down: zoom in/out
For a regular axes the first to should translate to pan and the third should translate to zoom. (We don't have this implemented right now, but it would be a nice addition if anybody is interested in picking this up.)
We'd have to think if these could be resaonbly be adapted for colorbars in the above context.
Maybe a use-case for why I wanted this in the first place would be helpful. When making figures, coauthors will often say, well what if vmin/vmax were just a little higher/lower, how would that look? That could be done with adding widgets/sliders to control the vmin/vmax, but I don't want to be adding components, so I figured this was a good way to fit it in. Zoom/pan seemed natural to me, but if it isn't to other people we can consider some other interactions as well.
-
set_xlim()
- interesting, I hadn't considered that. It seems likeset_xlim
/set_ylim
on a Colorbar axis should be dispatched tonorm.vmin
/norm.vmax
and not zoomed in to a smaller data range. For example, that could be misleading if you have extensions on your colorbar, but the limits at the top aren't actually the norm limits... -
Zoom out / extend limits - This should be right click and drag I believe, the same as a normal axis because I just took most of the code from there. This also works with the home, back, and forward buttons too.
-
Moving the ticks on the colorbar - I'm not sure I follow here? The ticks are moving in the above example. Do you mean you want to interact with just the ticks outside the axis and not inside the axis where the colors are?
As for the questions about whether this seems bearable for user expectations or not... I came up with this idea, so I'd say yes, but YMMV :) Another thing here is that you have to explicitly enter zoom/pan mode to get this available, so it isn't something that will just blindly be available for someone mousing over the image, so there is that safety net in a sense.
So maybe we start with set_xlim
/set_ylim
? Currently we do not have a colorbar.set_lim
because we usually expect the user to change the colorbar by changing the vmin/vmax on the Norm. Hence the colorbar is a subordinate to the Norm. This PR inverts that and makes the colorbar actively control the norm. I will say this is also a confusion I see on stackoverflow that folks think the colorbar should somehow control the colors in the image. Currently, it is really easy currently to say "no it doesn't", the colorbar passively reflects the norm. If we no longer make that the case, then we need to think this through very carefully what the back and forth is and have thorough tests.
Again, maybe this is OK, but can we develop a programatic API first that doesn't rely on pan and zoom? i.e. colorbar.set_lim or colorbar.set_vlims, or?? That would make me a lot more comfortable rather than jumping to a GUI-only API, and would make the discussion a lot easier...
Some questions. If I set the vlim on the colorbar, and then I set it on the mappable, which is used at draw time? What if someone flips the limits? Norms often have other parameters than limits, in particulate everyone's favourite is symlognorm, or centered norm. How will this interact with those? Will the center move as well?
I'll put on the dev call for today. Feel free to stop by and pitch it. I don't know if we should discuss all the nitty gritty, but putting the PR on folks' radar and asking for comment here (or if you make a simpler colorbar.set_vlims
PR) would be great.
I didn't find any tests of the Zoom/Pan tools, am I missing those somewhere, or do we not test those ones and only test the widgets separately?
Zoom is tested in lib/matplotlib/tests/test_backend_bases.py::test_interactive_zoom
.
I guess the other UI for this is instead of mucking with the colorbar directly, a norm editor pops up in a separate helper window. This could also change the colormap.
@jklymak, you missed a good lengthy discussion today :) I think what you're proposing is essentially @anntzer's project here: https://github.com/anntzer/mplinorm
I'll be giving this some more thought over the next week and try to improve the UI.
OK, so if I zoom, I change the color limits of the ScalarMappable, but if I set the limits of the axis, i.e. with set_xlim it zooms in on part of the colorbar (at least after #20054) but leaves the mappable unchanged.
I think this is a mistake, will open a different issue about that next, but resolving that is probably a pre-req for merging this.
Further, I don't see how to expand the limits with this GUI API.
I would assume that right click zoom and right-click pan "zoom out" like they do with normal axes?
I think that the pan/zoom verbiage is semantically correct (if a bit jarring on first pass), but the UI needs a bit of tuning to make it clearer what is happening.
Although we implement the colorbar on an (increasingly standard) Axes, it is much more like the tick labels than it is like an normal "image". Under the hood we have transforms that go from data space -> x/y. The absolute position within the figure is meaningless, but by putting ticks on the Axes we can locally give meaning to the (relative) position. Even if we remove the ticks, we the relative positions with in an Axes still have meaning. When we adjust these transforms (by adjusting the x and y limits) we call this "panning" and "zooming" (depending on if we change the dynamic range or not).
Similarly, when we color map we have a transform (implemented in 2 parts: the norm and the colormap) that take data from dataspace -> RGB. The absolute color has no meaning (change the color map does not change the inherent meaning of the data any more than translating the whole axes around the figure changes the meaning!), but the relative colors tell us something about the relations between the data. If we then add a colorbar then we can get back access to the absolute values the same way ticks do.
In the images below, I think it is un-controversial to say "the y-axis was zoomed" or "the y-axis was panned". Doing exactly the same thing to the clim I think makes sense to use the same wording (there are some slight differences like when the curve goes out of the bounds it gets clipped but the color mapping saturates, could set over/under colors to transparent to get the same effect). Given the limitations of screens (we can not get a color axis sticking out of the screen!), I think implementing pan/zoom on the colorbar to "pan" and "zoom" in norm-space is the next best thing.
Please forgive the copy-pasta nature of this code!
import matplotlib.pyplot as plt import numpy as np th = np.linspace(start := 0, stop := 2 * np.pi, N := 1024) data1d = np.sin(th) data2d = np.sin(th[:, np.newaxis]) * np.cos(th[np.newaxis, :]) curve_axes = ["1d", "y zoom in", "y zoom out"] image_axes = ["2d", "color zoom in", "color zoom out"] fig, ax_dict = plt.subplot_mosaic( [ curve_axes, image_axes, ], constrained_layout=True, ) for an in curve_axes: ax = ax_dict[an] ax.plot(th, data1d) ax.set_title(an) if an == "y zoom in": ax.set_ylim(-0.01, 0.01) elif an == "y zoom out": ax.set_ylim(-100, 100) else: ax.set_ylim(-1, 1) for an in image_axes: ax = ax_dict[an] im = ax.imshow( data2d, extent=[start - (stop - start) / (2 * N), stop + (stop - start) / (2 * N)] * 2, ) ax.set_title(an) fig.colorbar(im, ax=ax, aspect=7) if an == "color zoom in": im.set_clim(-0.01, 0.01) elif an == "color zoom out": im.set_clim(-100, 100) else: im.set_clim(-1, 1) fig.savefig("/tmp/zoom.png") curve_axes = ["1d", "y pan up", "y pan down"] image_axes = ["2d", "color pan up", "color pan down"] fig, ax_dict = plt.subplot_mosaic( [ curve_axes, image_axes, ], constrained_layout=True, ) for an in curve_axes: ax = ax_dict[an] ax.plot(th, data1d) ax.set_title(an) if an == "y pan up": ax.set_ylim(0, 2) elif an == "y pan down": ax.set_ylim(-2, 0) else: ax.set_ylim(-1, 1) for an in image_axes: ax = ax_dict[an] im = ax.imshow( data2d, extent=[start - (stop - start) / (2 * N), stop + (stop - start) / (2 * N)] * 2, ) ax.set_title(an) fig.colorbar(im, ax=ax, aspect=7) if an == "color pan up": im.set_clim(0, 2) elif an == "color pan down": im.set_clim(-2, 0) else: im.set_clim(-1, 1) fig.savefig("/tmp/pan.png") plt.show()
As mentioned on the call, this only makes sense for continuous (i.e. not categorical) norms.
I don't have any problem with the semantics of zooming. I have practical concerns about the complexity of two-way linking of the colorbar to the mappable/norm it represents.
I do have another issue with the GUI aspect of zooming on a colorbar. @greglucas has a nice hefty colorbar up there, and you can still see him struggle to get the drag box to stay in the colorbar. I almost never have such large colorbars as I consider them a waste of space so I would be even more screwed. Below is a published example:
struggle to get the drag box to stay in the colorbar.
I don't think you need to keep the cursor in the colorbar; as long as the initial click is in it, the drag box will just be clipped to the axes.
Thats good - but it's still a little fiddly...
@jklymak, the colorbar and mappable are currently directly linked. cb.norm is cb.mappable.norm
So, if you update the vmin/vmax of either the colorbar or the image mappable, that state should get propagated. I think this is the way it should be as well, I want my colorbar to represent the data in my image when I've updated the state and not lag behind or be out of sync somehow.
I suppose if you make your colorbar too small and can't interact with it, then that just means you can't take advantage of this feature as easily, it doesn't adversely affect anything. There was a discussion about trying to interact with the ticks instead of the axis/colored area to make it more explicit as well what we are changing.
@jklymak, the colorbar and mappable are currently directly linked.
cb.norm is cb.mappable.norm
So, if you update the vmin/vmax of either the colorbar or the image mappable, that state should get propagated. I think this is the way it should be as well, I want my colorbar to represent the data in my image when I've updated the state and not lag behind or be out of sync somehow.
Sure. But that does not necessarily have anything to do with the axes min/max. You are proposing that it does, and @tacaswell is proposing that zooming in on the colorbar is a "mistake" for some reason I've not heard. I'm not 100% against either proposal, but it needs some discussion. And just to reiterate, folks have asked to be able to zoom in on the colors on the colorbar, but to preserve the norm/colormap mapping. The ability to do so is new in #20054, so now is definitely the time to discuss if we don't want that behaviour, but except to implement these semantics, I'm not sure what the argument is against it.
I still feel the way forward here is if we are going to make this an action on the colorbar, we should first define the colorbar method that will do this, and leave the GUI discussion out of it.
Just a note about the limits corresponding to vmin/vmax. This is in the comments, which seems to indicate that a larger colorbar axes than just vmin and vmax is explicitly allowed in the old API (imagine contours on a plot, saturated at high values, but you still want them labeled on the colorbar.
# copy the norm and change the vmin and vmax to the vmin and
# vmax of the colorbar, not the norm. This allows the situation
# where the colormap has a narrower range than the colorbar, to
# accommodate extra contours:
I guess I still just don't quite understand this use case and wonder if it can't be solved in a different manner... Is the real request to be able to use less of the colormap than is available by default because you want it to saturate at a different value? I've used this crude hack in the past to change the saturation point, so maybe a public function on a colormap to do something similar would be the better place for it than messing with limits on an axes?
cmap = mpl.cm.get_cmap('inferno') # Remove the bottom 10% of the cmap cmap_small = mpl.colors.LinearSegmentedColormap.from_list('inferno_small', cmap(np.linspace(0.1, 1.0, 256)))
pc = ax.contour(np.arange(1, 11), np.arange(1, 11), np.arange(100).reshape(10, 10), levels=np.arange(0, 100, 10), vmin=0, vmax=50) cb = fig.colorbar(pc)
is a use case where the norm goes from 0 to 50 but the colorbar goes to 100.
BTW, just to be clear, I'm not saying the above functionality should trump the proposed functionality, just that it is something we will break...
I was mostly thinking about the case where you make the range of the color bar narrow than the vmin/vmax which seems like a miss-feature to me (because then we have put colors in the figure that do not have a key to indicate what that color means), but making the top/bottom of the color bar bigger than the norm (and basically implementing the extend arrows by having a constant area in the color bar).
I think I am going to back away from "the color bar limits should always exactly match the norm limits" to "the norm limits should be contained in the color bar limits".
The first thing that occurred to me under that framing is we have a norm_tracks_limits
setting on the color bar. If True
we get the behavior with pan/zoom that @greglucas is proposing here, if it is False
, then when the user sets the limits on the color bar we expand them to include vmin/vmax (there is precedent for this with the nonsingular step) and if they are bigger just let it happen. In all of these cases the "extend" arrows will still be consistent.
I think the constraints are:
- every possible color in the cmap should be shown with a correct value in the color bar
- ever tick / value along the axis of the color bar should be matched with the color that value would be in the data
every possible color in the cmap should be shown with a correct value in the color bar
That seems overly prescriptive. Folks have specifically asked to have the limits tighter than vmin and vmax. Why would we forbid that?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Just one suggestion about docstrings.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Just a couple of small question about the architecture.
lib/matplotlib/colorbar.py
Outdated
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is it actually possible to twin colorbar axises?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Possibly? I am not sure. This is more-so to keep the argument length/order the same as with normal axes so the call can be replicated. I removed the twinx and mode check in the code because those don't matter here for the one-dimensional colorbar.
Are we decided this should go in 3.5? Seems we have already made the beta, and this is kind of a major change to include after the beta?
The zoom and pan funcitons change the vmin/vmax of the norm attached to the colorbar. The colorbar is rendered as an inset axis, but the event handler is implemented on the parent axis.
This helps for subclasses in finding the zoom/pan locations by not having to duplicate the code used to determine the x/y locations of the zoom or pan.
Setting the zoom selector rectangle to the vertical/horizontal limits of the colorbar, depending on the orientation. Remove some mappable types from colorbar navigation. Certain mappables, like categoricals and contours shouldn't be mapped by default due to the limits of an axis carrying certain meaning. So, turn that off for now and potentially revisit in the future.
Adding tests for vertical and horizontal placements, zoom in, zoom out, and pan. Also verifying that a colorbar on a Contourset is not able to be interacted with.
This adds logic to remove the colorbar interactivity and replace the axes it is drawn in with the original interactive routines.
Colorbars are one-dimensional, so we don't want to cancel the zoom based on the short-axis. This also updates the test to account for this case.
d4ad3a1
to
21fc347
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We should still decide if this is 3.5 or 3.6.... I don't mind it going in for 3.5, but it hasn't had a lot of use yet...
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
... oh, as before, do we still need this song and dance as well?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, I didn't ever figure out a way to get the norm information up to the axes any other way. (This would be easier if a Colorbar inherited from some form of Axes so we could just override the methods directly)
My (biased) vote is for 3.5, but I do understand the hesitation. My thinking is: the betas and rc's probably don't get a lot of interactive testing use, so I'm not sure how much this feature would even be tested there. It only affects interactive users, not static figures/plots.
See https://stackoverflow.com/questions/69257255/change-colorbar-limits-without-changing-the-values-of-the-data-it-represents-in for another instance of someone wanting the colorbar limits to be different than the vmin/vmax of the norm. I'm not sure I completely sympathize, and such users always have the ability to create a custom colormap, but it is often requested that the colorbar be "zoomed" without affecting the norm.
It's fine for 3.5, but not after rc.
OK, well lets try it....
Sorry for the late comment but #20471 should probably be reverted before 3.5 lands if this feature is merged.
This feature was merged, so we should definitely think about what to do with that example.
When I run the example without any zoom/pan clicked and interact with the colorbar it seems to be very finnicky, so I'm not sure what the expected outcome is supposed to be? I'm wondering if some of the other norm updates messed with the example too...
Instead of removing the example, we could keep it and add two colorbars to the image and label one "standard zoom/pan" and the other "manual interactivity" to demonstrate multiple ways to do this as well. It may even be beneficial to put an example in demonstrating how to turn off zoom/pan on that specific axes colorbar.ax.can_zoom = colorbar.ax.can_pan = lambda: False
so that you can override it with your own methods.
(Side point, but I think(?) the "supported" API for disabling zoom/pan is ax.set_navigate(False)
?)
Yes, I agree, but that example is actually turning ax.set_navigate(True)
on that axes and then using pickers instead of zoom/pan. So, I was looking for a way to disable just the zoom/pan that was added on the colorbars. Not very elegant at all.
The easiest might be to remove that example, but I think we should make sure @richardsheridan's use-case is covered before just removing that.
Colorbar axis zoom and pan
Uh oh!
There was an error while loading. Please reload this page.
PR Summary
I've wanted to zoom/pan on values rather than extents of the image before and haven't figured out a clean way to do that without adding widgets. For example, if I set the range poorly the first time with my data and want to zoom in on a specific region of data, or if outliers made the vmin/vmax too large the first time. This adds zoom and pan capabilities to the colorbar. When zooming and panning on the axis, this updates the vmin/vmax of the scalar mappable norm associated with the colorbar. Currently, this changes the xlim/ylim of the axis holding the colorbar, which I don't think is desired behavior.
ezgif-6-88618ea58e72
For this to work, we need the parent axis for events, and the scalar mappable object. I wasn't sure where the best place to implement these methods is, as we need to override the parent axis event handlers, but that doesn't have access to the scalarmappable... I might be overlooking something obvious here too, so better suggestions welcome
.
Interactive example if you use the zoom/pan on the colorbar axis:
PR Checklist
pytest
passes).flake8
on changed files to check).flake8-docstrings
and runflake8 --docstring-convention=all
).doc/users/next_whats_new/
(follow instructions in README.rst there).doc/api/next_api_changes/
(follow instructions in README.rst there).