On 9/12/07, Michael Droettboom <md...@st...> wrote: > So, I feel like I'm going in a bit of a circle here, and I might need a > reality check. I thought I'd better check in and see where you guys > (who've thought about this a lot longer than I have) see this going. A > statement of objectives of this part of the task would be helpful. > (e.g. what's the biggest problem with how transforms work now, and what > model would be a better fit). John, I know you've mentioned some to me > before, e.g. the LazyValue concept is quirky and relies on C and the PDF > stateful transforms model is close, but not quite what we need, etc. I > feel I have a better sense of the overall code structure now, but you > guys may have a better "gut" sense of what will fit best. Here is a brief summary of what I see some of the problems to be with the existing approach to transformations, and what I would like to see improved in a refactoring. The three major objectives are clarity, extensibility and efficiency. Clarity: The existing transformation framework, written in C++ and making extensive use of deferred evaluation of binary operation trees and values by reference, is difficult for most developers to understand (and hence enhance). Additionally, since all the heavy lifting is done in C++, python developers who are not versed in C++ have an additional barrier to making contributions. Extensibilty: We would like to make it fairly easy for users to add additional non-linear transformations. The current framework requires adding a new function at the C++ layer, and hacking into axes.py to support additional functions. We would like the existing nonlinear transformations (log and polar) to be part of a general infrastructure where users could supply their own nonlinear functions which map (possibly nonseparable) (xhat, yhat) -> separable (x, y). There are two parts to this: one pretty easy and one pretty hard. The easy part is supporting a transformation which has a separation callable that takes, eg an Nx2 array and returns and Nx2 array. For log, this will simply be log(XY), for polar, it will be r*cos(X[:,0]), r*sin(X[:,1]). Presumably we will want to take advantage of masked arrays to support invalid transformations, eg log of nonpositive data. The harder part is to support axis, tick and label layout generically. Currently we do this by special casing log and polar, either with special tick locators and formatters (log) or special derived Axes (polar). Efficiency: There are three parts to the efficiency question: the efficiency of the transformation itself, the efficiency with which transformation data structures are updated in the presence of viewlim changes (panning and zooming, window resizing) and the efficiency in getting transformed data to the backends. My guess is that the new design may be slower or not dramatically faster for the first two (which are not the bottleneck in most cases anyhow) but you might get sigificant savings on the 3rd. What we would like to support is something like an operation which pushes the partially transformed data to the backend, and the backend then stores this data in a path or other data structure, and when the viewlimits are changed or the window is resized, the backend merely needs to get an updated affine to redraw the data. I say "partially transformed" because in the case of nonlinear or separable transformations, we will probably want to do the nonlinear/separation part first, and then push this to the backend which can build a path (eg an agg::path in agg) and on pan and zoom we would only need to let the backend know what the current affine is. There is more than one way to solve this problem: in mpl1 I used a path dictionary keyed off of a path id which was updated on renderer changes. Then the front end (eg Line2D) could do something like def on_renderer_change(self, renderer): # on renderer change; path data already has the # separable/nonlinear part handled self.pathid = renderer.push_path(pathdata) Additionally, you would need to track when either the data or nonlinear mapping function are changed in order to remove the old path and push out a new one. Then at draw time, you would not need to push any data to the backend def on_draw(self, renderer): # on draw the line just needs to inform the backend to draw the # cached path with the current separable transformation renderer.draw_path(self.pathid, gc, septrans) Ken I believe used some meta class magic to solve this problem. It may be that the ideal approach is different from either of these so feel free to experiment and I'll give it some more thought too. Ken will hopefully pipe in with his perspective too.