I would like to ask for some advice from those out there who have experience dealing with fonts and text layout. At the end of this message is the output of dvitype, which translates a dvi file (in this case a file typesetting "0.8" in ptmr7t fonts) into human readable output. I have considered writing a dvi parser for mpl, intending to extract the information necessary to render text with mpl's existing font and text support. There is some more information at http://en.wikipedia.org/wiki/DVI_(TeX) and http://www.math.umd.edu/~asnowden/comp-cont/dvi.html#setchar. Is this worth pursuing? If I could make it work, mpl's only external dependency would be TeX/LaTeX, even dvipng would not be required. Thanks, Darren (I'm sorry to beat this issue into the ground) $ dvitype 8b85d26da2410f54cd2d70976999b1d8.dvi This is DVItype, Version 3.6 (Web2C 7.5.5) Options selected: Starting page = * Maximum number of pages = 1000000 Output level = 4 (the works) Resolution = 300.00000000 pixels per inch numerator/denominator=25400000/473628672 magnification=1000; 0.00006334 pixels per DVI unit ' TeX output 2006年01月21日:1639' Postamble starts at byte 144. maxv=41484288, maxh=26673152, maxstackdepth=3, totalpages=1 Font 14: ptmr7t---loaded at size 655360 DVI units 42: beginning of page 1 87: down4 41484288 v:=0+41484288=わ41484288, vv:=2628 92: push level 0:(h=0,v=41484288,w=0,x=0,y=0,z=0,hh=0,vv=2628) 93: down4 -39649280 v:=41484288-ひく39649280=わ1835008, vv:=116 98: down4 37683200 v:=1835008+たす37683200=わ39518208, vv:=2503 103: push level 1:(h=0,v=39518208,w=0,x=0,y=0,z=0,hh=0,vv=2503) 104: down4 -35389440 v:=39518208-ひく35389440=わ4128768, vv:=262 109: push level 2:(h=0,v=4128768,w=0,x=0,y=0,z=0,hh=0,vv=262) 110: right3 5046272 h:=0+5046272=5046272, hh:=320 [ ] 114: fntdef1 14: ptmr7t 136: fntnum14 current font is ptmr7t 137: setchar48 h:=5046272+たす327680=わ5373952, hh:=341 138: setchar46 h:=5373952+たす163840=わ5537792, hh:=351 139: setchar56 h:=5537792+たす327680=わ5865472, hh:=372 [0.8] 140: pop level 2:(h=0,v=4128768,w=0,x=0,y=0,z=0,hh=0,vv=262) 141: pop level 1:(h=0,v=39518208,w=0,x=0,y=0,z=0,hh=0,vv=2503) 142: pop level 0:(h=0,v=41484288,w=0,x=0,y=0,z=0,hh=0,vv=2628) 143: eop
Darren Dale wrote: > I would like to ask for some advice from those out there who have experience > dealing with fonts and text layout. > > At the end of this message is the output of dvitype, which translates a dvi > file (in this case a file typesetting "0.8" in ptmr7t fonts) into human > readable output. I have considered writing a dvi parser for mpl, intending to > extract the information necessary to render text with mpl's existing font and > text support. There is some more information at > http://en.wikipedia.org/wiki/DVI_(TeX) and > http://www.math.umd.edu/~asnowden/comp-cont/dvi.html#setchar. > > Is this worth pursuing? If I could make it work, mpl's only external > dependency would be TeX/LaTeX, even dvipng would not be required. Quite probably. dvitype is new to me. Doing a DVI interpreter for mpl and Chaco has been on my list of things to look at for some time. The things stopping me (besides the density of the DVI standard) was that all of the readable code I found for interpreting DVI files has been GPLed. You will need to access the parameters defined in the TeX Font Metric files, I think. I recommend using tftopl(1) to convert them to a parseable form. -- Robert Kern rob...@gm... "In the fields of hell where the grass grows high Are the graves of dreams allowed to die." -- Richard Harter
On Monday 23 January 2006 18:27, Robert Kern wrote: > Darren Dale wrote: > > I would like to ask for some advice from those out there who have > > experience dealing with fonts and text layout. > > > > At the end of this message is the output of dvitype, which translates a > > dvi file (in this case a file typesetting "0.8" in ptmr7t fonts) into > > human readable output. I have considered writing a dvi parser for mpl, > > intending to extract the information necessary to render text with mpl's > > existing font and text support. There is some more information at > > http://en.wikipedia.org/wiki/DVI_(TeX) and > > http://www.math.umd.edu/~asnowden/comp-cont/dvi.html#setchar. > > > > Is this worth pursuing? If I could make it work, mpl's only external > > dependency would be TeX/LaTeX, even dvipng would not be required. > > Quite probably. dvitype is new to me. Doing a DVI interpreter for mpl and > Chaco has been on my list of things to look at for some time. The things > stopping me (besides the density of the DVI standard) was that all of the > readable code I found for interpreting DVI files has been GPLed. > > You will need to access the parameters defined in the TeX Font Metric > files, I think. I recommend using tftopl(1) to convert them to a parseable > form. How does matplotlib render glyphs, does it use freetype? Would matplotlib be able to render tex's fonts with the existing codebase, or would I have to delve into metafont?
>>>>> "Darren" == Darren Dale <dd...@co...> writes: Darren> How does matplotlib render glyphs, does it use freetype? Darren> Would matplotlib be able to render tex's fonts with the Darren> existing codebase, or would I have to delve into metafont? Yep, it uses freetype and no, we don't have any facility to render tex fonts. I think the dvi parsing project would be a very nice addition, but not a trivial task. The tex file formats are pretty arcane: Knuth is fond of using bytecode in his data files, so you would have to write a bytecode engine. Robert Kern did this on a rainy afternoon for the tfm files, so you could use that as an example http://sourceforge.net/mailarchive/message.php?msg_id=9894596 JDH
While the gnuplot approach of outputing an eps file plus a tex file that does all the axis labeling is probably lots easier to implement, I am nervous that if the tex file ever gets seperated from the eps file, I am left with a plot with no axis or tick mark labels - or labels that were accidentally editted. That may not be a big enough concern to justify the extra work to parse the dvi, but it is a concern of mine. It may be irrational on my part, but something about a single eps file feels cleaner to me than the seperate eps and tex file approach. Ryan On 1/23/06, Matt Newville <new...@ca...> wrote: > Hi Darren, > > It may be very different than what you're doing, but the hybrid > approach of gnuplot's 'pslatex' terminal device might be worth > considering and looking into. This writes postscript for the graphics > part and leaves simple lines (for the axes) and the text as plain > latex that overlays the postscript. I think for mpl, you might want > to have all the non-text go into the postscript and all the text go > into an accompanying latex file, but the idea is the same. > > One advantage there is portability, as the output is latex+embedded > postscript (using \special). It also allows pretty fine-grained > control on the output, including changing fonts or doing latex things > that mpl can't do (and this can all be done after the fact, so that > you can create the figure, and then change the fonts). It does > require latex to create the figure, but this step could be automated, > at least to stage of the dvi-with-eps-figure stage. Getting to ps, > pdf, or png would be less easy to automate but may be doable in a > portable way. > > That sounds easier than parsing a dvi file to me. And postscript > backend already exists. > > > --Matt > > > ------------------------------------------------------- > This SF.net email is sponsored by: Splunk Inc. Do you grep through log fi= les > for problems? Stop! Download the new AJAX search engine that makes > searching your log files as easy as surfing the web. DOWNLOAD SPLUNK! > http://sel.as-us.falkag.net/sel?cmdlnk&kid=103432&bid#0486&dat=121642 > _______________________________________________ > Matplotlib-users mailing list > Mat...@li... > https://lists.sourceforge.net/lists/listinfo/matplotlib-users >
On Monday 23 January 2006 19:06, Ryan Krauss wrote: > While the gnuplot approach of outputing an eps file plus a tex file > that does all the axis labeling is probably lots easier to implement, > I am nervous that if the tex file ever gets seperated from the eps > file, I am left with a plot with no axis or tick mark labels - or > labels that were accidentally editted. That may not be a big enough > concern to justify the extra work to parse the dvi, but it is a > concern of mine. > > It may be irrational on my part, but something about a single eps file > feels cleaner to me than the seperate eps and tex file approach. Actually, this was what the first incarnation of usetex looked like: a ps file and a separate latex file. In fact, it still does that, but I put a lot of work into hiding that from the user and making mpl do the extra work behind the scenes to deliver a single eps file. I didnt like the idea of having to insert an eps file and some latex code on top of it. What if the file is not destined for a latex document? I might want to use the image with all the fancy markup in a poster created with inkscape or adobe illustrator. > On 1/23/06, Matt Newville <new...@ca...> wrote: > > Hi Darren, > > > > It may be very different than what you're doing, but the hybrid > > approach of gnuplot's 'pslatex' terminal device might be worth > > considering and looking into. This writes postscript for the graphics > > part and leaves simple lines (for the axes) and the text as plain > > latex that overlays the postscript. I think for mpl, you might want > > to have all the non-text go into the postscript and all the text go > > into an accompanying latex file, but the idea is the same. > > > > One advantage there is portability, as the output is latex+embedded > > postscript (using \special). It also allows pretty fine-grained > > control on the output, including changing fonts or doing latex things > > that mpl can't do (and this can all be done after the fact, so that > > you can create the figure, and then change the fonts). It does > > require latex to create the figure, but this step could be automated, > > at least to stage of the dvi-with-eps-figure stage. Getting to ps, > > pdf, or png would be less easy to automate but may be doable in a > > portable way. > > > > That sounds easier than parsing a dvi file to me. And postscript > > backend already exists. > > > > > > --Matt > > > > > > ------------------------------------------------------- > > This SF.net email is sponsored by: Splunk Inc. Do you grep through log > > files for problems? Stop! Download the new AJAX search engine that > > makes searching your log files as easy as surfing the web. DOWNLOAD > > SPLUNK! http://sel.as-us.falkag.net/sel?cmdlnk&kid3432&bid#0486&dat1642 > > _______________________________________________ > > Matplotlib-users mailing list > > Mat...@li... > > https://lists.sourceforge.net/lists/listinfo/matplotlib-users > > ------------------------------------------------------- > This SF.net email is sponsored by: Splunk Inc. Do you grep through log > files for problems? Stop! Download the new AJAX search engine that makes > searching your log files as easy as surfing the web. DOWNLOAD SPLUNK! > http://sel.as-us.falkag.net/sel?cmd=lnk&kid3432&bid#0486&dat1642 > _______________________________________________ > Matplotlib-devel mailing list > Mat...@li... > https://lists.sourceforge.net/lists/listinfo/matplotlib-devel -- Darren S. Dale, Ph.D. Cornell High Energy Synchrotron Source Cornell University 200L Wilson Lab Rt. 366 & Pine Tree Road Ithaca, NY 14853 dd...@co... office: (607) 255-9894 fax: (607) 255-9001
Hi Darren,
It's reasonable to be concerned about having two files necessary to
make one plot. The gnuplot pslatex output doesn't do this: it writes
one output file that is essentially a latex "picture" environment
which includes an encapsulated postscript program for the "graphics
part" with a \special command, and uses \put(x,y){\makebox(0,0){TEXT}
for all the text. Since latex has a preferred orientation for text
flow, each piece of rotated text is handled with another \special
command. BTW, I was mistaken earlier when I said that axes were
drawn in latex with gnuplot's pslatex terminal -- they are included in
the postscript, and only text for axis labels and other text is added
in with latex. So I think that emulating this solution would not be
too hard.
The output latex files do not render a figure by themselves, as they
are intended to be used in-place in documents and latex insists on
exactly one \begin{document} / \end{document} pair. But it's trivial
to have a boilerplate outer latex file:
\documentclass[11pt]{article}
\usepackage{color,....}
\pagestyle{empty}
\begin{document}\begin{figure}{\input{fig.pstex}}\end{figure}\end{documen=
t}
for generating individual graphs, and then doing standard latex / dvips.
I am not necessarily advocating this as "the right way", but offer it
as a suggestion. I've made figures like this for many years. OK, I
admit that I still make many figures for papers and talks this way, as
I know gnuplot, have scripts I've used for years, and there are some
things I can do more easily in gnuplot/latex than mpl.
Advantages of this approach are that a) the latex is not interpreted
by gnuplot, but handed off to latex (any valid latex can be used),=20
and b) the output latex/postscript is easily hackable. The latex
parts (say, the labels) can be easily changed. Colors and linestyles
of the traces can also be changed simply by editng the postscript. Of
course, "Oh, just hack the postscript" is not what mpl should aspire
too, but it's nice to know it's there when you need it.
A sample gnuplot script and output pslatex are at
http://cars9.uchicago.edu/~newville/Python/MPlot/gnuplot-pslatex/
I also put a python script gp2ps (which requires Gnuplot.py) I use to
turn gnuplot command files with latex labels into eps (using dvips)
and png (using ImageMagick's convert) outputs. Feel free to use this
any way you want, or ignore it. Again, I'm not saying that this is
"doing tex support the right way", but it might beat the headache of
relying on version-specific features of gs.
Cheers,
--Matt
This weekend I took the first small step towards improving usetex and eliminating the need for PSFrag and dependencies like ghostscript. This is necessary in order to provide usetex support accross backends (like pdf and svg). The basic approach is to extract the font layout information from the dvi files. LaTeX could be the only dependency. Wikipedia has an article about the dvi specification: http://en.wikipedia.org/wiki/DVI_file_format, which has a link to a really useful file called dvitype.web. This file contains the full dvi specification and can be converted into a plain tex file using weave: http://www.ctan.org/tex-archive/systems/knuth/texware/dvitype.web dvitype.web also contains a well documented pascal program illustrating how to read dvi files correctly and convert them into symbolic form. It was meant as a guide to programmers developing dvi-related software. It includes a section on extracting the necessary information from tfm files, and other issues related to fonts. I also discovered that xdvi is OSS compliant, and have started studying that source code as well. I only started writing code to read the dvi information, but I post here to allow people to comment if they are so inclined, and to share the dvitype example with anyone who is interested. I have compiled the tex file, converted it to pdf, and posted the result at http://staff.chess.cornell.edu/~dale/matplotlib/dvitype.pdf. Darren
Thanks for doing this Darren. Keep up the good work! On 4/24/06, Darren Dale <dd...@co...> wrote: > This weekend I took the first small step towards improving usetex and > eliminating the need for PSFrag and dependencies like ghostscript. This i= s > necessary in order to provide usetex support accross backends (like pdf a= nd > svg). The basic approach is to extract the font layout information from t= he > dvi files. LaTeX could be the only dependency. > > Wikipedia has an article about the dvi specification: > http://en.wikipedia.org/wiki/DVI_file_format, which has a link to a reall= y > useful file called dvitype.web. This file contains the full dvi specifica= tion > and can be converted into a plain tex file using weave: > http://www.ctan.org/tex-archive/systems/knuth/texware/dvitype.web > > dvitype.web also contains a well documented pascal program illustrating h= ow to > read dvi files correctly and convert them into symbolic form. It was mean= t as > a guide to programmers developing dvi-related software. It includes a sec= tion > on extracting the necessary information from tfm files, and other issues > related to fonts. > > I also discovered that xdvi is OSS compliant, and have started studying t= hat > source code as well. I only started writing code to read the dvi informat= ion, > but I post here to allow people to comment if they are so inclined, and t= o > share the dvitype example with anyone who is interested. I have compiled = the > tex file, converted it to pdf, and posted the result at > http://staff.chess.cornell.edu/~dale/matplotlib/dvitype.pdf. > > Darren > > > ------------------------------------------------------- > Using Tomcat but need to do more? Need to support web services, security? > Get stuff done quickly with pre-integrated technology to make your job ea= sier > Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronim= o > http://sel.as-us.falkag.net/sel?cmd=3Dlnk&kid=3D120709&bid=3D263057&dat= =3D121642 > _______________________________________________ > Matplotlib-devel mailing list > Mat...@li... > https://lists.sourceforge.net/lists/listinfo/matplotlib-devel >
>>>>> "Darren" == Darren Dale <dd...@co...> writes: Darren> I also discovered that xdvi is OSS compliant, and have Darren> started studying that source code as well. I only started Darren> writing code to read the dvi information, but I post here Darren> to allow people to comment if they are so inclined, and to Darren> share the dvitype example with anyone who is interested. I Darren> have compiled the tex file, converted it to pdf, and Darren> posted the result at Darren> http://staff.chess.cornell.edu/~dale/matplotlib/dvitype.pdf. This is very good stuff Darren -- it would be really nice to have a python interface to dvi files under a BSD compatible license. matplotlib sitting on top of my two favorite runtimes: python and TeX. I think it is a shame that a lot tools built on top of TeX (web2c, pyx, ...) have a more restrictive license than TeX itself, so this would be a very useful piece of code. JDH
[This discussion somehow wandered onto matplotlib-user. I'm bringing it back over to the dev list.] On Monday 24 April 2006 16:30, Alan G Isaac wrote: > On 2006年4月24日, Christopher Barker apparently wrote: > > I think it might use dvips or something to do that. rather than > > reading and rendering the DVI itself. > > That is not my understanding, > which however is limited. PyX does read dvi files. I'll bet they ported dvitype to do so (PyX distributes a script called dvitype.py.) > PS I have tried to reopen the discussion with the PyX > developers. I'll post any useful outcomes. Their webpage states that they would consider relicensing, but when I talked to them, they were unwilling to release under a BSD-compatible license. However, we dont need all of PyX to be relicensed, and I did not make that clear to the PyX authors. If they would consider relicensing a subset of their code, it would be extremely helpful.
Darren Dale <dd...@co...> writes: > The basic approach is to extract the font layout information from the > dvi files. LaTeX could be the only dependency. Great! Another possibly useful reference is http://www.tug.org/tex-archive/dviware/driv-standard/level-0/dvistd0.pdf and the documents in driv-standard/papers. dvi is apparently pretty difficult to parse, as you have to implement a bytecode interpreter to keep track of the state. -- Jouni
On Monday 24 April 2006 10:48, Jouni K Seppanen wrote: > Darren Dale <dd...@co...> writes: > > The basic approach is to extract the font layout information from the > > dvi files. LaTeX could be the only dependency. > > Great! Another possibly useful reference is > > http://www.tug.org/tex-archive/dviware/driv-standard/level-0/dvistd0.pdf > > and the documents in driv-standard/papers. dvi is apparently pretty > difficult to parse, as you have to implement a bytecode interpreter to > keep track of the state. Thanks Jouni, I was not aware of this document. There are some other packages in dviware that could serve as useful examples, like dvitops (there are a lot of GPL'd packages too, like dvisvgm and dvipdfmx. Too bad.)