Andrew, I think I may have a clue how to fix the font discrepancies. Apple owns patents on some a byte code interpreter for hinting truetype fonts: http://freetype.sourceforge.net/patents.html and freetype can be built with the patented stuff turned on but the default is off -- freetype has non-patented auto-hinting which is what is used by default. However, according to a thread I found here (see the first post by RobK) ubuntu hardy heron, which is what you are using on the buildbot, has the patented stuff *turned on* by default: http://www.howtogeek.com/howto/ubuntu/enable-smooth-fonts-on-ubuntu-linux/ According to RobK, you can reconfigure your ubuntu system to turn these off. He suggests: To use autohinting, use the hint in this post, or just run the following command: sudo dpkg-reconfigure fontconfig-config then choose "autohinter", then choose "always", then choose "no" Give it a test and hopefully we will see both bots go green. Then we can focus on more tests and nightly builds... JDH
On Sun, Aug 30, 2009 at 4:17 PM, John Hunter<jd...@gm...> wrote: > According to RobK, you can reconfigure your ubuntu system to turn > these off. He suggests: > > To use autohinting, use the hint in this post, or just run the > following command: > > sudo dpkg-reconfigure fontconfig-config > > then choose "autohinter", then choose "always", then choose "no" If that doesn't work, this guy has more involved instructions on how to rebuild ubuntu libfreetype and disable the bytecode patch http://ubuntuforums.org/showthread.php?t=84359 JDH
John Hunter wrote: > On Sun, Aug 30, 2009 at 4:17 PM, John Hunter<jd...@gm...> wrote: >> According to RobK, you can reconfigure your ubuntu system to turn >> these off. He suggests: >> >> To use autohinting, use the hint in this post, or just run the >> following command: >> >> sudo dpkg-reconfigure fontconfig-config >> >> then choose "autohinter", then choose "always", then choose "no" > > If that doesn't work, this guy has more involved instructions on how > to rebuild ubuntu libfreetype and disable the bytecode patch > > http://ubuntuforums.org/showthread.php?t=84359 OK, I disabled all Ubuntu patches to libfreetype and recompiled and re-installed it. Unfortunately, I'm still getting the same failures. Then I additionally installed fontconfig-config and did the dpkg-reconfigure fontconfig-config step, setting everything to "Never". Same failures. These font errors make me unhappy. I think we should test some very simple pure libfreetype C program outputs generated on the various machines. I've just been playing with ftview, but that doesn't seem to have a command-line interface to save directly to a file. -Andrew
On Sun, Aug 30, 2009 at 5:47 PM, Andrew Straw<str...@as...> wrote: > John Hunter wrote: >> On Sun, Aug 30, 2009 at 4:17 PM, John Hunter<jd...@gm...> wrote: >>> According to RobK, you can reconfigure your ubuntu system to turn >>> these off. He suggests: >>> >>> To use autohinting, use the hint in this post, or just run the >>> following command: >>> >>> sudo dpkg-reconfigure fontconfig-config >>> >>> then choose "autohinter", then choose "always", then choose "no" >> >> If that doesn't work, this guy has more involved instructions on how >> to rebuild ubuntu libfreetype and disable the bytecode patch >> >> http://ubuntuforums.org/showthread.php?t=84359 > > OK, I disabled all Ubuntu patches to libfreetype and recompiled and > re-installed it. Unfortunately, I'm still getting the same failures. > > Then I additionally installed fontconfig-config and did the > dpkg-reconfigure fontconfig-config step, setting everything to "Never". > Same failures. > > These font errors make me unhappy. I think we should test some very > simple pure libfreetype C program outputs generated on the various > machines. I've just been playing with ftview, but that doesn't seem to > have a command-line interface to save directly to a file. Attached is "example1.c" from the freetype tutorial. When I wrote ft2font, I started with this tutorial and built around it, so some of the core logic is the same. A lot has been added since then, including a lot of stuff Michael has added to improve hinting, so we may not see any differences at this level. The program outputs an ascii-art file to stdout, so we could start by checking the md5 on the output. Since *most* of your unit tests agree with the baseline, we may struggle to find a difference. I compiled the example with gcc -I/Users/jdhunter/dev/include -I/Users/jdhunter/dev/include/freetype2 -L/Users/jdhunter/dev/lib -o example1 example1.c -lfreetype and ran it with ./example1 ~/mpl/lib/matplotlib/mpl-data/fonts/ttf/Vera.ttf "this is a test" > run.out my md5 on the output is home:~/tmp> md5 run.out MD5 (run.out) = 01868827436d858b4452ff03f80f7222 The example and output are attached -- we could easily replace the example "show_image" with a binary output file that we could read into numpy arrays to diff or display with imshow. I'm available to skype if you want to brainstorm this. JDH
You can also turn hinting off at runtime by passing a flag to freetype. matplotlib currently exposes this flag in the FT2Font extension, but we don't really expose the option to the user. It would be simple enough to make it an rcParam, though, which could then be set to "no hinting" by the test infrastructure just to rule out some of these moving parts. Of course, this doesn't rule out differences between different version numbers of freetype. Another thing to look into might be some sort of fuzzy or perceptual diffing approach. Freddie Witheren (our GSoC student) has been looking into pdiff (http://pdiff.sf.net) for testing mathtex. I don't know what the current status of that is. In general, I think I prefer the goal of exact determinism, but if this becomes a recurring problem down the road, it may be easier to just try to "gloss over" these minor font differences. Mike John Hunter wrote: > On Sun, Aug 30, 2009 at 5:47 PM, Andrew Straw<str...@as...> wrote: > >> John Hunter wrote: >> >>> On Sun, Aug 30, 2009 at 4:17 PM, John Hunter<jd...@gm...> wrote: >>> >>>> According to RobK, you can reconfigure your ubuntu system to turn >>>> these off. He suggests: >>>> >>>> To use autohinting, use the hint in this post, or just run the >>>> following command: >>>> >>>> sudo dpkg-reconfigure fontconfig-config >>>> >>>> then choose "autohinter", then choose "always", then choose "no" >>>> >>> If that doesn't work, this guy has more involved instructions on how >>> to rebuild ubuntu libfreetype and disable the bytecode patch >>> >>> http://ubuntuforums.org/showthread.php?t=84359 >>> >> OK, I disabled all Ubuntu patches to libfreetype and recompiled and >> re-installed it. Unfortunately, I'm still getting the same failures. >> >> Then I additionally installed fontconfig-config and did the >> dpkg-reconfigure fontconfig-config step, setting everything to "Never". >> Same failures. >> >> These font errors make me unhappy. I think we should test some very >> simple pure libfreetype C program outputs generated on the various >> machines. I've just been playing with ftview, but that doesn't seem to >> have a command-line interface to save directly to a file. >> > > Attached is "example1.c" from the freetype tutorial. When I wrote > ft2font, I started with this tutorial and built around it, so some of > the core logic is the same. A lot has been added since then, > including a lot of stuff Michael has added to improve hinting, so we > may not see any differences at this level. The program outputs an > ascii-art file to stdout, so we could start by checking the md5 on the > output. Since *most* of your unit tests agree with the baseline, we > may struggle to find a difference. > > I compiled the example with > > gcc -I/Users/jdhunter/dev/include > -I/Users/jdhunter/dev/include/freetype2 -L/Users/jdhunter/dev/lib -o > example1 example1.c -lfreetype > > and ran it with > > ./example1 ~/mpl/lib/matplotlib/mpl-data/fonts/ttf/Vera.ttf "this is a > test" > run.out > > my md5 on the output is > > home:~/tmp> md5 run.out > MD5 (run.out) = 01868827436d858b4452ff03f80f7222 > > The example and output are attached -- we could easily replace the > example "show_image" with a binary output file that we could read into > numpy arrays to diff or display with imshow. > > I'm available to skype if you want to brainstorm this. > > JDH > ------------------------------------------------------------------------ > > ------------------------------------------------------------------------------ > Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day > trial. Simplify your report design, integration and deployment - and focus on > what you do best, core application coding. Discover what's new with > Crystal Reports now. http://p.sf.net/sfu/bobj-july > ------------------------------------------------------------------------ > > _______________________________________________ > Matplotlib-devel mailing list > Mat...@li... > https://lists.sourceforge.net/lists/listinfo/matplotlib-devel -- Michael Droettboom Science Software Branch Operations and Engineering Division Space Telescope Science Institute Operated by AURA for NASA
Michael Droettboom wrote: > You can also turn hinting off at runtime by passing a flag to freetype. > matplotlib currently exposes this flag in the FT2Font extension, but we > don't really expose the option to the user. It would be simple enough > to make it an rcParam, though, which could then be set to "no hinting" > by the test infrastructure just to rule out some of these moving parts. > Of course, this doesn't rule out differences between different version > numbers of freetype. > I think that, for now, we're going have the buildslaves run the same version of freetype and have hinting disabled at freetype compile time. (Right now we're at freetype 2.3.5.) If you want to go ahead and add a "disable hinting" rcParam, it may come in handy in the future, but I don't think it's a high priority right now given that we have to ensure freetype version and thus will probably be compiling by hand it for any buildslave. > Another thing to look into might be some sort of fuzzy or perceptual > diffing approach. Freddie Witheren (our GSoC student) has been looking > into pdiff (http://pdiff.sf.net) for testing mathtex. I don't know what > the current status of that is. In general, I think I prefer the goal of > exact determinism, but if this becomes a recurring problem down the > road, it may be easier to just try to "gloss over" these minor font > differences. > The image difference approach we're currently using doesn't search for an exact duplication but just that the error is under some threshold. I haven't looked at the algorithm, but the printout says "RMS error". Now that all the tests are passing, I'm going to direct my energies to expanding the test infrastructure, especially making it easy for other devs to write tests. We may have to revisit these issues if, once we get a greater diversity, the buildslaves start failing again. -Andrew