@dstansby would you recommend appending to the test_math_fontfamily function?

fork

Or would you prefer delving into the pytest fixtures?

That would involve changing a test image, which seems unnecessary, so a different (and/or new) test would be better. Also, I don't know why you would have to delve into pytest fixtures?

test failure is real and appears related.... lib/matplotlib/tests/test_mathtext.py::test_mathtext_fontfamily

@jklymak should be fixed now

I'm not sure if I'm doing something wrong, but here is an example where the fix suggested by #17007 (comment) doesn't work:

import numpy as np
import matplotlib.pyplot as plt
from matplotlib.tight_layout import get_renderer
plt.rcParams["axes.formatter.use_mathtext"] = True
plt.rcParams["font.family"] = "serif"
plt.rcParams["font.serif"] = "cmr10"
plt.rcParams["mathtext.fontset"] = "cm"
fig, ax = plt.subplots()
image_artist = ax.imshow(np.random.uniform(-0.001, 0.001, (1000, 1000)))
colorbar = fig.colorbar(image_artist)
colorbar.ax.ticklabel_format(axis='y', scilimits=(-1, 0))
renderer = get_renderer(fig)
colorbar.ax.get_tightbbox(renderer)
offset_text = colorbar.ax.yaxis.get_offset_text()
exp = offset_text.get_text().split('e')[1].replace('+', '')
colorbar.ax.set_ylabel(
 f'Brightness [10ドル^{{{exp}}}$ W/m$^2$/SR/nm]'
)
colorbar.ax.yaxis.offsetText.set_visible(False)
plt.show()

I edited the mentioned lines in mathtext.py, but setting plt.rcParams["axes.formatter.use_mathtext"] = True in my case returns the following error:

pyparsing.ParseFatalException: Unknown symbol: \mathd, found '\' (at char 5), (line:1, col:6)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
 File "C:\Users\mapfu\Anaconda3\lib\site-packages\matplotlib\mathtext.py", line 2613, in parse
 result = self._expression.parseString(s)
 File "C:\Users\mapfu\Anaconda3\lib\site-packages\pyparsing.py", line 1955, in parseString
 raise exc
 File "C:\Users\mapfu\Anaconda3\lib\site-packages\pyparsing.py", line 4065, in parseImpl
 raise ParseSyntaxException._from_exception(pe)
pyparsing.ParseSyntaxException: Unknown symbol: \mathd, found '\' (at char 5), (line:1, col:6)
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
 File "C:/Users/mapfu/ownCloud/MPS/Python/CPT/core/temp2.py", line 26, in <module>
 plt.show()
 File "C:\Users\mapfu\Anaconda3\lib\site-packages\matplotlib\pyplot.py", line 353, in show
 return _backend_mod.show(*args, **kwargs)
 File "C:\Program Files\JetBrains\PyCharm 2020年1月1日\plugins\python\helpers\pycharm_matplotlib_backend\backend_interagg.py", line 29, in __call__
 manager.show(**kwargs)
 File "C:\Program Files\JetBrains\PyCharm 2020年1月1日\plugins\python\helpers\pycharm_matplotlib_backend\backend_interagg.py", line 112, in show
 self.canvas.show()
 File "C:\Program Files\JetBrains\PyCharm 2020年1月1日\plugins\python\helpers\pycharm_matplotlib_backend\backend_interagg.py", line 68, in show
 FigureCanvasAgg.draw(self)
 File "C:\Users\mapfu\Anaconda3\lib\site-packages\matplotlib\backends\backend_agg.py", line 407, in draw
 self.figure.draw(self.renderer)
 File "C:\Users\mapfu\Anaconda3\lib\site-packages\matplotlib\artist.py", line 41, in draw_wrapper
 return draw(artist, renderer, *args, **kwargs)
 File "C:\Users\mapfu\Anaconda3\lib\site-packages\matplotlib\figure.py", line 1864, in draw
 renderer, self, artists, self.suppressComposite)
 File "C:\Users\mapfu\Anaconda3\lib\site-packages\matplotlib\image.py", line 131, in _draw_list_compositing_images
 a.draw(renderer)
 File "C:\Users\mapfu\Anaconda3\lib\site-packages\matplotlib\artist.py", line 41, in draw_wrapper
 return draw(artist, renderer, *args, **kwargs)
 File "C:\Users\mapfu\Anaconda3\lib\site-packages\matplotlib\cbook\deprecation.py", line 411, in wrapper
 return func(*inner_args, **inner_kwargs)
 File "C:\Users\mapfu\Anaconda3\lib\site-packages\matplotlib\axes\_base.py", line 2747, in draw
 mimage._draw_list_compositing_images(renderer, self, artists)
 File "C:\Users\mapfu\Anaconda3\lib\site-packages\matplotlib\image.py", line 131, in _draw_list_compositing_images
 a.draw(renderer)
 File "C:\Users\mapfu\Anaconda3\lib\site-packages\matplotlib\artist.py", line 41, in draw_wrapper
 return draw(artist, renderer, *args, **kwargs)
 File "C:\Users\mapfu\Anaconda3\lib\site-packages\matplotlib\axis.py", line 1178, in draw
 self.label.draw(renderer)
 File "C:\Users\mapfu\Anaconda3\lib\site-packages\matplotlib\artist.py", line 41, in draw_wrapper
 return draw(artist, renderer, *args, **kwargs)
 File "C:\Users\mapfu\Anaconda3\lib\site-packages\matplotlib\text.py", line 681, in draw
 bbox, info, descent = textobj._get_layout(renderer)
 File "C:\Users\mapfu\Anaconda3\lib\site-packages\matplotlib\text.py", line 296, in _get_layout
 clean_line, self._fontproperties, ismath=ismath)
 File "C:\Users\mapfu\Anaconda3\lib\site-packages\matplotlib\backends\backend_agg.py", line 233, in get_text_width_height_descent
 self.mathtext_parser.parse(s, self.dpi, prop)
 File "C:\Users\mapfu\Anaconda3\lib\site-packages\matplotlib\mathtext.py", line 3337, in parse
 s, dpi, prop, rcParams['ps.useafm'], rcParams['mathtext.fontset'])
 File "C:\Users\mapfu\Anaconda3\lib\site-packages\matplotlib\mathtext.py", line 3360, in _parse_cached
 box = self._parser.parse(s, font_output, fontsize, dpi)
 File "C:\Users\mapfu\Anaconda3\lib\site-packages\matplotlib\mathtext.py", line 2618, in parse
 str(err)])) from err
ValueError: 
10^{s\mathd}
 ^
Unknown symbol: \mathd, found '\' (at char 5), (line:1, col:6)
Process finished with exit code 1

Not setting plt.rcParams["axes.formatter.use_mathtext"] = True simply renders the minus signs wrong and returns the familiar RuntimeWarning:

C:\Users\mapfu\Anaconda3\lib\site-packages\matplotlib\backends\backend_agg.py:238: RuntimeWarning: Glyph 8722 missing from current font.
 font.set_text(s, 0.0, flags=flags)
C:\Users\mapfu\Anaconda3\lib\site-packages\matplotlib\backends\backend_agg.py:201: RuntimeWarning: Glyph 8722 missing from current font.
 font.set_text(s, 0, flags=flags)

image

AFIACT this is unrelated. Check the value of exp...

What do you mean? For me it is -4 (str type). I don't have any issues if I use the standard font (i.e. disable all the rcParams changes):
image

EDIT

Oohh, now I get what you mean. With plt.rcParams["axes.formatter.use_mathtext"] = True:
offset_text.get_text() # $\times\mathdefault{10^{−4}}\mathdefault{}$
Without:
offset_text.get_text() # 1e−4
Sooo.. is there a clever work around? I mean I could of course hard code a solution, but I feel like the fix discussed here should work for every case that also works normally with any other font?

Is this still an issue? If so this PR needs to pass the tests (preferably rebased on our modern test).

It's still an issue but I've just been manually looping through axes & labels running .replace() whenever I use cmr10 to substitute out problematic glyphs.

I already had to change the tests once, not sure if I'll have time to rebase, get familiar with the "modern test"s and update again.

Yes sorry for the process. Things fall off the review queue so unless you ping, folks forget!

Well just rebased and naturally it fails. The fix no longer works. The tests are (correctly) failing.

Grrr.

@anntzer any ideas?

I don't know right now, but will throw the ball to gsoc-candidate @aitikgupta :-)

Hi sorry, because I'm still subscribed to this post I was reminded of it again due to the recent comments. So I thought I just point out one more isse that I ran into when using this fix, and which I forgot to report earlier. For some reason, the degree symbol ° is not displayed correctly, so in a polar plot e.g. you also need some manual work around:

import matplotlib.pyplot as plt
from matplotlib.ticker import FormatStrFormatter
# plt.rcParams["axes.formatter.use_mathtext"] = True
plt.rcParams["font.family"] = "serif"
plt.rcParams["font.serif"] = "cmr10"
plt.rcParams["mathtext.fontset"] = "cm"
fig = plt.figure()
ax = fig.add_subplot(111, projection='polar')
ax.set_title('Degree °')
# ax.get_xaxis().set_major_formatter(FormatStrFormatter('%d$,円\degree$'))
fig.show()
labels = [label.get_text() for label in ax.get_xticklabels()]
print(labels)

grafik

Unlike my previous issue this is not related to the use of mathtext. (削除) For some reason, it seems to only affect the tick labels (削除ここまで) (scrap that, I made a stupid mistake), but when printed in the console they look correct. I didn't manage to find a post regarding this specific problem, so i thought I make you aware of it.

For some reason, the degree symbol ° is not displayed correctly..

@mapfiable your bug more or less points to the fact that the glyph for ° just doesn't exist in "cmr10", and it is unrelated to mathtext since even the title (which is not wrapped with \mathtextdefault) can't render it.
This PR only implements falling to a different font file for a certain glyph using mathtext, what you need is a general-fallback. Related: #18883
(@anntzer can you confirm my assertion? - also, should this belong to a new issue?)

Hi @aitikgupta, thanks, but I think you're mistaken. This thread is dealing with the issue that the minus sign symbol (Glyph 8722) is missing from the cmr10 font and results in the error message pointed out in the title. The use of mathtext is then part of the proposed fix.
While my previous issue is a direct consequence of the fix using mathtext (and independed of the font used), it was argued that it is irrelevant to the problem (though I don't fully agree; while I guess it's fair to expect that you should be aware of the implications of using mathtext, I feel like a fix shouldn't produce unexpeced behaviour elsewhere, e.g. in my program I need to switch fonts occasionally and the fix forces me to implement font-dependent plotting routines, though ultimately I'm still very grateful for it).
The issue with the degree symbol however is not related to the use of mathtext, but to the use of the cmr10 font. Just like the minus sign, the degree symbol seems to be missing from the font, although curiously, there is no error message that is pointing this out. So I thought it might make sense to make you aware of the missing degree symbol here, because this thread is dealing with a very similar issue. Maybe this can be solved with the same fix by making some minor additions to it, but I really don't know.

From a very quick look (no guarantees...), I agree with @mapfiable here; I believe fixing the degree-sign problem basically requires a similar fix as for the minus sign, i.e. something like if font.family_name == "cmr10" and uniindex == ord("°"): font = <cmsy10.ttf>; uniindex = <whatever>.

As a reminder, the problem is the following:

• The cmr10 font is basically not intended for standalone use, because many required glyphs are not present in that file but in others (in particular cmsy10).
• Matplotlib non-mathtext rendering is absolutely incapable of rendering a string with glyph from multiple font files (Matplotlib would not try to apply all the font in font list to draw all characters in the given string. #18883 , etc.); fixing this may be a component of @aitikgupta's GSOC work, but in any case it is not an easy problem to fix.
• Therefore, cmr10 mostly only makes sense when used in mathtext (possibly, we could even emit a warning when cmr10 is used in a non-mathtext context, as that seems to be pretty common) -- possibly wrapped with \mathdefault{}, to look like normal text.
• However, even there, \mathdefault just means "use whatever glyphs are present in cmr10.ttf". So in order for glyphs present in other fonts to be used, we must add manual fallbacks to them (which is what this PR does for unicode minus, and could perhaps also do for degree).
• Even if we add a generic fallback system, it is unlikely to actually work for cmr10/cmsy10, because the font encoding of cm is basically ad hoc and different from all other fonts, so it'd need a separate fallback table at least, so we may as well start with some hard-coding here.

I think I misspoke in my statement, I agree with the fundamental issue here - with @mapfiable's example if we do a rcParams["axes.formatter.use_mathtext"] = True, the proposed codepath (that involves if font.family_name == "cmr10" and uniindex == ord("°"): font = <cmsy10.ttf>; uniindex = <whatever>) wouldn't be triggered - which is why I said it's unrelated to mathtext (which isn't entirely true, it's just true for this case because of below reason).

Axis ticks are wrapped around \mathdefault if we trigger the use_mathtext, however I don't think matplotlib.projections.polar.ThetaTick objects are.

To get around this, an interesting thing you did was to "force" the formatting to use mathtext, i.e., when you did:

ax.get_xaxis().set_major_formatter(FormatStrFormatter('%d$,円\degree$'))

However, the font-family in the if condition changed to STIXGeneral, so if you printed:

 if font is not None:
+ print("Not found!", font.family_name, uniindex)
 glyphindex = font.get_char_index(uniindex)

For °, It'll give you Not found! STIXGeneral 176. (somehow user's font family is overridden)
Ultimately, the reason why it "worked" for you is because matlpotlib used the STIX font and not "cmr10" for your ticks.

Overriding a user-defined font shouldn't be the default behaviour right? @anntzer

Could matplotlib perhaps roll its own standalone cmr10 (including missing symbols from cmsy10)?

Overriding a user-defined font shouldn't be the default behaviour right?

Possibly some bad interaction with rcParams["mathtext.fallback"]? (Not sure)

Could matplotlib perhaps roll its own standalone cmr10 (including missing symbols from cmsy10)?

I think this would more or less mean latin modern math (http://www.gust.org.pl/projects/e-foundry/lm-math), which yes, I agree, should be considered as a replacement (I have argued for it elsewhere).

To get around this, an interesting thing you did was to "force" the formatting to use mathtext, i.e., when you did: ...

That's super interesting. I didn't know this was happening at all. In fact, now that you've dissected it like that, I'm not even sure why I was thinking that it should work in the first place.. I guess I thought that ° and \degree are different symbols (what about ^\circ though?).. oh well I guess I just got lucky there. Though it irks me a bit now that I know that actually a different font is used.

EDIT ok just an aside, but something super strage just happened. I could have sworn that

ax.get_xaxis().set_major_formatter(FormatStrFormatter('%d$,円\degree$'))

did the trick, but I just ran it again and now somehow the values are not in degrees anymore but in multiples of pi (i.e. radians)?! I'm pretty sure this was not the case before because when I came up with the solution I had to run it multiple times to make it work and something this obvious should have caught my eye.. but maybe I'm just going crazy (though I really don't understand why the unit / values should change dependent on whether or not the StrFormatter is applied). Anyways, just for the record here is the non-radians solution:

fmt = lambda x, pos: f'{np.degrees(x):.0f}$\degree$'
ax.get_xaxis().set_major_formatter(FuncFormatter(fmt))

@aitikgupta If you had a chance to review this, that would be helpful. Thanks!

Note that the actual solution would still require the warning I mentioned in the above comment, to respect one of @anntzer's comments:

(probably a real patch would need to add a comment there)

Without the warning user wouldn't know that they need to pass rcParams["axes.formatter.use_mathtext"] = True when they use 'cmr10' font.

@aitikgupta Can you open a follow on PR with the warning?