homepage

http://bugs.python.org/issue11735 fixed in 2.7, but in 3.4 and 3.5 the same problem exists. Just crashes python.

Please follow Ned's suggestion that prompted you to open this issue and report the exact version of python, your OS, and how to reproduce it. I cannot reproduce it based on the existing issues using python 3.4.1 on Windows 7.

> I cannot reproduce it based on the existing issues using python 
> 3.4.1 on Windows 7.
Majeed could be referring to the ValueError that gets raised for this [only on Windows][1]. In Linux %f passes through silently. Maybe for 3.5 an alternate approach would be to disable the Windows CRT's invalid parameter handler using _Py_BEGIN_SUPPRESS_IPH and _Py_END_SUPPRESS_IPH.
Looking at this brought issue 10653 to my attention. It seems VC++ 14 has exacerbated that problem, but that the current solution no longer works even in 3.4.3 with VC++ 10. Should the issue be reopened or a new issue opened? 
[1]: https://hg.python.org/cpython/file/413e0e0004f4/Modules/timemodule.c#l611 

The ValueError is not a bug. Not all format codes are supported on all platforms (we start from what the platform supports, and then we have a few we have implemented cross platform, but that isn't one of them). We also pass through the behavior of the underlying system as far as handling unknown codes. Whether that is the best policy or not is an interesting question, but one that should probably be addressed on python-ideas if someone wants to tackle it, since it has backward compatibility implications (and the wider discussion would affect the the behavior on FreeBSD as well, where unknown codes are handled differently than they are on linux, though they don't produce an exception.)
For the other I think you should open a new issue referencing the old one.

When we have this for format:
 date_format = '%Y %m %d %H:%M:%S %z'
I see our logs as:
[2015 05 19 17:36:09 -0500] watcher modified
When I change the format to:
[2015 05 20 08:51:24 -0500] stderr AttributeError: 'StreamLogger' object has no attribute 'flush'
Unhandled exception in thread started by

Actually it is happening in Python 2.7 too..
Here is the format we are trying to use:
 date_format = '%Y %m %d %H:%M:%S:%f %z'
Also, is there something millisecond if not microsecond?

So, this is expected behavior on Windows. I'm inclined to close the issue as not a bug unless the Windows folks think eryksun suggestion is worth considering. (If the error is suppressed, does windows fill in the rest of the values and just leave the %f in the string?)

> If the error is suppressed, does windows fill in the rest of 
> the values and just leave the %f in the string?
No, it does not. I understand now why ValueError was an intentional choice here. The CRT actually breaks the loop with a goto if any call to expand_time fails. So raising a Python exception is the only reasonable course of action. (I still think maybe it's a good place to use the new _Py_BEGIN_SUPPRESS_IPH macro. That way instead of presuming what's a valid format code, it could just handle the EINVAL.)

_Py_*_SUPPRESS_IPH is the right thing to use here. It still displays an assertion dialog in debug builds, but ignoring it has the correct effect.
Patch attached for 3.5

What's the behavior after this patch? Does it just return the string unmodified? Or return a null string? Or is the traceback generated at a higher level (and if so what is the error message?)

It raises ValueError just like now (no visible change), but it won't crash and doesn't require being able to validate the complete format string.
If we want any different behaviour, we need to reimplement strftime for Python, which I'd be okay with, but I'm not the one to do it :)

OK. What made me wonder is that I saw that the 'invalid format string' exception was removed by the patch...I guess that is also raised at a higher level in the code.

It's raised by the existing handling for EINVAL at the end of the function. Previously we'd crash before getting that far because of the invalid parameter handler.

New changeset 254b8e68959e by Steve Dower in branch 'default':
Issue 24244: Prevents termination when an invalid format string is encountered on Windows.
https://hg.python.org/cpython/rev/254b8e68959e 

That handles Python 3.5 and future versions, even if the supported formats change.
Is there something that needs fixing in 3.4? I don't get a crash, just the ValueError...

timemodule.c no longer compiles on MacOSX:
gcc -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -Werror=declaration-after-statement -I./Include -I. -IInclude -I/usr/local/include -I/Users/yury/dev/py/cpython/Include -I/Users/yury/dev/py/cpython -c /Users/yury/dev/py/cpython/Modules/timemodule.c -o build/temp.macosx-10.10-x86_64-3.5/Users/yury/dev/py/cpython/Modules/timemodule.o
/Users/yury/dev/py/cpython/Modules/timemodule.c:656:9: error: use of undeclared identifier '_Py_BEGIN_SUPPRESS_IPH'
 _Py_BEGIN_SUPPRESS_IPH
 ^
/Users/yury/dev/py/cpython/Modules/timemodule.c:658:9: error: use of undeclared identifier '_Py_END_SUPPRESS_IPH'
 _Py_END_SUPPRESS_IPH
 ^
2 errors generated.

That change was in for beta 1, so we would have noticed if we didn't get Mac builds.
Something else has changed, probably some headers.

Yury, I'm not seeing that compile error with current head of default on OS X. Try a clean build, perhaps?

This exact sequence of commands
$ make clean
$ ./configure
$ make -j8
does not build.

$ hg status
shows nothing, branch is default (but 3.5 doesn't get built either) etc.

I think you have a Python installed in /usr/local that is interfering.

But why was it building just fine before this commit? I haven't updated my system packages in a while.

Those macros are only included if Py_BUILD_CORE is defined, regardless of platform (see Include/pyport.h).
Is it possible that's being undefined somehow?

Wild guess: perhaps you did a ./configure or the Makefile did an implicit call to configure recently and/or you did a make install (to /usr/local) before?

> Wild guess: perhaps you did a ./configure or the Makefile did an implicit call to configure recently and/or you did a make install (to /usr/local) before?
I don't have 'python' in /usr/local and /usr/local/bin

But do you have any Python header files in /usr/local/include? The gcc command you pasted shows -I/usr/local/include? Mine don't show that.

yury@ysmac ~/dev/py/cpython (HG: default?) $ ls /usr/local/include/
librtmp osxfuse pycairo

FWIW, I think that in order to use _Py_BEGIN_SUPPRESS_IPH timemodule.c should be compiled with PY_CORE_CFLAGS, and that should be reflected in the Makefile.

Looking at a `grep PY_CORE_CFLAGS`, that sounds reasonable to me.
I assumed that all core files were already being compiled with Py_BUILD_CORE (they certainly are for Windows), so this seems to be an oversight for timemodule.c.

Steve, maybe you can surround "_Py_BEGIN_SUPPRESS_IPH" with "#ifdef Py_BUILD_CORE"?

Yury, another (less) wild guess: do you have an out-of-date Modules/Setup or Setup.local? timemodule is defined in Setup.dist but that will be overridden by a locally modified copy in the Modules directory. Towards the end of the configured top-level Makefile, you should see:
# Rules appended by makedepend
[...]
Modules/timemodule.o: $(srcdir)/Modules/timemodule.c; $(CC) $(PY_CORE_CFLAGS) -c $(srcdir)/Modules/timemodule.c -o Modules/timemodule.o 

When do we build timemodule.c outside of core?

> Towards the end of the configured top-level Makefile, you should see:
Yes, I don't see that line. What should I do to regenerate it? And another question: what did go wrong with my checkout?

Check the times and contents on all of your Modules/Setup* files. Try deleting Setup.local for one.

Though %f is a valid format from Python's doc https://docs.python.org/2/library/datetime.html, the fix just ignores it on Windows? can we atleast get milliseconds on Windows and Micro on Linux?
%f 	Microsecond as a decimal number, zero-padded on the left. 	000000, 000001, ..., 999999 	(4)
%f is an extension to the set of format characters in the C standard (but implemented separately in datetime objects, and therefore always available). When used with the strptime() method, the %f directive accepts from one to six digits and zero pads on the right.
New in version 2.6.

> Check the times and contents on all of your Modules/Setup* files. Try deleting Setup.local for one.
I just made a clean checkout and that helped. I have no idea what caused this. Thank you, Ned, for troubleshooting this with me!

Note that when I run into build problems after an update, I generally run 'make distclean' and then redo the configure/make. This generally cleans up any problems like this (and I don't find that I need to do it very often.)

> Note that when I run into build problems after an update, I generally
> run 'make distclean' and then redo the configure/make.
Yes, but I believe that won't help with changed Modules/Setup* files and, because it is needed in the core interpreter executable, timemodule.c is supposed to be built using Setup and not setup.py. Setup and Setup.local are designed to be locally modified and thus not revision controlled nor cleaned by make clean:
$ cat Modules/Setup.dist
# The file Setup is used by the makesetup script to construct the files
# Makefile and config.c, from Makefile.pre and config.c.in,
# respectively. The file Setup itself is initially copied from
# Setup.dist; once it exists it will not be overwritten, so you can edit
# Setup to your heart's content. Note that Makefile.pre is created
# from Makefile.pre.in by the toplevel configure script.
[...]
$ hg stat --all Modules
I Modules/Setup
I Modules/Setup.config
I Modules/Setup.local 

Right, that's why I said make *dist*clean. That does delete any existing Setup file(s), which is what you want when working with a checkout for Python development purposes.

So you did, sorry! Another, perhaps evan more reliable option is (requires enabling the purge extension in hg):
hg purge --all

Finally got back to looking at this, and since %f works against MSVC 14.0 I'm just going to remove the part of the test that is currently failing and close this issue.

New changeset 2c10e9f62613 by Steve Dower in branch '3.5':
Closes #24244: Removes invalid test from test_time
https://hg.python.org/cpython/rev/2c10e9f62613
New changeset f0ca1fabb41f by Steve Dower in branch 'default':
Closes #24244: Removes invalid test from test_time
https://hg.python.org/cpython/rev/f0ca1fabb41f