homepage

Upon calling the select.poll objects poll method with no file descriptors registered, it will return an empty list immediately, regardless of the timeout given. This is a problem in the Mac OS X kernels implementation of the poll syscall, but the test can be easily fixed by binding a throwaway socket to the poll objects, and then using the poll method as usual.
Build details are attached

Patch exemplifying a fix for this in the test files

Fixed patch file, now in diff format

Thanks for the report. This seems to be a problem visible in the public betas of the upcoming macOS 12 release, not in previous releases (10.11 and earlier). Do you know if the problem has been reported to Apple? Also, I've uploaded a reviewable version of your patch.

Apple doesn’t allow access to other peoples bug reports, so I couldn’t say whether it’s been reported.
Many Thanks,
Ennis Massey
Wet Ferret Studios <http://wet-ferret-studios.azurewebsites.net/>

OK, if you get a chance, it would be helpful to submit a RADAR on it and update this issue. I neglected to mention that your patch does seem to work, e.g., test_eintr no longer fails.

Ok, cool
Many Thanks,
Ennis Massey
Wet Ferret Studios <http://wet-ferret-studios.azurewebsites.net/>

@Ned - any objection to my committing this at this point?

I'm not sure it should be a permanent fix. Perhaps you could add the test decorator so that it is only tested on 10.12. I don't have time to look at it more myself before b1.

(From https://github.com/curl/curl/issues/1057, the curl project has also seen this and opened an issue with Apple against macOS 10.12, RADAR 28372390.)

The failures in test_eintr and test_asyncio are confusing new contributors on Mac OS X - could we get some variant of this workaround merged until Apple fix their bug?

Sorry, I meant test_poll and test_asyncore as described in http://bugs.python.org/issue28456 (I misremembered due to the test_poll failure mentioning EINTR)

New changeset 1d391f926b37484b8d4b326003a72c0084db19ec by Ned Deily in branch '3.6':
[3.6] bpo-28087: Skip test_asyncore and test_eintr poll failures on macOS (#463)
https://github.com/python/cpython/commit/1d391f926b37484b8d4b326003a72c0084db19ec

New changeset de04644627f82d9dc48b3423def7ff5b4aa1926a by Ned Deily in branch 'master':
bpo-28087: Skip test_asyncore and test_eintr poll failures on macOS. (#462)
https://github.com/python/cpython/commit/de04644627f82d9dc48b3423def7ff5b4aa1926a

test_asyncore also fails on Python 2.7 on macOS Sierra. So I proposed a backport: see my PR 973.
======================================================================
FAIL: test_handle_expt (test.test_asyncore.TestAPI_UsePoll)
----------------------------------------------------------------------
Traceback (most recent call last):
 File "/Users/buildbot/buildarea/2.7.billenstein-sierra/build/Lib/test/test_asyncore.py", line 620, in test_handle_expt
 self.loop_waiting_for_flag(client)
 File "/Users/buildbot/buildarea/2.7.billenstein-sierra/build/Lib/test/test_asyncore.py", line 519, in loop_waiting_for_flag
 self.fail("flag not set")
AssertionError: flag not set
http://buildbot.python.org/all/builders/x86-64%20Sierra%202.7/builds/2/steps/test/logs/stdio 

New changeset 23d6eb656ec29140fcca1c9261b7953e2312b171 by Victor Stinner in branch '2.7':
bpo-28087: Skip test_asyncore and test_eintr poll failures on macOS. (#462) (#973)
https://github.com/python/cpython/commit/23d6eb656ec29140fcca1c9261b7953e2312b171

Ah! Python 2.7 tests succeeded on Sierra!
http://buildbot.python.org/all/builders/x86-64%20Sierra%202.7/builds/3 

Makes sense it would also fail. They both use the same syscall. Gud job on the backport
Sent from my iPhone
> On 4/04/2017, at 4:50 AM, STINNER Victor <report@bugs.python.org> wrote:
> 
> 
> STINNER Victor added the comment:
> 
> Ah! Python 2.7 tests succeeded on Sierra!
> 
> http://buildbot.python.org/all/builders/x86-64%20Sierra%202.7/builds/3
> 
> ----------
> 
> _______________________________________
> Python tracker <report@bugs.python.org>
> <http://bugs.python.org/issue28087>
> _______________________________________

Instead of skipping the test, it would be safer to not provide select.poll() if it's known to be broken. Curl added a check in configure:
https://github.com/curl/curl/commit/9297ca49f5f3caca938a679b9c1feeb719e61ddb
The problem is that I understood that Python provides a single binary for all supported macOS versions. So if we go for a check, it should be done as runtime, as the current select_have_broken_poll() test written for macOS.
On macOS without the bug, curl check takes 0.5 second which is not ideal :-/
Would it be possible to design a test which doesn't block?

> Would it be possible to design a test which doesn't block?
If there is no obvious way to detect poll() functionnaly, a workaround is to get the Darwin version from uname() and blacklist macOS versions known to have the bug (disable select.poll() on these versions).

https://github.com/python/cpython/pull/1426 implements the blacklist option.
I chose the conservative approach: hope that poll() will be fixed in 16.6.x. The problem of this approach is that test_asyncore and test_eintr will fail again if Apple doesn't fix poll().
Since I would like to backport this fix in 2.7, 3.5 and 3.6 and not have to modify the code at each broken macOS release, maybe we can use the pessimistic solution: blacklist Darwin >= 16.x, and only start again to whitelist once Apple releases a fixed macOS.

New changeset 23b312b087582cec5cc8c528d267eb42061601ac by Victor Stinner in branch '3.5':
[3.6] bpo-28087: Skip test_asyncore and test_eintr poll failures on macOS (#463) (#1424)
https://github.com/python/cpython/commit/23b312b087582cec5cc8c528d267eb42061601ac

Please DO NOT REMOVE select.poll() on MacOS. It is still useful.
Apple needs to live with their bug that they refuse to acknowledge by forcing developers to learn the special case that it now breaks in (0 fds). We should not pretend that poll() does not exist as a result.
For reference about the details of the MacOS change they were stupid enough to force upon the world, see this comment - which links to the source of their bug in the "open" source side of their OS.
 https://github.com/curl/curl/issues/1057#issuecomment-252286017 

The only thing CPython should do is apply workarounds to any standard library internal uses of poll with 0 fds guarded by a MacOS platform check.

Do we need to work around this issue at all?
Apple appears to have fixed it already:
 https://github.com/opensource-apple/xnu/commit/0cccba1c831cfe36f0d67d672f8aa67b4a7d4b6a#diff-e61c2932bb9d5cea2dd0732acd8ec626R1783
A comment in https://github.com/curl/curl/issues/1057 suggests Apple has shipped that change in 10.12.2.
If you still want to have Python deal with the 10.12.0 and 10.12.1 cases you could update your PR to check those versions, but given these are security fix releases as 10.12.4 is already current I think it is better to tell people just to apply their security updates and not bother with a workaround in Python.

You want to get poll() on macOS. I'm ok with that but I would like to see
tests for it. We should remove the skip in the test.
Ok, I will update my PR to just blacklist macOS 10.12.0 and 10.2.1, and
reenable the test. I just have to find the Darwin versions.
I don't think that we have to suggests Python users to upgrade, I expect
macOS popups requesting to apply upgrades are enough ;-)

If the problem is fixed in 10.12.2, I agree with gps that there is no need to add a workaround for 10.12.0 and .1. We should only ever need to support the most recent macOS point release; it is the user's responsibility to keep up-to-date and with most users these days that happens automatically.

I abandonned PR 1426 which proposed to blacklist bogus macOS versions, and instead I proposed a new PR to simply reenable previsouly skipped tests (because of the bug which is now fixed).
https://github.com/python/cpython/pull/1664
I'm unable to test my own PR right now. Ned Deily: would you mind to test it and review my change please?

Unfortunately, with the tests reenabled, they still fail with the most recent release of macOS 10.12 (10.12,5) so perhaps the curl issue was a different problem?

Ned Deily added the comment:
> Unfortunately, with the tests reenabled, they still fail with the most recent release of macOS 10.12 (10.12,5)
Oh. In that case, I will update my patch to blacklist macOS versions
with the bug (remove select.poll() depending on macOS version).

Actually, having managed to get macOS to update, it seems to be odd, as it correctly delays. Not sure what’s going on there, or whether it’s my setup. I’ll look into it some more. I’m also thinking of writing/finding a set of tests for core POSIX stuff like this, so we can avoid Apple screwing stuff up like they have.
Many Thanks,
Ennis Massey
ennisbaradine@gmail.com 

Well, the tests worked on macOS 10.12.6 beta, although my school network broke the urllib tests
Many Thanks,
Ennis Massey
ennisbaradine@gmail.com 

What is the status of this issue? Is there still something to do?
If you ask me my opinion, I would just suggest to remove select.poll() on macOS to stop to have to bother with poll() bugs which only trigger at runtime :-/

No update since 2017, I close the issue.