homepage

This issue tracker has been migrated to GitHub , and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: tuple function gives wrong answer when called on list subclass with custom __iter__
Type: behavior Stage: commit review
Components: Library (Lib) Versions: Python 3.4, Python 3.5, Python 2.7
process
Status: closed Resolution: fixed
Dependencies: Superseder: Concrete object C API considered harmful to subclasses of builtin types
View: 10977
Assigned To: rhettinger Nosy List: David MacIver, python-dev, r.david.murray, rhettinger, serhiy.storchaka, stub
Priority: high Keywords: easy, patch

Created on 2015年03月24日 08:42 by David MacIver, last changed 2022年04月11日 14:58 by admin. This issue is now closed.

Files
File name Uploaded Description Edit
listwithiter.py David MacIver, 2015年03月24日 08:42 Reproduction test case
fix_list_to_tuple.diff rhettinger, 2015年05月14日 00:24 Patch without the OP's test review
fix_list_to_tuple_2.diff serhiy.storchaka, 2015年05月17日 09:50 + tests review
Messages (11)
msg239098 - (view) Author: David MacIver (David MacIver) * Date: 2015年03月24日 08:42
Converting a list to a tuple appears to have an optimisation that is wrong in the presence of subclassing to override __iter__. It ignores the user defined iter and uses the normal list one. I've attached a file with a test case to demonstrate this.
I've verified this on both python 2.7 and python 3.4. It's presumably also the case on everything in between.
This was found because it caused a bug with a type that pytz uses, which lazily populates the list on iteration: https://bugs.launchpad.net/pytz/+bug/1435617 
msg239138 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2015年03月24日 15:06
This is a specific instance of the general problem covered by issue 10977.
msg239152 - (view) Author: David MacIver (David MacIver) * Date: 2015年03月24日 16:26
Ah, I hadn't seen that. Thanks for the link.
But... is it really? They have basically the same root cause, but the general problem seems to be hard to fix, while the specific problem here seems to be basically "don't use the concrete API here because it breaks things".
msg239154 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2015年03月24日 16:33
Right, but unless I miss my guess (I haven't looked at the code) it isn't that the concrete api is being used by the constructor, it's that the concrete API is being used by the iterator protocol called by the constructor. If I'm wrong it would be sensible to fix the constructor and we can reopen this. Even if I'm right perhaps there's a way to fix the constructor, but in that case it would be best addressed in the context of that issue.
msg239161 - (view) Author: David MacIver (David MacIver) * Date: 2015年03月24日 17:39
So as a data point, this problem seems to be unique to tuple. set(x), list(x), tuple(iter(x)) all seem to work as expected and respect the overridden __iter__ (set and list were both included in the test case I attached to demonstrated this, iter I just checked right now). This suggests that the problem is in what tuple is doing, not in some general iterator protocol,
msg243073 - (view) Author: Stuart Bishop (stub) Date: 2015年05月13日 11:34
Can we get this reopened? As David MacIver points out, this seems entirely a wart in tuple's constructor (compared to all the other builtin types), whereas 10977 is worrying about how 3rd party code using the C API can corrupt subclasses of builtin types (a much larger scope, and much less likely to be resolved in a good way).
Does it make sense to require python code wishing to case a tuple or tuple subclass do so using tuple(list(o)), or should tuple(o) work as expected? The primary use is of course converting a mutable sequence to an immutable representation to use as a dict key.
msg243158 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2015年05月14日 06:06
Added a patch. Needs to have the OP's test case added.
msg243159 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2015年05月14日 06:10
And it would be nice to add the same test for list, set, etc (if they don't exist).
msg243392 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2015年05月17日 09:50
Raymond's patch LGTM. Here is updated patch with tests.
msg243428 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2015年05月17日 21:37
New changeset b6121a4afad7 by Raymond Hettinger in branch '2.7':
Issue #23757: Only call the concrete list API for exact lists.
https://hg.python.org/cpython/rev/b6121a4afad7 
msg243429 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2015年05月17日 21:47
New changeset c79530e08985 by Raymond Hettinger in branch '3.4':
Issue #23757: Only call the concrete list API for exact lists.
https://hg.python.org/cpython/rev/c79530e08985 
History
Date User Action Args
2022年04月11日 14:58:14adminsetgithub: 67945
2015年05月17日 21:47:59rhettingersetstatus: open -> closed
resolution: fixed
2015年05月17日 21:47:12python-devsetmessages: + msg243429
2015年05月17日 21:37:48python-devsetnosy: + python-dev
messages: + msg243428
2015年05月17日 09:50:51serhiy.storchakasetfiles: + fix_list_to_tuple_2.diff
messages: + msg243392

assignee: rhettinger
keywords: + patch
stage: needs patch -> commit review
2015年05月14日 06:10:22serhiy.storchakasetmessages: + msg243159
2015年05月14日 06:06:42rhettingersetnosy: + rhettinger
messages: + msg243158
2015年05月14日 05:37:55serhiy.storchakasetnosy: + serhiy.storchaka
2015年05月14日 00:25:02rhettingersetkeywords: + easy, - patch
priority: normal -> high
versions: + Python 3.5
2015年05月14日 00:24:29rhettingersetfiles: + fix_list_to_tuple.diff
keywords: + patch
2015年05月13日 12:47:51r.david.murraysetstatus: closed -> open
type: behavior
resolution: duplicate -> (no value)
stage: resolved -> needs patch
2015年05月13日 11:34:00stubsetnosy: + stub
messages: + msg243073
2015年03月24日 17:39:05David MacIversetmessages: + msg239161
2015年03月24日 16:33:42r.david.murraysetmessages: + msg239154
2015年03月24日 16:26:35David MacIversetmessages: + msg239152
2015年03月24日 15:06:24r.david.murraysetstatus: open -> closed

superseder: Concrete object C API considered harmful to subclasses of builtin types

nosy: + r.david.murray
messages: + msg239138
resolution: duplicate
stage: resolved
2015年03月24日 08:42:29David MacIvercreate

AltStyle によって変換されたページ (->オリジナル) /