This issue tracker has been migrated to GitHub ,
and is currently read-only.
For more information,
see the GitHub FAQs in the Python's Developer Guide.
Created on 2013年06月21日 11:30 by shwouchk, last changed 2022年04月11日 14:57 by admin. This issue is now closed.
| Messages (10) | |||
|---|---|---|---|
| msg191566 - (view) | Author: (shwouchk) | Date: 2013年06月21日 11:30 | |
Consider this: $ python Python 2.7.4 (default, Apr 19 2013, 18:28:01) [GCC 4.7.3] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import multiprocessing as mp >>> q = mp.Queue() >>> while True: q.put(1) q.get_nowait() Traceback (most recent call last): File "<stdin>", line 3, in <module> File "/usr/lib/python2.7/multiprocessing/queues.py", line 152, in get_nowait return self.get(False) File "/usr/lib/python2.7/multiprocessing/queues.py", line 134, in get raise Empty Queue.Empty I believe that similar behavior could be seen in cPython 2.7.3 with the Queue.Queue implementation, but I can't reproduce it now and don't have the old version to test. And it is irrelevant anyway since it work "correctly" now. I think this behavior is counter intuitive and hampers the development of code that performs stuff with queues in a generic way and works in both single and multi-process environments. |
|||
| msg191609 - (view) | Author: Richard Oudkerk (sbt) * (Python committer) | Date: 2013年06月21日 20:26 | |
This is a very similar issue to #17985. While it may seem counter-intuitive, I don't see how it makes any difference. Another thread/process might remove the item before you can get it. I find it very difficult to imagine a real program where you can safely use get_nowait() without being prepared to handle an Empty exception. |
|||
| msg191613 - (view) | Author: (shwouchk) | Date: 2013年06月21日 22:12 | |
The major difference with the issue you referenced is that the behavior in #17985 is clearly stated and warned against in the docs (Queue.Empty being inconsistent with Queue.size), whereas this is not. As for the actual behavior problem: Imagine you build an abstraction atop Queue (For example and specifically, a queue iterator). This iterator might return None or a similar sentinel if there is nothing on the queue. Since you find the construct useful and would like to keep uniformity in the program, there is a place you use this abstraction in a fairly tight loop to pass messages from one part of the program to another part, in the same process and thread. Now the logic of the program does not work correctly as it was based on the assumption that in a single process Queue would work "as expected". The problem is not the exception but with having an abstraction built atop Queue that works right with respect to program logic both when you use it in one thread and in multiple ones. |
|||
| msg191620 - (view) | Author: Richard Oudkerk (sbt) * (Python committer) | Date: 2013年06月21日 22:47 | |
Why would you use a multi-process queue to "pass messages from one part of the program to another part, in the same process and thread"? Why not just use a deque? Is this something you actually did, or are you just trying to come up with a plausible example? And, of course, if you are sure there must be an item available, you could just use get() instead of get_nowait(). |
|||
| msg191621 - (view) | Author: (shwouchk) | Date: 2013年06月21日 23:04 | |
Richard,
I think you missed my point. First, yes I did do that.
Second ("the point"):
I did this to use the same abstraction that was used extensively for other purposes, instead of recreating the same abstraction with a deque as its basis. Component reusability is one of the main points of OOP, after all...
And no, an item is not necessarily available - sometimes there is a message and sometimes there isn't. But if one was put into the queue, I claim that I should be able to rely on it being available right away in the application logic.
|
|||
| msg191622 - (view) | Author: (shwouchk) | Date: 2013年06月21日 23:05 | |
Also, of course I did this or I would not have stumbled into this issue... |
|||
| msg191762 - (view) | Author: Roundup Robot (python-dev) (Python triager) | Date: 2013年06月24日 13:53 | |
New changeset 8dcc4e017d42 by Richard Oudkerk in branch '2.7': Issue #18277: Document quirks of multiprocessing queue. http://hg.python.org/cpython/rev/8dcc4e017d42 New changeset 0f921e73433a by Richard Oudkerk in branch '3.3': Issue #18277: Document quirks of multiprocessing queue. http://hg.python.org/cpython/rev/0f921e73433a New changeset 06b1447becdc by Richard Oudkerk in branch 'default': Issue #18277: Merge. http://hg.python.org/cpython/rev/06b1447becdc |
|||
| msg191764 - (view) | Author: Richard Oudkerk (sbt) * (Python committer) | Date: 2013年06月24日 14:00 | |
> I did this to use the same abstraction that was used extensively for > other purposes, instead of recreating the same abstraction with a deque > as its basis. So you wanted a FIFO queue and preferred the API of Queue to that of deque? Well it will be *much* less efficient. queue.Queue is also less efficient, but not by such a wide margin. I have added a documentation note. |
|||
| msg191783 - (view) | Author: (shwouchk) | Date: 2013年06月24日 16:22 | |
I agree it might be less efficient, but sometimes it is the price to pay for greater generality/simplicity. After all, If I *really* wanted efficiency perhaps I would have written everything in C++. Anyway, thanks! n.p. 1. "but should not cause any pratical difficulties" <-- you have a typo in 'pratical' there. 2. What exactly do you mean by "managed" queues in the new addition? Also, did part #2 of the note come up in other reports? It seemed somewhat trivial (can't hurt though)... Finally, I don't know whether I'm supposed to close the issue, but its a good solution as far as I'm concerned. Thanks again! |
|||
| msg191790 - (view) | Author: Richard Oudkerk (sbt) * (Python committer) | Date: 2013年06月24日 17:19 | |
> 1. "but should not cause any pratical difficulties" <-- you have a typo in 'pratical' there. > 2. What exactly do you mean by "managed" queues in the new addition? Woops. Fixed now see 860fc6a2bd21, 347647a1f798. A managed queue is one created like manager = multiprocessing.Manager() queue = manager.Queue() queue is a proxy for a conventional queue object in a "manager" process. > Also, did part #2 of the note come up in other reports? > It seemed somewhat trivial (can't hurt though)... Yes it did (but I can't find the report now). |
|||
| History | |||
|---|---|---|---|
| Date | User | Action | Args |
| 2022年04月11日 14:57:47 | admin | set | github: 62477 |
| 2013年06月24日 17:20:36 | sbt | set | status: open -> closed assignee: docs@python components: + Documentation, - Interpreter Core, IO nosy: + docs@python type: behavior -> resolution: fixed stage: resolved |
| 2013年06月24日 17:19:03 | sbt | set | messages: + msg191790 |
| 2013年06月24日 16:22:42 | shwouchk | set | messages: + msg191783 |
| 2013年06月24日 14:00:01 | sbt | set | messages: + msg191764 |
| 2013年06月24日 13:53:26 | python-dev | set | nosy:
+ python-dev messages: + msg191762 |
| 2013年06月21日 23:05:32 | shwouchk | set | messages: + msg191622 |
| 2013年06月21日 23:04:26 | shwouchk | set | messages: + msg191621 |
| 2013年06月21日 22:47:25 | sbt | set | messages: + msg191620 |
| 2013年06月21日 22:12:42 | shwouchk | set | messages: + msg191613 |
| 2013年06月21日 20:26:40 | sbt | set | messages: + msg191609 |
| 2013年06月21日 18:29:53 | ned.deily | set | nosy:
+ sbt |
| 2013年06月21日 11:30:16 | shwouchk | create | |