Timeline for How does asyncio actually work?
Current License: CC BY-SA 4.0
50 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Sep 4, 2024 at 12:50 | comment | added | wilson0x4d | "a new mechanism which uses kernel events called IOCP for Windows" IOCP are not new they are quite old (decades) and IOCP can delegate out to a pool of userspace threads (the use of a single thread would be an implementation-specific detail of the app using IOCP) and in k-space IOCP is triggered starting at ring0 (driver events, in the case of the IP stack NDIS drivers, thus starting their life as hardware events that thus signal directly into a waiting thread, and also why execution is handed off to a userspace thread) -- much of this is also why IOCP outperforms nearly everything. | |
| S Mar 12, 2024 at 16:31 | history | suggested | user16776498 | CC BY-SA 4.0 |
Remove unnecessary *the* . Use md annotations for code.
|
| Mar 12, 2024 at 7:38 | review | Suggested edits | |||
| S Mar 12, 2024 at 16:31 | |||||
| S Oct 5, 2022 at 18:51 | history | suggested | Andy | CC BY-SA 4.0 |
fixed some typos
|
| Oct 5, 2022 at 16:05 | review | Suggested edits | |||
| S Oct 5, 2022 at 18:51 | |||||
| Sep 5, 2021 at 11:20 | history | edited | Alex Waygood | CC BY-SA 4.0 |
Clarity
|
| Aug 11, 2021 at 13:24 | history | bounty awarded | Mike Williamson | ||
| Jul 12, 2021 at 3:20 | history | bounty awarded | paxdiablo | ||
| Jul 9, 2021 at 19:58 | comment | added | Bharel | Indeed, there is no need for mutex in most cases. The only place a mutex is used is for logical reasons (for example shared data structure that should be locked from connection start to finish) | |
| Jul 8, 2021 at 2:47 | comment | added | paxdiablo | Excellent answer. I'm just curious about the need for mutual exclusion here. It looks like call-backs are not executed concurrently, making the whole thing inherently "thread-safe" (because it has no threads). | |
| Feb 28, 2021 at 21:50 | history | bounty awarded | Mike Williamson | ||
| Feb 22, 2021 at 17:13 | comment | added | Mike Williamson | Great, thanks! SO says I have to wait until tomorrow to give it to you. Silly, since it actually asks me the reason, and I wrote, "to award an existing answer". | |
| Feb 22, 2021 at 14:52 | comment | added | Bharel | @MikeWilliamson Thank you :-) You can search for "Start a bounty", it's right right below the question, and select "award an existing answer". | |
| Feb 22, 2021 at 10:47 | comment | added | Mike Williamson | Can I offer a "bounty" of my own points to this answer? I already upvoted, but this is such an amazing answer that I would like to give more. :) | |
| S Jan 31, 2021 at 15:41 | history | suggested | Arnab De | CC BY-SA 4.0 |
fixed grammer
|
| Jan 31, 2021 at 15:30 | review | Suggested edits | |||
| S Jan 31, 2021 at 15:41 | |||||
| S Aug 17, 2020 at 2:59 | history | suggested | Wayde | CC BY-SA 4.0 |
fixed grammar of "it's" to "its" when necessary
|
| Aug 16, 2020 at 20:27 | review | Suggested edits | |||
| S Aug 17, 2020 at 2:59 | |||||
| Jun 20, 2020 at 9:12 | history | edited | Community Bot |
Commonmark migration
|
|
| Jun 17, 2020 at 19:43 | comment | added | Alex Povel | @Bharel Excellent, thanks for the very fast and clear answer. | |
| Jun 17, 2020 at 18:21 | comment | added | Bharel |
@AlexPovel During asyncio.sleep, the main thread is not blocked, and is free to process other async tasks. If for example you'll run several tasks of asyncio.sleep(2), they will all run at the same time, and take a total of 2 seconds. On the other hand, if you'll run several time.sleep(2), they won't run in parallel, but only one after the other, and will take 2*tasks seconds.
|
|
| Jun 17, 2020 at 12:30 | comment | added | Alex Povel |
Thank you @Bharel for providing this sample. Now, what I struggle with is: how is that provided example different from conventional subroutines? Consider these steps: simply remove await/async keywords, all loop stuff, replace asyncio.sleep with time.sleep and call print_sum normally. Normal subroutines are yielded (no pun intended), with what seems to be the same functionality. Where does asyncio come into play/improve things?
|
|
| May 5, 2020 at 14:51 | history | edited | Bharel | CC BY-SA 4.0 |
added 161 characters in body
|
| Apr 5, 2020 at 21:48 | comment | added | Bharel | @Lore I simplified it a bit, look at the example again. Yield from basically enters into another generator, until that generator is finished. | |
| Apr 5, 2020 at 21:47 | history | edited | Bharel | CC BY-SA 4.0 |
simplify
|
| Apr 4, 2020 at 19:16 | comment | added | Lore | I cannot understand the yield from example... can you explain me? | |
| Feb 18, 2020 at 12:30 | comment | added | Bharel |
@zonk Multiple tasks are handled like a fifo queue. Since it's a single thread, only one can finish at a time, therefore order is known and guaranteed. Resuming tasks after a socket received data, is not in any guaranteed order. (If 2 sockets return from select at the same time, the order is chosen by the operating system).
|
|
| Feb 17, 2020 at 16:24 | comment | added | zonk | @Bharel Thx for the quick response :). I'm a bit confused now. At 2) You say that multiple tasks are handled like a fifo queue and in 4) you say that order is not guaranted? | |
| Feb 17, 2020 at 16:01 | comment | added | Bharel | @zonk I can add more clarity as needed. Tell me if my answer helped and I'll see how to integrate it into my article. | |
| Feb 17, 2020 at 15:56 | comment | added | Bharel | @zonk 1. Event loop checks futures, and if nothing happens waits on sockets for data. 2. Multiple tasks are thrown into an internal list that acts as a fifo queue. Only tasks that are ready for execution are thrown in there. 3. Each cycle all tasks that are ready are executed, and ready sockets awake their relevant tasks. If a task waits for a socket, it won't be added to the queue, and as soon as the socket will be ready, the event loop will add it back. 4. Event loop does not guarantee order. Besides, apart from 2 sockets, there can't be 2 tasks that finish on the same time (Single thread). | |
| Feb 17, 2020 at 15:43 | comment | added | zonk | @Bharel The article itself is very helpfull,thx for that, but I was a bit lost at "Asyncio", when you started with the socket stuff. Is the Event loop itself a socket which is sending/receiving stuff, checks futures and if nothing happens it waits? I saw the nice picture of chained Tasks, but how are multiple tasks scheduled in the event loop itself? Is there a FIFO/FILO/whatever stack, where new tasks are thrown onto? When two tasks finish at the same time, how does the loop decide which comes first? | |
| S Feb 11, 2020 at 18:42 | history | suggested | aiven | CC BY-SA 4.0 |
clarification for async from example
|
| Feb 11, 2020 at 16:24 | review | Suggested edits | |||
| S Feb 11, 2020 at 18:42 | |||||
| Oct 31, 2018 at 22:27 | history | edited | Brad Solomon | CC BY-SA 4.0 |
small typo
|
| S Oct 23, 2018 at 18:10 | history | suggested | rkta | CC BY-SA 4.0 |
Change link to sequence diagram, original link redirected to newest version with out the diagramm; add direct link to diagram graphic
|
| Oct 23, 2018 at 17:34 | review | Suggested edits | |||
| S Oct 23, 2018 at 18:10 | |||||
| Sep 16, 2018 at 15:22 | comment | added | mjkim | This post has info of backbone of asynchronous I/O in Python. Thanks for such a kind explanation. | |
| Jul 9, 2018 at 12:00 | audit | Low quality answers | |||
| Jul 9, 2018 at 12:00 | |||||
| Jul 4, 2018 at 11:40 | comment | added | Alper t. Turker | @MisterMiyagi I guess it is a matter of expectations. Event loop alone is somewhat pedestrian and language constructs used are not a particularly interesting and could be implemented in a different way without a loss of functionality. Interaction with OS API is what makes asyncio useful in practice and hard to comprehend. Of course it just an opinion, and I am glad that bounty promoted a comprehensive answer. | |
| Jul 3, 2018 at 14:47 | comment | added | MisterMiyagi |
The things before the Asyncio section are perhaps the most critical, as they are the only thing the language actually does by itself. The select may qualify as well, since it is how non-blocking I/O system calls work on OS. The actual asyncio constructs and event loop are just app-level code built from these things.
|
|
| Jul 2, 2018 at 19:36 | comment | added | Bharel |
@user8371915 Always here to help :-) Keep in mind that in order to understand Asyncio you must know how generators, generator communication and yield from works. I did however note up top that it's skippable in case the reader already knows about it :-) Anything else you believe I should add?
|
|
| Jul 2, 2018 at 11:39 | comment | added | Alper t. Turker | Thanks for the answer Bharel. It is just an opinion, but everything before Asyncio seems superficial and doesn't contribute much to the answer (although it would serve as a nice intro in a more general context). | |
| Jul 1, 2018 at 8:28 | comment | added | Bharel | On an asynchronous socket, attempting to send or receive data checks the OS buffer first. If you're trying to receive and there's no data in the buffer, the underlying receive function will return an error value which will propagate as an exception in Python. Same with send and a full buffer. When the exception is raised, Python in turn sends those sockets over to the select function which suspends the process. But it's not how asyncio works, it's how select and sockets work which is highly OS specific as well. | |
| Jul 1, 2018 at 8:20 | comment | added | Bharel |
There's select for Windows which waits on sockets and wakes up the process epoll for Linux and a new mechanism which uses kernel events called IOCP for Windows as well. No different thread is used, the interpreter is process is suspended and waits for the kernel to wake it up. Python never calls any C procedure besides the selection for the waiting part, as calling open() will block and sending data on a full buffer will block.
|
|
| Jun 30, 2018 at 18:43 | comment | added | Bharel | If there's any more explanation needed, don't hesitate to comment. Btw, I'm not entirely sure if I should have written this as a blog article or an answer in stackoverflow. The question is a long one to answer. | |
| Jun 30, 2018 at 18:37 | history | edited | Bharel | CC BY-SA 4.0 |
added 2559 characters in body
|
| Jun 30, 2018 at 18:22 | history | edited | Bharel | CC BY-SA 4.0 |
added 2559 characters in body
|
| Jun 30, 2018 at 18:05 | history | edited | Bharel | CC BY-SA 4.0 |
added 2559 characters in body
|
| Jun 30, 2018 at 17:42 | history | edited | Bharel | CC BY-SA 4.0 |
added 2559 characters in body
|
| Jun 30, 2018 at 17:03 | history | answered | Bharel | CC BY-SA 4.0 |