Timeline for Running 1000 functions gracefully using python multi-processing
Current License: CC BY-SA 4.0
19 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Feb 1, 2021 at 17:22 | vote | accept | coderoftheday | ||
| Feb 1, 2021 at 17:09 | vote | accept | coderoftheday | ||
| Feb 1, 2021 at 17:22 | |||||
| Feb 1, 2021 at 15:16 | answer | added | putty | timeline score: 2 | |
| Jan 31, 2021 at 19:29 | vote | accept | coderoftheday | ||
| Feb 1, 2021 at 17:09 | |||||
| Jan 31, 2021 at 18:56 | answer | added | Lumber Jack | timeline score: 1 | |
| Jan 31, 2021 at 18:16 | comment | added | coderoftheday | @MisterMiyagi I’m very confused with what you’re trying to explain, sorry | |
| Jan 31, 2021 at 18:02 | comment | added | MisterMiyagi |
There are pre-existing tools for load-balanced for name_s in stocks: p = Process(target=info, args=(name_s, )), at least two in the standard library.
|
|
| Jan 31, 2021 at 17:51 | comment | added | coderoftheday | what do you mean 'there is no need to reinvent that'? @MisterMiyagi | |
| Jan 31, 2021 at 17:32 | comment | added | MisterMiyagi |
You probably just want concurrent.futures.ProcessPoolExecutor.map; there is no need to reinvent that. Be aware that as shown your database will not work – each process operates on a copy which is destroyed when the process ends.
|
|
| S Jan 31, 2021 at 17:27 | history | suggested | Diego | CC BY-SA 4.0 |
fix typo in title
|
| Jan 31, 2021 at 17:23 | comment | added | coderoftheday | @LumberJack I've edited the question with the error | |
| Jan 31, 2021 at 17:21 | review | Suggested edits | |||
| S Jan 31, 2021 at 17:27 | |||||
| Jan 31, 2021 at 17:17 | history | edited | coderoftheday | CC BY-SA 4.0 |
added 332 characters in body
|
| Jan 31, 2021 at 16:03 | review | Close votes | |||
| Feb 4, 2021 at 0:02 | |||||
| Jan 31, 2021 at 15:42 | comment | added | Lumber Jack | What do you mean by 'it crashes'? The kernel is dead or it seems to be endless? | |
| Jan 31, 2021 at 15:36 | comment | added | ggorlen | Does this answer your question? Multiprocessing Queue in Python | |
| Jan 31, 2021 at 15:36 | comment | added | coderoftheday | I wouldn't know how to implement this as of yet but I'll look into that thanks @ggorlen | |
| Jan 31, 2021 at 15:35 | comment | added | ggorlen |
Use a task queue that has a max number of workers, probably something like os.cpu_count() workers.
|
|
| Jan 31, 2021 at 15:32 | history | asked | coderoftheday | CC BY-SA 4.0 |