[Python-checkins] bpo-34134: Advise to use imap or imap_unordered when handling long iterables. (gh-8324) (gh-11673)

Fri Jan 25 07:08:18 EST 2019

https://github.com/python/cpython/commit/c2674bf11036af1e06c1be739f0eebcc72dfbf7a
commit: c2674bf11036af1e06c1be739f0eebcc72dfbf7a
branch: 3.7
author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com>
committer: Antoine Pitrou <pitrou at free.fr>
date: 2019年01月25日T13:08:14+01:00
summary:
bpo-34134: Advise to use imap or imap_unordered when handling long iterables. (gh-8324) (gh-11673)
(cherry picked from commit 3bab40db96efda2e127ef84e6501fda0cdc4f5b8)
Co-authored-by: Windson yang <wiwindson at outlook.com>
files:
M Doc/library/multiprocessing.rst

diff --git a/Doc/library/multiprocessing.rst b/Doc/library/multiprocessing.rst
index c50625dda320..6ed8f211f361 100644
--- a/Doc/library/multiprocessing.rst
+++ b/Doc/library/multiprocessing.rst
@@ -2148,6 +2148,10 @@ with the :class:`Pool` class.
 the process pool as separate tasks. The (approximate) size of these
 chunks can be specified by setting *chunksize* to a positive integer.
 
+ Note that it may cause high memory usage for very long iterables. Consider
+ using :meth:`imap` or :meth:`imap_unordered` with explicit *chunksize*
+ option for better efficiency.
+
 .. method:: map_async(func, iterable[, chunksize[, callback[, error_callback]]])
 
 A variant of the :meth:`.map` method which returns a result object.
@@ -2166,7 +2170,7 @@ with the :class:`Pool` class.
 
 .. method:: imap(func, iterable[, chunksize])
 
- A lazier version of :meth:`map`.
+ A lazier version of :meth:`.map`.
 
 The *chunksize* argument is the same as the one used by the :meth:`.map`
 method. For very long iterables using a large value for *chunksize* can
