This issue tracker has been migrated to GitHub ,
and is currently read-only.
For more information,
see the GitHub FAQs in the Python's Developer Guide.
Created on 2009年02月19日 01:39 by pitrou, last changed 2022年04月11日 14:56 by admin. This issue is now closed.
| Files | ||||
|---|---|---|---|---|
| File name | Uploaded | Description | Edit | |
| build_ext_parallel.patch | pitrou, 2013年10月26日 20:08 | review | ||
| build_ext_parallel2.patch | pitrou, 2013年10月26日 21:09 | review | ||
| build_ext_parallel3.patch | christian.heimes, 2013年10月26日 22:50 | review | ||
| build_ext_parallel4.patch | Sjlver, 2014年09月05日 09:48 | review | ||
| build_ext_parallel5.patch | pitrou, 2014年09月06日 23:16 | review | ||
| build_ext_parallel6.patch | pitrou, 2014年09月20日 22:26 | review | ||
| issue5309-doc.diff | berker.peksag, 2015年01月22日 21:52 | review | ||
| Messages (30) | |||
|---|---|---|---|
| msg82447 - (view) | Author: Antoine Pitrou (pitrou) * (Python committer) | Date: 2009年02月19日 01:39 | |
When run with "make -jN", the Python compilation process is able to parallelize across N concurrent processes for the interpreter core, but extension modules are still compiled sequentially. |
|||
| msg116603 - (view) | Author: Éric Araujo (eric.araujo) * (Python committer) | Date: 2010年09月16日 20:24 | |
Do you want to work on a patch? |
|||
| msg178993 - (view) | Author: Antoine Pitrou (pitrou) * (Python committer) | Date: 2013年01月03日 22:27 | |
The original request is really about setup.py, not packaging. I don't care about packaging, and it's not in the stdlib. |
|||
| msg179010 - (view) | Author: Christian Heimes (christian.heimes) * (Python committer) | Date: 2013年01月04日 10:01 | |
+1 I think it's sufficient to parallelize the compilation step (.c -> .o) and ignore the linker step. Linking is usually so fast that it doesn't matter. Idea: * separate compile step from link step * run all compile calls for all extensions in parallel until all objects of all extensions are generated * do linking and error reporting in serial mode |
|||
| msg201392 - (view) | Author: Antoine Pitrou (pitrou) * (Python committer) | Date: 2013年10月26日 20:08 | |
Here is a working patch. |
|||
| msg201396 - (view) | Author: Antoine Pitrou (pitrou) * (Python committer) | Date: 2013年10月26日 21:09 | |
Here is an updated patch which only enables parallel building in setup.py. |
|||
| msg201400 - (view) | Author: Christian Heimes (christian.heimes) * (Python committer) | Date: 2013年10月26日 22:50 | |
Antoine's patch doesn't work for a fresh copy of Python - add import error report to build_ext parallel (distutils doesn't use logging and doesn't have a working log.exception() function) - make time a builtin module - add hack for concurrent.futures.process - add --parallel=i / -ji (jobs) argument to build_ext |
|||
| msg226412 - (view) | Author: Jonas Wagner (Sjlver) * | Date: 2014年09月05日 09:48 | |
Is there a reason this has not landed? The patch works perfectly for me, except for one issue:
@@ -268,6 +275,9 @@
if self.undef:
self.undef = self.undef.split(',')
+ if self.parallel:
+ self.parallel = int(self.parallel)
+
if self.swig_opts is None:
self.swig_opts = []
else:
If self.parallel is True, this will set self.parallel to 1, causing it to use one worker instead of n (where n is the number of processors).
An updated patch is attached.
|
|||
| msg226415 - (view) | Author: Stefan Behnel (scoder) * (Python committer) | Date: 2014年09月05日 09:54 | |
Yes, please. The sequential build in distutils is very annoying. (too late for Py3.4, though, I guess...) |
|||
| msg226424 - (view) | Author: Jonas Wagner (Sjlver) * | Date: 2014年09月05日 13:59 | |
With this patch, and on Ubuntu 14.04, occasionally modules fail to build with the following error: *** WARNING: renaming "_testbuffer" since importing it failed: dlopen: cannot load any more object with static TLS I'm not 100% sure if this is really due to the patch... but I've never seen it before. I'll try investigate a bit more. |
|||
| msg226521 - (view) | Author: Antoine Pitrou (pitrou) * (Python committer) | Date: 2014年09月06日 23:16 | |
Updated patch: - add the -j option to the "build" command as well (even though it's only used by build_ext) - in setup.py, only force parallel compilation if make was called with -j |
|||
| msg226523 - (view) | Author: Arfrever Frehtes Taifersar Arahesis (Arfrever) * (Python triager) | Date: 2014年09月06日 23:37 | |
- Is support for infinite jobs planned? (GNU make supports it for -j without number argument.) - self.parallel = int(self.parallel) would raise ValueError when a non-number is passed. I suggest to print user-friendly error message. |
|||
| msg226546 - (view) | Author: Éric Araujo (eric.araujo) * (Python committer) | Date: 2014年09月07日 21:23 | |
> self.parallel = int(self.parallel) would raise ValueError when a > non-number is passed. I suggest to print user-friendly error message. The distutils idiom would be to catch the TypeError/ValueError and raise DistutilsOptionError. Higher layers convert that to a message. |
|||
| msg226646 - (view) | Author: Jonas Wagner (Sjlver) * | Date: 2014年09月09日 16:40 | |
I've checked the `dlopen` issue. This was due to a problem with UndefinedBehaviorSanitizer, and was solved by upgrading to Clang 3.5. It has little to do with this patch. Using this patch and http://bugs.python.org/issue22359 , I now get reliable parallel builds. |
|||
| msg227188 - (view) | Author: Antoine Pitrou (pitrou) * (Python committer) | Date: 2014年09月20日 22:26 | |
Updated patch raising DistutilsOptionError. I couldn't find any docs so I didn't update them :-) |
|||
| msg227651 - (view) | Author: Roundup Robot (python-dev) (Python triager) | Date: 2014年09月26日 21:32 | |
New changeset bbe57429eba0 by Antoine Pitrou in branch 'default': Issue #5309: distutils' build and build_ext commands now accept a ``-j`` https://hg.python.org/cpython/rev/bbe57429eba0 |
|||
| msg227652 - (view) | Author: Antoine Pitrou (pitrou) * (Python committer) | Date: 2014年09月26日 21:33 | |
This is now pushed. |
|||
| msg227877 - (view) | Author: STINNER Victor (vstinner) * (Python committer) | Date: 2014年09月30日 10:57 | |
It looks like compilation of Python 3.5 fails on FreeBSD 6.4 because of the changeset bbe57429eba0a9ec21fb0f1178f409f1bba44c22: http://buildbot.python.org/all/builders/x86%20FreeBSD%206.4%203.x/builds/5061 Compile log: --- building '_ctypes' extension creating build/temp.freebsd-6.4-RELEASE-i386-3.5-pydebug/usr/home/db3l/buildarea/3.x.bolen-freebsd/build/Modules/_ctypes/libffi creating build/temp.freebsd-6.4-RELEASE-i386-3.5-pydebug/usr/home/db3l/buildarea/3.x.bolen-freebsd/build/Modules/_ctypes/libffi/src creating build/temp.freebsd-6.4-RELEASE-i386-3.5-pydebug/usr/home/db3l/buildarea/3.x.bolen-freebsd/build/Modules/_ctypes/libffi/src/x86 gcc -pthread -fPIC -fno-strict-aliasing -Wsign-compare -g -O0 -Wall -Wstrict-prototypes -Ibuild/temp.freebsd-6.4-RELEASE-i386-3.5-pydebug/libffi/include -Ibuild/temp.freebsd-6.4-RELEASE-i386-3.5-pydebug/libffi -I/usr/home/db3l/buildarea/3.x.bolen-freebsd/build/Modules/_ctypes/libffi/src -I./Include -I. -IInclude -I/usr/local/include -I/usr/home/db3l/buildarea/3.x.bolen-freebsd/build/Include -I/usr/home/db3l/buildarea/3.x.bolen-freebsd/build -c /usr/home/db3l/buildarea/3.x.bolen-freebsd/build/Modules/_ctypes/_ctypes.c -o build/temp.freebsd-6.4-RELEASE-i386-3.5-pydebug/usr/home/db3l/buildarea/3.x.bolen-freebsd/build/Modules/_ctypes/_ctypes.o -Wall -fexceptions gcc -pthread -fPIC -fno-strict-aliasing -Wsign-compare -g -O0 -Wall -Wstrict-prototypes -Ibuild/temp.freebsd-6.4-RELEASE-i386-3.5-pydebug/libffi/include -Ibuild/temp.freebsd-6.4-RELEASE-i386-3.5-pydebug/libffi -I/usr/home/db3l/buildarea/3.x.bolen-freebsd/build/Modules/_ctypes/libffi/src -I./Include -I. -IInclude -I/usr/local/include -I/usr/home/db3l/buildarea/3.x.bolen-freebsd/build/Include -I/usr/home/db3l/buildarea/3.x.bolen-freebsd/build -c /usr/home/db3l/buildarea/3.x.bolen-freebsd/build/Modules/_ctypes/callbacks.c -o build/temp.freebsd-6.4-RELEASE-i386-3.5-pydebug/usr/home/db3l/buildarea/3.x.bolen-freebsd/build/Modules/_ctypes/callbacks.o -Wall -fexceptions gcc -pthread -fPIC -fno-strict-aliasing -Wsign-compare -g -O0 -Wall -Wstrict-prototypes -Ibuild/temp.freebsd-6.4-RELEASE-i386-3.5-pydebug/libffi/include -Ibuild/temp.freebsd-6.4-RELEASE-i386-3.5-pydebug/libffi -I/usr/home/db3l/buildarea/3.x.bolen-freebsd/build/Modules/_ctypes/libffi/src -I./Include -I. -IInclude -I/usr/local/include -I/usr/home/db3l/buildarea/3.x.bolen-freebsd/build/Include -I/usr/home/db3l/buildarea/3.x.bolen-freebsd/build -c /usr/home/db3l/buildarea/3.x.bolen-freebsd/build/Modules/_ctypes/callproc.c -o build/temp.freebsd-6.4-RELEASE-i386-3.5-pydebug/usr/home/db3l/buildarea/3.x.bolen-freebsd/build/Modules/_ctypes/callproc.o -Wall -fexceptions /usr/home/db3l/buildarea/3.x.bolen-freebsd/build/Modules/_ctypes/callproc.c: In function `py_dl_open': /usr/home/db3l/buildarea/3.x.bolen-freebsd/build/Modules/_ctypes/callproc.c:1336: warning: initialization discards qualifiers from pointer target type gcc -pthread -fPIC -fno-strict-aliasing -Wsign-compare -g -O0 -Wall -Wstrict-prototypes -Ibuild/temp.freebsd-6.4-RELEASE-i386-3.5-pydebug/libffi/include -Ibuild/temp.freebsd-6.4-RELEASE-i386-3.5-pydebug/libffi -I/usr/home/db3l/buildarea/3.x.bolen-freebsd/build/Modules/_ctypes/libffi/src -I./Include -I. -IInclude -I/usr/local/include -I/usr/home/db3l/buildarea/3.x.bolen-freebsd/build/Include -I/usr/home/db3l/buildarea/3.x.bolen-freebsd/build -c /usr/home/db3l/buildarea/3.x.bolen-freebsd/build/Modules/_ctypes/stgdict.c -o build/temp.freebsd-6.4-RELEASE-i386-3.5-pydebug/usr/home/db3l/buildarea/3.x.bolen-freebsd/build/Modules/_ctypes/stgdict.o -Wall -fexceptions gcc -pthread -fPIC -fno-strict-aliasing -Wsign-compare -g -O0 -Wall -Wstrict-prototypes -Ibuild/temp.freebsd-6.4-RELEASE-i386-3.5-pydebug/libffi/include -Ibuild/temp.freebsd-6.4-RELEASE-i386-3.5-pydebug/libffi -I/usr/home/db3l/buildarea/3.x.bolen-freebsd/build/Modules/_ctypes/libffi/src -I./Include -I. -IInclude -I/usr/local/include -I/usr/home/db3l/buildarea/3.x.bolen-freebsd/build/Include -I/usr/home/db3l/buildarea/3.x.bolen-freebsd/build -c /usr/home/db3l/buildarea/3.x.bolen-freebsd/build/Modules/_ctypes/cfield.c -o build/temp.freebsd-6.4-RELEASE-i386-3.5-pydebug/usr/home/db3l/buildarea/3.x.bolen-freebsd/build/Modules/_ctypes/cfield.o -Wall -fexceptions gcc -pthread -fPIC -fno-strict-aliasing -Wsign-compare -g -O0 -Wall -Wstrict-prototypes -Ibuild/temp.freebsd-6.4-RELEASE-i386-3.5-pydebug/libffi/include -Ibuild/temp.freebsd-6.4-RELEASE-i386-3.5-pydebug/libffi -I/usr/home/db3l/buildarea/3.x.bolen-freebsd/build/Modules/_ctypes/libffi/src -I./Include -I. -IInclude -I/usr/local/include -I/usr/home/db3l/buildarea/3.x.bolen-freebsd/build/Include -I/usr/home/db3l/buildarea/3.x.bolen-freebsd/build -c /usr/home/db3l/buildarea/3.x.bolen-freebsd/build/Modules/_ctypes/libffi/src/prep_cif.c -o build/temp.freebsd-6.4-RELEASE-i386-3.5-pydebug/usr/home/db3l/buildarea/3.x.bolen-freebsd/build/Modules/_ctypes/libffi/src/prep_cif.o -Wall -fexceptions gcc -pthread -fPIC -fno-strict-aliasing -Wsign-compare -g -O0 -Wall -Wstrict-prototypes -Ibuild/temp.freebsd-6.4-RELEASE-i386-3.5-pydebug/libffi/include -Ibuild/temp.freebsd-6.4-RELEASE-i386-3.5-pydebug/libffi -I/usr/home/db3l/buildarea/3.x.bolen-freebsd/build/Modules/_ctypes/libffi/src -I./Include -I. -IInclude -I/usr/local/include -I/usr/home/db3l/buildarea/3.x.bolen-freebsd/build/Include -I/usr/home/db3l/buildarea/3.x.bolen-freebsd/build -c /usr/home/db3l/buildarea/3.x.bolen-freebsd/build/Modules/_ctypes/libffi/src/closures.c -o build/temp.freebsd-6.4-RELEASE-i386-3.5-pydebug/usr/home/db3l/buildarea/3.x.bolen-freebsd/build/Modules/_ctypes/libffi/src/closures.o -Wall -fexceptions gcc -pthread -fPIC -fno-strict-aliasing -Wsign-compare -g -O0 -Wall -Wstrict-prototypes -Ibuild/temp.freebsd-6.4-RELEASE-i386-3.5-pydebug/libffi/include -Ibuild/temp.freebsd-6.4-RELEASE-i386-3.5-pydebug/libffi -I/usr/home/db3l/buildarea/3.x.bolen-freebsd/build/Modules/_ctypes/libffi/src -I./Include -I. -IInclude -I/usr/local/include -I/usr/home/db3l/buildarea/3.x.bolen-freebsd/build/Include -I/usr/home/db3l/buildarea/3.x.bolen-freebsd/build -c /usr/home/db3l/buildarea/3.x.bolen-freebsd/build/Modules/_ctypes/libffi/src/x86/ffi.c -o build/temp.freebsd-6.4-RELEASE-i386-3.5-pydebug/usr/home/db3l/buildarea/3.x.bolen-freebsd/build/Modules/_ctypes/libffi/src/x86/ffi.o -Wall -fexceptions gcc -pthread -fPIC -fno-strict-aliasing -Wsign-compare -g -O0 -Wall -Wstrict-prototypes -Ibuild/temp.freebsd-6.4-RELEASE-i386-3.5-pydebug/libffi/include -Ibuild/temp.freebsd-6.4-RELEASE-i386-3.5-pydebug/libffi -I/usr/home/db3l/buildarea/3.x.bolen-freebsd/build/Modules/_ctypes/libffi/src -I./Include -I. -IInclude -I/usr/local/include -I/usr/home/db3l/buildarea/3.x.bolen-freebsd/build/Include -I/usr/home/db3l/buildarea/3.x.bolen-freebsd/build -c /usr/home/db3l/buildarea/3.x.bolen-freebsd/build/Modules/_ctypes/libffi/src/x86/freebsd.S -o build/temp.freebsd-6.4-RELEASE-i386-3.5-pydebug/usr/home/db3l/buildarea/3.x.bolen-freebsd/build/Modules/_ctypes/libffi/src/x86/freebsd.o -Wall -fexceptions gcc -pthread -shared build/temp.freebsd-6.4-RELEASE-i386-3.5-pydebug/usr/home/db3l/buildarea/3.x.bolen-freebsd/build/Modules/_ctypes/_ctypes.o build/temp.freebsd-6.4-RELEASE-i386-3.5-pydebug/usr/home/db3l/buildarea/3.x.bolen-freebsd/build/Modules/_ctypes/callbacks.o build/temp.freebsd-6.4-RELEASE-i386-3.5-pydebug/usr/home/db3l/buildarea/3.x.bolen-freebsd/build/Modules/_ctypes/callproc.o build/temp.freebsd-6.4-RELEASE-i386-3.5-pydebug/usr/home/db3l/buildarea/3.x.bolen-freebsd/build/Modules/_ctypes/stgdict.o build/temp.freebsd-6.4-RELEASE-i386-3.5-pydebug/usr/home/db3l/buildarea/3.x.bolen-freebsd/build/Modules/_ctypes/cfield.o build/temp.freebsd-6.4-RELEASE-i386-3.5-pydebug/usr/home/db3l/buildarea/3.x.bolen-freebsd/build/Modules/_ctypes/libffi/src/prep_cif.o build/temp.freebsd-6.4-RELEASE-i386-3.5-pydebug/usr/home/db3l/buildarea/3.x.bolen-freebsd/build/Modules/_ctypes/libffi/src/closures.o build/temp.freebsd-6.4-RELEASE-i386-3.5-pydebug/usr/home/db3l/buildarea/3.x.bolen-freebsd/build/Modules/_ctypes/libffi/src/x86/ffi.o build/temp.freebsd-6.4-RELEASE-i386-3.5-pydebug/usr/home/db3l/buildarea/3.x.bolen-freebsd/build/Modules/_ctypes/libffi/src/x86/freebsd.o -L/usr/local/lib -o build/lib.freebsd-6.4-RELEASE-i386-3.5-pydebug/_ctypes.so *** WARNING: renaming "_ssl" since importing it failed: build/lib.freebsd-6.4-RELEASE-i386-3.5-pydebug/_ssl.so: Undefined symbol "X509_check_ca" *** WARNING: renaming "_ctypes" since importing it failed: build/lib.freebsd-6.4-RELEASE-i386-3.5-pydebug/_ctypes.so: Undefined symbol "ffi_call_win32" error: [Errno 2] No such file or directory: 'build/temp.freebsd-6.4-RELEASE-i386-3.5-pydebug/usr/home/db3l/buildarea/3.x.bolen-freebsd/build/Modules/_ctypes/_ctypes.o' *** Error code 1 --- Before: --- building '_ctypes' extension gcc -pthread -fPIC -fno-strict-aliasing -Wsign-compare -g -O0 -Wall -Wstrict-prototypes -Ibuild/temp.freebsd-6.4-RELEASE-i386-3.5-pydebug/libffi/include -Ibuild/temp.freebsd-6.4-RELEASE-i386-3.5-pydebug/libffi -I/usr/home/db3l/buildarea/3.x.bolen-freebsd/build/Modules/_ctypes/libffi/src -I./Include -I. -IInclude -I/usr/local/include -I/usr/home/db3l/buildarea/3.x.bolen-freebsd/build/Include -I/usr/home/db3l/buildarea/3.x.bolen-freebsd/build -c /usr/home/db3l/buildarea/3.x.bolen-freebsd/build/Modules/_ctypes/_ctypes.c -o build/temp.freebsd-6.4-RELEASE-i386-3.5-pydebug/usr/home/db3l/buildarea/3.x.bolen-freebsd/build/Modules/_ctypes/_ctypes.o -Wall -fexceptions gcc -pthread -fPIC -fno-strict-aliasing -Wsign-compare -g -O0 -Wall -Wstrict-prototypes -Ibuild/temp.freebsd-6.4-RELEASE-i386-3.5-pydebug/libffi/include -Ibuild/temp.freebsd-6.4-RELEASE-i386-3.5-pydebug/libffi -I/usr/home/db3l/buildarea/3.x.bolen-freebsd/build/Modules/_ctypes/libffi/src -I./Include -I. -IInclude -I/usr/local/include -I/usr/home/db3l/buildarea/3.x.bolen-freebsd/build/Include -I/usr/home/db3l/buildarea/3.x.bolen-freebsd/build -c /usr/home/db3l/buildarea/3.x.bolen-freebsd/build/Modules/_ctypes/callbacks.c -o build/temp.freebsd-6.4-RELEASE-i386-3.5-pydebug/usr/home/db3l/buildarea/3.x.bolen-freebsd/build/Modules/_ctypes/callbacks.o -Wall -fexceptions gcc -pthread -fPIC -fno-strict-aliasing -Wsign-compare -g -O0 -Wall -Wstrict-prototypes -Ibuild/temp.freebsd-6.4-RELEASE-i386-3.5-pydebug/libffi/include -Ibuild/temp.freebsd-6.4-RELEASE-i386-3.5-pydebug/libffi -I/usr/home/db3l/buildarea/3.x.bolen-freebsd/build/Modules/_ctypes/libffi/src -I./Include -I. -IInclude -I/usr/local/include -I/usr/home/db3l/buildarea/3.x.bolen-freebsd/build/Include -I/usr/home/db3l/buildarea/3.x.bolen-freebsd/build -c /usr/home/db3l/buildarea/3.x.bolen-freebsd/build/Modules/_ctypes/callproc.c -o build/temp.freebsd-6.4-RELEASE-i386-3.5-pydebug/usr/home/db3l/buildarea/3.x.bolen-freebsd/build/Modules/_ctypes/callproc.o -Wall -fexceptions /usr/home/db3l/buildarea/3.x.bolen-freebsd/build/Modules/_ctypes/callproc.c: In function `py_dl_open': /usr/home/db3l/buildarea/3.x.bolen-freebsd/build/Modules/_ctypes/callproc.c:1336: warning: initialization discards qualifiers from pointer target type gcc -pthread -fPIC -fno-strict-aliasing -Wsign-compare -g -O0 -Wall -Wstrict-prototypes -Ibuild/temp.freebsd-6.4-RELEASE-i386-3.5-pydebug/libffi/include -Ibuild/temp.freebsd-6.4-RELEASE-i386-3.5-pydebug/libffi -I/usr/home/db3l/buildarea/3.x.bolen-freebsd/build/Modules/_ctypes/libffi/src -I./Include -I. -IInclude -I/usr/local/include -I/usr/home/db3l/buildarea/3.x.bolen-freebsd/build/Include -I/usr/home/db3l/buildarea/3.x.bolen-freebsd/build -c /usr/home/db3l/buildarea/3.x.bolen-freebsd/build/Modules/_ctypes/stgdict.c -o build/temp.freebsd-6.4-RELEASE-i386-3.5-pydebug/usr/home/db3l/buildarea/3.x.bolen-freebsd/build/Modules/_ctypes/stgdict.o -Wall -fexceptions gcc -pthread -fPIC -fno-strict-aliasing -Wsign-compare -g -O0 -Wall -Wstrict-prototypes -Ibuild/temp.freebsd-6.4-RELEASE-i386-3.5-pydebug/libffi/include -Ibuild/temp.freebsd-6.4-RELEASE-i386-3.5-pydebug/libffi -I/usr/home/db3l/buildarea/3.x.bolen-freebsd/build/Modules/_ctypes/libffi/src -I./Include -I. -IInclude -I/usr/local/include -I/usr/home/db3l/buildarea/3.x.bolen-freebsd/build/Include -I/usr/home/db3l/buildarea/3.x.bolen-freebsd/build -c /usr/home/db3l/buildarea/3.x.bolen-freebsd/build/Modules/_ctypes/cfield.c -o build/temp.freebsd-6.4-RELEASE-i386-3.5-pydebug/usr/home/db3l/buildarea/3.x.bolen-freebsd/build/Modules/_ctypes/cfield.o -Wall -fexceptions gcc -pthread -fPIC -fno-strict-aliasing -Wsign-compare -g -O0 -Wall -Wstrict-prototypes -Ibuild/temp.freebsd-6.4-RELEASE-i386-3.5-pydebug/libffi/include -Ibuild/temp.freebsd-6.4-RELEASE-i386-3.5-pydebug/libffi -I/usr/home/db3l/buildarea/3.x.bolen-freebsd/build/Modules/_ctypes/libffi/src -I./Include -I. -IInclude -I/usr/local/include -I/usr/home/db3l/buildarea/3.x.bolen-freebsd/build/Include -I/usr/home/db3l/buildarea/3.x.bolen-freebsd/build -c /usr/home/db3l/buildarea/3.x.bolen-freebsd/build/Modules/_ctypes/libffi/src/prep_cif.c -o build/temp.freebsd-6.4-RELEASE-i386-3.5-pydebug/usr/home/db3l/buildarea/3.x.bolen-freebsd/build/Modules/_ctypes/libffi/src/prep_cif.o -Wall -fexceptions gcc -pthread -fPIC -fno-strict-aliasing -Wsign-compare -g -O0 -Wall -Wstrict-prototypes -Ibuild/temp.freebsd-6.4-RELEASE-i386-3.5-pydebug/libffi/include -Ibuild/temp.freebsd-6.4-RELEASE-i386-3.5-pydebug/libffi -I/usr/home/db3l/buildarea/3.x.bolen-freebsd/build/Modules/_ctypes/libffi/src -I./Include -I. -IInclude -I/usr/local/include -I/usr/home/db3l/buildarea/3.x.bolen-freebsd/build/Include -I/usr/home/db3l/buildarea/3.x.bolen-freebsd/build -c /usr/home/db3l/buildarea/3.x.bolen-freebsd/build/Modules/_ctypes/libffi/src/closures.c -o build/temp.freebsd-6.4-RELEASE-i386-3.5-pydebug/usr/home/db3l/buildarea/3.x.bolen-freebsd/build/Modules/_ctypes/libffi/src/closures.o -Wall -fexceptions gcc -pthread -fPIC -fno-strict-aliasing -Wsign-compare -g -O0 -Wall -Wstrict-prototypes -Ibuild/temp.freebsd-6.4-RELEASE-i386-3.5-pydebug/libffi/include -Ibuild/temp.freebsd-6.4-RELEASE-i386-3.5-pydebug/libffi -I/usr/home/db3l/buildarea/3.x.bolen-freebsd/build/Modules/_ctypes/libffi/src -I./Include -I. -IInclude -I/usr/local/include -I/usr/home/db3l/buildarea/3.x.bolen-freebsd/build/Include -I/usr/home/db3l/buildarea/3.x.bolen-freebsd/build -c /usr/home/db3l/buildarea/3.x.bolen-freebsd/build/Modules/_ctypes/libffi/src/x86/ffi.c -o build/temp.freebsd-6.4-RELEASE-i386-3.5-pydebug/usr/home/db3l/buildarea/3.x.bolen-freebsd/build/Modules/_ctypes/libffi/src/x86/ffi.o -Wall -fexceptions gcc -pthread -fPIC -fno-strict-aliasing -Wsign-compare -g -O0 -Wall -Wstrict-prototypes -Ibuild/temp.freebsd-6.4-RELEASE-i386-3.5-pydebug/libffi/include -Ibuild/temp.freebsd-6.4-RELEASE-i386-3.5-pydebug/libffi -I/usr/home/db3l/buildarea/3.x.bolen-freebsd/build/Modules/_ctypes/libffi/src -I./Include -I. -IInclude -I/usr/local/include -I/usr/home/db3l/buildarea/3.x.bolen-freebsd/build/Include -I/usr/home/db3l/buildarea/3.x.bolen-freebsd/build -c /usr/home/db3l/buildarea/3.x.bolen-freebsd/build/Modules/_ctypes/libffi/src/x86/freebsd.S -o build/temp.freebsd-6.4-RELEASE-i386-3.5-pydebug/usr/home/db3l/buildarea/3.x.bolen-freebsd/build/Modules/_ctypes/libffi/src/x86/freebsd.o -Wall -fexceptions gcc -pthread -shared build/temp.freebsd-6.4-RELEASE-i386-3.5-pydebug/usr/home/db3l/buildarea/3.x.bolen-freebsd/build/Modules/_ctypes/_ctypes.o build/temp.freebsd-6.4-RELEASE-i386-3.5-pydebug/usr/home/db3l/buildarea/3.x.bolen-freebsd/build/Modules/_ctypes/callbacks.o build/temp.freebsd-6.4-RELEASE-i386-3.5-pydebug/usr/home/db3l/buildarea/3.x.bolen-freebsd/build/Modules/_ctypes/callproc.o build/temp.freebsd-6.4-RELEASE-i386-3.5-pydebug/usr/home/db3l/buildarea/3.x.bolen-freebsd/build/Modules/_ctypes/stgdict.o build/temp.freebsd-6.4-RELEASE-i386-3.5-pydebug/usr/home/db3l/buildarea/3.x.bolen-freebsd/build/Modules/_ctypes/cfield.o build/temp.freebsd-6.4-RELEASE-i386-3.5-pydebug/usr/home/db3l/buildarea/3.x.bolen-freebsd/build/Modules/_ctypes/libffi/src/prep_cif.o build/temp.freebsd-6.4-RELEASE-i386-3.5-pydebug/usr/home/db3l/buildarea/3.x.bolen-freebsd/build/Modules/_ctypes/libffi/src/closures.o build/temp.freebsd-6.4-RELEASE-i386-3.5-pydebug/usr/home/db3l/buildarea/3.x.bolen-freebsd/build/Modules/_ctypes/libffi/src/x86/ffi.o build/temp.freebsd-6.4-RELEASE-i386-3.5-pydebug/usr/home/db3l/buildarea/3.x.bolen-freebsd/build/Modules/_ctypes/libffi/src/x86/freebsd.o -L/usr/local/lib -o build/lib.freebsd-6.4-RELEASE-i386-3.5-pydebug/_ctypes.so *** WARNING: renaming "_ctypes" since importing it failed: build/lib.freebsd-6.4-RELEASE-i386-3.5-pydebug/_ctypes.so: Undefined symbol "ffi_call_win32" --- |
|||
| msg227883 - (view) | Author: STINNER Victor (vstinner) * (Python committer) | Date: 2014年09月30日 12:04 | |
There is a similar error on OpenIndiana buildbot, Python cannot be compiled since this build: http://buildbot.python.org/all/builders/AMD64%20OpenIndiana%203.x/builds/8664/steps/compile/logs/stdio gcc -fPIC -Wsign-compare -g -O0 -Wall -Wstrict-prototypes -I/usr/local/include/ncursesw -m64 -Werror=declaration-after-statement -Ibuild/temp.solaris-2.11-i86pc.64bit-3.5-pydebug/libffi/include -Ibuild/temp.solaris-2.11-i86pc.64bit-3.5-pydebug/libffi -I/export/home/buildbot/64bits/3.x.cea-indiana-amd64/build/Modules/_ctypes/libffi/src -I./Include -I. -IInclude -I/usr/local/include -I/export/home/buildbot/64bits/3.x.cea-indiana-amd64/build/Include -I/export/home/buildbot/64bits/3.x.cea-indiana-amd64/build -c /export/home/buildbot/64bits/3.x.cea-indiana-amd64/build/Modules/_ctypes/libffi/src/x86/sysv.S -o build/temp.solaris-2.11-i86pc.64bit-3.5-pydebug/export/home/buildbot/64bits/3.x.cea-indiana-amd64/build/Modules/_ctypes/libffi/src/x86/sysv.o -Wall -fexceptions gcc -fPIC -Wsign-compare -g -O0 -Wall -Wstrict-prototypes -I/usr/local/include/ncursesw -m64 -Werror=declaration-after-statement -Ibuild/temp.solaris-2.11-i86pc.64bit-3.5-pydebug/libffi/include -Ibuild/temp.solaris-2.11-i86pc.64bit-3.5-pydebug/libffi -I/export/home/buildbot/64bits/3.x.cea-indiana-amd64/build/Modules/_ctypes/libffi/src -I./Include -I. -IInclude -I/usr/local/include -I/export/home/buildbot/64bits/3.x.cea-indiana-amd64/build/Include -I/export/home/buildbot/64bits/3.x.cea-indiana-amd64/build -c /export/home/buildbot/64bits/3.x.cea-indiana-amd64/build/Modules/_ctypes/libffi/src/x86/win32.S -o build/temp.solaris-2.11-i86pc.64bit-3.5-pydebug/export/home/buildbot/64bits/3.x.cea-indiana-amd64/build/Modules/_ctypes/libffi/src/x86/win32.o -Wall -fexceptions /var/tmp//ccqBa4n2.s: Assembler messages: /var/tmp//ccqBa4n2.s:8: Error: suffix or operands invalid for `push' /var/tmp//ccqBa4n2.s:17: Error: suffix or operands invalid for `push' /var/tmp//ccqBa4n2.s:18: Error: suffix or operands invalid for `push' /var/tmp//ccqBa4n2.s:79: Error: suffix or operands invalid for `jmp' /var/tmp//ccqBa4n2.s:138: Error: suffix or operands invalid for `pop' /var/tmp//ccqBa4n2.s:147: Error: suffix or operands invalid for `push' /var/tmp//ccqBa4n2.s:155: Error: suffix or operands invalid for `push' /var/tmp//ccqBa4n2.s:156: Error: suffix or operands invalid for `push' /var/tmp//ccqBa4n2.s:163: Error: suffix or operands invalid for `push' /var/tmp//ccqBa4n2.s:176: Error: suffix or operands invalid for `pop' /var/tmp//ccqBa4n2.s:210: Error: suffix or operands invalid for `jmp' /var/tmp//ccqBa4n2.s:251: Error: suffix or operands invalid for `pop' /var/tmp//ccqBa4n2.s:261: Error: suffix or operands invalid for `pop' /var/tmp//ccqBa4n2.s:268: Error: suffix or operands invalid for `push' /var/tmp//ccqBa4n2.s:272: Error: suffix or operands invalid for `push' /var/tmp//ccqBa4n2.s:315: Error: suffix or operands invalid for `jmp' /var/tmp//ccqBa4n2.s:358: Error: suffix or operands invalid for `pop' /var/tmp//ccqBa4n2.s:359: Error: suffix or operands invalid for `pop' /var/tmp//ccqBa4n2.s:370: Error: suffix or operands invalid for `push' /var/tmp//ccqBa4n2.s:383: Error: suffix or operands invalid for `pop' /var/tmp//ccqBa4n2.s:416: Error: suffix or operands invalid for `jmp' /var/tmp//ccqBa4n2.s:459: Error: suffix or operands invalid for `pop' /var/tmp//ccqBa4n2.s:460: Error: suffix or operands invalid for `pop' /var/tmp//ccqBa4n2.s:461: Error: suffix or operands invalid for `pop' /var/tmp//ccqBa4n2.s:471: Error: suffix or operands invalid for `jmp' *** WARNING: renaming "_curses" since importing it failed: ld.so.1: python: fatal: relocation error: file build/lib.solaris-2.11-i86pc.64bit-3.5-pydebug/_curses.so: symbol newscr: referenced symbol not found *** WARNING: renaming "_curses_panel" since importing it failed: No module named '_curses' error: [Errno 2] No such file or directory: 'build/temp.solaris-2.11-i86pc.64bit-3.5-pydebug/export/home/buildbot/64bits/3.x.cea-indiana-amd64/build/Modules/_decimal/_decimal.o' make: *** [sharedmods] Error 1 |
|||
| msg227885 - (view) | Author: Antoine Pitrou (pitrou) * (Python committer) | Date: 2014年09月30日 12:21 | |
Thanks for noticing this. Looking at an old build, the ctypes and curses compile errors under OpenIndiana are not new: http://buildbot.python.org/all/builders/AMD64%20OpenIndiana%203.x/builds/8658/steps/compile/logs/stdio Simply, it seems that a ctypes compile error is now interpreted as a failure of the compilation build step. I'll try to fix that. (the problem looks similar on the FreeBSD 6.4 buildbot) |
|||
| msg227887 - (view) | Author: STINNER Victor (vstinner) * (Python committer) | Date: 2014年09月30日 12:24 | |
> the ctypes and curses compile errors under OpenIndiana are not new FYI the compilation error of curses on OpenIndiana is *old*, at least 3 years old :-( http://bugs.python.org/issue13552 I just opened the issue #22521 for the compilation error of ctypes on FreeBSD. |
|||
| msg227888 - (view) | Author: Jonas Wagner (Sjlver) * | Date: 2014年09月30日 12:25 | |
I don't understand where this error comes from... The compilation commands are exactly the same in both the "before" and "after" logs. The order of commands is also the same. The only difference is this message: *** WARNING: renaming "_ssl" since importing it failed: build/lib.freebsd-6.4-RELEASE-i386-3.5-pydebug/_ssl.so: Undefined symbol "X509_check_ca" This message probably occurs because _ssl.so is now built in parallel with _ctypes. In the logs, one can see that the command to build _ctypes.o is executed; because of that, I wonder why the file is missing. Maybe we could check for wrong dependency tracking in the _ctypes makefile (similar to http://bugs.python.org/issue22359) |
|||
| msg227902 - (view) | Author: Roundup Robot (python-dev) (Python triager) | Date: 2014年09月30日 12:58 | |
New changeset 94af1af93670 by Antoine Pitrou in branch 'default': Remove pointless "vile hack" that can cause the build step to fail when some extension modules can't be imported. https://hg.python.org/cpython/rev/94af1af93670 |
|||
| msg227906 - (view) | Author: Antoine Pitrou (pitrou) * (Python committer) | Date: 2014年09月30日 13:01 | |
The change I just pushed should fix the failures, waiting for the buildbots to compile now. |
|||
| msg227907 - (view) | Author: Antoine Pitrou (pitrou) * (Python committer) | Date: 2014年09月30日 13:04 | |
Looks ok on OpenIndiana, closing now. |
|||
| msg234147 - (view) | Author: Julian Taylor (jtaylor) | Date: 2015年01月16日 19:59 | |
very nice, thanks for adding this. coincidentally numpy added the same to numpy.distutils independently just a week later, though numpy also accepts an environment variable to set the number of jobs. This is useful for e.g. pip installations where one does not control the command line. Also an environment variable allows parallel jobs in environments where it is not guaranteed that the feature is available. E.g. you could just put it into your .bashrc and when building with 3.5 it will just work and 2.7 will not fail. Is the naming --parallel/j already fixed? I'll change the numpy options to the same name then. Please also add it to the release notes so the feature can be discovered easier. |
|||
| msg234514 - (view) | Author: Berker Peksag (berker.peksag) * (Python committer) | Date: 2015年01月22日 21:52 | |
Here's a doc patch. |
|||
| msg234803 - (view) | Author: Roundup Robot (python-dev) (Python triager) | Date: 2015年01月27日 00:59 | |
New changeset 107669985805 by Berker Peksag in branch 'default': Add whatsnew entry for issue #5309. https://hg.python.org/cpython/rev/107669985805 |
|||
| msg240505 - (view) | Author: Roundup Robot (python-dev) (Python triager) | Date: 2015年04月11日 22:14 | |
New changeset fc28c67fbbd3 by doko in branch 'default': - Modules/Setup.dist: remove time extension duplicate, introduced by the fix for #5309. https://hg.python.org/cpython/rev/fc28c67fbbd3 |
|||
| msg276819 - (view) | Author: Giampaolo Rodola' (giampaolo.rodola) * (Python committer) | Date: 2016年09月17日 19:05 | |
I have 2 complaints about this: 1 - doc is missing: the only way to be aware of this is either by reading the 3.6 what's new doc or by checking the cmdline helper 2 - -j "N" parameter could be optional: if not specified os.cpu_count() can be used. |
|||
| History | |||
|---|---|---|---|
| Date | User | Action | Args |
| 2022年04月11日 14:56:45 | admin | set | github: 49559 |
| 2016年09月17日 19:05:32 | giampaolo.rodola | set | nosy:
+ giampaolo.rodola messages: + msg276819 |
| 2015年04月11日 22:14:07 | python-dev | set | messages: + msg240505 |
| 2015年01月27日 00:59:01 | python-dev | set | messages: + msg234803 |
| 2015年01月22日 21:52:57 | berker.peksag | set | files:
+ issue5309-doc.diff nosy: + berker.peksag messages: + msg234514 |
| 2015年01月16日 19:59:14 | jtaylor | set | nosy:
+ jtaylor messages: + msg234147 |
| 2014年09月30日 13:04:24 | pitrou | set | status: open -> closed resolution: fixed messages: + msg227907 |
| 2014年09月30日 13:01:04 | pitrou | set | messages: + msg227906 |
| 2014年09月30日 12:58:50 | python-dev | set | messages: + msg227902 |
| 2014年09月30日 12:25:06 | Sjlver | set | messages: + msg227888 |
| 2014年09月30日 12:24:40 | vstinner | set | messages: + msg227887 |
| 2014年09月30日 12:21:48 | pitrou | set | messages: + msg227885 |
| 2014年09月30日 12:04:45 | vstinner | set | messages: + msg227883 |
| 2014年09月30日 10:57:16 | vstinner | set | status: closed -> open nosy: + vstinner messages: + msg227877 resolution: fixed -> (no value) |
| 2014年09月26日 21:33:47 | pitrou | set | status: open -> closed resolution: fixed messages: + msg227652 stage: patch review -> resolved |
| 2014年09月26日 21:32:17 | python-dev | set | nosy:
+ python-dev messages: + msg227651 |
| 2014年09月20日 22:26:09 | pitrou | set | files:
+ build_ext_parallel6.patch messages: + msg227188 |
| 2014年09月09日 16:40:16 | Sjlver | set | messages: + msg226646 |
| 2014年09月07日 21:23:42 | eric.araujo | set | messages: + msg226546 |
| 2014年09月06日 23:37:08 | Arfrever | set | messages: + msg226523 |
| 2014年09月06日 23:23:45 | pitrou | set | stage: needs patch -> patch review |
| 2014年09月06日 23:16:48 | pitrou | set | files:
+ build_ext_parallel5.patch messages: + msg226521 |
| 2014年09月05日 13:59:52 | Sjlver | set | messages: + msg226424 |
| 2014年09月05日 09:54:23 | scoder | set | nosy:
+ scoder messages: + msg226415 versions: + Python 3.5, - Python 3.4 |
| 2014年09月05日 09:48:25 | Sjlver | set | files:
+ build_ext_parallel4.patch messages: + msg226412 |
| 2014年09月05日 09:25:55 | Sjlver | set | nosy:
+ Sjlver |
| 2013年10月26日 22:50:54 | christian.heimes | set | files:
+ build_ext_parallel3.patch messages: + msg201400 |
| 2013年10月26日 21:09:23 | pitrou | set | files:
+ build_ext_parallel2.patch messages: + msg201396 |
| 2013年10月26日 20:08:06 | pitrou | set | files:
+ build_ext_parallel.patch keywords: + patch messages: + msg201392 |
| 2013年10月25日 22:34:37 | Arfrever | set | nosy:
+ Arfrever title: setup.py doesn't parallelize extension module compilation -> distutils doesn't parallelize extension module compilation |
| 2013年01月04日 10:01:17 | christian.heimes | set | nosy:
+ christian.heimes messages: + msg179010 |
| 2013年01月03日 22:27:08 | pitrou | set | versions:
+ Python 3.4, - 3rd party title: packaging doesn't parallelize extension module compilation -> setup.py doesn't parallelize extension module compilation messages: + msg178993 assignee: tarek -> components: + Build, - Distutils2 |
| 2013年01月03日 21:24:53 | bfroehle | set | nosy:
+ bfroehle |
| 2011年04月08日 15:11:56 | eric.araujo | set | title: setup.py doesn't parallelize extension module compilation -> packaging doesn't parallelize extension module compilation |
| 2011年03月21日 22:26:30 | eric.araujo | set | keywords:
+ gsoc nosy: pitrou, tarek, eric.araujo |
| 2010年09月29日 23:51:32 | eric.araujo | set | versions: + 3rd party, - Python 2.6, Python 2.5, Python 3.1, Python 2.7, Python 3.2 |
| 2010年09月16日 20:24:33 | eric.araujo | set | stage: needs patch messages: + msg116603 versions: + Python 2.6, Python 2.5, Python 3.1, Python 2.7 |
| 2010年09月16日 16:18:40 | BreamoreBoy | set | nosy:
+ eric.araujo components: + Distutils2, - Distutils versions: + Python 3.2, - Python 3.1, Python 2.7 |
| 2009年02月19日 01:39:51 | pitrou | create | |