homepage

This issue tracker has been migrated to GitHub , and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: setup.py: do not add system header locations when cross compiling
Type: compile error Stage: patch review
Components: Cross-Build Versions: Python 3.7, Python 3.6
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: doko, hundeboll, python-dev, thomas-petazzoni, vstinner, yan12125, zach.ware
Priority: normal Keywords: patch

Created on 2014年01月09日 23:02 by thomas-petazzoni, last changed 2022年04月11日 14:57 by admin.

Files
File name Uploaded Description Edit
0001-setup.py-do-not-add-invalid-header-locations.patch thomas-petazzoni, 2014年01月09日 23:02
no-native-headers-libraries.patch xdegaye, 2016年12月10日 08:34 review
multiarch.patch xdegaye, 2016年12月14日 10:19
multiarch_2.patch xdegaye, 2016年12月14日 10:25 review
Messages (12)
msg207805 - (view) Author: Thomas Petazzoni (thomas-petazzoni) Date: 2014年01月09日 23:02
In the cross-compilation case, setup.py incorrectly adds /usr/include to self.compiler.include_dirs, and results in the following invalid compilation line:
/home/thomas/projets/buildroot/output/host/usr/bin/arm-none-linux-gnueabi-gcc
 -fPIC -Wno-unused-result -Werror=declaration-after-statement -DNDEBUG -g
 -O3 -Wall -Wstrict-prototypes -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE
 -D_FILE_OFFSET_BITS=64 -pipe -Os
 -I./Include -I/usr/include -I. -IInclude
 -I/home/thomas/projets/buildroot/output/host/usr/arm-buildroot-linux-gnueabi/sysroot/usr/include
 -I/home/thomas/projets/buildroot/output/build/python3-3.4.0b1/Include
 -I/home/thomas/projets/buildroot/output/build/python3-3.4.0b1
 -c /home/thomas/projets/buildroot/output/build/python3-3.4.0b1/Modules/_struct.c
 -o build/temp.linux-arm-3.4/home/thomas/projets/buildroot/output/build/python3-3.4.0b1/Modules/_struct.o
cc1: warning: include location "/usr/include" is unsafe for cross-compilation [-Wpoison-system-directories]
The -I/usr/include is wrong when cross compiling, so we disable adding INCLUDEDIR and LIBDIR from the host when cross compiling.
msg237380 - (view) Author: Mark Lawrence (BreamoreBoy) * Date: 2015年03月06日 20:55
Can we have a patch review on this issue please.
msg261435 - (view) Author: Martin Hundebøll (hundeboll) Date: 2016年03月09日 15:24
The patch looks good to me, and works in my setup.
msg282640 - (view) Author: (yan12125) * Date: 2016年12月07日 17:36
Got the same issue here on building Python for Android. See https://circleci.com/gh/yan12125/python3-android/11 for an example build log. /usr/include is erroneously included and leads to build failures:
building 'xxlimited' extension
/home/ubuntu/android-ndk-r13/toolchains/llvm/prebuilt/linux-x86_64/bin/clang -target aarch64-none-linux-android -gcc-toolchain /home/ubuntu/android-ndk-r13/toolchains/aarch64-linux-android-4.9/prebuilt/linux-x86_64 --sysroot=/home/ubuntu/android-ndk-r13/platforms/android-21/arch-arm64/usr -fPIC -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIE -fno-integrated-as -fPIE -fno-integrated-as -std=c99 -Wextra -Wno-unused-result -Wno-unused-parameter -Wno-missing-field-initializers -DPy_LIMITED_API=0x03050000 -I../Include -I/usr/include -IObjects -IPython -I. -I/home/ubuntu/python3-android/build/21-aarch64-linux-android-4.9/usr/include -I/home/ubuntu/python3-android/src/cpython/Include -I/home/ubuntu/python3-android/src/cpython/build-target -c /home/ubuntu/python3-android/src/cpython/Modules/xxlimited.c -o build/temp.linux-aarch64-3.7/home/ubuntu/python3-android/src/cpython/Modules/xxlimited.o
In file included from /home/ubuntu/python3-android/src/cpython/Modules/xxlimited.c:17:
In file included from ../Include/Python.h:11:
In file included from /usr/include/limits.h:25:
/usr/include/features.h:398:10: fatal error: 'gnu/stubs.h' file not found
#include <gnu/stubs.h>
 ^
1 error generated.
The patch looks reasonable and still applies to the default branch. With this patch the build is fine: https://circleci.com/gh/yan12125/python3-android/12 
msg282641 - (view) Author: (yan12125) * Date: 2016年12月07日 17:37
Modified this issue a bit and add some experts on the build system
msg282836 - (view) Author: Xavier de Gaye (xdegaye) * (Python triager) Date: 2016年12月10日 08:34
When cross compiling, setup.py must not look for headers in the directory for installing C header files and must not look for libraries in the directory for installing object code libraries of the _native_ interpreter which is being used to run setup.py.
Thomas patch updated to the tip of the default branch.
msg282837 - (view) Author: Matthias Klose (doko) * (Python committer) Date: 2016年12月10日 09:03
this assumption is wrong for the multiarch case.
msg282841 - (view) Author: (yan12125) * Date: 2016年12月10日 09:35
xdegaye's explanation is incorrect. The actual scene is much more complicated. First, let me explain why things may be broken. The wrong directory is included only if all of the following conditions are met:
1. PYTHON_FOR_BUILD is not installed with --prefix=/usr. A common example is those which are installed with pyenv. This condition is here simply because of `os.path.normpath(sys.base_prefix) != '/usr'`
2. The build script uses $DESTDIR convention to compile CPython
3. Header files in $DESTDIR$PREFIX/include are not equivalent to $PREFIX/include. Here "equivalent" means they are for the same set of architecture/kernel/... This is a common case when cross compiling, while a famous exception is the multiarch framework on Debian-based systems.
For example, if I use --prefix=/usr and DESTDIR=/tmp/python3-android/build, INCLUDEDIR will be /usr/include, which should not be included. On the other hand, if I use --prefix=/tmp/python3-android/build/usr and omit DESTDIR, INCLUDEDIR is correct /tmp/python3-android/build/usr/include. With $DESTDIR specified, the build script has the responsibility to specify $DESTDIR$PREFIX/include in CPPFLAGS, or CPython interpreter core won't even build, so skipping adding INCLUDEDIR in setup.py is safe. However, for the latter case (not using DESTDIR), the build script still needs to specify correct CPPFLAGS. I see it as a CPython limitation and worth documentation.
For Debian's multiarch framework, paths are already handled by add_multiarch_paths(). I'm not sure whether $PREFIX/include should still be included or not. (I don't use Debian-based systems)
msg282843 - (view) Author: Xavier de Gaye (xdegaye) * (Python triager) Date: 2016年12月10日 09:49
> this assumption is wrong for the multiarch case.
Please explain why it is wrong.
msg283182 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2016年12月14日 10:16
New changeset d1b400943483 by Xavier de Gaye in branch '3.6':
Issue #20211: Do not add the directory for installing C header files and
https://hg.python.org/cpython/rev/d1b400943483
New changeset fcc9f19fcc13 by Xavier de Gaye in branch 'default':
Issue #20211: Merge 3.6.
https://hg.python.org/cpython/rev/fcc9f19fcc13 
msg283183 - (view) Author: Xavier de Gaye (xdegaye) * (Python triager) Date: 2016年12月14日 10:19
>> this assumption is wrong for the multiarch case.
>
> Please explain why it is wrong.
Sadly no explanations have been given, we have to take Matthias word for it.
This new patch updates the add_multiarch_paths() method and includes LIBDIR and INCLUDEDIR in their corresponding search paths for the multiarch case.
msg283184 - (view) Author: Xavier de Gaye (xdegaye) * (Python triager) Date: 2016年12月14日 10:25
Same patch, but this one is properly set to be handled by Rietveld.
History
Date User Action Args
2022年04月11日 14:57:56adminsetgithub: 64410
2019年12月10日 08:12:27xdegayesetnosy: - xdegaye
2017年04月01日 05:47:06serhiy.storchakasetpull_requests: - pull_request867
2017年03月31日 16:36:11dstufftsetpull_requests: + pull_request867
2016年12月14日 10:30:02xdegayeunlinkissue26865 dependencies
2016年12月14日 10:25:38xdegayesetfiles: + multiarch_2.patch

messages: + msg283184
2016年12月14日 10:19:05xdegayesetfiles: + multiarch.patch
assignee: xdegaye ->
messages: + msg283183

stage: commit review -> patch review
2016年12月14日 10:16:42python-devsetnosy: + python-dev
messages: + msg283182
2016年12月10日 09:49:11xdegayesetmessages: + msg282843
2016年12月10日 09:35:00yan12125setmessages: + msg282841
2016年12月10日 09:03:02dokosetmessages: + msg282837
2016年12月10日 08:34:28xdegayesetfiles: + no-native-headers-libraries.patch
versions: - Python 3.5
messages: + msg282836

assignee: xdegaye
stage: commit review
2016年12月07日 20:30:50xdegayelinkissue26865 dependencies
2016年12月07日 17:40:09yan12125setnosy: + xdegaye
2016年12月07日 17:37:31yan12125setversions: + Python 3.5, Python 3.6, Python 3.7, - Python 3.4
nosy: + zach.ware
title: setup.py: do not add invalid header locations -> setup.py: do not add system header locations when cross compiling
messages: + msg282641

2016年12月07日 17:36:05yan12125setnosy: + yan12125
messages: + msg282640
2016年03月09日 22:07:15BreamoreBoysetnosy: - BreamoreBoy
2016年03月09日 15:24:27hundebollsetnosy: + hundeboll
messages: + msg261435
2016年03月09日 15:22:33berker.peksaglinkissue26443 superseder
2015年03月06日 20:55:09BreamoreBoysetnosy: + BreamoreBoy
messages: + msg237380
2014年02月17日 23:07:36vstinnersetnosy: + doko, vstinner
2014年01月09日 23:02:08thomas-petazzonicreate

AltStyle によって変換されたページ (->オリジナル) /