homepage

If you are fortunate enough to have all your third-party libraries in
a single quasi-standard location, say, /usr/local/lib, you will
probably have never encountered this problem, but setting environment
variables like LDFLAGS don't get translated into the relevant args for
the distutils build_ext command. The evidence for this is this output
at the end of the make process:
 Failed to find the necessary bits to build these modules:
 _hashlib _sqlite3 _ssl
 bsddb185 linuxaudiodev ossaudiodev
 To find the necessary bits, look in setup.py in detect_modules() for
the module's name.
 Failed to build these modules:
 _curses _curses_panel _tkinter
 gdbm readline
Here's the specific failure for readline:
 >>> import readline_failed
 Traceback (most recent call last):
 File "<stdin>", line 1, in <module>
	ImportError: ld.so.1: python: fatal: libreadline.so.4: open failed: No
such file or directory
yet if you poke around in my config.status file you'll see that I set
the relevant -L and -R flags:
 s,@LDFLAGS@,|#_!!_#|-L/opt/app/nonc++/ncurses-5.6/lib
-R/opt/app/nonc++/ncurses-5.6/lib -L/opt/app/nonc++/gdbm-1.8/lib
-R/opt/app/nonc++/gdbm-1.8/lib -L/opt/app/nonc++/readline-4.3/lib
-R/opt/app/nonc++/readline-4.3/lib -L/opt/app/nonc++/tcl-8.4/lib
-R/opt/app/nonc++/tcl-8.4/lib -L/opt/app/nonc++/BerkleyDB-4.3/lib
-R/opt/app/nonc++/BerkleyDB-4.3/lib,g
The workaround is to run the build_ext command separately:
 ./python ../setup.py build_ext --library-dirs=... --rpath=...
It's not hard to work around this problem, but it's tedious to build
the command line args from the massive list of directories I have to
educate distutils about.

Well, all of my modules are in a non-standard location and I have no
build issues on OS X 10.5. If you look at PyBuildExt.detect_modules()
you will see that the paths are at least added for the search path from
LDFLAGS and CPPFLAGS.

Brett> Well, all of my modules are in a non-standard location and I have
 Brett> no build issues on OS X 10.5. If you look at
 Brett> PyBuildExt.detect_modules() you will see that the paths are at
 Brett> least added for the search path from LDFLAGS and CPPFLAGS.
Ah, there's the problem. It picks through LDFLAGS looking for -L but fails
to pay attention to -R. I'll see if I can work up a patch.
Also, I'm seeing this problem on Solaris. I suppose it's possible that
linkage on OS X records the location of the libraries it finds, sort of
implicitly doing a -R.
Skip

Here's a patch. Works for me on Solaris 10. I'll try to check
it out on OS X 10.5. Would appreciate it if someone on Linux can
kick the tires too.

Confirmed that nothing seems broken on my OS X 10.5 laptop (doing a
unix-style build, not a framework build).

Seems to work for framework builds as well.

One of the problems that I see in that LDFLAGS is Makefile variable and
Makefile is part of distribution for posix build systems.
If you set specific LDFLAGS and you want to distribute own python build
user will get you specific settings.
One another environment variable is OPT (if compiler don't accept -R XXX
may be -Wl,-rpath,XXX ?) and we may use it. Also OPT environment
variable is also Makefile variable and user specific settings will go
into distribution too :( .
I quick look into distutils code show that sysconfig.py don't parse
LDFLAGS and append environment variable LDFLAGS as is to the ldshared
variable.
So the first question is why setup.py parse only -L flags instead to
pass variable value as is (aka sysconfig.py).
The second question is why LDFLAGS is a Makefile variable ? 
For now I would like to know if you set on Solaris suitable
LD_RUN_PATH(for the build) and LD_LIBRARY_PATH (when run) did you
succeed to build and run python ?

On Thu, Oct 2, 2008 at 1:57 PM, Roumen Petrov <report@bugs.python.org> wrote:
>
> Roumen Petrov <bugtrack@roumenpetrov.info> added the comment:
>
> One of the problems that I see in that LDFLAGS is Makefile variable and
> Makefile is part of distribution for posix build systems.
> If you set specific LDFLAGS and you want to distribute own python build
> user will get you specific settings.
> One another environment variable is OPT (if compiler don't accept -R XXX
> may be -Wl,-rpath,XXX ?) and we may use it. Also OPT environment
> variable is also Makefile variable and user specific settings will go
> into distribution too :( .
>
> I quick look into distutils code show that sysconfig.py don't parse
> LDFLAGS and append environment variable LDFLAGS as is to the ldshared
> variable.
> So the first question is why setup.py parse only -L flags instead to
> pass variable value as is (aka sysconfig.py).
Because that's the use case I was supporting when I added the support;
I just needed the -L flags for compiling the extensions. If someone
writes a patch that makes it more generic it has a chance of being
checked in.
> The second question is why LDFLAGS is a Makefile variable ?
Because there is no guarantee that someone specified the envvar when
running setup.py, but if they were set for configure you know that
Python was built using those values.

Lets see method customize_compiler(from sysconfig.py) 
The method search variables 'CPP', 'LDFLAGS' and 'CPPFLAGS' only in
environment. The variable CPP is not a Makefile variable. 'LDFLAGS' and
'CPPFLAGS' are makefile variables(macros). Usually makefile will export
to commands a macro only if exists environment variable with same name.
So to pass to distutils LDFLAGS is enough to to run as example
"LDFLAGS=fake make".
Issue3678 is about LDFLAGS for python shared library and now command
start with $(LDSHARED) $(LDFLAGS) ...
If we modify Makefile to use "...LDFLAGS=$(LDFLAGS) ... setup.py"
customize_compiler will append LDFLAGS to LDSHARED.
So this issue is about to use LDFLAGS for python modules.
Another point is that configure script append LDFLAGS to LDSHARED only
on certain platforms (all BSD based) and ignore all others.
What is preferred patch:
- a simple patch that just export LDFLAGS for setup.py;
- a patch that export LDFLAGS for all python programs, i.e. in Makefile
to append to RUNSHARED LDFLAGS=@LDFLAGS@ ;
- a patch that append LDFLAGS to LDSHARED in configure for all platforms
and remove use of LDFLAGS in Makefile and setup.py ?

I checked in r66823 to add the -R flag extraction like Brett's -L
extraction. I'll leave the issue open while folks discuss Roumen's
proposal. I have no particular desire to delve deeper into 
distutils.

I don't have rights to derive a new issue.
May be a new new title about distutils and makefile integration is more
appropriate.

I'm having similar problem with distutils.
http://mail.python.org/pipermail/python-dev/2008-November/083670.html
Is there any reason customize_compiler 
- only get CPPFLAGS in env, not ones from sysconfig?
- doesn't get OPT from from sysconfig, but it's used when env has CFLAGS?
I think this is a bug.
CPPFLAGS and OPT both should be used when building extension modules.
Just imagine a system that has include files in a non-standard locations
and explicitly added -I compiler flags to CPPFLAGS to adjust it.
When building an extension modules, the system should requires
the same CPPFLAGS settings. Without it, it won't compile.
In my opinion, this is rather serious bug.
So it would be nice to fix this before releasing Python 3.0/2.6.1/2.5.3

Is there anyone knowing the historical reason for this?
In that case I will take this and submit a patch here soon.

May I propose a patch. The patch is only for linker flags(LDFLAGS). If
is accepted I will prepare another one for compiler flags.
The patch is not minimal. It is mostly cleanup patch.
For minimal patch - only change in Makefile.pre.in is enough.
Cleanup patch remove work-arounds for the issue.

correct patch uploaded

Attached patch changes distutils to pass CPPFLAGS to compiler.

s/get_config_vars/get_config_var/

Updated issue4010 to honor os.environ['CPPFLAGS'].

Hi. We encountered this issue on a Solaris 10 while building python 2.6.2.
There's a problem with reptrov's patch from 2009年01月04日. The LDFLAGS in the 
Makefile.pre.in patch need to be quoted also. Like so:
LDFLAGS='$(LDFLAGS)'
This allows for multiple flags. Other than that, it works for us.

Furthermore, there's another bug in setup.py When extensions are built 
and CPPFLAGS (or other arguments) has multiple arguments, the order of 
the arguments are reversed. This is the wrong behavior. The specified 
directories should be used in the given order.
For example, if CPPFLAGS="-I/one -I/two -I/three", then when an 
extention is build, the order of the compile would be:
cc -I/three -I/two -I/one ...
There's this bit that seems reponsible in setup.py:
 for directory in reversed(options.dirs):
 add_dir_to_list(dir_list, directory)

The reversed call was added by Brett in r60537. Brett, can you comment?

When I did the patch they were coming in reversed, so I reversed the reversal. Obviously something changed so now the manual reversal is not necessary anymore.

I don’t think there can be a unit test for that, so do we agree on just removing the reversed call?

On Wed, Nov 17, 2010 at 17:19, Éric Araujo <report@bugs.python.org> wrote:
>
> Éric Araujo <merwok@netwok.org> added the comment:
>
> I don’t think there can be a unit test for that, so do we agree on just removing the reversed call?
As long as the person removing it verifies it and keeps an eye on the
buildbots, sure.

Distutils is now deprecated (see PEP 632) and all tagged issues are being closed. From now until removal, only release blocking issues will be considered for distutils.
If this issue does not relate to distutils, please remove the component and reopen it. If you believe it still requires a fix, most likely the issue should be re-reported at https://github.com/pypa/setuptools