homepage

(related to issue #24915)
I discovered that `make profile-opt` does not use the profile information for the builtin-modules (e.g. arraymodule or _pickle) because in the `profile-opt` target there is the following sequence of actions:
 ...
 $(MAKE) build_all_merge_profile
 @echo "Rebuilding with profile guided optimizations:"
 $(MAKE) clean
 $(MAKE) build_all_use_profile
 ...
The action `$(MAKE) clean` performs an `rm -rf build`, destroying among other things all the *.gcda files generated for the built-in modules.
On my Linux system with gcc, a kludge to `Makefile.pre.in` that works is:
 ...
 @echo "Rebuilding with profile guided optimizations:"
 find build -name \*.gcda -print | cpio -o >_modules.gcda.cpio # XXX
 $(MAKE) clean
 cpio -id <_modules.gcda.cpio # XXX
 $(MAKE) build_all_use_profile
 ...
but, like I said, it's a kludge and it's POSIX-only (-print0 can be avoided since I believe it's guaranteed that there will be no whitespace-containing-filenames).
Now, if this road is to be taken, I believe the most cross-platform method available to save the profile-generated files is to use a custom python script (at this point, the local profile-generating python executable is functional) and make a tar file, which will be untared back. However, I don't like it much.
I'm willing to provide any necessary patches if someone gives me a "proper" roadmap as to how to handle this issue.

Thank you for noticing and signaling this issue. Since I proposed the PGO patches, I will fix it in another patch, as I don't want it to break any builds.

I've added a fix for the PGO builds after the issue pointed out by you. Can you please test it also?

I've added the patch here also.

Pinging here. Christos could you test the patch?

New changeset 7d9cd4a0d488 by Gregory P. Smith in branch '3.5':
Fixes issue26307: The profile-opt build now applys PGO to the built-in
https://hg.python.org/cpython/rev/7d9cd4a0d488
New changeset bdc7292cf87e by Gregory P. Smith in branch 'default':
Fixes issue26307: The profile-opt build now applys PGO to the built-in modules.
https://hg.python.org/cpython/rev/bdc7292cf87e 

New changeset 75dae0b2ccb3 by Gregory P. Smith in branch '2.7':
Fixes issue26307: The profile-opt build now applys PGO to the built-in modules.
https://hg.python.org/cpython/rev/75dae0b2ccb3 

My change means that the build/ directory tree exists but won't have any files other than *.gc?? files in it after a make clean. I doubt this will bother anyone. That felt better for clean than just avoiding the removal of build/ entirely.