homepage

This issue tracker has been migrated to GitHub , and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: Implement atomic operations on non-x86 platforms
Type: enhancement Stage:
Components: Interpreter Core Versions: Python 3.4, Python 3.5
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: Arfrever, John.Malmberg, Vitor.de.Lima, gustavotemple, koobs, lbianc, neologix, python-dev, vstinner
Priority: normal Keywords: patch

Created on 2014年07月22日 17:24 by Vitor.de.Lima, last changed 2022年04月11日 14:58 by admin.

Files
File name Uploaded Description Edit
atomic.patch Vitor.de.Lima, 2014年07月22日 17:24 A patch implemeting atomic operating using the GCC builtins review
atomicv2.patch Vitor.de.Lima, 2014年07月29日 16:46 A patch implemeting atomic operating using the GCC builtins and the stdatomic.h header review
atomicv3.patch gustavotemple, 2014年12月18日 20:14 review
atomicv4.patch gustavotemple, 2015年01月13日 18:51 review
atomicv5.patch gustavotemple, 2015年01月14日 11:27 review
atomic_pointer.patch vstinner, 2015年01月14日 15:06
Messages (25)
msg223676 - (view) Author: Vitor de Lima (Vitor.de.Lima) Date: 2014年07月22日 17:24
The atomic operations listed in the pyatomic.h header file were implemented only for the x86 architecture, this patch uses the atomic bultins available in GCC >= 4.7 to implement such operations, allowing it to work properly in other platforms.
msg223740 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2014年07月23日 13:56
+ #define __ATOMIC_RELAXED 0
You should use the "_Py_" prefix for these constants, to avoid conflicts in applications.
(You may also replace tabs with spaces, the PEP 7 says "Use 4-space indents and no tabs at all." but I also prefer to avoid tabs in other places.)
I tested your patch on Fedora 20 (Linux kernel 3.14.8, GCC 4.8.2, glibc 2.18) on x86_64 ("Intel(R) Core(TM) i7-2600 CPU @ 3.40GHz") and the whole Python test suite pass.
GCC 4.9 (released a few month ago) provides the <stdatomic.h> header:
https://gcc.gnu.org/gcc-4.9/changes.html
pyatomic.h contains this comment:
/* XXX: When compilers start offering a stdatomic.h with lock-free
 atomic_int and atomic_address types, include that here and rewrite
 the atomic operations in terms of it. */
But using <stdatomic.h> header can be done in a separated issue.
msg224235 - (view) Author: Vitor de Lima (Vitor.de.Lima) Date: 2014年07月29日 16:46
Implemented a new version of the patch using either gcc builtins or the stdatomic.h header (this is detected by the configure script).
msg232813 - (view) Author: Gustavo Frederico Temple Pedrosa (gustavotemple) * Date: 2014年12月17日 12:22
ping
msg232836 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2014年12月17日 23:32
Fedora 20 provides GCC 4.7 but no stdatomic.c: HAVE_BUILTIN_ATOMIC set (HAVE_STD_ATOMIC unset).
Fedora 21 provides GCC 4.9 with stdatomic.c: HAVE_BUILTIN_ATOMIC and HAVE_STD_ATOMIC are set.
I tested atomicv2.patch on Fedora 20 (x86_64) and Fedora 21 (x86_64): the whole test suite pass.
msg232839 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2014年12月17日 23:53
atomicv2.patch:
> _Atomic int _value;
Why not using the atomic_int type from stdatomic.h here?
> https://gcc.gnu.org/onlinedocs/gcc/_005f_005fatomic-Builtins.html
"__atomic_store_n(): The valid memory model variants are __ATOMIC_RELAXED, __ATOMIC_SEQ_CST, and __ATOMIC_RELEASE."
I understand that _Py_atomic_store_explicit() only accept some values for order. An assertion should be added here, maybe for any implementation. Something like:
#define _Py_atomic_store_explicit(ATOMIC_VAL, NEW_VAL, ORDER) \
 (assert((ORDER) == __ATOMIC_RELAXED \
 || (ORDER) == __ATOMIC_SEQ_CST \
 || (ORDER) == __ATOMIC_RELEASE), \
 __atomic_store_n(&(ATOMIC_VAL)->_value, NEW_VAL, ORDER))
Same remark for _Py_atomic_load_explicit():
"__atomic_load_n(): The valid memory model variants are __ATOMIC_RELAXED, __ATOMIC_SEQ_CST, __ATOMIC_ACQUIRE, and __ATOMIC_CONSUME."
msg232901 - (view) Author: Gustavo Frederico Temple Pedrosa (gustavotemple) * Date: 2014年12月18日 19:47
@haypo, done: atomicv3.patch
msg233706 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2015年01月09日 01:14
New changeset fbe87fb071a6 by Victor Stinner in branch 'default':
Issue #22038: pyatomic.h now uses stdatomic.h or GCC built-in functions for
https://hg.python.org/cpython/rev/fbe87fb071a6 
msg233707 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2015年01月09日 01:16
atomicv3.patch is wrong for GCC builtin atomic operations: the parenthesis is not closed. I fixed this typo in the commit.
Vitor & Gustavo: Thanks for the patch, it's now applied to Python 3.5.
I tested it on Fedora 21 (x86_64). I disabled manually HAVE_STD_ATOMIC in pyconfig.h to test the two new options (stdatomic header & GCC builtins).
msg233745 - (view) Author: Gustavo Frederico Temple Pedrosa (gustavotemple) * Date: 2015年01月09日 11:44
Thank you, Victor!
msg233942 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2015年01月13日 15:19
I reopen the issue because Python cannot be compiled anymore on the Builder AMD64 FreeBSD 10.0 3.x:
http://buildbot.python.org/all/builders/AMD64%20FreeBSD%2010.0%203.x/builds/2947/steps/configure/logs/stdio
checking for stdatomic.h... yes
checking for GCC >= 4.7 __atomic builtins... yes
http://buildbot.python.org/all/builders/AMD64%20FreeBSD%2010.0%203.x/builds/2947/steps/compile/logs/stdio
--- Parser/tokenizer_pgen.o ---
In file included from Parser/tokenizer_pgen.c:2:
In file included from Parser/tokenizer.c:4:
In file included from Include/Python.h:53:
Include/pyatomic.h:37:5: error: _Atomic cannot be applied to incomplete type 'void' 
 _Atomic void *_value;
 ^
msg233944 - (view) Author: Gustavo Frederico Temple Pedrosa (gustavotemple) * Date: 2015年01月13日 15:27
@haypo, OK, I will investigate the problem.
msg233955 - (view) Author: Gustavo Frederico Temple Pedrosa (gustavotemple) * Date: 2015年01月13日 18:53
@haypo, done: atomicv4.patch
msg233958 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2015年01月13日 19:27
Can you please generate a patch for the default branch of Python?
msg233959 - (view) Author: Arfrever Frehtes Taifersar Arahesis (Arfrever) * (Python triager) Date: 2015年01月13日 19:28
Gustavo Temple: A patch against newest revision of default branch would be more useful.
msg233960 - (view) Author: Gustavo Frederico Temple Pedrosa (gustavotemple) * Date: 2015年01月13日 19:30
OK, I will do.
msg233991 - (view) Author: Kubilay Kocak (koobs) (Python triager) Date: 2015年01月14日 01:29
FreeBSD buildbots broken since fbe87fb071a67cb5e638b3496362b5aedc0fc9a7 
msg233993 - (view) Author: Kubilay Kocak (koobs) (Python triager) Date: 2015年01月14日 01:30
Oops, incomplete comment, apologies. Just noticed haypo has reported the issue here already
msg234023 - (view) Author: Gustavo Frederico Temple Pedrosa (gustavotemple) * Date: 2015年01月14日 12:28
@haypo, @Arfrever, done: atomicv5.patch
msg234029 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2015年01月14日 15:03
New changeset dacc944641b1 by Victor Stinner in branch 'default':
Issue #22038, configure: HAVE_STD_ATOMIC now also check that "atomic_int" and
https://hg.python.org/cpython/rev/dacc944641b1 
msg234030 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2015年01月14日 15:06
I commited atomicv5.patch because it's simple, but I'm not sure that we are using stdatomic.h "correctly". The current code looks to be written for GCC, Clang fails to compile it (FreeBSD 10 now uses Clang instead of GCC).
Maybe the "_Atomic void*" type is wrong, and we should use the "atomic_uintptr_t" type instead and cast to the expected type? Attached atomic_pointer.patch implements this idea. It works on FreeBSD 10 with Clang and on Fedora 21 with GCC 4.9, both have stdatomic.h.
msg234031 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2015年01月14日 15:14
At least, the latest change repaired FreeBSD 10 compilation.
msg234037 - (view) Author: Gustavo Frederico Temple Pedrosa (gustavotemple) * Date: 2015年01月14日 16:43
@haypo, I checked and your idea and implementation are very good, thank you very much.
Yes, there is a Clang-specific implementation of the stdatomic.h header [1].
The Musl libc for example created a stdatomic.h header with full compatibility [2].
[1] http://llvm.org/viewvc/llvm-project?view=revision&revision=218957
[2] http://www.openwall.com/lists/musl/2014/11/09/2 
msg237945 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2015年03月12日 15:05
New changeset eb48295e1f8b by Victor Stinner in branch 'default':
Issue #23644, #22038: Move #include <stdatomic.c> inside the extern "C" { ... }
https://hg.python.org/cpython/rev/eb48295e1f8b 
msg238348 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2015年03月17日 21:59
<stdatomic.h> is not compatible with C++: I disabled completly pyatomic.h on C++. pyatomic.h is only needed by Python core, not to compile Python extensions, so it's not an issue.
History
Date User Action Args
2022年04月11日 14:58:06adminsetgithub: 66237
2015年03月17日 21:59:03vstinnersetmessages: + msg238348
2015年03月12日 16:00:02lbiancsetnosy: + lbianc
2015年03月12日 15:05:34python-devsetmessages: + msg237945
2015年01月14日 16:43:46gustavotemplesetmessages: + msg234037
2015年01月14日 15:14:48vstinnersetmessages: + msg234031
2015年01月14日 15:06:22vstinnersetfiles: + atomic_pointer.patch

messages: + msg234030
2015年01月14日 15:03:15python-devsetmessages: + msg234029
2015年01月14日 12:28:53gustavotemplesetmessages: + msg234023
2015年01月14日 11:27:00gustavotemplesetfiles: + atomicv5.patch
2015年01月14日 01:34:26jyasskinsetnosy: - jyasskin
2015年01月14日 01:30:44koobssetmessages: + msg233993
2015年01月14日 01:29:45koobssetnosy: + koobs
messages: + msg233991
2015年01月13日 19:30:53gustavotemplesetmessages: + msg233960
2015年01月13日 19:28:00Arfreversetmessages: + msg233959
2015年01月13日 19:27:42vstinnersetmessages: + msg233958
2015年01月13日 18:53:21gustavotemplesetmessages: + msg233955
2015年01月13日 18:51:09gustavotemplesetfiles: + atomicv4.patch
2015年01月13日 18:37:25gustavotemplesetfiles: - atomicv4.patch
2015年01月13日 18:33:52gustavotemplesetfiles: + atomicv4.patch
2015年01月13日 15:27:48gustavotemplesetmessages: + msg233944
2015年01月13日 15:19:51vstinnersetstatus: closed -> open
resolution: fixed ->
messages: + msg233942
2015年01月09日 11:44:11gustavotemplesetmessages: + msg233745
2015年01月09日 01:16:08vstinnersetstatus: open -> closed
resolution: fixed
messages: + msg233707
2015年01月09日 01:14:32python-devsetnosy: + python-dev
messages: + msg233706
2014年12月19日 00:04:55Arfreversetnosy: + Arfrever
2014年12月18日 20:14:03gustavotemplesetfiles: + atomicv3.patch
2014年12月18日 20:13:53gustavotemplesetfiles: - atomicv3.patch
2014年12月18日 20:05:46gustavotemplesetfiles: + atomicv3.patch
2014年12月18日 20:05:07gustavotemplesetfiles: - atomicv3.patch
2014年12月18日 20:00:46gustavotemplesetfiles: + atomicv3.patch
2014年12月18日 20:00:14gustavotemplesetfiles: - atomicv3.patch
2014年12月18日 19:47:58gustavotemplesetmessages: + msg232901
2014年12月18日 19:47:16gustavotemplesetfiles: + atomicv3.patch
2014年12月17日 23:53:27vstinnersetmessages: + msg232839
2014年12月17日 23:32:15vstinnersetmessages: + msg232836
2014年12月17日 12:22:53gustavotemplesetnosy: + gustavotemple
messages: + msg232813
2014年09月02日 04:09:32John.Malmbergsetnosy: + John.Malmberg
2014年07月29日 16:46:07Vitor.de.Limasetfiles: + atomicv2.patch

messages: + msg224235
2014年07月23日 13:56:07vstinnersetnosy: + vstinner
messages: + msg223740
2014年07月22日 17:32:38pitrousetnosy: + jyasskin, neologix
2014年07月22日 17:24:12Vitor.de.Limacreate

AltStyle によって変換されたページ (->オリジナル) /