Skip to main content
Stack Overflow
  1. About
  2. For Teams

Return to Question

Notice removed Authoritative reference needed by Community Bot
Bounty Ended with DavidW's answer chosen by Community Bot
Notice added Authoritative reference needed by pprados
Bounty Started worth 50 reputation by pprados
added 723 characters in body
Source Link
pprados
  • 1.1k
  • 13
  • 21

The python incref is define like this

#define Py_INCREF(op) ( \
 _Py_INC_REFTOTAL _Py_REF_DEBUG_COMMA \
 ((PyObject *)(op))->ob_refcnt++)

With multi-core, the incrementation is only is L1 cache and not flushed to memory. Is

If two thread increment the refcnt at the same time, in differents core, without a flush to the real memory, for me, it's possible to lost one incrementation.

  • ob_refcnt=1
  • Core 1 increment, but not flush => ob_refcnt=2 in L1 cache of core 1
  • Core 2 increment, but not flush => ob_refcnt=2 in L1 cache of core 2
  • WTF

Is it a risk to use multi-core or multi-process ?

The PyObject was declared like this:

typedef struct _object {
 _PyObject_HEAD_EXTRA
 Py_ssize_t ob_refcnt;
 struct _typeobject *ob_type;
} PyObject

But Py_ssize_t is just a ssize_t or intptr_t.

The _Py_atomic* functions and attributes do not seem to be used.

How Python can manage this scenario ? How can it flush the cache between threads ?

The python incref is define like this

#define Py_INCREF(op) ( \
 _Py_INC_REFTOTAL _Py_REF_DEBUG_COMMA \
 ((PyObject *)(op))->ob_refcnt++)

With multi-core, the incrementation is only is L1 cache and not flushed to memory. Is it a risk to use multi-core or multi-process ?

The python incref is define like this

#define Py_INCREF(op) ( \
 _Py_INC_REFTOTAL _Py_REF_DEBUG_COMMA \
 ((PyObject *)(op))->ob_refcnt++)

With multi-core, the incrementation is only is L1 cache and not flushed to memory.

If two thread increment the refcnt at the same time, in differents core, without a flush to the real memory, for me, it's possible to lost one incrementation.

  • ob_refcnt=1
  • Core 1 increment, but not flush => ob_refcnt=2 in L1 cache of core 1
  • Core 2 increment, but not flush => ob_refcnt=2 in L1 cache of core 2
  • WTF

Is it a risk to use multi-core or multi-process ?

The PyObject was declared like this:

typedef struct _object {
 _PyObject_HEAD_EXTRA
 Py_ssize_t ob_refcnt;
 struct _typeobject *ob_type;
} PyObject

But Py_ssize_t is just a ssize_t or intptr_t.

The _Py_atomic* functions and attributes do not seem to be used.

How Python can manage this scenario ? How can it flush the cache between threads ?

Source Link
pprados
  • 1.1k
  • 13
  • 21

Python and multithreading

The python incref is define like this

#define Py_INCREF(op) ( \
 _Py_INC_REFTOTAL _Py_REF_DEBUG_COMMA \
 ((PyObject *)(op))->ob_refcnt++)

With multi-core, the incrementation is only is L1 cache and not flushed to memory. Is it a risk to use multi-core or multi-process ?

lang-py

AltStyle によって変換されたページ (->オリジナル) /