homepage

Alright, the current behaviour is quite strange:
we don't call msync() when closing the object, we just unmap() it:
mmap_close_method(mmap_object *self, PyObject *unused)
{
[...]
#ifdef UNIX
 if (0 <= self->fd)
 (void) close(self->fd);
 self->fd = -1;
 if (self->data != NULL) {
 munmap(self->data, self->size);
 self->data = NULL;
 }
#endif
[...]
}
But we set self->data to NULL to avoid calling munmap() a second time when deallocating the object:
static void
mmap_object_dealloc(mmap_object *m_obj)
{
[ ... ]
#ifdef UNIX
 if (m_obj->fd >= 0)
 (void) close(m_obj->fd);
 if (m_obj->data!=NULL) {
 msync(m_obj->data, m_obj->size, MS_SYNC);
 munmap(m_obj->data, m_obj->size);
 }
#endif /* UNIX */
[ ...]
}
So, if the object has been closed properly before being deallocated, msync() is _not_ called.
But, if we don't close the object, then msync() is called.
The attached test script shows the _huge_ performance impact of msync:
when only close() is called (no msync()):
$ ./python /home/cf/test_mmap.py
0.35829615593
when both flush() and close() are called (msync() called):
$ ./python /home/cf/test_mmap.py
4.95999493599
when neither is called, relying on the deallocation (msync() called):
$ ./python /home/cf/test_mmap.py
4.8811671257
And a strace leaves no doubt (called 10 times in a loop) :
mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb80b1000 <0.000019>
write(1, "4.12167286873\n"..., 144.12167286873
) = 14 <0.000012>
close(3) = 0 <0.000010>
munmap(0xb80b2000, 4096) = 0 <0.000023>
rt_sigaction(SIGINT, {SIG_DFL}, {0x811d630, [], 0}, 8) = 0 <0.000011>
close(5) = 0 <0.004889>
msync(0xb69f9000, 10000000, MS_SYNC) = 0 <0.584054>
munmap(0xb69f9000, 10000000) = 0 <0.000433>
See how expensive msync() is, and this is just for a 10MB file.
So the attached patch (mmap_msync.diff) removes the call to msync from mmap_object_dealloc(). Since UnmapViewOfFile() is only called inside flush() method, nothing to remove for MS Windows.
Here's the result of the same test script with the patch:
when only close() is called (no msync()):
$ ./python /home/cf/test_mmap.py
0.370584011078
when both flush() and close() are called (msync() called):
$ ./python /home/cf/test_mmap.py
4.97467517853
when neither is called, relying on the deallocation (msync() not called):
$ ./python /home/cf/test_mmap.py
0.390102148056
So we only get msync() latency when the user explicitely calls flush().