[Python-Dev] Re: radix tree arena map for obmalloc

2019年6月17日 02:11:37 -0700

Le 17/06/2019 à 10:55, Inada Naoki a écrit :
> On Mon, Jun 17, 2019 at 5:18 PM Antoine Pitrou <[email protected]> wrote:
>>
>> On 2019年6月17日 11:15:29 +0900
>> Inada Naoki <[email protected]> wrote:
>>>
>>> Increasing pool size is one obvious way to fix these problems.
>>> I think 16KiB pool size and 2MiB (huge page size of x86) arena size is
>>> a sweet spot for recent web servers (typically, about 32 threads, and
>>> 64GiB), but there is no evidence about it.
>>
>> Note that the OS won't give a huge page automatically, because memory
>> management becomes much more inflexible then.
> 
> When we used contiguous 2MB pages, Linux may use Huge Page
> implicitly when Transparent Huge Page is enabled.
But it's not enabled by default... And there are reasons for that (see
the manpage I quoted).
> In web applications, it's common to one Python worker process
> use 50~100+ MiB RSS. 2MB arena seems reasonable for those
> applications.
Perhaps, but what is the problem you are trying to solve? Do you have
evidence that memory management of those 50-100 MB is costly?
Perhaps those 50-100 MB are allocated at worker startup (module
initialization structures, docstrings...) and only get deallocated at
the end, so they aren't really a problem for arena allocation cost.
Regards
Antoine.
_______________________________________________
Python-Dev mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/[email protected]/message/Z2AO6AFBCYDJK7HM3NZ6SCTWXSUZ36TI/

Reply via email to