Re: [mildly OT] Some info about Python
[
Date Prev][
Date Next][
Thread Prev][
Thread Next]
[
Date Index]
[
Thread Index]
- Subject: Re: [mildly OT] Some info about Python
- From: Andre Leiradella <andre@...>
- Date: 2020年1月17日 14:24:19 +0000
On 16/01/20 13:30, Stefan wrote:
Am 16.01.2020 um 13:03 schrieb Andre Leiradella:
That was my use case. In my implementation, I didn't allocate new
memory when the block was shrinking, in this case I just updated the
block header with the new size.
I've also added dlmalloc [1] on top of the linear allocator, dlmalloc
would request new memory blocks from the linear allocator whenever it
couldn't satisfy an allocation request. You still have a hard limit
on the maximum memory for each state, but with 64-bit OSes you can in
theory have 4 billion+ 4 GiB linear allocators which should be enough
for everyone (TM). Both the Lua and the dlmalloc states were reset
with memcpy at the end of script execution, so it wasn't necessary to
event close the state to free up memory.
That sounds interesting! Do you want to share the sources?
Unfortunately I lost the source code during a PC migration. It wasn't
particularly difficult to implement, but I remember having to tweak
dlmalloc a bit to make it request more memory from its backing linear
allocator instead of using sbrk or mmap.
Btw. the 4 billion allocators won't work in practice. The paging
hardware of current processors seems to only support 48 bits
instead of the full 64 bit, so in fact you are limited to
2 ^ 48 / ( 4 * 1024 ^ 3 ) = 65536 states with 4GiB each.
Although this is still a lot, it can be exhausted.
Aren't those 48 bits the upper bits of the virtual address, allowing
2^48 pages of 64 KiB each? I think that's the case with AArch64.
Greetings
Cheers,
Andre