-
Notifications
You must be signed in to change notification settings - Fork 4
significantly reduce copies #31
Open
Description
There are several places in the code where copies are made just for ease of implementation. Eventually we will want to minimize these. Can we make a more sharing-friendly structure that minimizes copies, and supports Buf semantics? Rc<Box<Vec>> backed?
A write request involves at least these copies:
- sk_buff
- 1024 byte MutByteBuf
- Codec buffer
- deframed buffer
- parse into SetReq protobuf
- create Mutation protobuf
- for each peer, each Mutation is cloned and that value is placed into a new batch protobuf (is that one copy or two?)
- for each peer, the batch protobuf is converted to bytes
- for each peer, the bytes are framed using the Codec
- for each peer, the bytes are added to the outbound Vec
- for each peer, incrementally send bytes into sk_buff's for sockets to them
- for each peer, essentially repeat this process to parse and formulate a simple response
- for the leader, essentially repeat steps 1-5
So, for a cluster of 5, the write amplification is in the high dozens before we even talk about RocksDB. We are at least two orders of magnitude from being optimal.
Questions:
- can we implement a Codec that receives bytes directly from an sk_buff to skip basic mio loop and leader election #2 above?
- can we have a format that uses field offsets in a header to unsafely dereference interesting parts of requests?
- can we wrap this in a structure that is copyable without copying the underlying buffer? This would allow the structure to be copied all over the place without actually copying the data, just the read offsets for Buf. inner: Rc<Box<?>>
Activity
Metadata
Metadata
Assignees
Labels
No labels