TODO
| README.md | brain dump for hubur/channeler(/vessel) interaction and resulting hubur design | |
Hubur
These are currently just some high-level notes; some also refer to the interaction with channeler
- The general approach of serializing interests (with embedded caprock capabilities)
is fine, but a per-vessel-extent approach is unlikely to scale.
- Therefore, interests can either request:
- The named extent(s) only; this is for on-demand synchronization of known missing parts
- A request for the named extent(s) and their descendants; this is for retrieving an entire resource.
- A subscription to the extent, which is the same as above, but keeps a note to push data to the requester.
- Either of the above, but instead of starting with the extent itself, start with its most recent descendant - this permits streaming.
- Adding a filter for specific topic(s):
- if the topic is readable by the server, it can e.g. lead to retrieving AAA info completely,
but content separately
- This should mean an interest should refer to the same starting extent, but then contain separate sub-interests such as e.g. for the complete AAA topic, but for the latest content only.
- if the topic is not readable by the server, a response should indicate failure to comply
- Due to this, the sub-interests need to be identified separately as well.
- if the topic is readable by the server, it can e.g. lead to retrieving AAA info completely,
but content separately
- Cancellation of any of the above.
- Because permissions embeded into the extent may require the sending of other extents, it's
helpful to identify a request; this should be under the sender's control. A resource server
then can send multiple responses citing this request identifier.
- Multiple responses per interest also permits:
- Multiple response servers responding to the same request -> FEC?
- Extents fragmented over multiple response packets
- Multiple responses per interest also permits:
- Therefore, interests can either request:
Channeler/routing related:
- FEC:
- is generally a concern for channeler, so from peer A to peer B
- BUT if an interest carries a "redundancy" factor, a router can
- a) request a subset of FEC indices from upstream,
- and b) request different subsets from different upstream peers -> this bundles the stream in the router
- routers as handling all traffic related to an interest may not be perfect, so an interest should
flag whether the interest has a routing context, or a search context so to speak. If search, then
upstream should send to the peer directly; if router, then to the router.
- This affects the FEC notion above, and must be considered
- Note that a "router" can also be a function of the peer's own node
- Let's say:
- router-as-a-function lives on the peer and in upstream components.
- FEC is more about how to handle upstream; this is not really related to the interest as such, but rather to the request to a specific upstream.
- FEC factor indicates a tradeoff between bandwidth and level of security. Peer should provide this
as part of the request to router-as-a-function.
- local router-as-a-function treats this as the number of peers (or make that explicit?) from which
to request partial content.
- channeler: configure both FEC factor and subset(s) thereof, e.g.:
- factor of 5:
- peer A range: 1
- peer B range: 2-3
- peer C range: 4-5
- factor, range apply to user content; re-requests to packets. With factor/range set and no resends configured, this can work somehow.
- factor of 5:
- channeler: configure both FEC factor and subset(s) thereof, e.g.:
- remote router-as-a-function needs to know this only if local router-as-a-function forwards such
info.
- local router-as-a-function can forward this as part of its routing strategy, e.g. if client wants N peers and local router-as-a-function knowns N/3 peers, it can contact each for 1/3rd of the packets. Remote router-as-a-function may forward all or a subset to its upstreams, etc.
- Interests should have TTL.
- Could avoid cyclic routes by forwarding envelope; this allows TTL to remain signed and all peers to know whether to ignore something or not.
- local router-as-a-function treats this as the number of peers (or make that explicit?) from which
to request partial content.
API:
- Hubur router (local or remote) gets:
- Interest
- Interest name should be some kind of prefixed thing, e.g. a vessel extent ID or some other
string.
- Using some other string permits transparently resolving to extent IDs, which does mean that
the router must sign a new interest and map to the old interest
- that mapping is verifiable by the client: if they asked for an extent, simple. If they asked for something else... something more complex like a signature of sorts?
- allows API access in principle; needs request parameters of sorts, which might be a CapTP mapping.
- Using some other string permits transparently resolving to extent IDs, which does mean that
the router must sign a new interest and map to the old interest
- Interest name should be some kind of prefixed thing, e.g. a vessel extent ID or some other
string.
- Some FEC parameters
- A TTL for the interest
- Interest
- Hubur router implements a routing strategy; start with a simple one.
- Look into an index of peers that may fit.
- Adjust FEC parameters for peer(s).
- This can request a subset of FEC from individual peers, or pick a single peer. -> local decision by router; make configurable/pluggable
- Add forwarding envelope (effectively decreasing TTL)
- Send peers the result
- Hubur router sends responses:
- Either with data
- Or with (partial) errors
- BUT can do both sending data and forwarding/routing interest, in case not everything is local
- Hubur router API then produces extents (or errors) in response to an interest identifier
- recoverable errors should be notices.