ERIS over CoAP #8

emery commented 2022年09月12日 22:41:50 +02:00

Rendered view.

[Rendered view](https://codeberg.org/eris/eer/src/branch/coap/eer-coap/index.md).

emery commented 2022年09月12日 23:03:31 +02:00

I don't know how to express this normatively so I'll say it here. I don't want to specify any encryption here. If you need to keep secret the references of the ERIS blocks you are transporting over the wire than you have some serious problems and encryption would be a marginal improvement at best.

I don't know how to express this normatively so I'll say it here. I don't want to specify any encryption here. If you need to keep secret the references of the ERIS blocks you are transporting over the wire than you have some serious problems and encryption would be a marginal improvement at best.

emery commented 2022年09月18日 01:38:33 +02:00

Implemented for Python

[Implemented for Python](https://codeberg.org/eris/python-eris/src/commit/b4eb0d21ec6775b570fca71b177278acaf17ef26/eris/store.py#L181)

pukkamustard commented 2022年09月22日 11:24:53 +02:00

Some suggestions:

ERIS CoAP Store

I suggest defining something like a ERIS CoAP store that has a URL. This URL is then the base for paths/verbs defined.

E.g. A store URL could be coap://example.com/my-eris-store, coaps://store-host.com/8908a9b5-6ea6-4d28-88a6-a3a4669bcda0 or coap+ws://bli-blups.org/eris-coap-ws.

I think this is already implicitly assumed, but I think it's good to make it explicit.

This also makes definition of additional paths (in this EER or future EERs) easy. E.g.:

• filter path that returns a filter of all blocks in the store
• quota path that returns available quote for the specific store
• choke and interest subscription paths a la BitTorrent

This store URL is also something which could then be shared for peer discovery.

Block paths

Relative to the ERIS CoAP store URL, I would suggest moving the block paths to either blocks/ or blake2b/.

Block size

I'd suggest using the Size1 CoAP option (https://www.rfc-editor.org/rfc/rfc7252#section-5.10.9) in requests to get blocks and making it optional.

Some suggestions: ## ERIS CoAP Store I suggest defining something like a ERIS CoAP store that has a URL. This URL is then the base for paths/verbs defined. E.g. A store URL could be `coap://example.com/my-eris-store`, `coaps://store-host.com/8908a9b5-6ea6-4d28-88a6-a3a4669bcda0` or `coap+ws://bli-blups.org/eris-coap-ws`. I think this is already implicitly assumed, but I think it's good to make it explicit. This also makes definition of additional paths (in this EER or future EERs) easy. E.g.: - `filter` path that returns a filter of all blocks in the store - `quota` path that returns available quote for the specific store - `choke` and `interest` subscription paths a la BitTorrent This store URL is also something which could then be shared for peer discovery. ## Block paths Relative to the ERIS CoAP store URL, I would suggest moving the block paths to either `blocks/` or `blake2b/`. ## Block size I'd suggest using the `Size1` CoAP option (https://www.rfc-editor.org/rfc/rfc7252#section-5.10.9) in requests to get blocks and making it optional.

emery commented 2022年09月22日 17:18:12 +02:00

I suggest defining something like a ERIS CoAP store that has a URL. This URL is then the base for paths/verbs defined.
...
This also makes definition of additional paths (in this EER or future EERs) easy.

Yes, I would support namespacing the blocks into blake2b. We should probably be using the .well-known convention and service discovery to document and validate the protocol.

I'd suggest using the Size1 CoAP option in requests to get blocks and making it optional.

I saw this option and wasn't sure if using it in requests was appropriate, but looks like it should be fine.

> I suggest defining something like a ERIS CoAP store that has a URL. This URL is then the base for paths/verbs defined. > ... > This also makes definition of additional paths (in this EER or future EERs) easy. Yes, I would support namespacing the blocks into `blake2b`. We should probably be using the `.well-known` convention and [service discovery](https://www.rfc-editor.org/rfc/rfc7252#section-7) to document and validate the protocol. > I'd suggest using the Size1 CoAP option in requests to get blocks and making it optional. I saw this option and wasn't sure if using it in requests was appropriate, but looks like it should be fine.

emery commented 2022年09月23日 05:39:16 +02:00

How about constructing URLs with the block references as a URI-Query option and the block-size as Size1? - .../blocks?H77AGSYKAVTQPUHODJTQA7WZPTWGTTKLRB2GLMF5H53NEKFJ3FUQ

That seems more "Constrained RESTful". Now we can list the .../blocks path in /.well-known/coreand stores can be found with "resource discovery".

I like blocks better than blake2b (or blake2b256).

How about constructing URLs with the block references as a URI-Query option and the block-size as Size1? - `.../blocks?H77AGSYKAVTQPUHODJTQA7WZPTWGTTKLRB2GLMF5H53NEKFJ3FUQ` That seems more "Constrained RESTful". Now we can list the `.../blocks` path in `/.well-known/core`and stores can be found with "resource discovery". I like `blocks` better than `blake2b` (or `blake2b256`).

pukkamustard commented 2022年09月23日 08:47:09 +02:00

I saw this option and wasn't sure if using it in requests was appropriate, but looks like it should be fine.

Yeah, semantics of Size1 is not quite clear - well, confusing actually.

In RFC 7959: Block-Wise Transfers in the Constrained Application Protocol (CoAP) it says about Size1:

In a request carrying a Block1 Option, to indicate the current
estimate the client has of the total size of the resource
representation, measured in bytes ("size indication").

If you cut the Block1 thing out, it seems to match our usage. It's the estimated size of the requested block.

I think we're ok with using Size1.

> I saw this option and wasn't sure if using it in requests was appropriate, but looks like it should be fine. Yeah, semantics of `Size1` is not quite clear - well, confusing actually. In [RFC 7959: Block-Wise Transfers in the Constrained Application Protocol (CoAP)](https://datatracker.ietf.org/doc/html/rfc7959#section-4) it says about `Size1`: > In a request carrying a Block1 Option, to indicate the current > estimate the client has of the total size of the resource > representation, measured in bytes ("size indication"). If you cut the `Block1` thing out, it seems to match our usage. It's the estimated size of the requested block. I think we're ok with using `Size1`.

pukkamustard commented 2022年09月23日 09:15:53 +02:00

About Service Discovery: We're talking about service discovery as described in section 7 of the CoAP RFC (RFC 7262). Which defers to RFC 6690: Constrained RESTful Environments (CoRE) Link Format, where the .well-known/core enpdoint is defined.

So to discover a ERIS block store, a client first does a request to /.well-known/core:

REQ: GET /.well-known/core
RES: 2.05 Content
</eris/>;if="http://purl.org/eris/coap",
</blocks/sub-stores/12312312/>;if="http://purl.org/eris/coap"

This returns two block stores one at /eris and one at /block/sub-store/12331231/. http:/purl.org/eris/coap would link to the document that defines the ERIS CoAP block store.

Now, IMHO, it is ok if the endpoint to get a block is:

/eris/blocks/H77AGSYKAVTQPUHODJTQA7WZPTWGTTKLRB2GLMF5H53NEKFJ3FUQ/

or

/eris/blocks??H77AGSYKAVTQPUHODJTQA7WZPTWGTTKLRB2GLMF5H53NEKFJ3FUQ

I have a slight preference for the path vs query-option, but it's minor.

I think what you were suggesting is more like:

/eris/H77AGSYKAVTQPUHODJTQA7WZPTWGTTKLRB2GLMF5H53NEKFJ3FUQ/

or

/eris?H77AGSYKAVTQPUHODJTQA7WZPTWGTTKLRB2GLMF5H53NEKFJ3FUQ

I'm in favor of the top as it allows a bit more extensibility. We can add paths for block storage management and syncing, e.g.: /eris/filter or /eris/quota.

About Service Discovery: We're talking about service discovery as described in section 7 of the CoAP RFC ([RFC 7262](https://www.rfc-editor.org/rfc/rfc7252#section-7)). Which defers to [RFC 6690: Constrained RESTful Environments (CoRE) Link Format](https://www.rfc-editor.org/rfc/rfc6690#section-5), where the `.well-known/core` enpdoint is defined. So to discover a ERIS block store, a client first does a request to `/.well-known/core`: ``` REQ: GET /.well-known/core RES: 2.05 Content </eris/>;if="http://purl.org/eris/coap", </blocks/sub-stores/12312312/>;if="http://purl.org/eris/coap" ``` This returns two block stores one at `/eris` and one at `/block/sub-store/12331231/`. http:/purl.org/eris/coap would link to the document that defines the ERIS CoAP block store. Now, IMHO, it is ok if the endpoint to get a block is: `/eris/blocks/H77AGSYKAVTQPUHODJTQA7WZPTWGTTKLRB2GLMF5H53NEKFJ3FUQ/` or `/eris/blocks??H77AGSYKAVTQPUHODJTQA7WZPTWGTTKLRB2GLMF5H53NEKFJ3FUQ` I have a slight preference for the path vs query-option, but it's minor. I think what you were suggesting is more like: `/eris/H77AGSYKAVTQPUHODJTQA7WZPTWGTTKLRB2GLMF5H53NEKFJ3FUQ/` or `/eris?H77AGSYKAVTQPUHODJTQA7WZPTWGTTKLRB2GLMF5H53NEKFJ3FUQ` I'm in favor of the top as it allows a bit more extensibility. We can add paths for block storage management and syncing, e.g.: `/eris/filter` or `/eris/quota`.

emery commented 2022年09月23日 16:11:19 +02:00

I figured that with CoRE it wasn't possible to put a link in /.well-known/core and specify that there are more paths behind it (the blocks), but it would be proper to say there is an explicit path that can be queryied for blocks.

I want to be able to add /eris/filter and /eris/quota, what I mean is that there is no /eris/blocks/H77AGSYKAVTQPUHODJTQA7WZPTWGTTKLRB2GLMF5H53NEKFJ3FUQ rather /eris/blocks?H77AGSYKAVTQPUHODJTQA7WZPTWGTTKLRB2GLMF5H53NEKFJ3FUQ.

I figured that with CoRE it wasn't possible to put a link in `/.well-known/core` and specify that there are more paths behind it (the blocks), but it would be proper to say there is an explicit path that can be queryied for blocks. I want to be able to add `/eris/filter` and `/eris/quota`, what I mean is that there is no `/eris/blocks/H77AGSYKAVTQPUHODJTQA7WZPTWGTTKLRB2GLMF5H53NEKFJ3FUQ` rather `/eris/blocks?H77AGSYKAVTQPUHODJTQA7WZPTWGTTKLRB2GLMF5H53NEKFJ3FUQ`.

pukkamustard commented 2022年09月23日 19:42:22 +02:00

I figured that with CoRE it wasn't possible to put a link in /.well-known/core and specify that there are more paths behind it (the blocks), but it would be proper to say there is an explicit path that can be queryied for blocks.

Ah, I see. I assumed that it is possible to specify a top-path where resources are sub-paths. But I think you're right and it is more proper to indicate a complete path and every resource individually.

what I mean is that there is no /eris/blocks/H77AGSYKAVTQPUHODJTQA7WZPTWGTTKLRB2GLMF5H53NEKFJ3FUQ rather /eris/blocks?H77AGSYKAVTQPUHODJTQA7WZPTWGTTKLRB2GLMF5H53NEKFJ3FUQ.

Ok, makes sense now.

I want to be able to add /eris/filter and /eris/quota

These would be additional entries in the list returned by .well-known/core? If discovery even makes sense, probably no reason to make the qoota endpoint discoverable.

I might have had a slightly different peer-discover in mind. Use cases:

• Indicate the CoAP block store in a XMPP profile. This is to indicate that blocks of content published by this account can probably be dereferenced from this block store.
• In a Metalink file that specifies multiple URLs and a ERIS URN for a file (see #6).
• In an end-user application where a CoAP block store can be specified to get/put blocks.

In all cases it would be nice if a single URL suffices and the resources /blocks, /filter etc. can be assumed to be relative to the specified URL.

In the last case the URL might be a capability that allows a certain amount of blocks to be stored (e.g. coap+tcp://my-block-store.net/SECRET-THAT-ALLOWS-ME-TO-STORE-STUFF/).

For those use cases it would be nice to specify one "store URL".

> I figured that with CoRE it wasn't possible to put a link in `/.well-known/core` and specify that there are more paths behind it (the blocks), but it would be proper to say there is an explicit path that can be queryied for blocks. Ah, I see. I assumed that it is possible to specify a top-path where resources are sub-paths. But I think you're right and it is more proper to indicate a complete path and every resource individually. > what I mean is that there is no /eris/blocks/H77AGSYKAVTQPUHODJTQA7WZPTWGTTKLRB2GLMF5H53NEKFJ3FUQ rather /eris/blocks?H77AGSYKAVTQPUHODJTQA7WZPTWGTTKLRB2GLMF5H53NEKFJ3FUQ. Ok, makes sense now. > I want to be able to add /eris/filter and /eris/quota These would be additional entries in the list returned by `.well-known/core`? If discovery even makes sense, probably no reason to make the qoota endpoint discoverable. I might have had a slightly different peer-discover in mind. Use cases: - Indicate the CoAP block store in a XMPP profile. This is to indicate that blocks of content published by this account can probably be dereferenced from this block store. - In a Metalink file that specifies multiple URLs and a ERIS URN for a file (see #6). - In an end-user application where a CoAP block store can be specified to get/put blocks. In all cases it would be nice if a single URL suffices and the resources `/blocks`, `/filter` etc. can be assumed to be relative to the specified URL. In the last case the URL might be a capability that allows a certain amount of blocks to be stored (e.g. `coap+tcp://my-block-store.net/SECRET-THAT-ALLOWS-ME-TO-STORE-STUFF/`). For those use cases it would be nice to specify one "store URL".

emery commented 2022年09月24日 21:08:10 +02:00

In all cases it would be nice if a single URL suffices and the resources /blocks, /filter etc. can be assumed to be relative to the specified URL.

Yes, I think we should let the URL prefix be arbitrary for where /blocks are, if we understand each other correctly.

> In all cases it would be nice if a single URL suffices and the resources /blocks, /filter etc. can be assumed to be relative to the specified URL. Yes, I think we should let the URL prefix be arbitrary for where `/blocks` are, if we understand each other correctly.

pukkamustard commented 2022年09月26日 19:04:22 +02:00

In all cases it would be nice if a single URL suffices and the resources /blocks, /filter etc. can be assumed to be relative to the specified URL.

Yes, I think we should let the URL prefix be arbitrary for where /blocks are, if we understand each other correctly.

Yup, I think we agree on that.

What I would additionally like is to give the prefix a name. E.g.:

Store URL

The ERIS CoAP Store URL is an absolute URL. All resource endpoints defined for an ERIS CoAP store are defined relative to the store URL. For example if the store URL is coap+tls://discordia.net/stores/1312/ then the blocks endpoint is located at coap+tls://discordia.net/stores/1312/blocks.

A single host may host mutliple ERIS CoAP stores.

The ERIS CoAP Store URL is a convenient way to share the location of all store functionality and may be used in end-user applications or protocols to share locations of stores.

> > In all cases it would be nice if a single URL suffices and the resources /blocks, /filter etc. can be assumed to be relative to the specified URL. > > Yes, I think we should let the URL prefix be arbitrary for where `/blocks` are, if we understand each other correctly. Yup, I think we agree on that. What I would additionally like is to give the prefix a name. E.g.: > ## Store URL > > The ERIS CoAP Store URL is an absolute URL. All resource endpoints defined for an ERIS CoAP store are defined relative to the store URL. For example if the store URL is `coap+tls://discordia.net/stores/1312/` then the blocks endpoint is located at `coap+tls://discordia.net/stores/1312/blocks`. > > A single host may host mutliple ERIS CoAP stores. > > The ERIS CoAP Store URL is a convenient way to share the location of all store functionality and may be used in end-user applications or protocols to share locations of stores.

emery commented 2023年01月06日 00:42:32 +01:00

For the purposes of multicast requests we may want a common URL path for requesting blocks, so that a request for a block can be made directly at a multicast address rather than making a request to /.well-known/core and then parsing responses for other ERIS nodes. I need to read the specs again but I think we can have a dedicated multicast group (or something like that) to dispatch /blocks in, and that would essentially give us a dedicated namespace.

For the purposes of multicast requests we may want a common URL path for requesting blocks, so that a request for a block can be made directly at a multicast address rather than making a request to `/.well-known/core` and then parsing responses for other ERIS nodes. I need to read the specs again but I think we can have a dedicated multicast group (or something like that) to dispatch `/blocks` in, and that would essentially give us a dedicated namespace.

pukkamustard commented 2023年01月11日 10:34:05 +01:00

I've made some edits and pushed to a7cfe693c8. See diff and rendered view

Changes:

• Introduce a store URL concept. Resources (such as blocks) are defined relative to the store URL. Store URL can be capabilities that handle authorization and quota.
• Add a .well-known/eris store.
• Add examples to the GET and PUT requests on blocks.
• Add note on bi-directionality of CoAP
• Move security notes to a section "Security Considerations"

@emery: wdyt?

I've made some edits and pushed to a7cfe693c8b82fa447d3bf54a5f7d93b928fc52b. See [diff](https://codeberg.org/eris/eer/compare/coap...coap-pm-edits) and [rendered view](https://codeberg.org/eris/eer/src/commit/a7cfe693c8b82fa447d3bf54a5f7d93b928fc52b/eer/coap/index.md) Changes: - Introduce a *store URL* concept. Resources (such as `blocks`) are defined relative to the store URL. Store URL can be capabilities that handle authorization and quota. - Add a `.well-known/eris` store. - Add examples to the `GET` and `PUT` requests on blocks. - Add note on bi-directionality of CoAP - Move security notes to a section "Security Considerations" @emery: wdyt?

emery commented 2023年01月11日 18:22:51 +01:00

@pukkamustard Good by me.

@pukkamustard Good by me.

pukkamustard commented 2023年01月11日 19:18:01 +01:00

Merged with a7cfe693c8

Merged with a7cfe693c8b82fa447d3bf54a5f7d93b928fc52b