Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Sign up
Appearance settings

Use alternative Distribution Protocol #102

sleipnir started this conversation in Ideas
Discussion options

In order to achieve our goals of offering a way to reach the edges in a local-first approach, it is necessary that we offer an alternative to the old Erlang Distribution Protocol, because in environments that are really distributed in a fabric that deviates from the basic requirements of a datacenter or public private cloud if requires a protocol that is easily routable over the Internet or over unreliable connections.

I'm inclined to test quic_dist as an alternative to Erlang Distributed, but if anyone has another proposal feel free to present it.

For this to work, it is very important that the alternative protocol chosen is easily pluggable with Elixir and Libcluster.

You must be logged in to vote

Replies: 1 comment 2 replies

Comment options

sleipnir
Nov 20, 2022
Maintainer Author

I decided to turn the previous Issue into a discussion as I find it more appropriate to use this channel for this type of subject.
That said, I've found yet another good candidate for a distributed protocol and am experimenting with new ideas in the old eigr-labs/steinadler repository. It is about using the Nats broker as a basis for a distributed protocol. Nats has many interesting features mainly in the areas of security and topology and therefore I think it is a good candidate. However, an integration with it through Steinadler would result in something more invasive and would require more changes in the Spawn code base since Steinadler is a higher level abstraction than Erlang Distributed or quic_dist.
One of the benefits of using Steinadler via Nats is that in theory we would be able to support really large clusters without the cost of a mesh network where all nodes involved need to be directly connected to each other. Each Node would only need a single direct connection to the Nats cluster, which reduces the number of necessary connections and obviously increases the number of possible nodes to be connected. Depending on the implementation of Steinadler, it would also reduce the overhead in registering the processes and consequently improve the lookup of the processes.

You must be logged in to vote
2 replies
Comment options

Sounds like a single point of failure. Prefer hubs

Comment options

sleipnir Feb 23, 2023
Maintainer Author

Hi @heri16, glad to see someone moving around here.

Yes, there are tradeoffs, and all models will have their tradeoffs. You can operate on a geographically provisioned Nats cluster which would remove your single point of failure.

A fully connected network like the standard Erlang Dist is very convenient, but it comes with a very high cost in TCP connections and auto discovery is a very expensive operation for large clusters.

The idea of having more than one mode of operation is precisely to give freedom of choice to anyone using Spawn.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Category
Ideas
2 participants
Converted from issue

This discussion was converted from issue #90 on November 20, 2022 11:43.

AltStyle によって変換されたページ (->オリジナル) /