Elixir remote pubsub

Question

I'm learning Elixir and one of the things I would like to implement is a simple pubsub, where the publisher and consumer reside in different nodes, for now without the use of tooling like Redis.

What I am trying at the moment is do it without phoenix. I've looked at a few options, one of them was the new Registry, but it seams that it doesn't work remotely.

The other option I tried was gproc. Node.list() shows me my nodes, so the cluster seams to be ok, but :gproc.send({:p, :l, :event_manager}, {:message, "stuff"}) still only works within one node.

My question is this: is there any standard way of doing pubsub with Elixir I must have missed?

• Steve Pallen: I have not used :gproc so I can't comment what your missing. Have you looked at the various pubsub packages on hex.pm? Also phoenix_pubsub does not have any dependencies on phoenix, so my might be able to use it without the rest of phoenix.
• eduard: I have tried some, yes. About to try phoenix_pubsub and Pg2PubSub.
• Steve Pallen: Let us know how it goes
• eduard: I just did a quick and simple implementation based on :pg2. I basically start same group both on the publisher and consumer apps, on the consumer I join the group I would like consume in, then I just send a message from the producer to all nodes which have joined that group.

Answer 1

Have you looked at GenStage? It's built on top of GenServer, and it should work across nodes in a cluster.

There's an example of a producer to producer/consumer to consumer chain here, and you could modify that example to be a simple producer to consumer.

There are various dispatchers as well, like the GenStage.BroadcastDispatcher, if you need multiple consumers subscribed to the same types of messages.

Answer 2

Well, the simple answer is because

{:p, :l, :event_manager}

should be

{:p, :g, :event_manager}

The :l means local and :g means :global.

But, I would just suggest using phoenix_pubsub because it is extremely well written for a distributed pubsub system. gproc is a bit more complicated than just a pubsub system.

Also, phoenix_pubsub does something really clever by having only one process on each node handling their own local ets distribution table and those processes just talk to each other to achieve distribution.

• eduard: I missed the :g vs :l. Thanks for pointing that out. I will take another look at phoenix_pubsub