-
Notifications
You must be signed in to change notification settings - Fork 1.1k
Cluster Running with 2 nodes and having difficulty to add 3rd node in the cluster #5365
-
As per the below url ,noticed cluster is running with 2 node and we tried with adding 3rd nodes into the cluster via " curl -X PUT "http://xxx.xxx.xxx.xxx/_node/_local/_nodes/node2@yyy.yyy.yyy.yyy" -d {} " however it not working.
Moreover , we have noticed difference in " adminPassword,cookieAuthSecret,erlangCookie" for the 3rd node.Therefore we have manaually updated these values and after that all the nodes were restarted and 3rd node were joined into cluster .
However after some time,again noticed cluster membership is in degraded state . Any suggestion from here ?
version: 3.3.3
https://score-db.cec.net/_membership
{
"all_nodes": [
"couchdb@couchdb-couchdb-0.couchdb-couchdb.apache-couchdb.svc.cluster.local",
"couchdb@couchdb-couchdb-1.couchdb-couchdb.apache-couchdb.svc.cluster.local"
],
"cluster_nodes": [
"couchdb@couchdb-couchdb-0.couchdb-couchdb.apache-couchdb.svc.cluster.local",
"couchdb@couchdb-couchdb-1.couchdb-couchdb.apache-couchdb.svc.cluster.local",
"couchdb@couchdb-couchdb-2.couchdb-couchdb.apache-couchdb.svc.cluster.local"
]
}
https://score-db.cec.net/_cluster_setup
{
"state": "cluster_finished"
}
Logs :
[notice] 2024年12月16日T06:59:18.773373Z couchdb@couchdb-couchdb-2.couchdb-couchdb.apache-couchdb.svc.cluster.local <0.520.0> -------- mem3_reshard finished reloading jobs
[info] 2024年12月16日T06:59:18.796240Z couchdb@couchdb-couchdb-2.couchdb-couchdb.apache-couchdb.svc.cluster.local <0.527.0> -------- Apache CouchDB has started. Time to relax.
[info] 2024年12月16日T06:59:18.796317Z couchdb@couchdb-couchdb-2.couchdb-couchdb.apache-couchdb.svc.cluster.local <0.527.0> -------- Apache CouchDB has started on http://any:5984/
[notice] 2024年12月16日T06:59:18.863063Z couchdb@couchdb-couchdb-2.couchdb-couchdb.apache-couchdb.svc.cluster.local <0.619.0> -------- All system databases exist.
[warning] 2024年12月16日T06:59:18.866103Z couchdb@couchdb-couchdb-2.couchdb-couchdb.apache-couchdb.svc.cluster.local <0.617.0> -------- 6 shards in cluster with only 1 copy on nodes that are currently up
[warning] 2024年12月16日T06:59:18.868596Z couchdb@couchdb-couchdb-2.couchdb-couchdb.apache-couchdb.svc.cluster.local <0.617.0> -------- 6 shards in cluster with only 1 copy on nodes not in maintenance mode
[error] 2024年12月16日T06:59:19.760792Z couchdb@couchdb-couchdb-2.couchdb-couchdb.apache-couchdb.svc.cluster.local emulator -------- Error in process <0.688.0> on node 'couchdb@couchdb-couchdb-2.couchdb-couchdb.apache-couchdb.svc.cluster.local' with exit value:
{{rexi_DOWN,{'couchdb@couchdb-couchdb-0.couchdb-couchdb.apache-couchdb.svc.cluster.local',noconnect}},[{mem3_rpc,rexi_call,3,[{file,"src/mem3_rpc.erl"},{line,384}]},{mem3_seeds,'-start_replication/1-fun-0-',1,[{file,"src/mem3_seeds.erl"},{line,107}]}]}
[error] 2024年12月16日T06:59:19.760855Z couchdb@couchdb-couchdb-2.couchdb-couchdb.apache-couchdb.svc.cluster.local emulator -------- Error in process <0.688.0> on node 'couchdb@couchdb-couchdb-2.couchdb-couchdb.apache-couchdb.svc.cluster.local' with exit value:
{{rexi_DOWN,{'couchdb@couchdb-couchdb-0.couchdb-couchdb.apache-couchdb.svc.cluster.local',noconnect}},[{mem3_rpc,rexi_call,3,[{file,"src/mem3_rpc.erl"},{line,384}]},{mem3_seeds,'-start_replication/1-fun-0-',1,[{file,"src/mem3_seeds.erl"},{line,107}]}]}
[error] 2024年12月16日T06:59:20.761868Z couchdb@couchdb-couchdb-2.couchdb-couchdb.apache-couchdb.svc.cluster.local emulator -------- Error in process <0.692.0> on node 'couchdb@couchdb-couchdb-2.couchdb-couchdb.apache-couchdb.svc.cluster.local' with exit value:
{{rexi_DOWN,{'couchdb@couchdb-couchdb-1.couchdb-couchdb.apache-couchdb.svc.cluster.local',noconnect}},[{mem3_rpc,rexi_call,3,[{file,"src/mem3_rpc.erl"},{line,384}]},{mem3_seeds,'-start_replication/1-fun-0-',1,[{file,"src/mem3_seeds.erl"},{line,107}]}]}
Beta Was this translation helpful? Give feedback.
All reactions
Replies: 1 comment
-
can you double check that that new node is reachable via network from the others, that the required ports (see the docs) are all open and that the erlang cookie value is the same everywhere?
Beta Was this translation helpful? Give feedback.