bigchaindb集群部署(5节点)

在安装bigchainDB集群之前,我们需要了解bigchainDB的一些相关术语:

• bigchainDB节点:是一台或一群相近的节点的集合。bigchainDB节点上运行了mongoDB server、bigchainDB server及一些相关的软件
• bigchainDB集群:一群互联的bigchainDB节点组成了bigchainDB集群。集群中的节点运行相同的软件。一个集群可能需要有其他的额外的节点来完成集群监控
• bigchainDB联盟(Consortium):一个集群中管理节点的组织或个人属于bigchainDB联盟。当集群由一个公司管理运行,则联盟为该公司

bigchainDB的节点按照使用目的可以分为dev/test节点(开发与测试)、bare-bones节点(部署在云上)或者production节点(拥有更多的组件)。一个production节点必须包括BigchainDB Server、MongoDB Server 3.4+ (mongod)以及Scalable storage for MongoDB。这些关键组件的关系如下图所示。其中BigChainDB server必须能够与所有mongod实例相连,所有的mongod也相互连接。

这里写图片描述

安装环境

其中node-admin用来使用ansible来执行远程命令(为了部署便利),node1-node5为bigchain DB节点。所有机器用户均为root

为了使用ansible,我们首先在所有节点上安装openssh-server,并允许root用户ssh。具体方式为修改/etc/ssh/sshd_config,修改为PermitRootLogin yes,然后重启ssh服务。

安装与配置ansible

接下来在node-admin上安装与配置ansible。

apt-get update && apt-get upgrade
apt-get install ansible

修改/etc/ansible/hosts:

root@clean:~# grep -Ev "^$|#" /etc/ansible/hosts
[bigchain]
10.0.0.71 ansible_ssh_pass=123
10.0.0.72 ansible_ssh_pass=123
10.0.0.73 ansible_ssh_pass=123
10.0.0.74 ansible_ssh_pass=123
10.0.0.75 ansible_ssh_pass=123
root@clean:~#

设置为首次链接不需要key认证

sed -i "s/^#\(host_key_checking\).*/1円 = False/g" /etc/ansible/ansible.cfg

测试:ansible bigchain -m ping

安装bigchainDB

以下所有命令默认在node-admin上执行

• 同步时钟

# 测试是否能ping通cn.pool.ntp.org
ansible bigchain -m command -a "ping cn.pool.ntp.org -c 4"
# 同步
ansible bigchain -m command -a "ntpdate cn.pool.ntp.org"

• 安装mongoDB

编辑sources.list

echo "deb [ arch=amd64 ] http://repo.mongodb.org/apt/ubuntu trusty/mongodb-org/3.4 multiverse" > mongodb-org-3.4.list

备用 清华的源

echo "deb [ arch=amd64 ] https://mirrors.tuna.tsinghua.edu.cn/mongodb/apt/ubuntu trusty/mongodb-org/3.4 multiverse" > mongodb-org-3.4.list

编辑mongod.yml,内容如下:

---

- hosts: bigchain
 remote_user: root
 # invoke setup module to gather facts before executing tasks
 gather_facts: true
 tasks:
 - name: debug
 debug: msg="myhostname={{ansible_hostname}}"
 - name: apt-key
 command: sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv 0C49F3730359A14518585931BC711F9BA15703C6
 - name: sources.list
 template: 
 src: mongodb-org-3.4.list
 dest: /etc/apt/sources.list.d/mongodb-org-3.4.list
 owner: root
 group: root
 mode: 0644
 - name: update
 command: apt-get update -y
 - name: install packages
 apt:
 name: "{{item}}"
 force: yes
 with_items:
 - mongodb-org
 - g++ 
 - python3-dev
 - libffi-dev
 - python3-pip

 - name: setuptools
 command: pip3 install --upgrade pip setuptools
 - name: db
 command: mkdir -p /data/db

运行yml(耗时很长,因为需要完成apt-get update与安装许多包)

ansible-playbook mongod.yml

• 启动mongod

# 启动mongo
ansible bigchain -m command -a "mongod --replSet bigchain --fork --logpath /var/log/mongodb/mongod.log"

若出现address已经被绑定这种错误的话,可以在错误节点上执行

kill -9 $(ps -ef | grep mongod | awk 'NR>1{print p}{p=2ドル}')
# kill全部
ansible bigchain -m raw -a "kill -9 \$(ps -ef | grep mongod | awk 'NR>1{print p}{p=\2ドル}')"

若mongod出现错误:

2017年06月21日T09:56:34.552+0800 W - [initandlisten] Detected unclean shutdown - /var/lib/mongodb/mongod.lock is not empty.
2017-06-21T09:56:34.568+0800 E NETWORK [initandlisten] Failed to unlink socket file /tmp/mongodb-27017.sock Operation not permitted
2017-06-21T09:56:34.568+0800 I - [initandlisten] Fatal Assertion 28578 at src/mongo/util/net/listen.cpp 195

可以在错误节点上执行:

rm /data/db/mongod.lock 
mongod --repair
ansible bigchain -m raw -a "rm /data/db/mongd.lock"
ansible bigchain -m raw -a "mongod --repair"

• 创建副本集

在node1上执行

mongo
 config = {_id: 'bigchain', members: [{
 "_id": 0,
 "host":"10.0.0.71:27017"
 }]
 }
 rs.initiate(config);
 rs.add("10.0.0.72:27017")
 rs.add("10.0.0.73:27017")
 rs.add("10.0.0.74:27017")
 rs.add("10.0.0.75:27017")

• 安装bigchainDB

ansible bigchain -m command -a "pip3 install --upgrade bigchaindb"
ansible bigchain -m command -a "bigchaindb -y configure mongodb"

• 修改bigchainDB配置

允许接收所有地址信息

ansible bigchain -m raw -a "sed -i 's/\(\"bind\": \"\)localhost:9984\"/10円.0.0.0:9984\"/g' /root/.bigchaindb"

修改replSet名称为bigchain(之前mongod的replset名称):

ansible bigchain -m raw -a "sed -i 's/\(\"replicaset\": \"\).*/1円bigchain\",/g' /root/.bigchaindb"

还需要修改keyring选项,使之存储除本节点之外的所有其他的节点的keyring的公钥。先来获取所有节点的keyring的公钥

ansible bigchain -m raw -a "cat .bigchaindb | grep public | awk -F\\\" '{print \$4}'"

输出如下:

10.0.0.71 | success | rc=0 >>
7772APkwHENC8j3tDaUK2WJYPF3AMrTkVgR7sW1y3bkZ
10.0.0.72 | success | rc=0 >>
GRTkTmFuYETDaXAftSZW1SdCMMwaYs6p6yhAn5C4QBZv
10.0.0.74 | success | rc=0 >>
Eok1FnDbKpak9t6SpJVpFsMqkvNiVGsys6BP8UbSiCTv
10.0.0.73 | success | rc=0 >>
8bXEbEJVCDNhptYyAJ5WWHCngiie6VuwTKF5NmZ4Fazv
10.0.0.75 | success | rc=0 >>
GH3uAPwi1MzXsxy4PJdj4p5m55nXuLAakNtpFNJw7cqH

我们将这些信息写入一个脚本(conf.py)里

import sys
import json
keyring = {
 "10.0.0.71": "7772APkwHENC8j3tDaUK2WJYPF3AMrTkVgR7sW1y3bkZ",
 "10.0.0.72": "GRTkTmFuYETDaXAftSZW1SdCMMwaYs6p6yhAn5C4QBZv",
 "10.0.0.74": "Eok1FnDbKpak9t6SpJVpFsMqkvNiVGsys6BP8UbSiCTv",
 "10.0.0.73": "8bXEbEJVCDNhptYyAJ5WWHCngiie6VuwTKF5NmZ4Fazv",
 "10.0.0.75": "GH3uAPwi1MzXsxy4PJdj4p5m55nXuLAakNtpFNJw7cqH"
}
rets = []
for key, value in keyring.items():
 if key != sys.argv[1]:
 rets.append(value)
conf = json.load(open("/root/.bigchaindb"))
conf['keyring'] = rets
json.dump(conf, open("/root/.bigchaindb", "w"), indent=2)

编辑bigchain.yml,用来分发该脚本,并执行该脚本

---

- hosts: bigchain
 remote_user: root
 # invoke setup module to gather facts before executing tasks
 gather_facts: true
 tasks:
 - name: debug
 debug: msg="my ip of eth0 is {{ansible_eth0.ipv4.address}}"
 - name: copy file
 template:
 src: conf.py
 dest: /root/conf.py
 owner: root
 group: root
 mode: 0644
 - name: modify configuration
 command: python conf.py {{ansible_eth0.ipv4.address}}

执行:

ansible-playbook bigchain.yml

执行成功后,可以看到node1-node5的.bigchaindb里keyring均成功写入。

• 启动bigchainDB

任选一节点(如node1上)执行

bigchaindb init

然后启动所有节点的bigchaindb

bigchaindb start
# 或者后台启动
nohup bigchaindb start > /dev/null 2>&1 &
# 启动全部
ansible bigchain -m shell -a "nohup bigchaindb start > /dev/null 2>&1 &"
# kill全部
ansible bigchain -m raw -a "kill -9 \$(ps -ef | grep bigchaindb | awk 'NR>1{print p}{p=\2ドル}')"

使用bigchaindb

首先得安装bigchaindb的python driver

ansible bigchain -m command -a "apt-get install git -y"
ansible bigchain -m command -a "apt-get install libssl-dev -y"
ansible bigchain -m command -a "pip3 install --process-dependency-links git+https://github.com/bigchaindb/bigchaindb-driver.git"

官网上的测试用例:https://docs.bigchaindb.com/projects/py-driver/en/latest/usage.html

# coding=utf-8
# author: Wu Luo
import bigchaindb_driver
from bigchaindb_driver import BigchainDB
from bigchaindb_driver.crypto import generate_keypair
from time import sleep
import json
from sys import exit
import sys
class BigChainAPI(object):
 def __init__(self, host, port=9984, conf='/root/.bigchaindb', timeout=60):
 '''
 :param host: cluster host of bigchaindb
 :param port: port of bigchaindb
 :param conf: the configuration file of bigchaindb
 :param timeout: waiting vote for `timeout` seconds
 :return:
 '''
 self.host = host
 self.port = int(port)
 self.bdb = BigchainDB("http://%s:%d" % (self.host, self.port))
 self.conf = conf
 self.timeout = int(timeout)
 self.loadUserKey()
 def loadUserKey(self):
 '''
 load user key from configuration files, as the valid keypairs are
 designated in our deployment
 :return:
 '''
 configuration = json.load(open(self.conf, "r"))
 self.user = configuration['keypair']
 def waitVote(self, txid):
 '''
 wait the transaction to be voted as valid
 :param txid: the transaction id
 :return: True/False
 '''
 print("txid is ", txid)
 trials = 0
 while trials < self.timeout:
 try:
 if self.bdb.transactions.status(txid).get('status') == 'valid':
 print('Tx valid in:', trials, 'secs')
 break
 elif self.bdb.transactions.status(txid).get('status') == 'invalid':
 print('Tx invalid in:', trials, 'secs')
 print(self.bdb.transactions.status(txid))
 break
 else:
 trials += 1
 print("trials " + str(trials))
 sleep(1)
 except bigchaindb_driver.exceptions.NotFoundError:
 trials += 1
 sleep(1)
 if trials == self.timeout:
 print('Tx is still being processed... Bye!')
 return False
 return True
 def writeTransaction(self, data, metadata=None):
 '''
 write data into bigchaindb
 :param data: the data to write. it should be a dict
 :param metadata: the metadata to write. it should be a dict
 :return: the transaction id
 '''
 _asset = {
 'data': data
 }
 prepared_creation_tx = self.bdb.transactions.prepare(
 operation='CREATE',
 signers=self.user['public'],
 asset=_asset,
 metadata=metadata
 )
 fulfilled_creation_tx = self.bdb.transactions.fulfill(
 prepared_creation_tx,
 private_keys=self.user['private']
 )
 sent_creation_tx = self.bdb.transactions.send(fulfilled_creation_tx)
 txid = fulfilled_creation_tx['id']
 if not self.waitVote(txid):
 return False
 return txid
 def transferAsset(self, asset_id, publicKeyOfTargetUser):
 '''
 transfer an asset to another user
 :param asset_id: the asset to transfer
 :param publicKeyOfTargetUser: the public key of the target user
 It should be generated by generate_keypair()
 :return: the transaction id
 '''
 transfer_asset = {
 'id': asset_id
 }
 # query the transaction which contains the asset
 data = self.bdb.transactions.retrieve(asset_id)
 output_index = 0
 output = data['outputs'][output_index]
 transfer_input = {
 'fulfillment': output['condition']['details'],
 'fulfills': {
 'output': output_index,
 # in other version of BigchainDB, the key should be 'txid'
 'transaction_id': data['id']
 },
 'owners_before': output['public_keys']
 }
 prepared_transfer_tx = self.bdb.transactions.prepare(
 operation='TRANSFER',
 asset=transfer_asset,
 inputs=transfer_input,
 recipients=publicKeyOfTargetUser,
 )
 fulfilled_transfer_tx = self.bdb.transactions.fulfill(
 prepared_transfer_tx,
 private_keys=self.user['private'],
 )
 sent_transfer_tx = self.bdb.transactions.send(fulfilled_transfer_tx)
 txid = fulfilled_transfer_tx['id']
 if not self.waitVote(txid):
 return False
 return txid
 def queryTransaction(self, txid):
 '''
 retrieve a transaction
 :param txid: the txid of the transaction to query
 :return: the data of the retrieved transaction
 '''
 data = self.bdb.transactions.retrieve(txid)
 return data['asset']
def test():
 bigchainAPI = BigChainAPI(host='10.0.0.71', port=9984, timeout=300)
 # write an transaction
 data = {
 'author': 'Wu Luo'
 }
 txid = bigchainAPI.writeTransaction(data)
 if txid == False:
 print(">>> waiting for vote")
 return False
 # query this transaction
 query_data = bigchainAPI.queryTransaction(txid)
 print(">>> txid is " + txid)
 print(">>> data retrieved from bigchainDB")
 print(query_data)
 # transfer
 user = generate_keypair()
 newid = bigchainAPI.transferAsset(txid, user.public_key)
 print(txid, newid)
 transfer_data = bigchainAPI.queryTransaction(newid)
 print(transfer_data)
test()

运行后

root@node1:~# python3 bigchainAPI.py
txid is 6a82600a7cd8683f8e83cc6cfb765b219a8543404da2a9ad5b56114b9f1a8dc5
trials 1
trials 2
Tx valid in: 2 secs
>>> txid is 6a82600a7cd8683f8e83cc6cfb765b219a8543404da2a9ad5b56114b9f1a8dc5
>>> data retrieved from bigchainDB
{'data': {'author': 'Wu Luo'}}
txid is c6a7b301d485243a7404fd048d0cec3912410f58ccd20701bc8937d7587516a8
trials 1
trials 2
Tx valid in: 2 secs
6a82600a7cd8683f8e83cc6cfb765b219a8543404da2a9ad5b56114b9f1a8dc5 c6a7b301d485243a7404fd048d0cec3912410f58ccd20701bc8937d7587516a8
{'id': '6a82600a7cd8683f8e83cc6cfb765b219a8543404da2a9ad5b56114b9f1a8dc5'}

部署成功

有疑问加站长微信联系(非本文作者)

• 请尽量让自己的回复能够对别人有帮助
• 支持 Markdown 格式, **粗体**、~~删除线~~、`单行代码`
• 支持 @ 本站用户;支持表情(输入 : 提示),见 Emoji cheat sheet
• 图片支持拖拽、截图粘贴等方式上传