I am trying to setup a hyperledger network with 2 orgs running on 2 different physical machine. So machine 1 has orderer, peer0.org1, peer1.org1, ca-org1, couchdb1, couchdb2 and cli containers. Machine 2 has peer0.org2, peer1.org2, ca-org2, couchdb3, couchdb4 and cli container.
Below is the docker-compose-org1.yaml file -
version: '2'
volumes:
orderer.example.com:
peer0.org1.example.com:
peer1.org1.example.com:
networks:
byfn:
services:
ca0.example.com:
image: hyperledger/fabric-ca:$IMAGE_TAG
environment:
- FABRIC_CA_HOME=/etc/hyperledger/fabric-ca-server
- FABRIC_CA_SERVER_CA_NAME=ca-org1
- FABRIC_CA_SERVER_TLS_ENABLED=true
- FABRIC_CA_SERVER_TLS_CERTFILE=/etc/hyperledger/fabric-ca-server-config/ca.org1.example.com-cert.pem
- FABRIC_CA_SERVER_TLS_KEYFILE=/etc/hyperledger/fabric-ca-server-config/65cc7012f64e12926807f9b731d3f4dbbda27dc577cb5a6afdb3730fcf53c48c_sk
ports:
- '7054:7054'
command: sh -c 'fabric-ca-server start --ca.certfile /etc/hyperledger/fabric-ca-server-config/ca.org1.example.com-cert.pem --ca.keyfile /etc/hyperledger/fabric-ca-server-config/65cc7012f64e12926807f9b731d3f4dbbda27dc577cb5a6afdb3730fcf53c48c_sk -b admin:adminpw -d'
volumes:
- ./crypto-config/peerOrganizations/org1.example.com/ca/:/etc/hyperledger/fabric-ca-server-config
container_name: ca0.example.com
extra_hosts:
- "peer0.org2.example.com:10.160.151.114"
- "peer1.org2.example.com:10.160.151.114"
- "ca1.example.com:10.160.151.114"
networks:
- byfn
orderer.example.com:
container_name: orderer.example.com
image: hyperledger/fabric-orderer:$IMAGE_TAG
environment:
- ORDERER_GENERAL_LOGLEVEL=INFO
- ORDERER_GENERAL_LISTENADDRESS=0.0.0.0
- ORDERER_GENERAL_GENESISMETHOD=file
- ORDERER_GENERAL_GENESISFILE=/var/hyperledger/orderer/orderer.genesis.block
- ORDERER_GENERAL_LOCALMSPID=OrdererMSP
- ORDERER_GENERAL_LOCALMSPDIR=/var/hyperledger/orderer/msp
# enabled TLS
- ORDERER_GENERAL_TLS_ENABLED=true
- ORDERER_GENERAL_TLS_PRIVATEKEY=/var/hyperledger/orderer/tls/server.key
- ORDERER_GENERAL_TLS_CERTIFICATE=/var/hyperledger/orderer/tls/server.crt
- ORDERER_GENERAL_TLS_ROOTCAS=[/var/hyperledger/orderer/tls/ca.crt]
working_dir: /opt/gopath/src/github.com/hyperledger/fabric
command: orderer
volumes:
- ./channel-artifacts/genesis.block:/var/hyperledger/orderer/orderer.genesis.block
- ./crypto-config/ordererOrganizations/example.com/orderers/orderer.example.com/msp:/var/hyperledger/orderer/msp
- ./crypto-config/ordererOrganizations/example.com/orderers/orderer.example.com/tls/:/var/hyperledger/orderer/tls
- orderer.example.com:/var/hyperledger/production/orderer
extra_hosts:
- "peer0.org2.example.com:10.160.151.114"
- "peer1.org2.example.com:10.160.151.114"
- "ca1.example.com:10.160.151.114"
ports:
- 7050:7050
networks:
- byfn
peer0.org1.example.com:
container_name: peer0.org1.example.com
image: hyperledger/fabric-peer:$IMAGE_TAG
environment:
- CORE_PEER_ID=peer0.org1.example.com
- CORE_PEER_ADDRESS=peer0.org1.example.com:7051
- CORE_PEER_GOSSIP_BOOTSTRAP=peer1.org1.example.com:7051
- CORE_PEER_GOSSIP_EXTERNALENDPOINT=peer0.org1.example.com:7051
- CORE_PEER_LOCALMSPID=Org1MSP
- CORE_VM_ENDPOINT=unix:///host/var/run/docker.sock
- CORE_VM_DOCKER_HOSTCONFIG_NETWORKMODE=${COMPOSE_PROJECT_NAME}_byfn
- CORE_LOGGING_LEVEL=INFO
- CORE_PEER_TLS_ENABLED=true
- CORE_PEER_GOSSIP_USELEADERELECTION=true
- CORE_PEER_GOSSIP_ORGLEADER=false
- CORE_PEER_PROFILE_ENABLED=true
- CORE_PEER_TLS_CERT_FILE=/etc/hyperledger/fabric/tls/server.crt
- CORE_PEER_TLS_KEY_FILE=/etc/hyperledger/fabric/tls/server.key
- CORE_PEER_TLS_ROOTCERT_FILE=/etc/hyperledger/fabric/tls/ca.crt
- CORE_LEDGER_STATE_STATEDATABASE=CouchDB
- CORE_LEDGER_STATE_COUCHDBCONFIG_COUCHDBADDRESS=couchdb0:5984
- CORE_LEDGER_STATE_COUCHDBCONFIG_USERNAME=
- CORE_LEDGER_STATE_COUCHDBCONFIG_PASSWORD=
volumes:
- /var/run/:/host/var/run/
- ./crypto-config/peerOrganizations/org1.example.com/peers/peer0.org1.example.com/msp:/etc/hyperledger/fabric/msp
- ./crypto-config/peerOrganizations/org1.example.com/peers/peer0.org1.example.com/tls:/etc/hyperledger/fabric/tls
- peer0.org1.example.com:/var/hyperledger/production
working_dir: /opt/gopath/src/github.com/hyperledger/fabric/peer
extra_hosts:
- "peer0.org2.example.com:10.160.151.114"
- "peer1.org2.example.com:10.160.151.114"
- "ca1.example.com:10.160.151.114"
command: peer node start
ports:
- 7051:7051
- 7053:7053
depends_on:
- couchdb0
networks:
- byfn
couchdb0:
container_name: couchdb0
image: hyperledger/fabric-couchdb
environment:
- COUCHDB_USER=
- COUCHDB_PASSWORD=
ports:
- '5984:5984'
networks:
- byfn
peer1.org1.example.com:
container_name: peer1.org1.example.com
image: hyperledger/fabric-peer:$IMAGE_TAG
environment:
- CORE_PEER_ID=peer1.org1.example.com
- CORE_PEER_ADDRESS=peer1.org1.example.com:7051
- CORE_PEER_GOSSIP_BOOTSTRAP=peer0.org1.example.com:7051
- CORE_PEER_GOSSIP_EXTERNALENDPOINT=peer1.org1.example.com:7051
- CORE_PEER_LOCALMSPID=Org1MSP
- CORE_VM_ENDPOINT=unix:///host/var/run/docker.sock
- CORE_VM_DOCKER_HOSTCONFIG_NETWORKMODE=${COMPOSE_PROJECT_NAME}_byfn
- CORE_LOGGING_LEVEL=INFO
- CORE_PEER_TLS_ENABLED=true
- CORE_PEER_GOSSIP_USELEADERELECTION=true
- CORE_PEER_GOSSIP_ORGLEADER=false
- CORE_PEER_PROFILE_ENABLED=true
- CORE_PEER_TLS_CERT_FILE=/etc/hyperledger/fabric/tls/server.crt
- CORE_PEER_TLS_KEY_FILE=/etc/hyperledger/fabric/tls/server.key
- CORE_PEER_TLS_ROOTCERT_FILE=/etc/hyperledger/fabric/tls/ca.crt
- CORE_LEDGER_STATE_STATEDATABASE=CouchDB
- CORE_LEDGER_STATE_COUCHDBCONFIG_COUCHDBADDRESS=couchdb1:5984
- CORE_LEDGER_STATE_COUCHDBCONFIG_USERNAME=
- CORE_LEDGER_STATE_COUCHDBCONFIG_PASSWORD=
volumes:
- /var/run/:/host/var/run/
- ./crypto-config/peerOrganizations/org1.example.com/peers/peer1.org1.example.com/msp:/etc/hyperledger/fabric/msp
- ./crypto-config/peerOrganizations/org1.example.com/peers/peer1.org1.example.com/tls:/etc/hyperledger/fabric/tls
- peer1.org1.example.com:/var/hyperledger/production
working_dir: /opt/gopath/src/github.com/hyperledger/fabric/peer
extra_hosts:
- "peer0.org2.example.com:10.160.151.114"
- "peer1.org2.example.com:10.160.151.114"
- "ca1.example.com:10.160.151.114"
command: peer node start
ports:
- 8051:7051
- 8053:7053
depends_on:
- couchdb1
networks:
- byfn
couchdb1:
container_name: couchdb1
image: hyperledger/fabric-couchdb
environment:
- COUCHDB_USER=
- COUCHDB_PASSWORD=
ports:
- '6984:5984'
networks:
- byfn
cli:
container_name: cli
image: hyperledger/fabric-tools:$IMAGE_TAG
tty: true
stdin_open: true
environment:
- GOPATH=/opt/gopath
- CORE_VM_ENDPOINT=unix:///host/var/run/docker.sock
#- CORE_LOGGING_LEVEL=DEBUG
- CORE_LOGGING_LEVEL=INFO
- CORE_PEER_ID=cli
- CORE_PEER_ADDRESS=peer0.org1.example.com:7051
- CORE_PEER_LOCALMSPID=Org1MSP
- CORE_PEER_TLS_ENABLED=true
- CORE_PEER_TLS_CERT_FILE=/opt/gopath/src/github.com/hyperledger/fabric/peer/crypto/peerOrganizations/org1.example.com/peers/peer0.org1.example.com/tls/server.crt
- CORE_PEER_TLS_KEY_FILE=/opt/gopath/src/github.com/hyperledger/fabric/peer/crypto/peerOrganizations/org1.example.com/peers/peer0.org1.example.com/tls/server.key
- CORE_PEER_TLS_ROOTCERT_FILE=/opt/gopath/src/github.com/hyperledger/fabric/peer/crypto/peerOrganizations/org1.example.com/peers/peer0.org1.example.com/tls/ca.crt
- CORE_PEER_MSPCONFIGPATH=/opt/gopath/src/github.com/hyperledger/fabric/peer/crypto/peerOrganizations/org1.example.com/users/[email protected]/msp
working_dir: /opt/gopath/src/github.com/hyperledger/fabric/peer
extra_hosts:
- "peer0.org2.example.com:10.160.151.114"
- "peer1.org2.example.com:10.160.151.114"
- "ca1.example.com:10.160.151.114"
command: /bin/bash
volumes:
- /var/run/:/host/var/run/
- ./../chaincode/:/opt/gopath/src/github.com/chaincode
- ./crypto-config:/opt/gopath/src/github.com/hyperledger/fabric/peer/crypto/
- ./scripts:/opt/gopath/src/github.com/hyperledger/fabric/peer/scripts/
- ./channel-artifacts:/opt/gopath/src/github.com/hyperledger/fabric/peer/channel-artifacts
depends_on:
- orderer.example.com
- peer0.org1.example.com
- peer1.org1.example.com
networks:
- byfn
docker-compose-org2.yaml file -
version: '2'
volumes:
peer0.org2.example.com:
peer1.org2.example.com:
networks:
byfn:
services:
ca1.example.com:
image: hyperledger/fabric-ca
environment:
- FABRIC_CA_HOME=/etc/hyperledger/fabric-ca-server
- FABRIC_CA_SERVER_CA_NAME=ca-org2
- FABRIC_CA_SERVER_TLS_ENABLED=true
- FABRIC_CA_SERVER_TLS_CERTFILE=/etc/hyperledger/fabric-ca-server-config/ca.org2.example.com-cert.pem
- FABRIC_CA_SERVER_TLS_KEYFILE=/etc/hyperledger/fabric-ca-server-config/16acd23be9bbaf215828ddb410074828ce2b87ab68dff5976725b36f439f8c47_sk
ports:
- '7054:7054'
command: sh -c 'fabric-ca-server start --ca.certfile /etc/hyperledger/fabric-ca-server-config/ca.org2.example.com-cert.pem --ca.keyfile /etc/hyperledger/fabric-ca-server-config/16acd23be9bbaf215828ddb410074828ce2b87ab68dff5976725b36f439f8c47_sk -b admin:adminpw -d'
volumes:
- ./crypto-config/peerOrganizations/org2.example.com/ca/:/etc/hyperledger/fabric-ca-server-config
container_name: ca0.example.com
extra_hosts:
- 'peer0.org1.example.com:10.160.151.121'
- 'peer1.org1.example.com:10.160.151.121'
- 'ca0.example.com:10.160.151.121'
- 'orderer.example.com:10.160.151.121'
networks:
- byfn
peer0.org2.example.com:
container_name: peer0.org2.example.com
image: hyperledger/fabric-peer:latest
environment:
- CORE_PEER_ID=peer0.org2.example.com
- CORE_PEER_ADDRESS=peer0.org2.example.com:7051
- CORE_PEER_GOSSIP_BOOTSTRAP=peer1.org2.example.com:7051
- CORE_PEER_GOSSIP_EXTERNALENDPOINT=peer0.org2.example.com:7051
- CORE_PEER_LOCALMSPID=Org2MSP
- CORE_VM_ENDPOINT=unix:///host/var/run/docker.sock
- CORE_VM_DOCKER_HOSTCONFIG_NETWORKMODE=${COMPOSE_PROJECT_NAME}_byfn
- CORE_LOGGING_LEVEL=INFO
- CORE_PEER_TLS_ENABLED=true
- CORE_PEER_GOSSIP_USELEADERELECTION=true
- CORE_PEER_GOSSIP_ORGLEADER=false
- CORE_PEER_PROFILE_ENABLED=true
- CORE_PEER_TLS_CERT_FILE=/etc/hyperledger/fabric/tls/server.crt
- CORE_PEER_TLS_KEY_FILE=/etc/hyperledger/fabric/tls/server.key
- CORE_PEER_TLS_ROOTCERT_FILE=/etc/hyperledger/fabric/tls/ca.crt
- CORE_LEDGER_STATE_STATEDATABASE=CouchDB
- CORE_LEDGER_STATE_COUCHDBCONFIG_COUCHDBADDRESS=couchdb2:5984
- CORE_LEDGER_STATE_COUCHDBCONFIG_USERNAME=
- CORE_LEDGER_STATE_COUCHDBCONFIG_PASSWORD=
volumes:
- /var/run/:/host/var/run/
- ./crypto-config/peerOrganizations/org2.example.com/peers/peer0.org2.example.com/msp:/etc/hyperledger/fabric/msp
- ./crypto-config/peerOrganizations/org2.example.com/peers/peer0.org2.example.com/tls:/etc/hyperledger/fabric/tls
- peer0.org2.example.com:/var/hyperledger/production
extra_hosts:
- 'peer0.org1.example.com:10.160.151.121'
- 'peer1.org1.example.com:10.160.151.121'
- 'ca0.example.com:10.160.151.121'
- 'orderer.example.com:10.160.151.121'
working_dir: /opt/gopath/src/github.com/hyperledger/fabric/peer
command: peer node start
ports:
- 9051:7051
- 9053:7053
depends_on:
- couchdb2
networks:
- byfn
couchdb2:
container_name: couchdb2
image: hyperledger/fabric-couchdb
environment:
- COUCHDB_USER=
- COUCHDB_PASSWORD=
ports:
- '7984:5984'
networks:
- byfn
peer1.org2.example.com:
container_name: peer1.org2.example.com
image: hyperledger/fabric-peer:latest
environment:
- CORE_PEER_ID=peer1.org2.example.com
- CORE_PEER_ADDRESS=peer1.org2.example.com:7051
- CORE_PEER_GOSSIP_BOOTSTRAP=peer0.org2.example.com:7051
- CORE_PEER_GOSSIP_EXTERNALENDPOINT=peer1.org2.example.com:7051
- CORE_PEER_LOCALMSPID=Org2MSP
- CORE_VM_ENDPOINT=unix:///host/var/run/docker.sock
- CORE_VM_DOCKER_HOSTCONFIG_NETWORKMODE=${COMPOSE_PROJECT_NAME}_byfn
- CORE_LOGGING_LEVEL=INFO
- CORE_PEER_TLS_ENABLED=true
- CORE_PEER_GOSSIP_USELEADERELECTION=true
- CORE_PEER_GOSSIP_ORGLEADER=false
- CORE_PEER_PROFILE_ENABLED=true
- CORE_PEER_TLS_CERT_FILE=/etc/hyperledger/fabric/tls/server.crt
- CORE_PEER_TLS_KEY_FILE=/etc/hyperledger/fabric/tls/server.key
- CORE_PEER_TLS_ROOTCERT_FILE=/etc/hyperledger/fabric/tls/ca.crt
- CORE_LEDGER_STATE_STATEDATABASE=CouchDB
- CORE_LEDGER_STATE_COUCHDBCONFIG_COUCHDBADDRESS=couchdb3:5984
- CORE_LEDGER_STATE_COUCHDBCONFIG_USERNAME=
- CORE_LEDGER_STATE_COUCHDBCONFIG_PASSWORD=
volumes:
- /var/run/:/host/var/run/
- ./crypto-config/peerOrganizations/org2.example.com/peers/peer1.org2.example.com/msp:/etc/hyperledger/fabric/msp
- ./crypto-config/peerOrganizations/org2.example.com/peers/peer1.org2.example.com/tls:/etc/hyperledger/fabric/tls
- peer1.org2.example.com:/var/hyperledger/production
extra_hosts:
- 'peer0.org1.example.com:10.160.151.121'
- 'peer1.org1.example.com:10.160.151.121'
- 'ca0.example.com:10.160.151.121'
- 'orderer.example.com:10.160.151.121'
working_dir: /opt/gopath/src/github.com/hyperledger/fabric/peer
command: peer node start
ports:
- 10051:7051
- 10053:7053
depends_on:
- couchdb3
networks:
- byfn
couchdb3:
container_name: couchdb3
image: hyperledger/fabric-couchdb
environment:
- COUCHDB_USER=
- COUCHDB_PASSWORD=
ports:
- '8984:5984'
networks:
- byfn
cli:
container_name: cli
image: hyperledger/fabric-tools:latest
tty: true
stdin_open: true
environment:
- GOPATH=/opt/gopath
- CORE_VM_ENDPOINT=unix:///host/var/run/docker.sock
#- CORE_LOGGING_LEVEL=DEBUG
- CORE_LOGGING_LEVEL=INFO
- CORE_PEER_ID=cli
- CORE_PEER_ADDRESS=peer0.org2.example.com:7051
- CORE_PEER_LOCALMSPID=Org2MSP
- CORE_PEER_TLS_ENABLED=true
- CORE_PEER_TLS_CERT_FILE=/opt/gopath/src/github.com/hyperledger/fabric/peer/crypto/peerOrganizations/org2.example.com/peers/peer0.org2.example.com/tls/server.crt
- CORE_PEER_TLS_KEY_FILE=/opt/gopath/src/github.com/hyperledger/fabric/peer/crypto/peerOrganizations/org2.example.com/peers/peer0.org2.example.com/tls/server.key
- CORE_PEER_TLS_ROOTCERT_FILE=/opt/gopath/src/github.com/hyperledger/fabric/peer/crypto/peerOrganizations/org2.example.com/peers/peer0.org2.example.com/tls/ca.crt
- CORE_PEER_MSPCONFIGPATH=/opt/gopath/src/github.com/hyperledger/fabric/peer/crypto/peerOrganizations/org2.example.com/users/[email protected]/msp
working_dir: /opt/gopath/src/github.com/hyperledger/fabric/peer
extra_hosts:
- 'peer0.org1.example.com:10.160.151.121'
- 'peer1.org1.example.com:10.160.151.121'
- 'ca0.example.com:10.160.151.121'
- 'orderer.example.com:10.160.151.121'
command: /bin/bash
volumes:
- /var/run/:/host/var/run/
- ./chaincode/:/opt/gopath/src/github.com/chaincode
- ./crypto-config:/opt/gopath/src/github.com/hyperledger/fabric/peer/crypto/
- ./scripts:/opt/gopath/src/github.com/hyperledger/fabric/peer/scripts/
- ./channel-artifacts:/opt/gopath/src/github.com/hyperledger/fabric/peer/channel-artifacts
depends_on:
- peer0.org2.example.com
- peer1.org2.example.com
networks:
- byfn
As you can see I am using "extra_hosts" to connect to the docker container in other machine. This way I am able to create channel from one machine and join the channel for both peers from that machine. In the other machine I fetched the same channel config and join the same channel for both peers. I checked that all 4 peers in both machines have joined the same channel by below command -
peer channel list
peer channel getinfo -c mychannel
Now from machine 1 I have installed and instantiated chaincode. This is working fine. Used below commands to install and instantiate -
peer chaincode install -n mycc -v 1.0 -l golang -p github.com/chaincode/chaincode_example02/go/
peer chaincode instantiate -o orderer.example.com:7050 --tls true --cafile /opt/gopath/src/github.com/hyperledger/fabric/peer/crypto/ordererOrganizations/example.com/orderers/orderer.example.com/msp/tlscacerts/tlsca.example.com-cert.pem -C mychannel -n mycc -l golang -v 1.0 -c '{"Args":["init","a","100","b","200"]}' -P "AND ('Org1MSP.peer','Org2MSP.peer')"
If I query the chaincode from the same machine it is working fine and I am getting proper result.
peer chaincode query -C mychannel -n mycc -c '{"Args":["query","a"]}'
But when I am executing the same chaincode query command from different machine it is giving below error -
Error: endorsement failure during query. response: status:500 message:"failed to execute transaction 2444b829db1485ad097b2e11442d65cb36f02de1c6a3f3372ad53594af4d1a4d: [channel mychannel] failed to get chaincode container info for mycc:1.0: could not get chaincode code: chaincode fingerprint mismatch: data mismatch"
I can see the hyperledger fabric is syncing all peers properly in both machines. Because when I run peer channel getinfo -c mychannel
I can match the currentBlockHash, previousBlockHash and height value for all 4 peers. I am just not able to execute the peer chaincode query
command from the other machine.
When I run peer chaincode instantiate
command I can see a new docker container is created in machine 1 which is for running chaincode. But there is no such container in machine 2.
Am I missing any configuration here so that the chaincode container created on machine 1 can be accessed from machine 2?
New findings:
I have installed chaincode on all 4 peers in both machines by running peer chaincode install
command from the CLI container. I have two CLI container running in two machines.
When I run the command peer chaincode list --installed
. In machine 1 I get result like below
Name: mycc, Version: 1.0, Path: github.com/chaincode/chaincode_example02/go/, Id: 333a19b11063d0ade7be691f9f22c04ad369baba15660f7ae9511fd1a6488209
When I run the same command in machine 2. The result is same except the Id
value is different. But I also found that the Id
value is same for both peers on same machine. Probably that's why I am able to execute the chaincode query from both peers on same machine. But not able to run it from other machine. Looks like that's the root cause of this problem.
I was able to figure out the problem ultimately so posting an answer here. The problem was that when I was installed the chaincode from two different machine the chaincode Id
was different. Previously I was using the command peer chaincode install -n mycc -v 1.0 -l golang -p github.com/chaincode/chaincode_example02/go/
to install the chaincode seperately on two machines. So when I do peer chaincode list --installed
it was showing the Id
value as different.
I was able to solve it using slightly a different approach. First created chaincode package using the command peer chaincode package ccpack.out -n mycc -l golang -p github.com/chaincode/chaincode_example02/go/ -v 1.0
. It will generate a file ccpack.out
. Moved this file physically from one machine to another. Then installed the same package file in both machines using the command peer chaincode install ccpack.out
. In that way the chaincode Id
remains same and it works perfectly.