Hyperledger Fabric: Buildpacks & External Chaincode Server Builder

Detect, Build, Release, Run and Repeat!

Reference an external chaincode server builder written in Go at https://github.com/arsulegai/fabric-cc-external-builder. Feel free to raise a PR, extend the source code to include additional scenarios or use it in your development cycle.

How does a Hyperledger Fabric `Peer` node build the chaincode?

Have you wondered what happens in the background when you run peer chaincode install command? Well, look no further. Consider that the peer node is doing more than magic for you.

Hyperledger Fabric Peer node chaincode build process

• The peer CLI packages the chaincode source, adds a metadata file to it.
• The package is then sent over to the peer node by the peer CLI.
• The peer node stores the submitted package, but before that it computes the SHA256of the submitted package.
• The peer node stores the submitted chaincode package as is using name , version , SHA256 . This together constitutes a chaincode package ID.
• The peer node runs the package through a list of buildpacks .
• The first builder that detects the metadata successfully picks up the code and runs through the build process.
• The build generates artefacts using appropriate language compilers.
• The release process copies the generated artefacts along with related metadata files for execution. This is when the peer node generates a chaincode’s connection.json file.
• Finally run will package the generated artefacts on top of a chaincode base image for execution.

Note: In case of failure to detect any of the custom builders. The chaincode package is sent to the in-built builders. These in-built builders are language specific and they try to package the chaincode as a container image and bring up the peer’s host.

What are buildpacks and why external chaincode builder?

Consider buildpacks to be a collection of executable binaries. These binaries work together to achieve the task of build. The detect and build mechanism of buildpacks allows one to write code once, leave the build and deployment to the buildpacks agent.

In case of Hyperledger Fabric peer node, a detect process can read and identify which language is the chaincode written in. If an administrator supplies additional builders, those will be prioritized for detect phase over the in-built builders.

Buildpacks in Hyperledger Fabric Peer node

There can be multiple reasons to run chaincode as a server. In a Kubernetes based deployment, administrators would like to have chaincode spun up and in control of the controller manager. If a peer node brings up the chaincode then it is tied to the host where peer node runs. Kubernetes is deprecating docker daemon support from 1.20 version onwards.

All these leads to one adding buildpacks for external chaincode server build. Though the chaincode itself is not packaged, there is no code to build, one still have to package and place the metadata information in appropriate places.

Chaincode Server

Tip: If you are writing a new builder make sure to dump all the output to STDERR. Hyperledger Fabric peer node does not consider STDOUT logs. Refer to the example implementation to know how to control log levels.

A chaincode builder becomes mandatory in case of chaincode run as a server. With all the information related to how to connect to a chaincode server. A peer node can establish connection to the chaincode server when either invoke or equivalent actions are called on the chaincode.

Note: Chaincode’s connection.json is different from peer’s connection.json . Unlike chaincode run by the peer node, in the case of chaincode server these parameters can be passed in any form, this is controlled by the administrator and not by the peer node.

Hyperledger Fabric follows CNCF buildpacks principles. Read more about CNCF’s buildpacks at https://buildpacks.io/. The explanation in this article is overly simplified for the purpose of understanding.

How to support multiple buildpacks?

The peer node does run through the list of detect processes that are listed in the core.yaml configuration file. The first detect process that returns an error status 0 would be considered an ideal match. Make sure to read enough metadata information before considering the builder a match.

Good practice is to match type information in metadata.json file. In addition to this, one can pass in extra data to be consumed in the custom builders. The extra metadata can optionally also include dependencies if any.

Using the Buildpack to detect, build and release an External Chaincode servers’ artefacts

Read in detail about chaincode installation in my other blog article https://arsulegai.medium.com/hyperledger-fabric-chaincode-install-upgrade-and-revert-af52015c4007.

How to keep the Chaincode ID consistent across all the peer nodes in an organization?

If your chaincode server package contents vary across multiple peers belonging to same organization. The SHA256 checksum of the chaincode package varies. Thus you cannot have the same package ID to approve on.

Key challenges for adoption of chaincode server and external builders

1. Unique key-pair for TLS client authentication by the chaincode server i.e., a chaincode server shall be aware of which peer it is talking to.
2. If an organization has multiple peers and each peer would like to have its own chaincode server instance, participate in the channel.
3. If an organization has multiple peers and each peer wishes to use different timeout option.

In each of these scenarios, the chaincode package contents vary. How to get a unique chaincode ID? Let’s look at the following diagram

Building the External Chaincode server the Right way!

In this case, all of the variadic parameters are passed as an environment variable. The actual connection.json file expected by the peer node is built as part of the buildpacks’ build process. The challenge however is still in passing these environment variables to the external builder. First you have to inject them to the peer process environment, from there you shall be passing them to the external builders via the peer’s environment variable passing the core.yaml file.

Hope this article was helpful, happy coding!