Skip to main content

Distributed Ledger Technologies (DLT)

Now that ledgers have been introduced, centralised ledgers and distributed ledgers, let's start to dive deeper into the different components of distributed ledger technologies.

Overview​

  • 1.5 Distributed Ledger Technologies and Networks
  • 1.6 DLT Network Connections
  • 1.7 Establish Your DLT Network Connection
  • 1.8 Permissionless and Permissioned DLT
  • 1.9 Distributed Ledger Technology Components
  • 1.10 Read Data from Distributed Ledgers
  • 1.11 DLT Nodes

[1.5] Distributed Ledger Technologies and Networks​

Distributed Ledgers are implemented via Distributed Ledger Technologies. There are many Distributed Ledger Technologies (DLTs), such as Hyperledger Fabric, Corda and Ethereum. Each DLT can be further defined by version, e.g. Hyperledger Fabric v2.3, Corda v5 or Ethereum London Upgrade. These technology and version combinations describe:

  1. Identity Management: How ledger users are identified;
  2. Ledger State: How the ledger’s data can be stored (i.e. it’s state machine definition);
  3. Ledger Updates: How transactions are created and how they affect the current ledger state (i.e. it’s state transition function);
  4. Transaction Ordering: How transactions are ordered (e.g. via a blockchain or otherwise); and
  5. Consensus Protocol: How ledger data held by different nodes becomes consistent over time.

DLT Networks

Each DLT can be deployed via multiple DLT networks. For instance, Hyperledger Fabric v1.4 might be used for different DLT networks of disjoint consortia (e.g. for a supply chain consortium in the EU, and a separate healthcare consortium in the UK). Whereas the Ethereum London Upgrade is currently utilised in Ethereum’s main DLT network (where the cryptocurrency ETH resides), as well as Ethereum’s associated test DLT networks (such as Ropsten, Rinkeby, etc).

Each DLT network is made up of a set of DLT nodes, connected in a mesh manner. These nodes run the same underlying technology and one of a set of compatible technology versions. DLT nodes in the same DLT network will gossip to each other various messages, such as transactions to update the distributed ledger.

DLT Modularity

Some DLTs offer no flexibility on how the above points 1 to 5 must be implemented (e.g. Bitcoin). Other DLTs offer options for how these points can be implemented before a DLT network is running, but any further changes require a major upgrade to the node software. For example, the Ethereum DLT provides a few consensus protocols options when an Ethereum DLT network is being established, but a change of consensus protocol while the network is active requires a major upgrade. Furthermore, other DLTs offer options for how points 1 to 5 can be implemented before the DLT network is running, and allow further changes to be made without a major upgrade to the node software. For example, the Substrate DLT allows the state transition function of the ledger to be changed without a major upgrade.

[1.6] DLT Network Connections​

Connecting to a DLT network requires establishing a connection to at least one DLT node on the network.

DLT Network Connection Options

When connecting an application to one or more DLT networks, a developer has a few choices:

  • DLT Node Only: The application could connect directly to a DLT node. This is not crash resilient and not scalable from a load balancing perspective.
  • DLT Node Proxy: A DLT node proxy manages the routing and load balancing issues between an application and one or more DLT nodes, creating logical separation. To an application, interacting with a DLT node proxy is near identical to interacting with a DLT node as the message requests and responses will be virtually the same. The only possible difference may be some identifying meta data in the messages in order to track the DLT node proxy users (e.g. for rate limiting reasons).
  • DLT Gateway: Like a DLT node proxy, a DLT gateway also manages the routing and load balancing issues between an application and one or more DLT nodes, creating logical separation. A DLT gateway differs from a DLT node proxy because a gateway provides additional functionality not offered by the DLT node itself. Such additional functionality can include a standardised data model, meaning that unlike DLT node proxies, a DLT gateway can be used to connect to many DLT networks of multiple DLT types.

Security of Connection

In an adversarial environment like a DLT Network, securing your infrastructure is of the highest importance.

Should you run a DLT node it is a requirement that you open a network connection to connect to other DLT nodes of the network. Yet it is equally necessary to control access to any other aspect of your DLT node.

To do so, the authentication and authorisation procedures of your DLT network connection architecture should be well planned and thought out.

We recommend utilising a DLT node proxy or a DLT gateway in order to secure your DLT network infrastructure.

[1.7] Establish Your DLT Network Connection​

Note that the section below contains external content and is not managed by FutureLearn. Interacting with the content may require or result in personal data transfer to this external website; FutureLearn advises you to check the external website’s privacy policy before use.

Overledger

In this exercise you will set up your connection to DLT nodes via the Overledger DLT gateway. Overledger provides access to multiple DLT networks of different DLT types, allowing multi-ledger applications to be built. Additionally Overledger utilises a standardised data model, easing the learning process for software developers. This means that, by using a unique data model for connecting to all DLTs, developers will not need to deal with the huge heterogeneity of DLTs and their available APIs.

Overledger Signup

To use Overledger, you will need to sign up on the Overledger Dev Portal.

Once signed up, there are a few steps you need to perform to select the correct environment and add a wallet:

  • Firstly select the “Sandbox V2.0” environment.
  • Then scroll to the “Wallets” tab and click on the “Add Wallet” button.
  • Select an “Application” wallet.
  • Give your wallet a name.
  • Add two different Ethereum addresses in your wallet. The specific addresses only become relevant should you want to access DLT mainnets through Overledger. Mainnets are the DLT networks that contain real token value. Note that all of the developer exercises in this course only use the DLT testnets. Therefore, for these exercises, you can add any Ethereum addresses. Should you not know how to generate ethereum addresses at this time, you can enter a dummy address, e.g. 0x123. This will only need to be updated when you require access to mainnet.

After a wallet has been added, you next need to setup an application, to do so:

  • Scroll to the “Applications” tab.
  • Click on the “register mDapp” button.
  • Give your application a name and select the wallet you just created.
  • Note that a callback URL is not needed for these developer exercises, as it is used for more advanced subscription features.

Now you have setup Overledger server side.

Setting up your Development Environment and Testing your Connection

To test your connection, go to the first exercise link on our Github repository in a new tab and follow the instructions. This exercise will set up your client side environment and allow you to generate Overledger access tokens, which is a requirement for the later development exercises.

[1.8] Permissionless and Permissioned DLT​

Network Categories

In permissionless DLT networks, every compatible DLT node can join the network. Whereas in permissioned DLT networks, only compatible DLT nodes with particular authorisations can join the network. Permissioned DLT networks can additionally assign a specific role to each DLT node, restricting the functions it can perform.

There are two sub categories of permissioned DLT networks:

  • Private Permissioned DLT networks: where the DLT nodes do not provide public access to the data contained in the distributed ledger; and
  • Public Permissioned DLT networks: where the DLT nodes do provide public access to the data, such as via block explorers.

Example use cases

Permissionless

  • Currency without banks.
  • Voting systems without election authorities.
  • Betting systems without brokers.

Permissioned

  • Efficient inter-bank transfers.
  • Voting systems with a group of election authorities.
  • Supply chain systems with a group of suppliers.

Permissioned DLT Networks are suited for situations where there is a desire to decentralise the control to some degree but not completely. For example, to move from a single authority model to a consortium model. This consortium model can have efficiency benefits if consortium members currently spend significant resources to keep their private ledgers up to date. This model can also have automation benefits if consortium members currently interact with each other in bespoke ways to synchronise their private ledgers.

Alternatively, if an open system with the maximum possible decentralisation is required, then a permissionless DLT network is more appropriate. This model can have transparency benefits as any individual in the world has the capability to access information on a permissionless DLT network via their own DLT node. There is also increased censorship resistance, as it is more difficult to restrict information from being added to a permissionless DLT network compared to being added to a permissioned version. Finally, this model can have organic ecosystem growth benefits. This is because adding information onto a permissionless DLT network means that anyone in the world can use this data to build any application they can imagine.

Data Privacy on DLT Networks

It is a common misconception that data can only be private between users on permissioned DLT networks. Firstly, it is true that permissioned DLT networks can offer more features to enable data privacy, such as private channels between DLT nodes. Secondly, Permissioned DLT networks can enforce data privacy solutions that compile with regulations.

But data privacy solutions are also available on many permissionless DLT networks. This is because smart contracts can enable any type of application to be built, and this includes privacy promoting applications. For example, Zero Knowledge Proof techniques can be utilised in smart contracts to build a private payment mechanism on a permissionless DLT network. See for example Aztec on Ethereum. There are also DLTs built specifically for privacy, such as the ZCash and Monero, which are implemented on permissionless DLT networks.

[1.9] Distributed Ledger Technology Components​

Any Distributed Ledger Technology (DLT) has a few specific components that we will discuss in this section.

Identity Management

In a distributed ledger, all data is linked to an identity (usually referred to as an account or address), to establish ownership. Accounts can be controlled by cryptographic key pairs or smart contract code.

In a permissionless DLT network, users can compute a cryptographic key pair locally on their machine (or outsource this feature to a centralised wallet provider). In permissionless DLT networks, an individual user can control one or more ledger identities.

In permissioned DLT networks, identity is usually more tightly controlled. Even though users can still compute a cryptographic key pair locally on their machine, they may also be required to be certified by a certificate authority before they can submit transactions to change the ledger.

Identity will be further discussed in section 3.

Ledger State and Ledger Updates

In a distributed ledger, its current and possible future states are defined by its virtual machine. Originally, each DLT could only have one associated virtual machine, but nowadays there is more flexibility. Properties of distributed ledger virtual machines, include:

  • State Model: How does the virtual machine model the current and future state?
  • State Transition Function: How are valid transactions formed and how are they processed by the virtual machine?
  • User Defined Code: Can user defined code (commonly referred to as smart contracts) be added to the ledger and processed by the virtual machine?
  • Turing Complete Smart Contracts: Can users write smart contracts using a Turing Complete language?

The Ethereum virtual machine was the first instance of a virtual machine that allowed Turing complete smart contracts to be added onto a permissionless DLT network, but nowadays there are a few others such as the web assembly virtual machine used by the Substrate DLT.

Ledger state and updates will be discussed in section 2

Transaction Ordering

Distributed Ledger Technologies can have different ways to order the transactions to update the ledger.

Blockchain Ordering

A blockchain requires transactions to be grouped together in blocks. Each block must be linked to one previously accepted block. An accepted block is one that the DLT nodes have agreed to process to update the distributed ledger. For a blockchain, every DLT node must process each block in sequence order and for each block, every transaction must be processed in sequence order. Examples of DLTs using a blockchain include Bitcoin, Ethereum and the XRP ledger.

Directed Acyclic Graph Ordering

A Directed Acyclic Graph (DAG) does not group transactions into blocks. Instead, each transaction must be linked to other accepted transactions. An accepted transaction is one that the DLT nodes have agreed to process to update the distributed ledger. This DAG structure allows each DLT node to process transactions in different orders. In fact, a DLT node may not need to process large subgraphs of the DAG at all, if the transactions in these subgraphs do not affect parts of the ledger that the DLT node is concerned with. An example of a DLT using a DAG is Corda.

Block DAG Ordering

A block DAG groups transactions into blocks. Each block must be linked to at least one accepted block. Therefore, blocks become connected in a DAG structure. This block DAG structure allows DLT nodes to process blocks in different orders. Similarly to DAGs, a block DAG allows a DLT node to not process large subgraphs if the transactions in the blocks of the subgraph do not affect parts of the ledger that the DLT node is concerned with. An Example of a DLT using a block DAG is Hashgraph (where blocks are called events or transaction containers).

Selectable Ordering

Finally, some DLTs allow a selectable transaction ordering method. An example of a DLT with a selectable ordering system is Avalanche, which orders via a blockchain on it’s c-chain implementation and via a DAG on its x-chain implementation.

Consensus Protocol

The DLT nodes of each DLT network must come to a system wide agreement on the order of transactions to update the distributed ledger. This allows each DLT node’s ledger data to be (at least eventually) consistent. To do so, the DLT nodes follow a set of rules and procedures defined in the DLT network’s consensus protocol (sometimes referred to as a consensus mechanism).

Each consensus protocol must have the following two main components.

Anti-Sybil Mechanism
The first main component is the anti-sybil mechanism. The purpose of an anti-sybil mechanism is to prevent individual DLT nodes from replicating themselves to increase their influence on how the entire DLT network reaches consensus. There are many proposed anti-sybil mechanisms, the most prevalent ones in permissionless DLT networks are currently:

  • Proof-of-Work: Where each single DLT node is weighted according to the amount of computation power it has. Should the operator of this single DLT node instead create multiple DLT nodes, the operator would now have to divide his available computation power between all of his DLT nodes. This would mean that the collection of all the DLT nodes would have an equal weight when compared to the single DLT node deployment, which implies that the operator gains no benefit, from a consensus perspective, of deploying multiple nodes.
  • Proof-of-Stake: Where each single DLT node is weighted according to the amount of a certain token it has. Should the operator of this single DLT node instead create multiple DLT nodes, the operator would now have to divide his available tokens between all of his DLT nodes. This would mean that the collection of all the DLT nodes would have an equal weight when compared to the single DLT node deployment, which implies that the operator gains no benefit, from a consensus perspective, of deploying multiple nodes.

Communication Rules

A consensus protocol’s second main component is its distributed ledger data communication rules. These rules detail what functions a DLT node should run:

  • Upon Receiving Messages: When receiving distributed ledger data (transactions, blocks) from other nodes or direct from ledger users. For instance, what checks should a node perform in order to verify a transaction or block received from another node before using it to update the ledger?
  • Before Sending Messages: When broadcasting distributed ledger data (transactions, blocks) to other DLT nodes. For instance, how can a correctly formatted block of transactions be created?

Finality of Ledger Updates

A consensus protocol’s communication rules will be designed for deterministic or probabilistic transaction finality.

  • Deterministic Transaction Finality: When a transaction is accepted to update the ledger, it can never be rejected.
  • Probabilistic Transaction Finality: When a transaction is accepted to update the ledger, it maybe eventually rejected. Overtime, the likelihood of rejection decreases.

Probabilistic transaction finality generally promotes liveness of the ledger. Where the ledger can always be read and updated. Whereas deterministic transaction finality generally promotes safety, where correct data will always be read from the ledger (but there can be periods when the ledger cannot be updated).

Naming

Consensus protocols are commonly referred to according to their anti-sybil mechanism, even though the actual communication rules may differ. For example, both Bitcoin and Bitcoin Cash use a proof-of-work anti-sybil mechanism, and both are usually referred to as having a proof-of-work consensus protocol. But Bitcoin and Bitcoin Cash differ in their communication rules, where nodes of a Bitcoin Cash DLT network are allowed to create and send larger blocks of transactions compared to nodes of a Bitcoin DLT network.

Incentivisation

DLT nodes are incentivised to follow a consensus protocol. If a DLT node deviates from the consensus protocol, it’s copy of the ledger will go out of sync with others, leading to this DLT node being removed from its original DLT network and instead establishing a new DLT network for itself and any other DLT nodes that follow this deviation.

There is also a positive motivation for DLT nodes to follow a consensus protocol of a permissionless DLT network. That is, if a DLT node successfully produces an accepted block (or an accepted transaction for DAG ordering systems), then it will receive a cryptocurrency reward and this cryptocurrency will hold real economic value. This reward comes from:

  • The generation of new cryptocurrency coins; and/or
  • Collecting transaction fees from the newly accepted transactions.

If a DLT node deviates from a consensus protocol, thereby establishing a new DLT network, by definition the cryptocurrency given on the new DLT network as a reward will not be the same coin as on the original DLT network. This new cryptocurrency will usually be valued significantly less compared to the reward on the original DLT network.

In permissioned DLT networks, nodes will not have these cryptocurrency rewards. But DLT nodes in permissioned DLT networks have their identity established and verified, therefore these nodes have additional incentives from matters such as reputation or legal responsibility.

[1.10] Read Data from Distributed Ledgers​

Note that the external exercise linked to in this page is a non-essential element of the course. It is provided for your benefit to build your blockchain developer skills. FutureLearn advises you that interacting with the content may require or result in personal data transfer to this external website; FutureLearn recommend that you check external websites privacy policies before use.

Overledger API Interaction

Overledger allows two forms of interaction:

  • Preparation and Execution: One API call to prepare the request. Another API call to confirm the request.
  • AutoExecute: One API call to prepare and confirm the request.

Overledger requires a preparation phase as Overledger’s API data model is standardised across different DLTs, and so the preparation allows a mapping to occur between the Overledger data model and the native data model of each DLT.

The preparation phase can be split from the execution phase, if the user wants to check these mappings have occurred correctly. These split phases also allow a client side audit trail of the mappings to be saved.

For the rest of the examples in the course we will use the AutoExecute interaction type as our development exercises are only for training and not for production systems.

Reading DLT Network Data via Overledger

In our first example, we will use Overledger to read blocks from multiple DLT networks: Bitcoin testnet, Ethereum Ropsten testnet and the XRP Ledger testnet. We will be using the permissionless testnet versions of these DLTs, due to the assumed DLT inexperience of developers using this course, and we don’t want any real value to accidentally be sent in later exercises.

For this task, go the second exercise link on our Github repository in a new tab and follow the step by step instructions.

Top tip: Consider copying objects printed out in the terminal to a JSON formatter such as here in order for you to better visualise the objects being returned from Overledger.

[1.11] DLT Nodes​

One Distributed Ledger Technology (DLT) can have many different DLT nodes implementations. For example, Ethereum has Geth, Besu and Nethermind implementations (among others). Each implementation must follow the DLT consensus protocol, otherwise they will not be compatible with other nodes on the network (leading to a network partition). But each implementation has the flexibility to implement the internal aspects of the consensus protocol in different ways (such as using a more efficient internal database).

A benefit to the DLT of having multiple DLT node implementations is that a technical issue in one version has a less chance to bring down the entire DLT network. Whereas a negative of having multiple implementations is that there is more technical debt for every change to the DLT consensus protocol.

Regardless of implementation, each DLT node must include the following components.

Network Connections

DLT nodes contain the functionality to establish network connections to other DLT nodes. That is, to search for other available DLT nodes and verify them. In order to simplify this discovery process, DLT node software may contain hardcoded information on the location of reliable DLT nodes (known as boot nodes).

Once network connections to other DLT nodes are established, messages can be propagated to and received from other DLT nodes. The creation or processing of these messages that relate to the distributed ledger are handled by the node’s consensus protocol implementation.

Identity Management

DLT nodes can contain ledger identities (e.g. cryptographic key pairs that control an account in the ledger) in a wallet stored locally. Additionally, DLT nodes can contain the ability to create and authorise new identities.

The fact that DLT nodes are heavily involved in ledger identity management is one of the key reasons that access to DLT nodes needs to be protected.

Ledger State

Different categories of DLT nodes correspond to the amount of state that they hold:

  • Light nodes: Light nodes only hold a reference to the ledger state (e.g. a Merkle tree root of the state). To find out further information on the current state, the light node must ask a node of a different category.
  • Full nodes: Full nodes hold information on the current ledger state. They may also hold information on some recent historical ledger states.
  • Archive nodes: Archive nodes hold information on the current ledger state and also many (usually all) historical ledger states. Light nodes vastly cut down on the storage space required to run the DLT node software, but they do so at the expense of having to rely on other nodes for data queries.

Note that full and archive nodes are not guaranteed to hold the entire ledger state at any particular time. Instead, this is a DLT network specific issue as each network may have DAG or block DAG based transaction ordering, allowing state to be partitioned between the DLT nodes. Additionally, there could also be private channels between DLT nodes obscuring some data.

Ledger Updates, Transaction Ordering & Consensus Protocols

Each full or archive DLT node has its own (at least partial) copy of the ledger. A DLT node will update its local copy, when they receive a new accepted block of transactions (if using the blockchain or block DAG transaction ordering systems), or when they receive a new accepted transaction (if using the DAG transaction ordering system).

Each DLT node must follow the rules of the consensus protocol in order to keep their copy of the ledger in sync with the copies of other DLT nodes.

Finally, DLT nodes may also keep their own local copy of the transactions that are waiting to be ordered to update the ledger (known as unconfirmed transactions). As the set of unconfirmed transactions is unordered, this set will therefore differ between DLT nodes. Each DLT node’s unconfirmed transaction set will include transactions this node has either received from other nodes or received directly from a user.