Expand description
A Rust Programming Language library for Byzantine Fault Tolerant (BFT) State Machine Replication (SMR), intended for production systems.
§Features
- Pluggable components: library users get to provide their own business logic
(
app
), key-value storage (kv_store
), and peer-to-peer networking (network
). HotStuff-rs is agnostic to these details and can therefore be adapted to many use-cases. - Dynamic validator sets:
App
s can control and change the set of replicas (“validators”) that vote to replicate the state machine without any downtime or manual configuration. Unlike older generations of SMR, “validator set updates” are not stop-the-world operations. - View synchronization: HotStuff-rs implements Byzantine View Synchronization in the
pacemaker
module. This enables it to quickly bring replicas’ view number in sync and therefore recover quickly from a network disruption. - Rigorous specification: HotStuff-rs is not only rigorously specified, but its specification is integrated into its Rustdocs so that they remain up-to-date with the implementation. This includes proofs of correctness for each of its subprotocols.
§Getting started
To replicate your application using HotStuff-rs, you need to represent it as a type and then
have the type implement the App
trait.
Then, initialize
the replica’s storage,
build
a ReplicaSpec
,
and then start
the replica.
An example on how to start a replica can be found in the rustdocs for
the replica
module.
§Understanding HotStuff-rs
The best way to understand HotStuff-rs is by reading these Rustdocs (so be happy, you are already at the right place!).
HotStuff-rs’ Rustdocs are designed to be both documentation, and specification. This means that it does not only describe the current (v0.4.0) version of this (Rust) implementation of HotStuff-rs, but also prescribes a protocol that developers can use to implement BFT SMR libraries that are compatible with HotStuff-rs.
Before starting to read these Rustdocs, it is helpful to have a good high-level awareness of the concepts behind HotStuff-rs, as well as a high-level picture of how the library’s modules are organized.
The Concepts subsection helps with the former by introducing a slew of the important concepts in a way gives a sense of the relationships between them, as well as linking to further reading resources whenever they are available, while the Organization subsection helps with the latter by briefly describing how the module tree is structured.
§Concepts
The library user implements their business logic in an App
and provides a handle to it
to HotStuff-rs via the app
setter on ReplicaSpecBuilder
. The
library user then start
s a replica.
Upon startup, the replica starts a background thread called the algorithm
thread. The algorithm
thread then creates instances of HotStuff-rs’ three subprotocols–namely hotstuff
,
pacemaker
, and block_sync
–and starts to poll these instances in an infinite loop, driving
each subprotocol forward.
The Pacemaker subprotocol works to keep track of an increasing counter called the “view number”, which is a sort of logical clock that the algorithm thread uses to decide the replica should do at any moment in time. At any given view, a replica could either be a validator, or a listener. Suppose that it is a validator. Then, it needs to play a role in the HotStuff subpprotocol.
The two main roles validator can play in the HotStuff subprotocol are: Proposer, and Phase-Voter. These two roles are not mutually exclusive. For example, it could be that in a given view, a validator is both a proposer, and a phase-voter.
Suppose that a validator is a proposer. Then, it will begin the view by calling
produce_block
to ask the App
to generate the contents of a new
Block
. Then, it will pack the block in a
Proposal
and broadcast this to all other replicas through the
user-provided P2P Network
.
Suppose further that the validator is also phase-voter. Then, after broadcasting the proposal, the
validator will wait to receive a proposal from the Network
. Upon receiving a Proposal
(which
could be the same Proposal
it just itself proposed), it will call
validate_block
to ask the App
to ask it to validate the block
according to user-defined application-level semantics.
If the block is valid, then the replica will create a PhaseVote
by signing a message with its SigningKey
, and send this
to the phase_vote_recipient
. Then, it will insert the
block to its Block Tree, which the library stores in the
KVStore
provided by the user.
The overall result is an immutable “block chain” that is guaranteed to satisfy safety and liveness
invariants
as long as no more than 1/3rd of the
TotalPower
of a
ValidatorSet
is faulty.
As all of this is happening, the algorithm constantly emits events
, which user code can receive
as they occur by registering event handlers.
§Module organization
HotStuff-rs’ modules are organized into two levels depending on whether the definitions they contain are subprotocol-specific, or commonly used across multiple subprotocols.
The three subprotocols of HotStuff-rs and their associated modules are:
- HotStuff (
hotstuff
): the subprotocol for committing blocks. - Pacemaker (
pacemaker
): the subprotocol for bringing replicas into the same view so that they can start committing blocks. - Block Sync (
block_sync
): the subprotocol for bringing replicas’ block trees up-to-date after periods of time in which they miss messages.
Modules not listed above not subprotocol-specific.
Modules§
- Pluggable, replicable applications.
- Subprotocol that “catches-up” the local block tree in case the replica misses out on messages.
- The persistent state of a replica.
- Thread that receives events emitted by the algorithm and block sync server threads and passes them to event handlers.
- Subscribable events that are published when significant things happen in the replica.
- Subprotocol for committing
Block
s. - Functions that log out events.
- Pluggable peer-to-peer (P2P) networking.
- Subprotocol for synchronizing view numbers.
- Methods for building and running a replica as well as initializing its storage.
- Types and traits that are used across multiple sub-protocols or components of HotStuff-rs.