Soramitsu Kagome Save

Intro

KAGOME is a Polkadot Host (former Polkadot Runtime Environment) developed by Quadrivim and funded by a Web3 Foundation grant and Treasury proposals ( 1, 2).

Status

• JSON-RPC (compatible with Polkadot JS)
  • New unstable JSON-RPC
• Scale codec
• Synchronizer
  • Full sync
  • Fast sync
  • Warp sync
• Transaction pool
• Consensus
  • BABE
  • GRANDPA
  • SASSAFRAS
• Storage
  • Blockchain
    • Block storage
    • Block tree
    • Digest tracker
  • Trie storage (merkle trie)
  • RocksDB
  • Dynamic pruning
  • Trie nodes caches
  • State caches
• Runtime
  • Host API
  • WASM engine
    • Binaryen
    • WAVM
    • WasmEdge
• Parachains core
  • Non-asynchronous Backing
  • Data availability
  • Approval voting
  • Disputes resolution
  • Async-Backing
  • Elastic scaling
• Networking
  • Peer manager
    • /dot/block-announces/1
    • /paritytech/grandpa/1
    • /polkadot/validation/1
    • /polkadot/collation/1
    • /dot/transactions/1
    • /polkadot/req_collation/1
    • /dot/light/2
    • /polkadot/req_pov/1
    • /dot/state/2
    • /dot/sync/warp
    • /polkadot/req_statement/1
    • /dot/sync/2
    • /polkadot/req_available_data/1
    • /polkadot/req_chunk/1
    • /polkadot/send_dispute/1
  • Libp2p
    • Transport
      • TCP
      • QUIC
      • WebRTC
    • Secure connection
      • Noise
      • TLS
    • Multiplexing
      • Yamux
    • Multiselect protocol
    • Peer discovery
      • Kademlia
    • Ping protocol
    • Identify protocol
• Offchain workers
• Keystore
• Telemetry support
• Prometheus metrics

More details of KAGOME development can be found within the supported features section and in projects board

Getting Started

Build

Prerequisites

If you are using a Debian Linux system, the following command allows you to build KAGOME:

git clone https://github.com/qdrvm/kagome
cd kagome
sudo chmod +x scripts/init.sh scripts/build.sh

sudo ./scripts/init.sh
./scripts/build.sh

You will get KAGOME binary in the build/node/ folder

Other make commands are:

make docker
make command args="gcc --version"
make release
make release_docker
make debug_docker
make clear

Using KAGOME

Obtaining database snapshot (optional)

In order to avoid syncing from scratch we are maintaining the most recent snapshot of Polkadot network for KAGOME node available for anyone here: https://drive.google.com/drive/folders/1pAZ1ongWB3_zVPKXvgOo-4aBB7ybmKy5?usp=sharing

After downloading the snapshot you can extract it in the folder where the node will be running:

unzip polkadot-node-1.zip

Execute KAGOME Polkadot full syncing node

You can synchronize with Polkadot using KAGOME and obtain an archive node that can be used to query the Polkadot network at any state.

To launch KAGOME Polkadot syncing node execute:

cd examples/polkadot/
PATH=$PATH:../../build/node/
kagome --chain polkadot.json --base-path polkadot-node-1

Note: If you start KAGOME for the first time, you can add the --sync Fast flag to synchronize using Fast sync

After this command KAGOME will connect with other nodes in the network and start importing blocks. You may play with your local node using polkadot js apps: https://polkadot.js.org/apps/?rpc=ws%3A%2F%2F127.0.0.1%3A9944#/explorer

You will also be able to see your node on https://telemetry.polkadot.io/. If you need to identify it more easily you can add --name <node-name> flag to node's execution command and find your node in telemetry by typing its name.

Run kagome --help to explore other CLI flags.

Execute KAGOME block_validator node in development mode

The easiest way to get started with KAGOME is to run it in development mode, which is a single node network:

kagome --dev

That executes node with default accounts Alice and Bob. You can read about these accounts here.

To launch with wiping existing data you can do:

kagome --dev-with-wipe

Run KAGOME node in validator mode

To start the KAGOME validator:

cd examples/first_kagome_chain
PATH=$PATH:../../build/node/
kagome --validator --chain localchain.json --base-path base_path

This command executes a KAGOME full node with an authority role.

Run KAGOME with collator

Read this tutorial

Configuration Details

To run a KAGOME node, you need to provide it with a genesis config, cryptographic keys, and a place to store db files.

• Example of a genesis config file can be found in examples/first_kagome_chain/localchain.json
• Example of a base path dir can be found in examples/first_kagome_chain/base_path
• To create database files, just provide any base path into kagome executable (mind that start with authority role requires keys to start).

Contributing Guides

Please refer to the Contributor Documentation.

Modules

• api
  • JSON-RPC based on specification from PSP and Polkadot JS documentation. Uses HTTP and WebSockets for transport.
• application
  • Implements logic for running KAGOME node, such as processing CLI flags and defining an execution order of different modules
• assets
  • Artifacts needed to run in development mode
• authority_discovery
  • Logic for finding peer information by authority id provided
• authoriship
  • Mechanism for building blocks from extrinsics provided by the transaction pool
• blockchain
  • Implements blockchain logic such as fork handling and block digest information processing
• clock
  • Implements a clock interface that is used to access the system time
  • Implements timer interface that is used to schedule events
• common
  • A set of miscellaneous primitives, data structures, and helpers that are widely used in the project to simplify development
• consensus
  • Implementation of BABE block production mechanism
  • Implementation of Grandpa finality gadget
• containers
  • An implementation of a container that serves as an allocator for frequently created objects
• crypto
  • Crypto primitives implementation such as DSAs, Hashers, VRF, Random generators
• filesystem
  • A convenient interface for working with directories in the project
• host_api
  • Host APIs exposed by the Polkadot runtime as WASM import functions needed to handle storage, cryptography, and memory
• injector
  • A mechanism for matching class interfaces to their corresponding implementations using the Boost dependency injector
• log
  • A configuration of KAGOME logger
• macro
  • Convenience macros
• metrics
  • Prometheus metrics to retrieve KAGOME node execution statistics
• network
  • A set of networking substream protocols implementation on top of cpp-libp2p library
• offchain
  • Offchain worker implementation
• outcome
  • Convenient functions for boost outcome
• parachain
  • Parachains logic such as collation protocol, backing, availability and validity
• runtime
  • Integration of Binaryen and WAVM WebAssembly engines with Host APIs
• scale
  • Scale codec for some primitives
• storage
  • Storage and trie interfaces with RocksDB implementation
• subscription
  • Subscription engine
• telemetry
  • Connection to telemetry such as https://telemetry.polkadot.io/
• transaction_pool
  • Pool of transactions to be included into the block
• utils
  • Utils such as profiler, pruner, thread pool

You can find more information about the components by checking reference documentation. Check out tutorials and more examples in official documentation: https://kagome.readthedocs.io/

KAGOME in media

• Press-release: Soramitsu to implement Polkadot Runtime Environment in C++
• KAGOME: C++ implementation of PRE presentation at DOTCon (18.08.19)
• KAGOME and consensus in Polkadot presentation (in Russian) during Innopolis blockchain meetup (28.10.19)
• Web3 Builders: Soramitsu | C++ Implementation of Polkadot Host Polkadot's Web3 builders online presentation
• Building alternative clients Polkadot Decoded 2023 presentation