A robust execution client for Ethereum node operators.
This version introduces a new approach to the state database: half-path. It replaces the previous approach, which we will call hash for these notes.
The goal of half-path is to improve the performance of the existing database without significant codebase changes. It is a middleware between the hash and full-path-based designs (which we are currently working on). It is mainly aimed at validators' performance (we observed a significant improvement in effectiveness) and, as a side effect, improving archive nodes' sync.
--Init.StateDbKeyScheme Hash
.Env variable equivalent of this flag is: NETHERMIND_INITCONFIG_STATEDBKEYSCHEME=HalfPath/Hash
--Init.StateDbKeyScheme HalfPath
— without it, pruning will behave as before, so it will recreate the state with the previous database approach. To reduce slowdown, the user should increase the default full pruning memory budget to at least 8 GB: --Pruning.FullPruningMemoryBudgetMb 8000
.Learn more
More details about the half-path and other state redesigns which we are still exploring, can be found here. Please also refer to #6331, which contains all of the changes around the half-path and more in-depth details about specific performance metrics.
We're excited to share that Nethermind now supports snap serving, a capability previously unique to Geth. This is a big step for the Ethereum ecosystem, offering redundancy and easing Geth's considerable responsibility in maintaining network health.
This achievement doesn't diminish Geth's critical role but reinforces our shared goal of a resilient Ethereum. We see this as an opportunity to distribute the workload, ensuring the network's strength through diversity.
This feature was previously present but required a change to the database layout for efficient performance. Nodes running on the old hash approach will not enable snap serving by default. Snap serving's performance is too low on this design and may affect the node's overall performance. If a node operator wants to enable it on hash design, there is an option --Sync.SnapServingEnabled true
, although we don't recommend it.
During snap serving for other nodes, the overall performance of the node may be slightly worse. Please reach out to us on Discord with all observations so more optimizations can be discovered and applied. At the same time, it is advised to keep it enabled for the sake of network health.
This release introduces a new chiseled rootless Docker image for enhanced security.
These image tags have the -chiseled
suffix and run the Nethermind process on behalf of the non-root app
user with UID/GID of 64198
.
Nethermind no longer needs the Snappy (libsnappy-dev) to be installed on Linux.
--Sync.FastBlocks
has been deprecated. It is advised to remove this option from your custom configs, environment variables, or flags passed to the client. See #6792 for more details.
Multiple database and networking metrics are now deprecated and will be removed in the next release. These metrics follow the format:
nethermind_*_db_size
nethermind_*_db_reads
nethermind_*_db_writes
nethermind_*_disconnects
nethermind_*_received
nethermind_*_sent
They have been consolidated to labeled metrics:
nethermind_db_size{db=*}
nethermind_db_reads{db=*}
nethermind_db_writes{db=*}
nethermind_local_disconnects_total{reason=*}
nethermind_remote_disconnects_total{reason=*}
nethermind_incoming_p2p_messages{message=*}
nethermind_outgoing_p2p_messages{message=*}
New features
Optimism
Performance
Bug fixes and stability
Other changes
readonly struct
BlockExecutionContext
via in
by @benaadams in https://github.com/NethermindEth/nethermind/pull/6469
Full Changelog: https://github.com/NethermindEth/nethermind/compare/1.25.4...1.26.0
This version introduces a new approach to the State database - HalfPath. It replaces the previous approach which for a purpose of these notes we will call “Hash”.
The goal of HalfPath is to improve performance of the existing database without major codebase changes. It is a middleware between Hash design and a full Path-based design (on which we are currently working on). It is mostly aimed for validators performance (observed major effectiveness improvement) and as a side effect improve archive nodes sync.
Major achievements:
Decisions made by the team to make a transition to HalfPath as smooth as possible:
Init.StateDbKeyScheme=Hash
.Migration to HalfPath can be done in two ways:
Init.StateDbKeyScheme=HalfPath
- without it pruning will behave as before so will recreate state with previous DB approach. User should increase the default full pruning memory budget to at least 8GB (—Pruning.FullPruningMemoryBudgetMb 8000
) to reduce slowdown.Upgrading from previous version to 1.26.0 will result with:
Learn more
More details about HalfPath and other state redesigns which we are still exploring can be found in this [Medium blogpost](https://medium.com/nethermind-eth/nethermind-client-3-experimental-approaches-to-state-database-change-8498e3d89771). Also please refer to this [Pull Request](https://github.com/NethermindEth/nethermind/pull/6331) which contains all of the changes around HalfPath and more in-depth details about specific performance metrics.
We're excited to share that Nethermind now supports Snap Sync, a capability previously unique to Geth. This is a big step for Ethereum's ecosystem, offering redundancy and easing the considerable responsibility Geth has held in maintaining network health.
This achievement doesn't diminish Geth's critical role but reinforces our shared goal of a resilient Ethereum. We see this as an opportunity to distribute the workload, ensuring the network's strength through diversity.
The feature was previously present, but it required a change to the database layout for efficient performance.
Nodes running on old Hash approach will not have a SnapServing enabled by default. Performance of SnapServing is very bad on this design and may overall performance of a node.
If for any reason node operator wants to enable it on Hash db, there is a flag Sync.SnapServingEnabled=true
which will make it possible but is not advised.
Important During snap serving for other nodes in network overall performance of node may be slightly worse. Please reach to the team on Discord with all observations out there so more optimizations around it can be discovered and applied - at the same time it is advised to keep it enabled to keep the network more healthy.
New features
Optimism
Performance
Bug fixes and stability
Other changes
readonly struct
BlockExecutionContext
via in
by @benaadams in https://github.com/NethermindEth/nethermind/pull/6469
Full Changelog: https://github.com/NethermindEth/nethermind/compare/1.25.4...1.26.0-exp.4
🚧 THIS IS AN EXPERIMENTAL RELEASE. We don't recommend running it on your main validator infrastructure. It brings major improvements, but we're still unsure about its downsides and long-term effects. The latest stable version is 1.25.4.
IMPORTANT To fully benefit from this release, you must resync your nodes from scratch. Half-path is now the default sync mode for new nodes. If you start this version on a node with an old database schema, it will continue using the old schema without improvements. This is not recommended.
Next HalfPath upgrade - This is a next release which brings an experimental state redesign called "HalfPath". More details can be found in 1.26.0-exp.2 and in our blog post
SNAP SERVING - This version also introduces experimental (still under testing) mechanism for serving snap data! This is very important milestone done which will improve health of Ethereum (now only Geth is capable of serving Snap data so Nethermind can reduce very high responsibility of Geth and serve this data as well). Since we are still working on it users can experience some worse performance when snap data is served - we are working on optimizing it.
Dencun ready version for Experimental users - All users which are running experimental version are asked to migrate to this version since this is the only experimental version with Dencun support. All previous versions will stop working once hard fork occurs.
Full pruning doesn't work properly. You may experience very long pruning times, which at the end may not work as expected causing big block processing spikes - we are working on solution but it is not advised to use it at all at the moment.
⚠️
This release is a mandatory upgrade for all nodes operating on the Ethereum Mainnet and Gnosis. Please update your node to this version to ensure correct node functionality.
The Mainnet Dencun hard fork is scheduled on Mar 13, 2024 at 13:55:35 UTC (epoch 269568) ⚠️ Execution client and consensus client database sizes can increase by about 150GB over time - please prepare for that!
The Gnosis Dencun hard fork is scheduled on Mar 11, 2024 at 18:30:20 UTC (slot 14237696) ⚠️ Execution client and consensus client database sizes can increase by about 150GB over time - please prepare for that!
The PPA package has been revised, including the version and installers. Since the version has been fixed to be 1.25.4 instead of 1.2540, installing the latest version requires the manual removal of the previous one as follows:
# ⚠️ If your data directory is in the default location of /usr/share/nethermind, back it up before the package removal.
sudo apt-get purge -y nethermind
sudo apt-get update
debug_getBadBlocks
support by @kjazgar and @Marchhill in https://github.com/NethermindEth/nethermind/pull/3838
Metrics.ExposeHost
option so a hostname other than "0.0.0.0" can be exposed by @tgerring in https://github.com/NethermindEth/nethermind/pull/6528
:warning: For now we decided to keep default behavior so ExposeHost will default to "+" which means "0.0.0.0" but in the future we are going to change default of that flag to "127.0.0.1" to reduce risk of remote access to metrics.
Full Changelog: https://github.com/NethermindEth/nethermind/compare/1.25.3...1.25.4
⚠️
This release is a mandatory upgrade for all nodes operating on the following chains: Sepolia, Holesky, and Chiado.
Please update your node to this version to ensure correct node functionality.
The Sepolia Dencun hard fork is scheduled on Jan 30, 2024 at 22:51:12 UTC (epoch 132608)
The Chiado Dencun hard fork is scheduled on Jan 31, 2024 at 18:15:40 UTC (epoch 516608)
The Holesky Dencun hard fork is scheduled on Feb 07, 2024 at 11:34:24 UTC (epoch 29696)
Full Changelog: https://github.com/NethermindEth/nethermind/compare/1.25.2...1.25.3
🚧 THIS IS AN EXPERIMENTAL RELEASE. We don't recommend running it on your main validator infrastructure. It brings major improvements, but we're still unsure about its downsides and long-term effects. The latest stable version is 1.25.3.
This version brings many performance improvements, the most important being a major state database optimization called Half-path. This new optimization is not a full path-based storage, but the way it arranges data helps with the performance of block processing and state database size.
For more details on Half-path, see our blog post.
This hotfix addresses the consensus issue in Nethermind that was introduced in v1.23.0. This release is mandatory for all Nethermind users.
No resync required. A consensus client restart is required without the need to resync it. The v1.22.0 and below weren't affected.
A full postmortem will be published soon.
⛔ THIS VERSION HAS A CONSENSUS ISSUE. Use v1.25.2 or later instead.
⚠️ This version supports the upcoming Goerli Dencun hard fork that is scheduled on Jan 17, 2024 at 06:32:00 UTC (epoch 231680)
Please update your node to this version to ensure correct node functionality.
⚠️ This version supports the upcoming Chiado Dencun hard fork that is scheduled on Jan 31, 2024 at 18:15:40 UTC (epoch 516608)
Please update your node to this version to ensure correct node functionality.
eth_syncing
invalid behaviorIn v1.25.0, the eth_syncing
method misbehaves for those who upgraded from the older version to the latest one (returned "syncing" when a node was fully synced). This version addresses this issue, so it properly returns the sync status.
In v1.25.0, in some cases, the old bodies and old receipts started to download even though it should not have been the case on the synced nodes. This usually happens for older nodes that were synced a long time ago without any resync in the meantime.
Full Changelog: https://github.com/NethermindEth/nethermind/compare/1.25.0...1.25.1
⛔ THIS VERSION HAS A CONSENSUS ISSUE. Use v1.25.2 or later instead.
⚠️ This version supports the upcoming Goerli Dencun hard fork that is scheduled on Jan 17, 2024 at 06:32:00 UTC (epoch 231680)
Please update your node to this version to ensure correct node functionality.
The initial support of the OP in Nethermind was implemented right before the Canyon hard fork happened. Because of that, Nethermind nodes could not follow the chain after Canyon activation. Since this version, Nethermind supports Canyon hard fork on all OP-related chains.
We replaced the famous Json.NET library with the System.Text.Json implementation. As a result, we drastically reduced the memory overhead, improved the block processing time, and sped up JSON-RPC handling in general.
The debug_trace*
JSON-RPC methods now support custom tracers written in JavaScript. This allows custom tracing logic and is in line with the Geth implementation.
eth_getBlockReceipts
that is based on the previous Parity implementation and gives an easier way to get all receipts for transactions within a specified blockdebug_getRawBlock
debug_getRawReceipts
debug_getRawHeader
debug_getRawTransaction
These methods have been added to have a possibility to analyze encoded block/receipt/header/transactionOn newly synced nodes, instead of using barriers set for a block with deposit contract for beacon chain (11052984), the node will sync all bodies and receipts till genesis:
With the migrations to .NET 8, we got a performance boost in various areas and reduced memory usage, which is especially beneficial for validators.
New features
Cancun
ChainSpecBasedSpecProvider
by @smartprogrammer93 in https://github.com/NethermindEth/nethermind/pull/6409
BlobBaseFee
not consistent with geth when ExcessBlobGas
is null by @smartprogrammer93 in https://github.com/NethermindEth/nethermind/pull/6320
Optimism
Performance
Metrics
Logging
Bug fixes and stability
Other changes
Full Changelog: https://github.com/NethermindEth/nethermind/compare/1.24.0...1.25.0
⛔ THIS VERSION HAS A CONSENSUS ISSUE. Use v1.25.2 or later instead.
⚠️ WARNING
This version cannot be downgraded. Once you upgrade to this version or sync from scratch with it, you cannot downgrade to any previous version.
Improvements to the headers database
Supported networks adjustments
For more info, see the -c, --config <value>
flag and supported networks.
For Optimism networks, ensure that networking is disabled to reduce unnecessary hardware usage.
New features and important changes
Optimism
Bug fixes and stability
Full Changelog: https://github.com/NethermindEth/nethermind/compare/1.23.0...1.24.0