Splunk Connect for Ethereum
Splunk Connect for Ethereum has not been fully tested with changes resulting from the move from Proof-of-Work (POW) to Proof-of-Stake (POS), more commonly known as the Merge, that occurred on September 15, 2022. There should be no impact to permissioned ethereum-based networks, but POS-based Ethereum mainnet has not been fully tested.
Splunk Connect for Ethereum (aka ethlogger
) makes it easy to ingest data about Ethereum ledgers, node information, and node metrics into another system for analysis. Currently, it can log to the Splunk HTTP Event Collector and stdout. It can also deserialize transactions and events if given a contract ABI. Some benefits include:
If you want to learn more about Splunk's efforts to make blockchains stable, secure, and scalable raise an issue here or email us at [email protected].
Prior to running Splunk Connect for Ethereum you will need to ensure your Splunk server has been configured with an HTTP Event Collector (HEC) endpoint and token and that the token specified has the ability to write data to the indexes provided in your configuration settings.
You will need 2 separate indexes; 1 for events and 1 for metrics.
$ ethlogger [...options]
Details about ethlogger's command-line usage in the CLI docs
Find out how to configure ethlogger in the configuration docs.
Splunk Connect for Ethereum can be run in a docker container. You can pull the image from the GitHub container registry.
Example:
$ docker run -it ghcr.io/splunkdlt/ethlogger:latest \
--eth-rpc-url=https://rpc.gnosischain.com\
--start-at-block=latest \
--hec-url=https://mysplunkserver.com:8088 \
--hec-token=123-123-123-123 \
--hec-events-index=main \
--hec-metrics-index=metrics
There is also an example on how to run ethlogger in docker-compose. This example will spin up a container for Splunk, install the Splunk App for Ethereum, and start a container for Splunk Connect for Ethereum. Ethereum data will start flowing into Splunk within a few minutes starting from the latest block. To get started follow these steps:
$ curl -O https://raw.githubusercontent.com/splunk/splunk-connect-for-ethereum/main/examples/docker-compose-basic/docker-compose.yaml
$ docker-compose up
Wait for all containers to start (typically just a few minutes).
You can rely on the output of docker ps
to see the state of services. To see the logs use docker logs ethlogger
and docker logs splunk
.
Login to Splunk and check out the Splunk App for Ethereum to explore the data ingested by ethlogger. Login using user admin
and password changeme
and start exploring! The Splunk App for Ethereum has a number of dashboards to help get you started quickly.
Note: if you change the index name please update the ethereum_index
macro so the dashboards will populate.
Create a new index named metrics
that is a metric
type of index. This will be used to gather monitoring metrics for ethlogger itself.
You can create the index by choosing Settings and then Indexes from the menu in Splunk. Once created, check out the Node Health - SC4Ethereum dashboard under the Node Monitoring
menu.
Shut it down and clean up for next time.
$ docker-compose down
$ rm checkpoints.json
$ docker volume prune
There's a lot of information available via RPC on Ethereum nodes, but they need to be enabled via the command line on startup for ethlogger
to connect. See the relevant docs for:
--http.api admin,db,eth,debug,miner,net,shh,txpool,personal,web3
--http.api quorum,istanbul,raft
(note: only one of istanbul
or raft
is active depending on the consensus method)ethlogger
is running on is whitelisted in the --http.vhosts
cli setting.--jsonrpc-apis APIs
--rpc-http-enabled --rpc-http-api admin,eth,debug,miner,net,txpool,priv,trace,web3
Thank you for considering to contribute to Splunk Connect for Ethereum! Please read the contribution guidelines to get started.