Sentry

Client that is run along side an Ethereum consensus client and collects data via the consensus client's Beacon API. You must run your own consensus client and this sentry will connect to it via the consensus client's http server.

This sentry can output events to various sinks and it is not a hard requirement to run the Xatu server.

Usage
Requirements
Configuration
Running locally

Usage

Sentry requires a config file.

Usage:
  xatu sentry [flags]

Flags:
      --config string   config file (default is sentry.yaml) (default "sentry.yaml")
  -h, --help            help for sentry

Requirements

Ethereum consensus client with exposed http server.
Optional server running with the Coordinator service enabled.

Configuration

Sentry requires a single yaml config file. An example file can be found here

Name	Type	Default	Description
logging	string	`info`	Log level (`panic`, `fatal`, `warn`, `info`, `debug`, `trace`)
metricsAddr	string	`:9090`	The address the metrics server will listen on
pprofAddr	string		The address the pprof server will listen on. When ommited, the pprof server will not be started
name	string		Unique name of the sentry
labels	object		A key value map of labels to append to every sentry event
ethereum.beaconNodeAddress	string		Ethereum consensus client http server endpoint
ethereum.beaconSubscriptions	array		List of topcis to subscribe to. If empty, all subscriptions are subscribed to.
ntpServer	string	`time.google.com`	NTP server to calculate clock drift for events
proposerDuty.enabled	bool	`true`	Publish each epoch's proposer duties (one duties request and 32 events per epoch)
beaconCommittees.enabled	bool	`false`	Fetch and publish each epoch's beacon committees
attestationData.enabled	bool	`false`	Fetch attestation data from the beacon node on a schedule
attestationData.interval.every	string	`12s`	How often to fetch attestation data when `attestationData.interval.enabled` is set
forkChoice.enabled	bool	`false`	Fetch the beacon node's fork choice on a schedule
forkChoice.interval.every	string	`12s`	How often to fetch the fork choice when `forkChoice.interval.enabled` is set
execution.enabled	bool	`false`	Watch the execution client's mempool. A watcher whose execution client is unreachable retries in the background. `xatu_sentry_execution_watcher_up{watcher}` is 1 while it runs with its websocket or block subscription up; one that stays down for 2 minutes rebuilds the execution client and every watcher
execution.rpcAddress	string		Execution client JSON-RPC address, required when `execution.enabled` is set
execution.websocketEnabled	bool	`false`	Subscribe to new pending transactions over the WebSocket
execution.wsAddress	string		Execution client WebSocket address, required when `execution.websocketEnabled` is set
execution.txPoolContentEnabled	bool	`true`	Poll `txpool_content` for pending transactions. Set to `false` on Besu
execution.fetchInterval	int	`45`	Seconds between `txpool_content` polls
execution.processingInterval	int	`3000`	Milliseconds between transaction processing batches
execution.rpcBatchSize	int	`15`	Transactions per batched RPC call
execution.maxConcurrency	int	`3`	Maximum concurrent batched RPC calls
execution.queueSize	int	`10000`	Size of the transaction processing queue
execution.processorWorkerCount	int	`40`	Transaction processing workers
execution.pruneDuration	int	`384`	Seconds to keep pending transactions before pruning
execution.stateSize.enabled	bool	`false`	Poll the execution client's state size (`debug_stateSize`)
execution.stateSize.triggerMode	string	`head`	When to poll the state size: `head`, `block` (needs `execution.websocketEnabled`) or `interval`
execution.stateSize.intervalSeconds	int	`12`	Seconds between state size polls in `interval` mode
outputs	array		List of outputs for the sentry to send data to
outputs[].name	string		Name of the output
outputs[].type	string		Type of output (`xatu`, `http`, `kafka`, stdout`)
outputs[].config	object		Output type configuration `xatu`/`http`/`kafka`

Output `xatu` configuration

Output configuration to send sentry events to a Xatu server.

Name	Type	Default	Description
outputs[].config.address	string		The address of the server receiving events
outputs[].config.tls	bool		Server requires TLS
outputs[].config.headers	object		A key value map of headers to append to requests
outputs[].config.maxQueueSize	int	`51200`	The maximum queue size to buffer events for delayed processing. If the queue gets full it drops the events
outputs[].config.batchTimeout	string	`5s`	The maximum duration for constructing a batch. Processor forcefully sends available events when timeout is reached
outputs[].config.exportTimeout	string	`30s`	The maximum duration for exporting events. If the timeout is reached, the export will be cancelled
outputs[].config.maxExportBatchSize	int	`512`	MaxExportBatchSize is the maximum number of events to process in a single batch. If there are more than one batch worth of events then it processes multiple batches of events one batch after the other without any delay

Output `http` configuration

Output configuration to send sentry events to a http server.

Name	Type	Default	Description
outputs[].config.address	string		The address of the server receiving events
outputs[].config.headers	object		A key value map of headers to append to requests
outputs[].config.maxQueueSize	int	`51200`	The maximum queue size to buffer events for delayed processing. If the queue gets full it drops the events
outputs[].config.batchTimeout	string	`5s`	The maximum duration for constructing a batch. Processor forcefully sends available events when timeout is reached
outputs[].config.exportTimeout	string	`30s`	The maximum duration for exporting events. If the timeout is reached, the export will be cancelled
outputs[].config.maxExportBatchSize	int	`512`	MaxExportBatchSize is the maximum number of events to process in a single batch. If there are more than one batch worth of events then it processes multiple batches of events one batch after the other without any delay

Output `kafka` configuration

Output configuration to send sentry events to a kafka server.

Name	Type	Default	Allowed Values	Description
outputs[].config.brokers	string			Comma delimited list of brokers. Eg: `localhost:19091,localhost:19092`
outputs[].config.topic	string			Name of the topic. Mutually exclusive with `topicPattern`.
outputs[].config.topicPattern	string			Topic template with variables `${EVENT_NAME}` or `${event-name}`. Mutually exclusive with `topic`.
outputs[].config.flushFrequency	string	`3s`		The maximum time a single batch can wait before a flush. Producer flushes the batch when the limit is reached.
outputs[].config.flushBytes	int	`1000000`		The maximum size (in bytes) of a single batch before a flush. Producer flushes the batch when the limit is reached.
outputs[].config.maxRetries	int	`3`		The maximum retries allowed for a single batch delivery. The batch would be dropped, if the producer fails to flush with-in this limit.
outputs[].config.compression	string	`none`	`none` `gzip` `snappy` `lz4` `zstd`	Compression to use.
outputs[].config.requiredAcks	string	`leader`	`none` `leader` `all`	Number of ack's required for a succesful batch delivery.
outputs[].config.partitioning	string	`none`	`none` `random`	Paritioning to use for the distribution of messages across the partitions.
outputs[].config.encoding	string	`json`	`json` `protobuf`	Serialization format for message values.

Simple example

name: example-instance-001

ethereum:
  beaconNodeAddress: http://localhost:5052

outputs:
- name: standard-out
  type: stdout

Xatu server output example

name: example-instance-002

ethereum:
  beaconNodeAddress: http://localhost:5052

outputs:
- name: xatu-output
  type: xatu
  config:
    address: localhost:8080

http server output example

name: example-instance-003

ethereum:
  beaconNodeAddress: http://localhost:5052

outputs:
- name: http-basic-auth
  type: http
  config:
    address: http://localhost:8080
    headers:
      authorization: "Basic Someb64Value"

kafka server output example

name: example-instance-004

ethereum:
  beaconNodeAddress: http://localhost:5052

outputs:
- name: kafka-sink
  type: kafka
  config:
    brokers: localhost:19092
    topic: events

Complex example with multiple outputs example

logging: "debug"
metricsAddr: ":9090"
pprofAddr: ":6060"

name: example-instance

labels:
  ethpandaops: rocks

ntpServer: time.google.com

ethereum:
  beaconNodeAddress: http://localhost:5052

outputs:
- name: log
  type: stdout
- name: xatu-server
  type: xatu
  config:
    address: localhost:8080
    headers:
      authorization: Someb64Value
    maxQueueSize: 51200
    batchTimeout: 5s
    exportTimeout: 30s
    maxExportBatchSize: 512
- name: kafka-sink   
  type: kafka
  config:
    brokers: localhost:19092
    topic: events

Running locally

# docker
docker run -d --name xatu-sentry -v $HOST_DIR_CHANGE_ME/config.yaml:/opt/xatu/config.yaml -p 9090:9090 -it ethpandaops/xatu:latest sentry --config /opt/xatu/config.yaml
# build
go build -o dist/xatu main.go
./dist/xatu sentry --config sentry.yaml
# dev
go run main.go sentry --config sentry.yaml

Navigation Menu

Repository navigation

sentry.md

sentry.md

Sentry

Table of contents

Usage

Requirements

Configuration

Output `xatu` configuration

Output `http` configuration

Output `kafka` configuration

Simple example

Xatu server output example

http server output example

kafka server output example

Complex example with multiple outputs example

Running locally

FilesExpand file tree

sentry.md

Latest commit

History

sentry.md

File metadata and controls

Sentry

Table of contents

Usage

Requirements

Configuration

Output xatu configuration

Output http configuration

Output kafka configuration

Simple example

Xatu server output example

http server output example

kafka server output example

Complex example with multiple outputs example

Running locally

Output `xatu` configuration

Output `http` configuration

Output `kafka` configuration