diff --git a/_docs-4/configuration/client-properties.md b/_docs-4/configuration/client-properties.md index d5618f84e..d7a8f05f7 100644 --- a/_docs-4/configuration/client-properties.md +++ b/_docs-4/configuration/client-properties.md @@ -17,7 +17,7 @@ Below are properties set in `accumulo-client.properties` that configure [Accumul | auth.type | password | 2.0.0 | Authentication method (i.e password, kerberos, PasswordToken, KerberosToken, etc) | | auth.principal | *empty* | 2.0.0 | Accumulo principal/username for chosen authentication method | | auth.token | *empty* | 2.0.0 | Authentication token (ex. mypassword, /path/to/keytab) | -| batch.writer.durability | default | 2.0.0 | The durability used to write to the write-ahead log. Legal values are: none, which skips the write-ahead log; log, which sends the data to the write-ahead log, but does nothing to make it durable; flush, which pushes data to the file system; and sync, which ensures the data is written to disk. Setting this property will change the durability for the BatchWriter session. A value of "default" will use the table's durability setting. | +| batch.writer.durability | default | 2.0.0 | The durability of writes to tables includes ensuring that mutations written by clients are persisted in the write-ahead log and that files written during a compaction are persisted to disk successfully. This property only configures the durability used to write to the write-ahead log. Legal values are: none, which skips the write-ahead log; log, which sends the data to the write-ahead log, but does nothing to make it durable; flush, which pushes data out of the JVM (likely to page cache); and sync, which ensures that each mutation is written to the physical disk. To configure the durability of files written during minor and major compactions, set the Hadoop property "dfs.datanode.synconclose" to "true". This will ensure that the blocks of the files in HDFS are written to the physical disk as the compaction output files are written (Note that this may only apply to replicated files in HDFS). Setting this property will change the durability for the BatchWriter session. A value of "default" will use the table's durability setting. | | batch.writer.latency.max | 120s | 2.0.0 | Max amount of time (in seconds) to hold data in memory before flushing it | | batch.writer.memory.max | 50M | 2.0.0 | Max memory (in bytes) to batch before writing | | batch.writer.threads.max | 3 | 2.0.0 | Maximum number of threads to use for writing data to tablet servers. | diff --git a/_docs-4/configuration/metrics.md b/_docs-4/configuration/metrics.md index 46da8f233..e752b0eaf 100644 --- a/_docs-4/configuration/metrics.md +++ b/_docs-4/configuration/metrics.md @@ -33,6 +33,46 @@ Metrics that are generated across all server types. **Description:** Reports 1 when process memory usage is above the threshold, reports 0 when memory is okay.
+### accumulo.recoveries.sorts.avg.progress +**Type:** GAUGE +**Description:** The average percentage (0.0 - 99.9) of the in progress log sorts. +
+
+### accumulo.recoveries.sorts.in.progress +**Type:** GAUGE +**Description:** The number of log sorts in progress. +
+
+### accumulo.recoveries.sorts.runtime.longest +**Type:** GAUGE +**Description:** The time (in milliseconds) of the longest running log sort. +
+
+### accumulo.recoveries.tablets.completed +**Type:** GAUGE +**Description:** The number of tablet recoveries completed +
+
+### accumulo.recoveries.tablets.failed +**Type:** GAUGE +**Description:** The number of tablet recoveries failed +
+
+### accumulo.recoveries.tablets.in.progress +**Type:** GAUGE +**Description:** The number of tablet recoveries in progress +
+
+### accumulo.recoveries.tablets.mutations.replayed +**Type:** GAUGE +**Description:** The number of mutations replayed for tablet recovery +
+
+### accumulo.recoveries.tablets.started +**Type:** GAUGE +**Description:** The number of tablet recoveries started +
+
### accumulo.server.idle **Type:** GAUGE **Description:** Indicates if the server is idle or not. The value will be 1 when idle and 0 when not idle. @@ -47,6 +87,16 @@ Metrics that are generated across all server types. **Type:** DISTRIBUTION_SUMMARY **Description:** Time waiting to execute an RPC request.
+
+### executor.completed +**Type:** FUNCTION_COUNTER +**Description:** Task completed by a thread pool. Each thread pool emits this metric w/ a different tag. +
+
+### executor.queued +**Type:** GAUGE +**Description:** Task queued for a thread pool. Each thread pool emits this metric w/ a different tag. +
{::options parse_block_html="false" /} @@ -66,6 +116,26 @@ Metrics specific to compactions, both minor and major. Metrics for major compact **Description:** Number of entries written by all compactions that have run on this compactor (majc) or tserver (minc).
+### accumulo.compaction.majc.cancelled +**Type:** FUNCTION_COUNTER +**Description:** Number compactions that have been cancelled on this compactor +
+
+### accumulo.compaction.majc.completed +**Type:** FUNCTION_COUNTER +**Description:** Number compactions that have succeeded on this compactor +
+
+### accumulo.compaction.majc.failed +**Type:** FUNCTION_COUNTER +**Description:** Number compactions that have failed on this compactor +
+
+### accumulo.compaction.majc.failures.consecutive +**Type:** GAUGE +**Description:** Number of consecutive compaction failures. Resets to zero on a successful compaction +
+
### accumulo.compaction.majc.in_progress **Type:** GAUGE **Description:** Indicator of whether a compaction is in-progress (value: 1) or not (value: 0). An in-progress compaction could also be stuck. @@ -96,6 +166,11 @@ Metrics specific to compactions, both minor and major. Metrics for major compact **Description:** Number and duration of stuck minor compactions.
+### accumulo.compaction.process.terminated +**Type:** FUNCTION_COUNTER +**Description:** Will report 1 if the Compactor terminates due to consecutive failures, else 0. Emitting this metric is a best effort before the process terminates +
+
### accumulo.compaction.queue.count **Type:** GAUGE **Description:** Number of priority queues for compaction jobs. @@ -146,7 +221,17 @@ Metrics specific to compactions, both minor and major. Metrics for major compact **Description:** Size of queued jobs in bytes.
-### accumulo.compaction.svc.misconfigured +### accumulo.compaction.svc.meta.misconfigured +**Type:** GAUGE +**Description:** A value of 1 indicates a misconfiguration in the compaction service, while a value of 0 indicates that the configuration is valid. +
+
+### accumulo.compaction.svc.root.misconfigured +**Type:** GAUGE +**Description:** A value of 1 indicates a misconfiguration in the compaction service, while a value of 0 indicates that the configuration is valid. +
+
+### accumulo.compaction.svc.user.misconfigured **Type:** GAUGE **Description:** A value of 1 indicates a misconfiguration in the compaction service, while a value of 0 indicates that the configuration is valid.
@@ -207,9 +292,19 @@ Metrics that are generated by the Fate component in the Manager process. **Description:** Number of FATE operations in progress. The op type is designated by the `op.type` tag.
+### accumulo.fate.ops.threads.inactive +**Type:** GAUGE +**Description:** Keeps track of the number of idle threads (not working on a fate operation) in the thread pool. The pool name can be found in the pool.name tag. The fate instance type can be found in the instanceType tag. +
+
+### accumulo.fate.ops.threads.total +**Type:** GAUGE +**Description:** Keeps track of the total number of threads in the thread pool. The pool name can be found in the pool.name tag. The fate instance type can be found in the instanceType tag. +
+
### accumulo.fate.tx **Type:** GAUGE -**Description:** The state is now in a tag (e.g., state=new, state=in.progress, state=failed, etc.). +**Description:** Count of FATE operations in a certain state. The state is now in a tag (e.g., state=new, state=in.progress, state=failed, etc.).
{::options parse_block_html="false" /} @@ -402,62 +497,67 @@ Metrics specific to scans, which can be executed in the ScanServer or the Tablet {::options parse_block_html="true" /}
### accumulo.scan.busy.timeout.count -**Type:** COUNTER +**Type:** FUNCTION_COUNTER **Description:** Count of the scans where a busy timeout happened.
### accumulo.scan.close -**Type:** COUNTER +**Type:** FUNCTION_COUNTER **Description:** Number of calls to close a scan or multiscan.
### accumulo.scan.continue -**Type:** COUNTER +**Type:** FUNCTION_COUNTER **Description:** Number of calls to continue a scan or multiscan.
+### accumulo.scan.errors +**Type:** FUNCTION_COUNTER +**Description:** Number of scan task that had an exception. +
+
### accumulo.scan.files.open **Type:** GAUGE **Description:** Number of files open for scans.
### accumulo.scan.paused.for.memory -**Type:** COUNTER +**Type:** FUNCTION_COUNTER **Description:** Count of scans paused due to server being low on memory.
### accumulo.scan.queries -**Type:** GAUGE +**Type:** FUNCTION_COUNTER **Description:** Number of queries made during scans.
### accumulo.scan.query.results -**Type:** GAUGE +**Type:** FUNCTION_COUNTER **Description:** Query count. The rate can be derived from this metric.
### accumulo.scan.query.results.bytes -**Type:** GAUGE +**Type:** FUNCTION_COUNTER **Description:** Query byte count. The rate can be derived from this metric.
### accumulo.scan.query.scanned.entries -**Type:** GAUGE +**Type:** FUNCTION_COUNTER **Description:** Count of scanned entries. The rate can be derived from this metric.
### accumulo.scan.result -**Type:** GAUGE +**Type:** DISTRIBUTION_SUMMARY **Description:** Results per scan.
### accumulo.scan.return.early.for.memory -**Type:** COUNTER +**Type:** FUNCTION_COUNTER **Description:** Count of scans that returned results early due to server being low on memory.
### accumulo.scan.start -**Type:** COUNTER +**Type:** FUNCTION_COUNTER **Description:** Number of calls to start a scan or multiscan.
@@ -467,7 +567,7 @@ Metrics specific to scans, which can be executed in the ScanServer or the Tablet
### accumulo.scan.yields -**Type:** GAUGE +**Type:** DISTRIBUTION_SUMMARY **Description:** Counts scans that have yielded.
@@ -570,19 +670,39 @@ Metrics that are generated by the Manager process. **Description:** The number of migrations that need to complete before the system is balanced.
+### accumulo.manager.goal.state +**Type:** GAUGE +**Description:** Manager goal state: -1=unknown, 0=CLEAN_STOP, 1=SAFE_MODE, 2=NORMAL. +
+
### accumulo.tabletmgmt.meta.errors **Type:** GAUGE **Description:** Error count encountered by the TabletGroupWatcher for the META data level.
+### accumulo.tabletmgmt.meta.recovery +**Type:** GAUGE +**Description:** Recovery count encountered by the TabletGroupWatcher for the META data level. +
+
### accumulo.tabletmgmt.root.errors **Type:** GAUGE **Description:** Error count encountered by the TabletGroupWatcher for the ROOT data level.
+### accumulo.tabletmgmt.root.recovery +**Type:** GAUGE +**Description:** Recovery count encountered by the TabletGroupWatcher for the ROOT data level. +
+
### accumulo.tabletmgmt.user.errors **Type:** GAUGE **Description:** Error count encountered by the TabletGroupWatcher for the USER data level.
+
+### accumulo.tabletmgmt.user.recovery +**Type:** GAUGE +**Description:** Recovery count encountered by the TabletGroupWatcher for the USER data level. +
{::options parse_block_html="false" /} diff --git a/_docs-4/configuration/server-properties.md b/_docs-4/configuration/server-properties.md index edb83b21c..a6ef3533a 100644 --- a/_docs-4/configuration/server-properties.md +++ b/_docs-4/configuration/server-properties.md @@ -10,37 +10,41 @@ Below are properties set in `accumulo.properties` or the Accumulo shell that con | Property | Description | |--------------|-------------| -| **compaction.*** | **Available since:** 3.1.0
Both major and minor compaction properties can be included under this prefix. | +| **compaction.*** | **Available since:** 4.0.0
Both major and minor compaction properties can be included under this prefix. | | **compaction.coordinator.*** | **Available since:** 2.1.0
Properties in this category affect the behavior of the accumulo compaction coordinator server. | -| compaction.coordinator.compactor.dead.check.interval | **Available since:** 2.1.0
The interval at which to check for dead compactors.
**type:** TIMEDURATION, **zk mutable:** no, **default value:** `5m` | -| compaction.coordinator.reservation.threads.meta | **Available since:** 4.0.0
The number of threads used to reserve files for compaction in a tablet for accumulo.metadata tablets.
**type:** COUNT, **zk mutable:** no, **default value:** `1` | -| compaction.coordinator.reservation.threads.root | **Available since:** 4.0.0
The number of threads used to reserve files for compaction in a tablet for the root tablet.
**type:** COUNT, **zk mutable:** no, **default value:** `1` | -| compaction.coordinator.reservation.threads.user | **Available since:** 4.0.0
The number of threads used to reserve files for compaction in a tablet for user tables.
**type:** COUNT, **zk mutable:** no, **default value:** `64` | -| **compaction.service.*** | **Available since:** 3.1.0
This prefix should be used to define all properties for the compaction services.See {% jlink -f org.apache.accumulo.core.spi.compaction.RatioBasedCompactionPlanner %}.
A new external compaction service would be defined like the following:
`compaction.service.newService.planner="org.apache.accumulo.core.spi.compaction.RatioBasedCompactionPlanner".`
`compaction.service.newService.opts.groups="[{"group": "small", "maxSize":"32M"},{ "group":"medium", "maxSize":"512M"},{"group":"large"}]`
`compaction.service.newService.opts.maxOpen=50`.
Additional options can be defined using the `compaction.service..opts.