-
Notifications
You must be signed in to change notification settings - Fork 75
HDDS-15864. Branch merge checklist for Short Circuit read #503
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
ChenSammi
wants to merge
11
commits into
apache:master
Choose a base branch
from
ChenSammi:master
base: master
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
+132
−0
Open
Changes from 2 commits
Commits
Show all changes
11 commits
Select commit
Hold shift + click to select a range
f7e5fe6
HDDS-15864. Branch merge checklist for Short Circuit read
ChenSammi 1b302c6
address comments
ChenSammi 7235fa3
HDDS-15864. Fix CI for short-circuit merge checklist doc.
jojochuang ad0e9b1
Remove images, convert to tables
errose28 2a55e10
Fix spelling
errose28 1a98f08
address comments
ChenSammi c87df0f
Merge remote-tracking branch 'origin/master' into ChenSammi/master
adoroszlai 23b423b
fix lint reports
ChenSammi debe295
fix new lint reports
ChenSammi 2f3b32b
Revert "fix new lint reports"
ChenSammi 174ef9b
fix lint issue not reported locally
ChenSammi File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
113 changes: 113 additions & 0 deletions
113
...-git/03-feature-branches/03-merged-branches/17-hdds-10685-short-circuit-read.md
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,113 @@ | ||
| # HDDS-10685 : Short-Circuit Read | ||
|
|
||
| Epic: [HDDS-10685](https://issues.apache.org/jira/browse/HDDS-10685) | ||
| Feature branch: https://github.com/apache/ozone/tree/HDDS-10685 | ||
|
|
||
| ## 1. Builds/intermittent test failures | ||
|
|
||
| There are no intermittent failures specific to the HDDS-10685 branch as of now. During the development, it was ensured all the CI checks were clean prior to every commit merge. | ||
|
|
||
| The plan is to run repeated CI checks on the merge commit to master. | ||
|
|
||
| ## 2. Documentation | ||
|
|
||
| [User Documentation](https://ozone.apache.org/docs/next/administrator-guide/configuration/performance/short-circuit-local-reads/) of Short Circuit Read has been added. | ||
|
|
||
| ## 3. Design, attached the docs | ||
|
|
||
| Design document can be found here : [Short Circuit Read Support](https://github.com/apache/ozone/blob/HDDS-10685/hadoop-hdds/docs/content/design/short-circuit-read.md). | ||
|
|
||
| ## 4. S3 compatibility | ||
|
|
||
| N/A, S3 compatibility remains the same. Short Circuit Read only affects the client and DataNode read path. | ||
|
|
||
| ## 5. Docker-compose / Acceptance tests | ||
|
|
||
| New robot test [short-circuit.robot](https://github.com/apache/ozone/blob/HDDS-10685/hadoop-ozone/dist/src/main/smoketest/short-circuit/short-circuit.robot) is being added. | ||
|
|
||
| New acceptance tests are added, mainly tests the Short Circuit Read metrics. It does not test fault injection. | ||
|
|
||
| ## 6. Support of containers / Kubernetes | ||
|
|
||
| No addition. No change in existing support. | ||
|
|
||
| ## 7. Coverage / Code quality | ||
|
|
||
| [New Code Coverage](https://sonarcloud.io/summary/new_code?id=hadoop-ozone&branch=HDDS-10685) for Short Circuit Read Support (HDDS-10685) is 80.78%% and [Overall Code Coverage](https://sonarcloud.io/summary/overall?id=hadoop-ozone&branch=HDDS-10685) is 78.5%. | ||
| [Overall Code Coverage](https://sonarcloud.io/summary/overall?id=hadoop-ozone&branch=master) for master is 78.5%%. | ||
|
|
||
| ## 8. Build time | ||
|
|
||
| [Build time for the latest commit](https://github.com/apache/ozone/actions/runs/27126447271/job/80056268213) from HDDS-10685 Branch is 9m 59s. | ||
| [Build time for the latest commit](https://github.com/apache/ozone/actions/runs/29915076929/job/88926295859) from the master branch is 10m 43s. | ||
|
|
||
| ## 9. Possible incompatible changes/used feature flag | ||
|
|
||
| Short-circuit read is gated by the non-rolling upgrade framework as HDDSLayoutFeature.SHORT_CIRCUIT_READS (layout version 11). | ||
|
errose28 marked this conversation as resolved.
Outdated
|
||
|
|
||
| It remains inactive until the cluster upgrade is finalized. A global enable/disable switch is provided via ozone.client.read.short-circuit (default: false). | ||
|
|
||
| To enable the feature (after finalization), add the following to the client and Datanode ozone-site.xml to enable the feature: | ||
|
|
||
| ```xml | ||
| <property> | ||
| <name>ozone.client.read.short-circuit</name> | ||
| <value>true</value> | ||
| <tag>CLIENT, DATANODE</tag> | ||
| <description>Disable or enable the short-circuit local read feature. By default it is disabled.</description> | ||
| </property> | ||
| ``` | ||
| And the following to the client and Datanode ozone-site.xml, to specify the path of the UNIX domain socket: | ||
| ```xml | ||
| <property> | ||
| <name>ozone.domain.socket.path</name> | ||
| <value></value> | ||
| <tag>CLIENT, DATANODE</tag> | ||
| <description>UNIX domain socket path for co-located client–Datanode short-circuit communication.</description> | ||
| </property> | ||
| ``` | ||
|
|
||
| ## 10. Third-party dependencies/License changes | ||
|
|
||
| There are no third party dependencies introduced by this feature. | ||
|
|
||
| ## 11. Performance | ||
|
|
||
| The major workflow of Short-Circuit Read is: The client detects a local Datanode replica → opens a UNIX domain socket → sends GetBlock with requestShortCircuitAccess=true → Datanode passes a file descriptor → The client reads block data directly from the local disk via FileChannel. | ||
|
|
||
| End-to-end read performance is therefore dominated by local disk I/O, similar to a direct file read, rather than by gRPC data transfer. | ||
|
|
||
| A benchmark was run against feature branch HDDS-10685, testing short-circuit performance with the `ozone fs` command and YCSB on a ycloud cluster. Short-circuit read was toggled with `ozone.client.read.short-circuit` and `ozone.domain.socket.path`. | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. please replace ycloud with "Cloudera's internal test bed" |
||
|
|
||
| ### Cluster configuration | ||
|
|
||
|  | ||
|
|
||
| ### Ozone FS | ||
|
|
||
| Use `ozone fs -get ofs://ozone1733996033/vol-scr/buck/scr/file33 ./file33` to download a 10 GB file. Node9 and node7 have the datanode role; node8 does not. | ||
|
|
||
|  | ||
|
|
||
| ### YCSB | ||
|
|
||
| The tests are performed by running 3 consecutive iterations after changing the `ozone.client.read.short-circuit` configuration and restarting all related services. HBase `l1CacheHitRatio` is around 90% during the test. | ||
|
|
||
| **Workload C** | ||
|
|
||
|  | ||
|
|
||
| **Workload A** | ||
|
|
||
|  | ||
|
|
||
| Metrics (`ContainerLocalOps`, local op latencies, local bytes stats) are added in Datanode to observe the runtime state/latency. | ||
|
|
||
| ## 12. Security considerations | ||
|
|
||
| Short-Circuit Read does not introduce any new CLI or admin command. | ||
|
|
||
| Short-circuit communication uses a UNIX domain socket (`ozone.domain.socket.path`) between the client and Datanode. | ||
|
|
||
| It follows the same [Hadoop Socket Path Security](https://cwiki.apache.org/confluence/spaces/HADOOP2/pages/120730260/SocketPathSecurity) rules as HDFS short-circuit reads. | ||
|
errose28 marked this conversation as resolved.
Outdated
|
||
|
|
||
Binary file added
BIN
+13.7 KB
...ure-branches/03-merged-branches/short-circuit-read-benchmark/cluster-config.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added
BIN
+9.27 KB
...ature-branches/03-merged-branches/short-circuit-read-benchmark/ozone-fs-get.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added
BIN
+8.44 KB
...feature-branches/03-merged-branches/short-circuit-read-benchmark/workload-a.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added
BIN
+19 KB
...feature-branches/03-merged-branches/short-circuit-read-benchmark/workload-c.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.