CLDSRV-899: Log replication events in server access logs#6164
CLDSRV-899: Log replication events in server access logs#6164dvasilas wants to merge 5 commits intodevelopment/9.2from
Conversation
Replication writes via the backbeat putData route are now visible in bucket server access logs with operation = REST.PUT.OBJECT.
Hello dvasilas,My role is to assist you with the merge of this Available options
Available commands
Status report is not available. |
Incorrect fix versionThe
Considering where you are trying to merge, I ignored possible hotfix versions and I expected to find:
Please check the |
|
LGTM |
Codecov Report❌ Patch coverage is
Additional details and impacted files
... and 2 files with indirect coverage changes @@ Coverage Diff @@
## development/9.2 #6164 +/- ##
===================================================
- Coverage 84.50% 84.41% -0.10%
===================================================
Files 204 204
Lines 13180 13215 +35
===================================================
+ Hits 11138 11155 +17
- Misses 2042 2060 +18
Flags with carried forward coverage won't be shown. Click here to find out more. 🚀 New features to boost your workflow:
|
dvasilas
force-pushed
the
improvement/CLDSRV-899-log-replication-events
branch
from
b922c2f to
803285c
Compare
Request integration branchesWaiting for integration branch creation to be requested by the user. To request integration branches, please comment on this pull request with the following command: Alternatively, the |
|
LGTM |
1 similar comment
|
LGTM |
Destination-side delete-marker replication is now logged as REST.DELETE.OBJECT, same as AWS.
Destination-side PutObjectTagging and DeleteObjectTagging replication are now logged as REST.PUT.OBJECT_TAGGING (same as AWS). AWS replicates a DeleteObjectTagging on the destination as REST.PUT.OBJECT_TAGGING with an empty tag set and identical URI shape (PUT /<bucket>/<key>?tagging&versionId=<vid>).
dvasilas
force-pushed
the
improvement/CLDSRV-899-log-replication-events
branch
from
0c52210 to
812257c
Compare
Incorrect fix versionThe
Considering where you are trying to merge, I ignored possible hotfix versions and I expected to find:
Please check the |
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
| const { headers, bucketName, objectKey } = request; | ||
|
|
||
| // Destination-side delete-marker replication. | ||
| // We need the REPLICA status to distinquish from |
There was a problem hiding this comment.
Typo: "distinquish" should be "distinguish".
```suggestion
// We need the REPLICA status to distinguish from
| query = versionId ? `?tagging&versionId=${versionId}` : '?tagging'; | ||
| } | ||
| logEntry.requestURI = | ||
| `${method} /${params.bucketName}/${params.objectKey}${query} HTTP/${req.httpVersion ?? '1.1'}`; |
There was a problem hiding this comment.
params.objectKey is URI-decoded (via _decodeURI at line 79 of routeBackbeat.js), so keys with special characters (spaces, unicode) will appear unencoded here (e.g. my file.txt), while normal S3 request URIs logged via getURI(req) preserve the raw HTTP encoding (my%20file.txt). This could cause inconsistent parsing for log consumers. Consider re-encoding the key to match non-replication entries.
— Claude Code
Review by Claude Code |
Destination-side PutObjectAcl replication is now logged as REST.PUT.ACL (same as AWS). The discriminator is replicationInfo.status === 'REPLICA' combined with omVal.originOp === 's3:ObjectAcl:Put', set on the source by objectPutACL and preserved through ObjectQueueEntry.toReplicaEntry()'s clone. AWS replicates a PutObjectAcl on the destination as REST.PUT.ACL with URI shape PUT /<bucket>/<key>?acl&versionId=<srcVersionId> and aclRequired = 'Yes'. Verified empirically against AWS in eu-north-1 on 2026-05-08.
dvasilas
force-pushed
the
improvement/CLDSRV-899-log-replication-events
branch
from
812257c to
2c9ff6d
Compare
| const { headers, bucketName, objectKey } = request; | ||
|
|
||
| // Destination-side delete-marker replication. | ||
| // We need the REPLICA status to distinquish from |
There was a problem hiding this comment.
Typo: "distinquish" should be "distinguish".
```suggestion
// We need the REPLICA status to distinguish from
| query = versionId ? `?acl&versionId=${versionId}` : '?acl'; | ||
| } | ||
| logEntry.requestURI = | ||
| `${method} /${params.bucketName}/${params.objectKey}${query} HTTP/${req.httpVersion ?? '1.1'}`; |
There was a problem hiding this comment.
The synthesized requestURI uses the decoded objectKey directly, but for non-replication requests getURI() uses the raw request.url which is URL-encoded. An object key like my file.txt would appear as PUT /bucket/my%20file.txt HTTP/1.1 in normal logs but PUT /bucket/my file.txt HTTP/1.1 here. Consider URL-encoding bucketName and objectKey to stay consistent with the non-replication log format.
— Claude Code
|
2 participants
Replication writes via the backbeat
putDataroute are now visible inthe destination bucket's server access logs.
Reads from the source bucket are already logged as
REST.GET.OBJECT.MPU replication
MPU replication produces N ×
REST.PUT.OBJECTon the destination, not AWS's 1 ×REST.POST.UPLOADS+ N ×REST.PUT.PART+ 1 ×REST.POST.UPLOAD.Backbeat replicates an MPU as N
PUT /_/backbeat/datacalls + one stitchingPUT /_/backbeat/metadata. The destination has no MPU lifecycle — parts land as raw data blobs and the metadata write turns them into a multipart-shaped object. AWS's initiate/parts/complete events correspond to operations that don't happen here, and thePutDataAPI carries nouploadId/partNumberfor cloudserver to even relabel parts asREST.PUT.PART.