Perftest: Fix immediate data handling and post list scatter entry - #407
Open
NitzanLavy wants to merge 4 commits into
Open
NitzanLavy wants to merge 4 commits into
NitzanLavy wants to merge 4 commits into
Conversation
added 4 commits
September 16, 2026 11:02
Send with immediate is dispatched per transport type and SRD was left out, so selecting it reported unsupported post send properties. Derive the operation from the selected verb, as the other transports already do. Fixes: 1f1271c ("Perftest: Add support for Send with Immediate verb") Reviewed-by: Ramon Fried <friedram@amazon.com> Reviewed-by: Yonatan Nachum <ynachum@amazon.com> Signed-off-by: Nitzan Lavy <nlavy@amazon.com>
Tests using send or write with immediate only verified that a completion reports immediate data as present, never its content. Set a known value on these work requests, honour the value they carry, and verify it on receive. Reviewed-by: Michael Margolin <mrgolin@amazon.com> Reviewed-by: Yonatan Nachum <ynachum@amazon.com> Signed-off-by: Nitzan Lavy <nlavy@amazon.com>
The post send path of the ibv_wr API reads the scatter entry once before walking the work request list, so every request in a post list is posted with the address of the first one. Only the atomic opcodes refresh it, which leaves send and write requests transmitting the same buffer region repeatedly instead of the region each request refers to. Restore reading the scatter entry from the work request being posted, as it was before the entry was hoisted out of the loop. Fixes: 4f48008 ("Perftest: Add real-time data validation for RDMA write and read operations") Reviewed-by: Michael Margolin <mrgolin@amazon.com> Reviewed-by: Yonatan Nachum <ynachum@amazon.com> Signed-off-by: Nitzan Lavy <nlavy@amazon.com>
Send with immediate was removed from the bandwidth loop's exclusion from advancing the remote address. For UD and SRD that increment lands on the address handle field inside struct ibv_send_wr, which shares a union with the remote address, corrupting it after the first work request. Only the old post send path reads that field, and it segfaults. Exclude it as well, matching other loops behavior. Fixes: 4f48008 ("Perftest: Add real-time data validation for RDMA write and read operations") Reviewed-by: David Shoolman <shoolman@amazon.com> Reviewed-by: Firas Jahjah <firasj@amazon.com> Signed-off-by: Nitzan Lavy <nlavy@amazon.com>
NitzanLavy
force-pushed
the
imm_and_post_list_fixes
branch
from
September 16, 2026 11:03
e301c1c to
67b8885
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Four fixes and updates to the immediate-data and post list paths.
Perftest: Support send with immediate over SRD
Send with immediate is dispatched per transport type and SRD was left out, so selecting it reported unsupported post send properties. The operation is now derived from the selected verb, as the other transports already do.
Perftest: Verify the immediate data value
Completion handling reports whether immediate data is present but never checks its content. Set a known value on these work requests, honour a value the caller supplies, and verify it on receive.
Perftest: Set the scatter entry per work request in a post list
The scatter entry is set once before the work request list is walked, so every request in a post list is posted with the entry belonging to the first request, or to the most recent atomic operation, rather than its own. With a post list of one this is invisible; with more, requests carry data that is not theirs.
Perftest: Do not advance the remote address for send with immediate
Send with immediate was removed from the bandwidth loop's exclusion from advancing the remote address. For UD and SRD that increment lands on the address handle, which shares a union with the remote address, so every request after the first carries a corrupted handle. Only the old post send path reads that field, which is where it surfaces, as a segfault once the first request has been posted.
This and the scatter entry patch above both restore behaviour lost in 4f48008.
A follow-up PR for synchronous data validation depends on these patches. #364 will be rebased on top once this lands.