[client] Concurrent remote log downloads can cause Unexpected EOF

[fxbing]

Search before asking

• I searched in the issues and found nothing similar.

Fluss version

main (development)

Please describe the bug 🐞

When multiple scanners read the same remote log segment, duplicate downloads may target the same local file.

LogFetcher may reuse a fetchableBuckets() snapshot captured before an asynchronous metadata update, causing an already buffered bucket to be fetched again. RemoteFileDownloader then writes directly to the final path, truncating the file while another scanner is reading it.

The reader may fail with CorruptMessageException: Unexpected EOF.

Expected: an open reader can finish reading the downloaded segment.

Actual: a duplicate download may truncate the segment being read.

Solution

• Recheck fetchableBuckets() under the LogFetcher lock before creating fetch requests.
• Download to a unique temporary file and atomically move the completed file to the final path.
• Add regression tests for both races.

This does not change any public API or storage format.

Are you willing to submit a PR?

• I'm willing to submit a PR!