Skip to content

feat(services/baidu-netdisk): add Baidu Netdisk support - #8516

Open
suyanhanx wants to merge 5 commits into
apache:mainfrom
suyanhanx:feat/baidu-netdisk
Open

suyanhanx wants to merge 5 commits into
apache:mainfrom
suyanhanx:feat/baidu-netdisk

Conversation

@suyanhanx

@suyanhanx suyanhanx commented Nov 13, 2026 •

Copy link
Copy Markdown
Member

Which issue does this PR close?

Closes #8481.

Rationale for this change

Add access to Baidu Netdisk through the official Open Platform APIs. The implementation was exercised against a real application and account before and during development.

What changes are included in this PR?

  • Add opendal-service-baidu-netdisk, the services-baidu-netdisk facade feature, and scheme registration.
  • Support stat, paginated list, ranged reads, directory creation, writes, copy, rename and delete. Reject directory sources for copy/rename and existing directory destinations for copy/rename/write. Preserve directory-path semantics for recursive deletion, support file-like prefix deletion, and check directory emptiness for ordinary deletes.
  • Keep authentication inside the service: access tokens, serialized OAuth refresh, single-use refresh-token rotation, and an optional persistence callback.
  • Upload complete buffers through precreate, upload-server discovery, 4 MiB parts and create, using the actual block hashes. Preserve UTF-8 download filenames and allow RetryLayer to reconstruct a copy after temporary filemanager failures.
  • Document real-service test setup and explicitly skip the existing emoji copy case for Baidu, with a warning and the original filenames unchanged.

Validation

  • Real-account probes covered OAuth refresh and persistence, multipart upload above 4 MiB, exact full/range readback across a block boundary, empty files, and file operations. Test data was cleaned up.

  • Full behavior run, single-threaded with a 300-second I/O timeout and unchanged random payload sizes: 128 passed, 1 failed, 0 ignored, 0 filtered, in 1190.54 seconds. Of the harness passes, 58 were early returns: the emoji compatibility skip and 57 unsupported-capability guards. The source/capability breakdown is therefore 70 checks passed, 1 failed, 58 early returns.

  • The failure was test_copier_completion: filemanager returned busy error errno=111, and the one-shot future made it persistent. Switched to OneShotCopier::new_with so RetryLayer can recreate the operation. The original focused case then passed against Baidu. The full suite was not rerun after the copier or directory-operation fixes, and the focused rerun did not establish another occurrence of error 111.

  • Focused all-target Clippy with -D warnings, facade behavior compilation, and formatting checks passed.

  • Latest real-account regression run: 27 operation checks passed on the updated source, including empty/non-empty destination-directory protection for copy/rename/write, directory-source rejection, non-recursive directory protection, slash-terminated deletion preserving same-named files, and recursive prefix deletion with both missing and existing exact targets. Checks verified preserved source/destination bytes, parent scope, nonmatching siblings, file overwrite, and repeated deletion. The harness exited successfully and its root was independently confirmed absent through the parent listing.

  • Nine unchanged existing behavior cases passed on the updated source: file, empty-directory, missing-path and recursive deletion; nested and overwrite copy/rename; empty-file write. These ran single-threaded with original random payload sizes and a 300-second I/O timeout. The isolated test root was cleaned and independently confirmed absent.

Are there any user-facing changes?

A new baidu-netdisk service is available. Configure a root beneath /apps/<application name>/ and an authorized access token; automatic renewal additionally requires the application credentials and refresh token. Persist rotated tokens with on_token_refresh before restarting.

Writes require the complete file in memory and are limited to 4 GiB; streaming writes and append are unsupported. Rename uses server-side copy followed by delete and is not atomic: live native move/rename rejected existing destinations despite requesting overwrite. Baidu also rejects the tested emoji and some supplementary Unicode filenames at final creation; names are never silently rewritten. Recursive deletion of a path ending in / selects that directory only; without /, it removes all matching paths in the parent directory and matching directory trees. Non-recursive deletion checks directory emptiness before calling Baidu's recursive delete API. Emptiness checks, path-type checks and mutations are separate requests: concurrently added children or changed path types are not protected atomically. Account limits and download speed still apply.

Breaking changes

AI Usage Statement

  • Harness: Codex desktop.
  • Model: gpt-6-astra.
  • Effort: high.
  • Role: Assisted implementation, real-service probes, failure analysis, tests and documentation. The contributor supplied the test application/account and actively guided the work.

@github-actions github-actions Bot added releases-note/feat The PR implements a new feature or has a title that begins with "feat" size:XXL This PR changes 1000+ lines, ignoring generated files. labels Nov 13, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

releases-note/feat The PR implements a new feature or has a title that begins with "feat" size:XXL This PR changes 1000+ lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

new feature: Baidu Netdisk service

1 participant