refactor(clp-package)!: Remove legacy Spider build and runtime code.

[sitaowang1998]

Description

Closes #2440.

This PR removes the remaining legacy build, packaging and Python runtime code. The legacy Docker Compose deployment and configuration were already removed in #2452, leaving these components unreachable but still built and shipped.

The followings are removed:

• Legacy Spider C++ build tasks and pinned source dependency.
• Legacy spider_scheduler and spider_worker package binaries.
• The Spider-only mariadb-connector-cpp build dependency.
• libmariadbcpp.so from the CLP package image.
• The yscope-spider-py Python dependency and its transitive lockfile entries.
• Legacy Python runtime modules:
  • Spider worker launcher.
  • Spider compression task adapter.
  • Spider compression task manager.
• The legacy Spider build directory and package dependency wiring.

This also removes the Ubuntu-only restriction imposed by the legacy Spider build.

Checklist

• The PR satisfies the contribution guidelines.
• This is a breaking change and that has been indicated in the PR title, OR this isn't a
  breaking change.
• Necessary docs have been updated, OR no docs need to be updated.

Validation performed

• Compression and search on Docker Compose works end-to-end.
• GitHub workflows pass.

Summary by CodeRabbit

• Removed Features

  • Removed support for the Spider-based compression workflow, including task scheduling, compression execution, and worker management.
  • Removed Spider-specific build and runtime components from package creation.

• Chores

  • Removed obsolete Spider and MariaDB Connector/C++ build dependencies and image assets.
  • Simplified dependency setup and packaging configuration.

• Documentation

  • Updated container tooling documentation to reflect the removal of obsolete runtime configuration.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: afd8b87f-30ff-4e4a-9a3b-22a011f2c098

📥 Commits

Reviewing files that changed from the base of the PR and between 753048a and f6f1fac.

📒 Files selected for processing (2)

• docs/src/dev-docs/tooling-containers.md
• tools/docker-images/clp-package/Dockerfile

💤 Files with no reviewable changes (1)

• docs/src/dev-docs/tooling-containers.md

---

Walkthrough

The legacy Spider compression orchestration, worker launcher, dependencies, build tasks, container image integration, and related documentation were removed.

Changes

Legacy Spider removal

Layer / File(s)	Summary
Spider orchestration removal components/job-orchestration/job_orchestration/executor/compress/spider_compress.py, components/job-orchestration/job_orchestration/executor/start-spider-worker.py, components/job-orchestration/job_orchestration/scheduler/compress/task_manager/spider_task_manager.py	Removes Spider compression execution, worker process management, task submission, result handling, and parameter serialization.
Build and package integration cleanup components/job-orchestration/pyproject.toml, taskfile.yaml, taskfiles/deps/main.yaml, tools/docker-images/clp-package/Dockerfile, docs/src/dev-docs/tooling-containers.md	Removes the Spider dependency, build tasks, build path, package dependency, MariaDB connector setup, container libraries and binaries, and LD_LIBRARY_PATH documentation.

Estimated code review effort: 2 (Simple) | ~10 minutes

Possibly related PRs

• y-scope/clp#2232: Removes or replaces legacy Spider compression orchestration.
• y-scope/clp#2401: Introduces replacement compression coordinator APIs and task I/O types.
• y-scope/clp#2452: Removes related Spider task-management and scheduling support.

Suggested reviewers: linzhihao-723

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly identifies the removal of legacy Spider build and runtime code.
Linked Issues check	✅ Passed	The changes complete the remaining Spider build, packaging, dependency, and runtime removals requested by issue #2440.
Out of Scope Changes check	✅ Passed	All changes support the removal of legacy Spider integration and its obsolete library path configuration.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[jackluo923]

LD_LIBRARY_PATH="${CLP_HOME}/lib" (tools/docker-images/clp-package/Dockerfile:22) was added
alongside the libmariadbcpp.so* COPY this PR removes (1b92ca8, #1414) and has no other
consumer — after this change nothing but python3/site-packages lives under ${CLP_HOME}/lib.

Worth dropping it here too, along with its row in the clp-package "Notable configurations" table in
docs/src/dev-docs/tooling-containers.md.