Skip to content

chore: migrate DuckDuckGo tool to ddg-kit - #1572

Open
lennney wants to merge 1 commit into
i-am-bee:mainfrom
lennney:codex/migrate-ddg-kit
Open

chore: migrate DuckDuckGo tool to ddg-kit#1572
lennney wants to merge 1 commit into
i-am-bee:mainfrom
lennney:codex/migrate-ddg-kit

Conversation

@lennney

@lennney lennney commented Jul 29, 2026

Copy link
Copy Markdown

Why

BeeAI currently uses duck-duck-scrape@^2.2.7 directly from
DuckDuckGoSearchTool. This patch replaces it with ddg-kit@^0.1.0 while
preserving the tool name, input schema, throttle, pagination, safe-search and
offset handling, AbortSignal propagation, and result mapping.

The main behavior gain is explicit provider failure handling. ddg-kit exposes
DdgError for bot challenges, rate limits, HTTP failures, timeouts, and parser
changes, and its default client applies a bounded request deadline and
challenge cooldown. This tool already passes its caller AbortSignal and proxy
option; the migration keeps those paths intact.

I maintain ddg-kit and disclose that relationship here. It is an unofficial,
greenfield compatibility implementation, has no affiliation with DuckDuckGo,
and is not an official duck-duck-scrape release. This PR gives BeeAI
maintainers room to review the dependency, the transport-option change, and the
maintenance model before deciding whether to adopt it.

Why use ddg-kit here

  • The tool already depends on the search, SearchOptions, and
    SafeSearchType surface; ddg-kit preserves those exports and the
    SearchResults fields consumed here.
  • Typed failures make provider, challenge, timeout, HTTP, and parser outcomes
    distinguishable from a valid empty result.
  • RequestOptions provides a small portable contract for proxy, signal,
    and timeoutMs, matching the options this tool can safely preserve.
  • The package has npm provenance, packed ESM/CommonJS checks, and a recorded
    Node 18/20/22 compatibility matrix.

Evidence:

Downstream adoption evidence:

  • OpenCandle OOTB Granite Bee Agent #145 was merged by maintainer Kahtaf on July 29, 2026. The merge
    commit is c7a7af0.
  • Five checks passed, and the maintainer reported live Web and News validation.
  • This is evidence of one downstream maintainer's review and adoption. It is
    not broad production proof or a provider SLA.

Changes

  • Replace duck-duck-scrape with ddg-kit@^0.1.0.
  • Change the public httpClientOptions type from NeedleOptions to
    ddg-kit.RequestOptions.
  • Keep proxy and caller AbortSignal forwarding, throttle, pagination, search
    options, and output mapping.
  • Remove Needle-only headers, uri_modifier, and rejectUnauthorized, plus
    the now-unused header-generator and @types/needle dependencies.
  • Update the DuckDuckGo tool test mock and Yarn lockfile.

The old BEEAI_DDG_TOOL_PROXY_VERIFY setting is no longer applicable because
RequestOptions does not expose a TLS-verification override. The PR does not
silently claim to preserve that setting.

Validation

The prepared downstream canary recorded:

  • DuckDuckGo tool and similarity tests: 8/8 passed
  • TypeScript check: passed
  • ESLint and Prettier: passed
  • CJS/ESM bundling: passed

In this follow-up sparse checkout, ESLint passed for both changed files. A full
TypeScript/Vitest run was not reproducible locally because the canary does not
contain the repository's complete workspace source and peer-provided
dependencies (zod, vite, and related setup). BeeAI CI is therefore the
authoritative full validation. No live DuckDuckGo response is included.

Limits and rollback

DuckDuckGo does not publish a supported contract for these endpoints. The
migration does not provide a provider SLA, prove production stability, or
preserve Needle-specific headers and TLS options. The package's typed errors
make failures visible, but BeeAI still needs to decide how those failures
should be surfaced or retried at the tool/workflow layer.

The migration has no persistent data change. A rollback can restore
duck-duck-scrape, the Needle/HeaderGenerator imports and options, the removed
dependencies, and the prior lockfile.

Additional downstream adoption evidence

  • intercept-mcp PR #6 has now merged the Web-search migration in merge commit 700b10a.
  • Together with OpenCandle PR #145, this gives two merged downstream maintainer examples for the focused Web/News compatibility seam.
  • These merges are adoption evidence, not a production SLA or a claim of broad provider stability.

@gemini-code-assist

Copy link
Copy Markdown
Contributor

Caution

The consumer version of Gemini Code Assist on GitHub has been sunset. All code review activity has officially ceased.

@github-actions github-actions Bot added the typescript Typescript related functionality label Jul 29, 2026
Signed-off-by: lennney <lennney@example.com>
@lennney
lennney force-pushed the codex/migrate-ddg-kit branch from c932f40 to 218dcbb Compare July 29, 2026 05:20
@lennney
lennney marked this pull request as ready for review July 29, 2026 05:28
@lennney
lennney requested a review from a team as a code owner July 29, 2026 05:28
@gemini-code-assist

Copy link
Copy Markdown
Contributor

Caution

The consumer version of Gemini Code Assist on GitHub has been sunset. All code review activity has officially ceased.

@dosubot dosubot Bot added the size:S This PR changes 10-29 lines, ignoring generated files. label Jul 29, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size:S This PR changes 10-29 lines, ignoring generated files. typescript Typescript related functionality

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant