Skip to content

test: add credential-gated Pinecone integration test - #74

Open
pinecone-groundskeeper[bot] wants to merge 1 commit into
mainfrom
agent/maintenance/issue-25-123d8286f9bf9d98
Open

test: add credential-gated Pinecone integration test#74
pinecone-groundskeeper[bot] wants to merge 1 commit into
mainfrom
agent/maintenance/issue-25-123d8286f9bf9d98

Conversation

@pinecone-groundskeeper

@pinecone-groundskeeper pinecone-groundskeeper Bot commented Aug 29, 2026

Copy link
Copy Markdown
Contributor

⚠️ This PR modifies CI workflow files. They execute on the PR branch before merge — review the workflow diff directly; do not treat a passing check as review.

Problem

The suite under tests/ is unit-only — every Pinecone call is mocked (see tests/chunkedUpsert.test.ts). Nothing exercises a real upsert/query round-trip through the SDK, which is exactly the kind of gap that let a v2→v8 response-shape mismatch (#5) ship undetected: a unit test can't catch a wire-format change, only a real call can.

Solution

Added tests/integration/pinecone.test.ts:

  • Wrapped in describe.skipIf(!process.env.PINECONE_API_KEY), so it's a clean no-op without credentials.
  • Creates a uniquely-named throwaway serverless index (recommender-it-<8 hex chars>), using PINECONE_CLOUD/PINECONE_REGION for the spec, matching the pattern already used in src/index.ts.
  • Upserts 5 hand-built 384-dim records through the real chunkedUpsert path (the same helper src/index.ts uses), then queries and asserts on the returned records' shape (id, 384-length values, metadata).
  • Polls the query (up to 10 attempts / 3s apart) instead of a fixed sleep, to ride out Pinecone's eventual consistency after an upsert.
  • Tears the index down in afterAll, gated on creation having actually succeeded, so a failed beforeAll doesn't throw again on cleanup.

Kept this off the default npm test path so it can never accidentally run credentialed or uncredentialed inside a PR:

  • vitest.config.ts now excludes tests/integration/** explicitly (spread onto vitest's own configDefaults.exclude rather than replacing it).
  • New vitest.integration.config.ts scopes to tests/integration/**/*.test.ts with a 120s test/hook timeout (serverless index creation is slow).
  • New npm run test:integration script runs that config.
  • ci.yml gets a workflow_dispatch trigger and a new integration-test job gated if: github.event_name != 'pull_request' (push to main or manual dispatch only — a fork PR must never get access to PINECONE_API_KEY). The job passes PINECONE_API_KEY/PINECONE_CLOUD/PINECONE_REGION from secrets; if those secrets aren't configured yet the test's own skipIf makes the job a harmless no-op rather than a failure.

User impact

Before: npm test only ever touches mocks; a real SDK regression is invisible until someone runs npm run index/npm run recommend by hand.

After: npm run test:integration (and CI's integration-test job, once PINECONE_API_KEY/PINECONE_CLOUD/PINECONE_REGION secrets exist on this repo) actually round-trips a vector through a live index. npm test is unaffected — still 5 files / 23 tests, no network.

Follow-ups

  • The repo doesn't yet have PINECONE_API_KEY/PINECONE_CLOUD/PINECONE_REGION configured as Actions secrets (out of my reach — that's a repo-settings change a human needs to make). Until then integration-test runs but skips.

Verification

  • npm run typecheck — clean.
  • npm run lint / npm run format:check — clean.
  • npm test — 5 files, 23 tests, unchanged; integration file untouched.
  • npx vitest run -c vitest.integration.config.ts — 1 test, 1 skipped (no PINECONE_API_KEY in this sandbox), confirming the self-skip path works.
  • Could not exercise the actual live-index path (no Pinecone credentials available in this environment) — that requires the secrets above to be configured, at which point the integration-test CI job will run it for real.

Closes #25

Generated by GitHub Maintenance Fixer · sonnet50 · 124.1 AIC · ⌖ 12.8 AIC · ⊞ 9.4K ·


Note

Low Risk
Changes are test and CI-only with PR/fork secret isolation and self-skip when credentials are absent; no production runtime paths are modified.

Overview
Adds a live Pinecone upsert/query check so SDK/wire-format regressions are not limited to mocked unit tests. A new tests/integration/pinecone.test.ts creates a throwaway serverless index, runs chunkedUpsert with real records, polls until queries see the upserts, and asserts match shape (id, vector length, metadata).

Default npm test stays offline: vitest.config.ts excludes tests/integration/**, and npm run test:integration uses new vitest.integration.config.ts (longer timeouts).

CI gains workflow_dispatch and an integration-test job that runs only on push to main or manual dispatch (never on pull_request), injecting PINECONE_* secrets. The suite **skipIf**s when the API key is missing so missing secrets do not fail the job.

Reviewed by Cursor Bugbot for commit 5b57c29. Bugbot is set up for automated code reviews on this repo. Configure here.

@pinecone-groundskeeper pinecone-groundskeeper Bot added the agent:maintenance Opened/maintained by the maintenance agent; it recognizes its own work by this label label Aug 29, 2026
@pinecone-groundskeeper
pinecone-groundskeeper Bot requested review from a team as code owners August 29, 2026 12:22
@pinecone-groundskeeper pinecone-groundskeeper Bot added the agent:maintenance Opened/maintained by the maintenance agent; it recognizes its own work by this label label Aug 29, 2026

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Want higher recall? High effort reviews run extra passes and find more bugs. A team admin can switch effort levels in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 5b57c29. Configure here.

if (indexCreated) {
await pinecone.deleteIndex(indexName);
}
});

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Throwaway indexes can leak

Medium Severity

indexCreated flips only after createIndex with waitUntilReady resolves, and teardown runs only in afterAll. A hook timeout or a cancelled CI job (this workflow uses cancel-in-progress) leaves the live recommender-it-* index behind, which can exhaust the project index quota and fail later creates.

Additional Locations (1)
Fix in Cursor Fix in Web

Reviewed by Cursor Bugbot for commit 5b57c29. Configure here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

agent:maintenance Opened/maintained by the maintenance agent; it recognizes its own work by this label

Projects

None yet

Development

Successfully merging this pull request may close these issues.

test: add credential-gated integration test against a live Pinecone index

0 participants