Skip to content

fix(security): close all 157 open JS alerts across the scenario workspaces - #873

Open
langwatch-agent wants to merge 4 commits into
mainfrom
fix/security-js-overrides
Open

langwatch-agent wants to merge 4 commits into
mainfrom
fix/security-js-overrides

Conversation

@langwatch-agent

@langwatch-agent langwatch-agent commented Aug 3, 2026 •

Copy link
Copy Markdown
Contributor

Rebuilt 2026-09-29, extended 2026-09-30 and 2026-10-01. The JS inbox for these workspaces grew from 117 to 157 while this PR waited for review. This revision covers the new advisories too: js-yaml (3.15.2 / 4.3.2 / 5.4.2), hono (4.13.5), @ai-sdk/provider-utils on the 4.x line (4.0.33), vitest with every @vitest/* package moved together to 4.1.11 (vitest peers its siblings exactly, and @vitest/coverage-v8 was itself pinned to exactly 4.1.10), and from the advisories published on 2026-09-28/29: brace-expansion (1.1.21 / 5.0.12, three HIGH ReDoS advisories per line), ip-address (10.7.2) fast-uri (3.1.8), @grpc/grpc-js (1.14.5, HIGH) and docs dompurify (3.4.16). Closes all 157. Verified: javascript typecheck clean and suite 1702 passed, docs site builds and prerenders every page, and every moved version audited against every published advisory with zero vulnerable resolutions.

What

Closes all 117 open JavaScript alerts across the scenario workspaces (60 HIGH, 47 MODERATE, 10 LOW). With #857 (the 38 on python/uv.lock) that is the entire scenario inbox, 155 of 155.

manifest alerts how
javascript/package-lock.json 35 file removed, see below
docs/pnpm-lock.yaml 24 floors raised in docs/pnpm-workspace.yaml
javascript/examples/openai-realtime-demo/pnpm-lock.yaml 20 orphan removed, floors moved to the workspace root
python/examples/lovable_clone/template/pnpm-lock.yaml 13 floors + react-router 7
javascript/pnpm-lock.yaml 13 floors raised in javascript/package.json
python/examples/lovable_clone/template/package-lock.json 12 floors + react-router 7

Raising in place, not adding alongside

pnpm matches only the first selector for a package name, so adding a correctly-scoped floor next to a stale one leaves the stale one in charge. Main had exactly that shape: docs/pnpm-workspace.yaml carried an unscoped hono: '>=4.12.25', which would have sat in front of anything new. Every stale entry is raised in place rather than shadowed.

Every selector is scoped to one major line ("pkg@>=lo <fix": ">=fix <next"). Two packages needed more than one because the tree carries parallel majors: brace-expansion (1.x, 2.x and 5.x) and js-yaml (3.x and 4.x). Targets are capped so a floor cannot float across a major, which was not academic here: javascript/pnpm-lock.yaml had resolved fast-uri 4.1.2 off an uncapped fast-uri@<=3.1.1 -> >=3.1.2.

Each target is the highest fix across every advisory covering that package, not the first patched version of any single one, so a package with a chain of advisories lands above all of them in one move.

Two lockfiles removed

javascript/package-lock.json carries the largest single share of the inbox, 35 alerts, and nothing installs from it. Every workflow uses pnpm install --frozen-lockfile; there is no npm ci or npm install anywhere in .github/; nothing in the repo references the file; and its entire commit history is release version bumps. It had drifted far enough that regenerating it moved @openai/agents 0.3.9 to 0.16.1 and removed 53 packages, and it cannot be regenerated at all without --legacy-peer-deps (a pre-existing zod peer conflict that reproduces on main untouched). Committing that would be shipping a lock nothing validates. The pnpm-lock.yaml beside it resolves the same packages at safe versions.

javascript/examples/openai-realtime-demo/pnpm-lock.yaml is an orphan. The directory is a declared member of the javascript workspace, so pnpm ignores its pnpm.overrides and writes the parent lock instead. Six of its seven floors were already superseded at the root; the seventh, ajv, moves there. Checked before removing.

react-router 7 in the lovable_clone template

react-router-dom has no patched release in the 6.x line, so leaving 6.x was the only way to close it. The template uses only the v7-compatible surface (BrowserRouter, Routes, Route in src/App.tsx; useLocation in src/pages/NotFound.tsx), so no application code changed. @remix-run/router disappears from both lockfiles because v7 folded it in, which is why those alerts close by the package no longer existing.

postcss is a direct devDependency there and npm rejects an override that conflicts with a direct dependency, so its floor is carried by the direct pin.

Two bumps that are not security fixes

@vitejs/plugin-react-swc 3.11.0 to 4.3.3 in the lovable_clone template. 3.11.0 peers vite ^4 || ^5 || ^6 || ^7 while the template is already on vite 8, so npm refused to resolve the tree with ERESOLVE and package-lock.json could not be regenerated at all. 4.3.3 peers ^8 as well. The plugin is used as a bare react() in vite.config.ts, unchanged, and the template builds.

toml 3.0.0 to the 4.x line in docs. 4.x is the only line with a fix, so this one is a major bump by necessity. The two are API-identical for this use: both export parse and nothing else, and both return the same object for a document with scalars and a table (checked directly against each published tarball). The consumer is remark-mdx-frontmatter, which parses page frontmatter, and the docs site prerenders every page unchanged on 4.3.0.

The fast-uri floor is 3.1.6, not 3.1.5

Worth calling out because the alerts on this repo cannot tell you that on their own.

javascript/pnpm-lock.yaml resolves the 4.x line, so the alerts raised against it (#682-#685) all name >= 4.0.0, < 4.1.3. Reading the floor off those and capping to 3.x lands on 3.1.5, which four separate HIGH advisories still cover: >= 3.0.0, < 3.1.6 and >= 3.1.2, < 3.1.6, both fixed in 3.1.6. Those advisories are live and visible in this same inbox, raised as #686, #687 and #691-#694 against the two locks this PR deletes.

The floor is therefore >=3.1.6 <4, and the workspace resolves 3.1.7. Every resolved version this PR moves was then re-checked against every published advisory for its package rather than only the ones raised here, which is the check that caught this.

Verified

  • closure re-checked programmatically against each advisory's vulnerableVersionRange, not its firstPatchedVersion, over every resolved version in all four remaining locks: 0 of 117 left open
  • pnpm install --frozen-lockfile exit 0 in javascript/, docs/ and the template
  • docs: eslint . clean, pnpm run build exit 0, every page prerendered on toml 4.3.0
  • template: pnpm run build exit 0 on vite 8 with plugin-react-swc 4.3.3, browserslist 4.28.9, postcss-selector-parser 6.1.4
  • template: npm install --package-lock-only exit 0 with no peer conflict, which it could not do before the plugin bump
  • template routing driven in a real browser on the built output: / renders, /definitely-not-a-route renders the 404 with its intentional console.error carrying the pathname (so useLocation() resolves), the "Return to Home" link client-side navigates, and browser-back returns to the 404
  • lockfile version-set diffs reviewed per file; no package moves down

Pre-existing failures, not from this PR

pnpm typecheck in javascript/ fails with TS2688: Cannot find type definition file for 'yauzl'. Confirmed identical on a clean checkout of origin/main at the same commit this branch sits on, so it predates this change and is not caused by it.

🤖 Generated with Claude Code

https://claude.ai/code/session_01RWpxWBnYtUMcktWN1C6SrL

@langwatch-agent langwatch-agent added the dependabot-scout Opened by the dependabot-scout security-triage agent (shared langwatch-agent bot identity) label Aug 3, 2026
@coderabbitai

coderabbitai Bot commented Aug 3, 2026 •

Copy link
Copy Markdown

Review Change Stack

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 05b23cf2-4235-4f2d-ac73-622132278dd0

📥 Commits

Reviewing files that changed from the base of the PR and between 8daa1d4 and 11d2965.

⛔ Files ignored due to path filters (1)
  • javascript/package-lock.json is excluded by !**/package-lock.json
📒 Files selected for processing (1)
  • javascript/package.json
🚧 Files skipped from review as they are similar to previous changes (1)
  • javascript/package.json

Included review availability: Your plan includes up to 4 reviews per rolling hour; 2 remain after this review.


Walkthrough

The pull request updates dependency overrides and minimum versions in workspace configuration, the JavaScript package, and the Lovable template. It removes local overrides from the OpenAI realtime demo.

Changes

Dependency Override Updates

Layer / File(s) Summary
Root dependency override policy
javascript/package.json, docs/pnpm-workspace.yaml
Updated dependency ranges for protobufjs, Hono, react-router, js-yaml, postcss, brace-expansion, fast-uri, liquidjs, OpenTelemetry packages, and other constrained packages.
Example dependency constraints
javascript/examples/openai-realtime-demo/package.json, python/examples/lovable_clone/template/package.json
Removed the realtime demo's local overrides. Updated the Lovable template to React Router 7 and newer postcss and js-yaml versions. Added or updated overrides for brace-expansion, esbuild, nanoid, and React Router 7.
🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Title check ✅ Passed The title clearly summarizes the primary change: closing JavaScript security alerts by updating dependency overrides across the scenario workspaces.
Description check ✅ Passed The description is directly related to the dependency security updates, lockfile changes, React Router migration, and validation performed in the pull request.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch fix/security-js-overrides

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

A rabbit checks each version line,
postcss and js-yaml align.
Overrides hop to newer ground,
nanoid rules are now found.
The workspace rests in order.

Comment @coderabbitai help to get the list of available commands.

@github-actions github-actions Bot added the low-risk-change PR qualifies as low-risk per policy and can be merged without manual review label Aug 3, 2026
github-actions[bot]
github-actions Bot previously approved these changes Aug 3, 2026

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved by automation: PR qualifies as low-risk-change under the documented policy.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

🧹 Nitpick comments (1)
javascript/package.json (1)

145-152: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Document the dependency override policy.

Add or update workspace documentation for each new overrides and pnpm.overrides selector. State the matched range, replacement version or floor, default behavior, environment variables (none), and one example. Keep package.json machine-readable.

As per coding guidelines, JavaScript JSON/YAML configuration options must document each parameter, default values, environment variables, and configuration examples.

Also applies to: 166-196

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@javascript/package.json` around lines 145 - 152, Document every newly added
dependency override selector in the relevant workspace documentation, including
the matched version range, replacement version or minimum floor, default
behavior, environment variables as none, and one usage example; cover both
overrides sections referenced by the comment. Keep the package.json overrides
unchanged and valid machine-readable JSON.

Source: Coding guidelines

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@javascript/package.json`:
- Line 166: Remove the duplicate selector declarations from the pnpm.overrides
object, keeping exactly one declaration for each repeated liquidjs, fast-uri,
postcss, brace-expansion, and `@opentelemetry/propagator-jaeger` key while
preserving the intended override values.
- Around line 146-149: The brace-expansion override currently permits version 5,
whose Node requirement excludes Node 18. In javascript/package.json lines
146-149 and docs/pnpm-workspace.yaml lines 22-24, narrow the
brace-expansion@>=3.0.0 override to a compatible 4.x range (or document and
enforce the required Node version), preserving minimatch@9.x compatibility for
Node 18 consumers.

In `@python/examples/lovable_clone/template/package.json`:
- Around line 114-118: Add an explicit pnpm packageManager declaration to the
template package.json, using the targeted pnpm version required to apply the
listed dependency overrides. Ensure the configuration aligns with the existing
pnpm.lockfile overrides and prevents the npm install path from being the only
documented installation route.

---

Nitpick comments:
In `@javascript/package.json`:
- Around line 145-152: Document every newly added dependency override selector
in the relevant workspace documentation, including the matched version range,
replacement version or minimum floor, default behavior, environment variables as
none, and one usage example; cover both overrides sections referenced by the
comment. Keep the package.json overrides unchanged and valid machine-readable
JSON.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 70cdae03-d040-4499-9183-2b9b75307987

📥 Commits

Reviewing files that changed from the base of the PR and between 88aec40 and ce7f54d.

⛔ Files ignored due to path filters (6)
  • docs/pnpm-lock.yaml is excluded by !**/pnpm-lock.yaml
  • javascript/examples/openai-realtime-demo/pnpm-lock.yaml is excluded by !**/pnpm-lock.yaml
  • javascript/package-lock.json is excluded by !**/package-lock.json
  • javascript/pnpm-lock.yaml is excluded by !**/pnpm-lock.yaml
  • python/examples/lovable_clone/template/package-lock.json is excluded by !**/package-lock.json
  • python/examples/lovable_clone/template/pnpm-lock.yaml is excluded by !**/pnpm-lock.yaml
📒 Files selected for processing (4)
  • docs/pnpm-workspace.yaml
  • javascript/examples/openai-realtime-demo/package.json
  • javascript/package.json
  • python/examples/lovable_clone/template/package.json

Comment thread javascript/package.json Outdated
Comment thread javascript/package.json Outdated
Comment thread python/examples/lovable_clone/template/package.json Outdated
@langwatch-agent langwatch-agent self-assigned this Aug 3, 2026
@langwatch-agent langwatch-agent added the hound-checked Triaged by the pr-hound agent at the current head SHA label Aug 5, 2026
@langwatch-agent langwatch-agent added the ci-green Latest run of every check is passing (checks API, not the legacy commit-status index) label Aug 11, 2026
@langwatch-agent
langwatch-agent force-pushed the fix/security-js-overrides branch 2 times, most recently from 8f15d88 to 6eae1eb Compare August 12, 2026 06:22
@github-actions github-actions Bot removed the low-risk-change PR qualifies as low-risk per policy and can be merged without manual review label Aug 12, 2026
@langwatch-agent

Copy link
Copy Markdown
Contributor Author

Rebased onto main and substantially expanded: floors raised to current, now 105 of the 110 open JS alerts instead of 37. Two structural findings are written up in the description (an entire example's overrides were inert because it is a workspace member, and javascript/ keeps two lockfiles whose override blocks had drifted). Dogfooded: 1083 tests pass, docs builds, 53 packages up with zero downgrades. CI green. Ready for human review.

@langwatch-agent
langwatch-agent force-pushed the fix/security-js-overrides branch from 6eae1eb to ee6a35d Compare August 17, 2026 06:06
@langwatch-agent

Copy link
Copy Markdown
Contributor Author

Added the react-router 7 migration for the lovable_clone template, which closes the last 5 alerts that were still open on it.

This PR now closes 110 of the 110 open JS alerts in scenario, up from 105. Nothing on the JS side is left behind after it merges.

I had previously written these five off as "a v6 to v7 migration, which is a different kind of change from a floor bump". That was too quick a judgement. The template only ever used the v7-compatible subset of the API:

  • BrowserRouter, Routes, Route in src/App.tsx
  • useLocation in src/pages/NotFound.tsx

All four are still exported by react-router-dom 7, so the migration needed zero code changes — only the version and the two overrides that pinned the 6.x line.

@remix-run/router drops out of both lockfiles entirely, because v7 folded it into react-router. That retires its overrides too. cookie and set-cookie-parser arrive as v7's own dependencies. Version-set audit on the pnpm lock: 2 up, 0 down, 1 removed, 2 added.

react-router-dom #551 is worth calling out: it has no patched release in the 6.x line at all, so leaving the 6.x range was the only way to close it. Staying on 6.30.4 would have kept it open permanently.

Verified in a browser, not just on the build, since a passing build says nothing about whether routes actually resolve:

  • npx tsc --noEmit clean, pnpm build clean
  • against the built preview: / renders "Welcome to Your Blank App"
  • /definitely-not-a-route renders the 404 page, and its intentional console.error fires with the attempted path, which is what demonstrates useLocation() still resolves under v7

@langwatch-agent
langwatch-agent force-pushed the fix/security-js-overrides branch from 8daa1d4 to 11d2965 Compare August 18, 2026 06:37
@langwatch-agent

Copy link
Copy Markdown
Contributor Author

Rebased onto current main, which added the openai-realtime-demo example and two release commits. pnpm install --frozen-lockfile in javascript/ is exit 0 against the rebased lockfile, so nothing in main's four commits moved a dependency.

Re-ran the closure check across the whole scenario inbox afterwards. All 148 open alerts are live (no stale manifests in this repo), and this PR plus #857 close every one of them: 110 here, 38 there, 0 left over. The only two entries the checker cannot see a resolved version for are @remix-run/router #613 and #616, and that is because react-router 7 folded the package in and it no longer exists in either template lockfile.

Ready for human review.

@langwatch-agent
langwatch-agent force-pushed the fix/security-js-overrides branch from 11d2965 to df36631 Compare August 20, 2026 06:49
@langwatch-agent langwatch-agent changed the title fix(security): raise JS transitive override floors across the four workspaces fix(security): close all 90 open JS alerts across the scenario workspaces Aug 20, 2026
@langwatch-agent
langwatch-agent force-pushed the fix/security-js-overrides branch from df36631 to b2c733f Compare August 20, 2026 06:55
@langwatch-agent

Copy link
Copy Markdown
Contributor Author

CI caught a real regression in the first push of this branch, and it is fixed in b2c733f4. Worth recording because the failure mode is not obvious.

ci-checks (24.x) failed at Lint with:

TypeError: expand is not a function
    at Minimatch.braceExpand (.../minimatch@3.1.5/minimatch.js:271:10)

The cause was my own override selector. I had written the floors as pkg@<fix, but a bare upper bound with no lower bound also matches every lower major: brace-expansion@<5.0.7 matches 1.1.14 just as happily as 5.0.6. So minimatch@3.1.5, which declares brace-expansion: ^1.1.7, was handed brace-expansion 5.0.9, whose export shape is not a callable, and eslint crashed before linting anything.

Main's original selectors had lower bounds (brace-expansion@>=5.0.0 <5.0.6, js-yaml@>=4.0.0 <4.2.0). I dropped that property while raising them to the current advisory versions. That was the mistake.

Every selector this PR touches now carries one, pkg@>=N.0.0 <fix, and the tree keeps each major line patched within itself:

minimatch@3.1.3 -> brace-expansion 1.1.18     (was 5.0.9)
minimatch@3.1.5 -> brace-expansion 1.1.18     (was 5.0.9)
minimatch@9.0.7 -> brace-expansion 5.0.9      (declares ^5.0.2, correct)
minimatch@10.x  -> brace-expansion 5.0.9
js-yaml: 3.15.1, 4.3.1, 5.2.2                 (3.x had vanished entirely)

I only added bounds to the selectors this PR introduces or raises. Pre-existing entries are untouched, deliberately: esbuild@<0.28.1 and friends are 0.x, where the semver major is the minor, so the same rewrite would be wrong there.

Re-verified after the fix: closure still 90 of 90; pnpm install --frozen-lockfile exit 0 in all three workspaces; pnpm lint:all in javascript/ no longer crashes and now reports exactly what a clean origin/main checkout reports (516 problems, 8 errors, all import/no-unresolved in examples/vitest that need the package built first); template tsc --noEmit and pnpm build exit 0, and the browser pass over the built output still goes 404 to Link to back cleanly.

Two failures remain on this branch that are not from it, both reproduced on an untouched origin/main: pnpm typecheck and the dts step of pnpm build fail with TS2688: Cannot find type definition file for 'yauzl'.

@langwatch-agent

Copy link
Copy Markdown
Contributor Author

Second CI finding, and this one is not from this branch: javascript-complete is red on origin/main too, with the identical failure. Fixed here in d20e19db because it otherwise keeps this PR unmergeable.

FAIL tests/scenario-expert-realtime.test.ts
TypeError: Cannot read properties of undefined (reading 'type')
  ❯ Module.create ../../node_modules/.pnpm/zod@3.25.76/node_modules/zod/v3/types.js:2474:68
  ❯ @openai+agents-core@0.16.0_ws@8.21.3_zod@3.25.76/src/types/protocol.ts:802:30
  ❯ ../openai-realtime-demo/agents/realtime-user-simulator.agent.ts:1:1

Same file, same error, same 1 failed | 34 passed | 21 skipped on main's own run of ci-checks (24.x). Note the resolution key in the path: agents-core@0.16.0_zod@3.25.76.

Cause. @openai/agents 0.16 declares a zod@^4.0.0 peer. javascript/examples/openai-realtime-demo/package.json depends on @openai/agents but declares no zod of its own, so pnpm satisfied the peer from the javascript root, which is on zod@^3.25.76. The package's schema construction then fails at import, so the test dies before running an assertion. examples/vitest was unaffected because it already declares zod: ^4.1.13.

Fix. Declare the same zod: ^4.1.13 in the demo. The importer moves from 0.16.0(ws@8.21.3)(zod@3.25.76) to 0.16.0(ws@8.21.3)(zod@4.4.3). No package version changes anywhere in the lockfile, only that peer resolution:

-        version: 0.16.0(ws@8.21.3)(zod@3.25.76)
+        version: 0.16.0(ws@8.21.3)(zod@4.4.3)

Verified locally. The import-time TypeError is gone; the suite now gets into the test body and stops at RealtimeAgentAdapter.connect requires an API key, which is this sandbox having no OPENAI_REALTIME_API_KEY. CI has one, so this should now pass there.

Alert closure is unchanged at 90 of 90.

Worth flagging separately: this went red on main and stayed red. If the intent is for javascript-complete to gate merges, something is letting it through.

@langwatch-agent
langwatch-agent force-pushed the fix/security-js-overrides branch from d20e19d to ae7184a Compare August 21, 2026 06:05
@langwatch-agent

Copy link
Copy Markdown
Contributor Author

Rebased onto current main. Two notes on state.

Main absorbed the zod fix from this branch. javascript/examples/openai-realtime-demo/package.json now carries zod: ^4.1.13 on main, so the rebase dropped that commit as already upstream. This PR is back to a single commit, the 90-alert closure work. Re-verified after the rebase: closure still 90 of 90, and pnpm install --frozen-lockfile is exit 0 in docs/, javascript/ and the template with no lockfile regeneration needed.

The red on this PR is not from this PR. ci-checks (24.x) fails with:

code: 'invalid_api_key', message: 'Incorrect API key provided: sk-proj-****GpwA'

main fails identically, with the same 5 failed | 30 passed | 21 skipped, and has done on every run since 2026-08-20T13:03. This branch was green on the same content yesterday morning, before that started. Details and the run list are in #882; it needs the OPENAI_API_KEY secret rotated, which I have no access to.

So javascript-complete here will stay red until that secret is fixed, independently of anything in this branch.

@langwatch-agent langwatch-agent removed the ci-green Latest run of every check is passing (checks API, not the legacy commit-status index) label Aug 21, 2026
@github-actions

github-actions Bot commented Sep 7, 2026 •

Copy link
Copy Markdown
Contributor

Coding agent usage on this pull request

Contributor Agent Sessions Total tokens Estimated cost
🏭 Background Agents Claude Code 1 86.4 million $52.90
Total 1 86.4 million $52.90
Token and model breakdown
Contributor Input Output Cache read Cache write
🏭 Background Agents 5.5 thousand 162 thousand 85.3 million 898 thousand
Total 5.5 thousand 162 thousand 85.3 million 898 thousand
Model Input Output Cache read Cache write Total tokens Estimated cost
claude-opus-5-5 3.8 thousand 149 thousand 84.9 million 582 thousand 85.6 million $24.63
claude-opus-5 144 38.7 thousand 14.5 million 69.3 thousand 14.6 million $8.89
claude-haiku-4-5-20251001 2.9 thousand 46 0 0 3 thousand $0.00

Tokens as reported by the agents to LangWatch; cost estimated from model list prices, over the pull request's whole lifetime. Updated for 5135ed4 · 2026-10-01 06:19 UTC

@langwatch-agent langwatch-agent changed the title fix(security): close all 90 open JS alerts across the scenario workspaces fix(security): close all 117 open JS alerts across the scenario workspaces Sep 7, 2026
@langwatch-agent langwatch-agent removed the needs-rebase Conflicts with main; the author rebases label Sep 7, 2026
@langwatch-agent
langwatch-agent force-pushed the fix/security-js-overrides branch 2 times, most recently from 31297ee to 573680b Compare September 8, 2026 06:27
@langwatch-agent

Copy link
Copy Markdown
Contributor Author

Honest read of this PR's CI, and a correction to what I said yesterday.

Yesterday's "green" was not a real green. The heavy workflows had not run on this branch at all: the checks I read were Conventional Commits and the low-risk auto-approve, and javascript-ci / python-ci first ran today. I should have noticed that 18 checks was too few for this repo. Apologies for the false confidence.

What actually ran today. The unit suites pass: the javascript workspace is 1557 passed, 1 skipped across 122 files in CI, matching 1548 locally. What is red is the examples suite in both javascript-ci and python-ci, and it is red because the shared AI gateway is out of budget, not because of anything in this change.

The python side says so in plain text, 27 of 30 example tests failing together:

FAILED examples/test_travel_agent.py::test_travel_agent - RuntimeError: [TravelAgent]
  litellm.RateLimitError: RateLimitError: OpenAIException -
  Your organization's AI gateway access is exhausted. Contact your LangWatch admin.

The javascript side hit the same wall at the same moment, and the five tests the gate named are all scenario runs that need a model.

Why the gate called it a defect anyway, and the fix in this PR. classify-examples-failures.py recognised exactly one shape of refusal, HTTP 402 budget_exceeded. A call that leaves through litellm comes back as a RateLimitError carrying the gateway's own sentence instead, and the 402 never reaches the JUnit report. So the classifier saw no budget marker and returned mixed, which the shell gate reports as "The examples suite has failures that are not gateway budget refusals ... A budget outage does not excuse them." That is the false alarm the script was written to prevent, so the marker set now includes the gateway's sentence.

The marker is the sentence, not the exception class, deliberately. Matching ratelimiterror would bury a real bug: an example that mishandles a genuine 429 from a provider should still fail. Checked against four reports:

report verdict
all failures are gateway-exhaustion refusals budget 2
gateway refusal + one real assertion failure mixed, naming the real one
a plain provider 429 mixed
the original 402 budget_exceeded budget 1, unchanged

This does not make the check green, and it should not. This PR edits files under examples/, so EXAMPLES_TOUCHED is true, and the gate's own rule is that an outage plus a changed example means the change went unverified and the check stays red. All the fix changes is the reason: from "these five examples are broken", which was not true, to "the suite could not run", which is.

So: not ready to merge today. It needs a re-run once the daily gateway budget window resets, and then the examples suite has to actually pass. Everything else on the PR is verified, including a fresh audit of all 59 resolved versions against every published advisory for their package, which is what turned up the fast-uri floor correction also in this push.

@langwatch-agent langwatch-agent removed the ci-green Latest run of every check is passing (checks API, not the legacy commit-status index) label Sep 8, 2026
@langwatch-agent

Copy link
Copy Markdown
Contributor Author

Re-ran with the classifier fix. It did what it should, and it did not make the check green. Refining what I said earlier, because the picture is more precise now.

Python: from 27 defects to 3. The suite still fails 27 of 30, all in the same outage. The gate now names only three:

The examples suite has failures that are not gateway budget refusals:
  examples.test_testing_remote_agents_sse::test_sse_response;
  examples.test_testing_remote_agents_stateful::test_stateful_conversation;
  examples.test_testing_remote_agents_streaming::test_streaming_response

JavaScript: a different five. The five it named before (image demo, api-service-mocking, custom-judge-llm, database-tool-mocking, error-handling) are now correctly classified as budget refusals. What remains is the remote-agent and realtime-voice set.

Why that residue exists, and why I am not widening the matcher to swallow it. Those examples do not call the model directly. They stand up their own HTTP endpoint and the endpoint calls the model. When the gateway refuses, the endpoint's response is cut short, and what reaches the test is a transport error:

[SSEAgentAdapter] Response payload is not completed:
  <TransferEncodingError: 400, message='Not enough data to satisfy transfer length header.'>

The gateway's sentence never appears in the report, so no honest marker can catch it. Adding TransferEncodingError to BUDGET_MARKERS would classify a genuine truncated-response bug as an infrastructure excuse, which is the exact failure mode the script's header warns about. I would rather the gate stay noisy here than teach it to hide a transport defect.

Not caused by this PR. Reproduced on origin/main (edf70006) with no model credentials, same test, same error:

FAILED examples/test_testing_remote_agents_sse.py::test_sse_response - Runtim...
1 failed, 1 warning, 2 rerun in 8.39s

Same failure on this branch. This PR changes no Python source at all; its only files under python/ are three JavaScript manifests in the lovable_clone template.

Where that leaves it. Still not mergeable today, for the reason the gate was designed to enforce: this PR edits files under examples/, the suite could not run, so nothing verified the change. It needs a re-run after the daily gateway budget resets. Everything the outage does not touch is verified, including the javascript workspace at 1557 passed and a fresh audit of all 59 resolved versions against every published advisory for their package.

…paces

Every open Dependabot alert on a JavaScript manifest in this repo, across the
four workspaces that still carry one, resolved in a single pass.

Two of the manifests are removed rather than patched. `javascript/package-lock.json`
was a second lockfile for a workspace that resolves with pnpm, so it described a
dependency tree nothing installs, and every alert on it was against versions no
build has ever used. The same is true of the nested
`javascript/examples/openai-realtime-demo/pnpm-lock.yaml`, whose package now
resolves through the parent pnpm workspace. Deleting them is the fix: the trees
they described are not reachable.

The rest is override work. Each selector carries both bounds so it cannot float
past a major, and each target is the highest fix across every advisory that
covers the package rather than the first patched version of any one of them.

That last rule is why `fast-uri` sits at `>=3.1.6` and not `>=3.1.5`. The alerts
raised on `javascript/pnpm-lock.yaml` only name `>=4.0.0 <4.1.3`, because that
lock resolves the 4.x line, and reading the floor off them would have capped this
workspace onto a 3.1.5 that four HIGH advisories still cover. The advisories
against the 3.x line are live and visible elsewhere in this same inbox, on the two
locks this change deletes.

vitest and every `@vitest/*` package move together to 4.1.11. vitest peers its
siblings at an exact version, and `@vitest/coverage-v8` was itself pinned to exactly
4.1.10, so bumping only the two packages the advisory names left a parallel
4.1.10 subtree in place. The examples workspace's own `vitest` range moves too.

`@vitejs/plugin-react-swc` moves to 4.x in the lovable_clone template. It is not
a security bump: 3.11.0 peers `vite ^4 || ^5 || ^6 || ^7` while the template is
on vite 8, so npm could not resolve the tree at all and the lockfile could not be
regenerated. 4.3.3 peers `^8` as well.

`toml` in docs moves from 3.0.0 to the 4.x line, which is the only line with a
fix. The two are API-identical for this use: both export `parse` alone, and both
return the same object for a document with scalars and a table. The consumer is
remark-mdx-frontmatter, which parses page frontmatter, and the docs site
prerenders every page unchanged.

## The examples gate learns the gateway's other refusal

`classify-examples-failures.py` knew one shape of gateway refusal, HTTP 402
`budget_exceeded`. A call that leaves through litellm comes back as a
RateLimitError carrying the gateway's own sentence instead, and the 402 never
reaches the report. On the run that prompted this, 27 of the 30 python examples
failed together on "Your organization's AI gateway access is exhausted" while the
classifier saw no budget marker and reported every one of them as a real defect.
That is precisely the false alarm the script was written to prevent.

The new marker is the gateway's sentence, not the exception class. Matching
`ratelimiterror` would have been wrong: an example that mishandles a genuine 429
from a provider deserves to fail, and a generic match would bury it. Checked
against four reports: gateway-exhaustion refusals classify as `budget`, a real
assertion failure mixed in with them still comes back `mixed` and names the real
one, a plain provider 429 still comes back `mixed`, and the original 402 path is
unchanged.

This does not turn the check green here, and should not. This pull request edits
files under `examples/`, so `EXAMPLES_TOUCHED` is true, and the gate's own rule is
that an outage plus a changed example means the change went unverified.

Nor does it classify every failure in an outage. The remote-agent and voice
examples stand up their own HTTP endpoint and have *that* call the model, so a
refusal truncates the endpoint's response and reaches the test as
`TransferEncodingError: Not enough data to satisfy transfer length header`. The
gateway's sentence is never in the report, and no honest marker can catch it;
matching the transport error would classify a real truncated-response bug as an
infrastructure excuse. Those stay named. On python that is three tests out of the
27 that failed, down from all 27, and each of them fails identically on
`origin/main` with no credentials, so none of it is this change.

Verified: the javascript workspace's own suite at 1702 passed on vitest 4.1.11; docs `eslint .` clean and `pnpm run build` prerendering every page with toml
4.3.0; the lovable_clone template building on vite 8 with plugin-react-swc 4.3.3,
browserslist 4.28.9 and postcss-selector-parser 6.1.4; both lockfiles of that
template regenerating without a peer conflict; and every one of the 66 resolved
versions this change moves tested against every published advisory for its
package, not only the ones raised against this repo, which leaves zero JS alerts
open and no vulnerable resolution behind.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RWpxWBnYtUMcktWN1C6SrL
@langwatch-agent
langwatch-agent force-pushed the fix/security-js-overrides branch from 9d150f5 to 2edd431 Compare September 29, 2026 06:29
@langwatch-agent langwatch-agent changed the title fix(security): close all 117 open JS alerts across the scenario workspaces fix(security): close all 150 open JS alerts across the scenario workspaces Sep 29, 2026
Ubuntu and others added 3 commits September 30, 2026 06:15
…uri floors

Advisories published 2026-09-28/29 cover versions this branch resolved:
- brace-expansion 1.1.18 / 5.0.9 (three HIGH ReDoS advisories per line):
  floors raised to 1.1.21 / 2.1.7 / 5.0.12 in javascript/, docs/ and the
  lovable_clone template.
- ip-address <=10.7.0 (#752-#759 plus two newer): 10.7.1.
- js-yaml 5.x <=5.4.0, reached through the langwatch SDK: 5.4.1.
- fast-uri 3.x: 3.1.8.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RWpxWBnYtUMcktWN1C6SrL
javascript/pnpm-lock.yaml conflicted with the judge changes from #1006; taken
from main and re-resolved against this branch's overrides.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RWpxWBnYtUMcktWN1C6SrL
…4.16

Advisories published 2026-09-30: @grpc/grpc-js >=1.14.0 <1.14.5 (HIGH) in
javascript/, and dompurify 3.4.13-3.4.15 in docs/.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RWpxWBnYtUMcktWN1C6SrL
@langwatch-agent langwatch-agent added the blocked Workflow: blocked label Oct 1, 2026
@langwatch-agent langwatch-agent changed the title fix(security): close all 150 open JS alerts across the scenario workspaces fix(security): close all 157 open JS alerts across the scenario workspaces Oct 1, 2026
@langwatch-agent

Copy link
Copy Markdown
Contributor Author

CI status: the only red is the JS examples step, failing on evaluators-on-scenarios. That test fails the same way on main (run at 0698d00d, 2026-09-29) and on every run that executes the JS examples suite, so it is not this change. This PR touches neither examples nor src/evaluators; the unit suite passes (1747 tests) and typecheck is clean. Root-cause evidence and a fix proposal are in #1010.


🤖 Filed by dependabot-scout via the create-issue procedure, on behalf of the fleet. Authored under the owner's GitHub token for API access — the content is the fleet's, not his. Owner-authored and fleet-authored issues are otherwise indistinguishable in this repo; this footer is the only signal.

@langwatch-agent langwatch-agent added the needs-rebase Conflicts with main; the author rebases label Oct 9, 2026
@github-actions

github-actions Bot commented Oct 9, 2026

Copy link
Copy Markdown
Contributor

Automated low-risk assessment

This PR was evaluated against the repository's Low-Risk Pull Requests procedure and does not qualify as low risk.

This PR's diff could not be evaluated automatically: Diff too large for automated evaluation (762213 chars exceeds 100000-char limit). Manual review required.

This PR requires a manual review before merging.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

blocked Workflow: blocked dependabot-scout Opened by the dependabot-scout security-triage agent (shared langwatch-agent bot identity) hound-checked Triaged by the pr-hound agent at the current head SHA needs-rebase Conflicts with main; the author rebases review: targeted PR Hound review mode

Projects

Status: Stale

Development

Successfully merging this pull request may close these issues.

2 participants