Skip to content

test(css): replace the random css-fuzz loops with a deterministic table #38500

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
robobun wants to merge 2 commits into from
Closed

test(css): replace the random css-fuzz loops with a deterministic table #38500

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
959bd60
test(css): run fewer css-fuzz iterations on debug builds
robobun Aug 14, 2026
499ac4e
test(css): replace the random css-fuzz loops with a deterministic table
robobun Aug 14, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
364 changes: 118 additions & 246 deletions test/js/bun/css/css-fuzz.test.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,254 +1,126 @@
import { expect, test } from "bun:test";
import { isCI, isDebug } from "harness";

interface InvalidFuzzOptions {
maxLength: number;
strategy: "syntax" | "structure" | "encoding" | "memory" | "all";
iterations: number;
import { describe, expect, test } from "bun:test";

// Malformed stylesheets the CSS parser has to get through. Each one is bundled
// from memory as written and again after every corruption below. The build may
// succeed or report errors; the assertions only check that it came back with a

Check warning on line 5 in test/js/bun/css/css-fuzz.test.ts

View check run for this annotation

Claude / Claude Code Review

PR title/description describe the first commit, not the final rewrite 

The PR title and description still describe commit 959bd608 (the `isDebug`-gated 50/10 iteration-count tweak, `CSS_FUZZ_SHUTUP`, the 0.77^50 flake-probability argument), but the diff under review is commit 499ac4e4 — a full rewrite to a deterministic table with no `isDebug`, no iteration counts, no random corruption, the `!isCI` gate removed so the file now runs in CI, and the file removed from `no-validate-leaksan.txt`. Per `.claude/docs/landing-prs.md` the description becomes the squash-commit
Comment thread
robobun marked this conversation as resolved.
Resolved
// well-formed result, the process surviving is the point.

const invalid: Record<string, string[]> = {
selectors: [
"}{color:red}",
"&*#@.class{color:red}",
"..double.dot{color:red}",
".{color:red}",
"#{color:red}",
".outer { .inner { color: red } }",
],
declarations: [".test{color:}", ".test{:red}", ".test{color::red}", ".test{;color:red}", ".test{color:red;;;}"],
"unclosed blocks and comments": [
".test { color: red\n.another { padding: 10px }",
"/* unclosed comment .test{color:red}",
".test{color:red} /* unclosed",
"/**//**//* .test{color:red}",
"@media screen { @media print { } ",
],
"at-rules": [
"@media ;",
"@{color:red}",
"@media screen and and {color:red}",
"@keyframes { @keyframes { } }",
"@import url('test.css'",
"@import 'file' 'screen';",
"@import url(;",
"@import url('test.css') print",
],
"null bytes": [".test{color:red\0;}", ".te\0st{color:red}", "\0.test{color:red}"],
};

const corruptions: ((css: string) => string)[] = [
css => css.replaceAll("{", "}"),
css => css.replaceAll("}", "{"),
css => css.replaceAll(":", ";"),
css => css.replaceAll(";", ":"),
css => css.slice(0, css.length >> 1),
css => css + "}}}}",
css => Array.from(css).reverse().join(""),
];

const invalidUtf8: Record<string, number[]> = {
"overlong two byte NUL": [0xc0, 0x80],
"overlong three byte NUL": [0xe0, 0x80, 0x80],
"overlong four byte NUL": [0xf0, 0x80, 0x80, 0x80],
"truncated three byte sequence": [0xe2],
};

function splice(before: string, bytes: number[], after: string): Uint8Array {
return new Uint8Array([...Buffer.from(before), ...bytes, ...Buffer.from(after)]);
}

const shutup = process.env.CSS_FUZZ_SHUTUP === "1";
const log = shutup ? () => {} : console.log;

// Collection of invalid CSS generation strategies
const invalidGenerators = {
// Syntax errors
syntax: {
unclosedRules: () => `
.test { color: red
.another { padding: 10px }`,
invalidSelectors: () => [
"}{color:red}",
"&*#@.class{color:red}",
"..double.dot{color:red}",
".{color:red}",
"#{color:red}",
],
malformedProperties: () => [
".test{color:}",
".test{:red}",
".test{color::red}",
".test{;color:red}",
".test{color:red;;;}",
],
unclosedComments: () => [
"/* unclosed comment .test{color:red}",
".test{color:red} /* unclosed",
"/**//**//* .test{color:red}",
],
} as const,

// Structural errors
structure: {
nestedRules: () => [
".outer { .inner { color: red } }", // Invalid nesting without @rules
"@media screen { @media print { } ", // Unclosed nested at-rule
"@keyframes { @keyframes { } }", // Invalid nesting of @keyframes
],
malformedAtRules: () => ["@media ;", "@import url('test.css'", "@{color:red}", "@media screen and and {color:red}"],
invalidImports: () => ["@import 'file' 'screen';", "@import url(;", "@import url('test.css') print"],
} as const,

// Encoding and character issues
encoding: {
invalidUTF8: () => [
`.test{content:"${Buffer.from([0xc0, 0x80]).toString()}"}`,
`.test{content:"${Buffer.from([0xe0, 0x80, 0x80]).toString()}"}`,
`.test{content:"${Buffer.from([0xf0, 0x80, 0x80, 0x80]).toString()}"}`,
],
nullBytes: () => [`.test{color:red${"\0"};}`, `.te${"\0"}st{color:red}`, `${"\0"}.test{color:red}`],
controlCharacters: () => {
const controls = Array.from({ length: 32 }, (_, i) => String.fromCharCode(i));
return controls.map(char => `.test{color:${char}red}`);
},
} as const,

// Memory and resource stress
memory: {
deepNesting: (depth: number = 300) => {
let css = "";
for (let i = 0; i < depth; i++) {
css += "@media screen {";
}
css += ".test{color:red}";
for (let i = 0; i < depth; i++) {
css += "}";
}
return css;
},
longSelectors: (length: number = 100000) => {
const selector = ".test".repeat(length);
return `${selector}{color:red}`;
},
manyProperties: (count: number = 10000) => {
const properties = Array(count).fill("color:red;").join("\n");
return `.test{${properties}}`;
},
} as const,
} as const;

// Helper to randomly corrupt CSS
function corruptCSS(css: string): string {
const corruptions = [
(s: string) => (s + "").replace(/{/g, "}"),
(s: string) => (s + "").replace(/}/g, "{"),
(s: string) => (s + "").replace(/:/g, ";"),
(s: string) => (s + "").replace(/;/g, ":"),
(s: string) => (s + "").slice(Math.floor(Math.random() * (s + "").length)),
(s: string) => s + "" + "}}".repeat(Math.floor(Math.random() * 5)),
(s: string) => (s + "").split("").reverse().join(""),
(s: string) => (s + "").replace(/[a-z]/g, c => String.fromCharCode(97 + Math.floor(Math.random() * 26))),
];

const numCorruptions = Math.floor(Math.random() * 3) + 1;
let corrupted = css;

for (let i = 0; i < numCorruptions; i++) {
const corruption = corruptions[Math.floor(Math.random() * corruptions.length)];
corrupted = corruption(corrupted);
}
// The sources only exist inside the `files` option; nothing is written to disk.
const virtualDir = import.meta.dir.replaceAll("\\", "/") + "/css-fuzz-input";

// Builds every source in one Bun.build call. A batch produces no output once any
// of its sources is rejected, so a source that should also reach the printer has
// to be built on its own.
async function build(sources: (string | Uint8Array)[]) {
const files: Record<string, string | Uint8Array> = {};
sources.forEach((source, i) => {
files[`${virtualDir}/${i}.css`] = source;
});
const entrypoints = Object.keys(files);

return corrupted;
const result = await Bun.build({ entrypoints, files, throw: false });

for (const log of result.logs) {
expect(log.message).toBeString();
}
if (result.success) {
expect(result.outputs).toHaveLength(entrypoints.length);
} else {
expect(result.logs.length).toBeGreaterThan(0);
}
return result;
}

// TODO:
if (!isCI) {
// Main fuzzing test suite for invalid inputs
test.each(
[["syntax", 1000], ["structure", 1000], ["encoding", 500], !isDebug ? ["memory", 100] : []].filter(
xs => xs.length > 0,
),
)(
"CSS Parser Invalid Input Fuzzing - %s (%d iterations)",
async (strategy, iterations) => {
const options: InvalidFuzzOptions = {
maxLength: 10000,
strategy: strategy as any,
iterations,
};

let crashCount = 0;
let errorCount = 0;
const startTime = performance.now();

for (let i = 0; i < options.iterations; i++) {
let invalidCSS = "";

switch (strategy) {
case "syntax":
invalidCSS =
invalidGenerators.syntax[
Object.keys(invalidGenerators.syntax)[
Math.floor(Math.random() * Object.keys(invalidGenerators.syntax).length)
]
]()[Math.floor(Math.random() * 5)];
break;

case "structure":
invalidCSS =
invalidGenerators.structure[
Object.keys(invalidGenerators.structure)[
Math.floor(Math.random() * Object.keys(invalidGenerators.structure).length)
]
]()[Math.floor(Math.random() * 3)];
break;

case "encoding":
invalidCSS =
invalidGenerators.encoding[
Object.keys(invalidGenerators.encoding)[
Math.floor(Math.random() * Object.keys(invalidGenerators.encoding).length)
]
]()[0];
break;

case "memory":
const memoryFuncs = Object.keys(invalidGenerators.memory);
const selectedFunc = memoryFuncs[Math.floor(Math.random() * memoryFuncs.length)];
invalidCSS = invalidGenerators.memory[selectedFunc]();
break;
}

// Further corrupt the CSS randomly
if (Math.random() < 0.3) {
invalidCSS = corruptCSS(invalidCSS);
}

log("--- CSS Fuzz ---");
invalidCSS = invalidCSS + "";
log(JSON.stringify(invalidCSS, null, 2));
await Bun.write("invalid.css", invalidCSS);

try {
const result = await Bun.build({
entrypoints: ["invalid.css"],
});

// We expect the parser to either throw an error or return a valid result
// If it returns undefined/null, that's a potential issue
if (result === undefined || result === null) {
crashCount++;
console.error(`Parser returned ${result} for input:\n${invalidCSS.slice(0, 100)}...`);
}
} catch (error) {
// Expected behavior for invalid CSS
errorCount++;

// Check for specific error types we want to track
if (error instanceof RangeError || error instanceof TypeError) {
console.warn(`Unexpected error type: ${error.constructor.name} for input:\n${invalidCSS.slice(0, 100)}...`);
}
}

// Memory check every 100 iterations
if (i % 100 === 0) {
const heapUsed = process.memoryUsage().heapUsed / 1024 / 1024;
expect(heapUsed).toBeLessThan(500); // Alert if memory usage exceeds 500MB
}
}

const endTime = performance.now();
const duration = endTime - startTime;

console.log(`
Strategy: ${strategy}
Total iterations: ${iterations}
Crashes: ${crashCount}
Expected errors: ${errorCount}
Duration: ${duration.toFixed(2)}ms
Average time per test: ${(duration / iterations).toFixed(2)}ms
`);

// We expect some errors for invalid input, but no crashes
expect(crashCount).toBe(0);
expect(errorCount).toBeGreaterThan(0);
},
10 * 1000,
);

// Additional test for mixed valid/invalid input
test("CSS Parser Mixed Input Fuzzing", async () => {
const validCSS = ".test{color:red}";

for (let i = 0; i < 100; i++) {
const mixedCSS = `
${validCSS}
${corruptCSS(validCSS)}
${validCSS}
`;

console.log("--- Mixed CSS ---");
console.log(JSON.stringify(mixedCSS, null, 2));
await Bun.write("invalid.css", mixedCSS);

try {
await Bun.build({
entrypoints: ["invalid.css"],
});
} catch (error) {
// Expected to throw, but shouldn't crash
expect(error).toBeDefined();
}
test("a valid stylesheet builds through the same path", async () => {
const result = await build([".test{color:red}"]);
expect(result.success).toBe(true);
expect(await result.outputs[0].text()).toContain("color: red");
});

for (const [family, stylesheets] of Object.entries(invalid)) {
describe(family, () => {
for (const css of stylesheets) {
test(JSON.stringify(css), async () => {
await build([css]);
await build(corruptions.map(corrupt => corrupt(css)));
});
}
});
}

test("every control character inside a declaration value", async () => {
await build(Array.from({ length: 32 }, (_, i) => `.test{color:${String.fromCharCode(i)}red}`));
});

describe("invalid UTF-8", () => {
for (const [name, bytes] of Object.entries(invalidUtf8)) {
test(`${name} inside a string`, async () => {
await build([splice('.test{content:"', bytes, '"}')]);
});

test(`${name} inside an identifier`, async () => {
await build([splice(".te", bytes, "st{color:red}")]);
});
}
});

describe("oversized input", () => {
test("a selector with 100000 compound parts", async () => {
await build([Buffer.alloc(".test".length * 100_000, ".test").toString() + "{color:red}"]);
});

test("a rule with 10000 declarations", async () => {
await build([".test{" + Buffer.alloc("color:red;\n".length * 10_000, "color:red;\n").toString() + "}"]);
});
});

Check warning on line 126 in test/js/bun/css/css-fuzz.test.ts

View check run for this annotation

Claude / Claude Code Review

Deep-nesting @media stress case dropped on release too 

The old `memory.deepNesting` case (300-deep `@media screen {` nesting) is dropped from the rewrite entirely, but the PR description still says "the memory strategy stays skipped on debug builds" — implying release keeps it. Either restore it in the `oversized input` block gated on `!isASAN`, or update the description to say release drops it too pending the separate parser-stack fix.
Comment thread
robobun marked this conversation as resolved.
Resolved
1 change: 0 additions & 1 deletion test/no-validate-leaksan.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -58,7 +58,6 @@ test/integration/next-pages/test/dev-server-ssr-100.test.ts
test/integration/next-pages/test/dev-server.test.ts
test/integration/next-pages/test/next-build.test.ts
test/integration/vite-build/vite-build.test.ts
test/js/bun/css/css-fuzz.test.ts
test/js/bun/ffi/cc.test.ts
test/js/bun/http/bun-serve-html-manifest.test.ts
test/js/bun/http/bun-server.test.ts
Expand Down
Loading