Skip to content

About

Web Streams (TransformStream) and async iterator library for RFC 7464 (json-seq) and NDJSON/jsonl with zero dependencies.

Topics

Resources

Stars

1 star

Watchers

0 watching

Forks

Repository files navigation

json-seq-stream

application/json-seq / jsonl (NDJSON) support stream / async iterator library

This library provides composable TransformStream / ReadableWritablePair classes and async generator utilities to work with streaming JSON data formats.

Features

  • Supports RFC 7464 (application/json-seq), including multi-line JSON and truncated record recovery
  • Supports JSON Lines / NDJSON (application/jsonl, application/x-ndjson)
  • Fully stream-compatible (Fetch API, File API, Node.js 20+, Bun, Deno, etc.)
  • Composable with standard Web Streams (pipeThrough()) and AsyncIterable / for await...of
  • Supports async parse / stringify functions and maximum buffer length (maxLength) protection
  • Content-Type based format detection for fetch() responses (parseJsonResponse)
  • Customizable backpressure via writableStrategy / readableStrategy queuing strategy options
  • Truncated final record handling (truncationFallback, RFC 7464 §2.4)
  • Format constants (MIME types, separators) via subpath exports (json-seq-stream/rfc7464, json-seq-stream/jsonlines)

Usage Examples

JSON Sequence (RFC 7464)

import { InputJsonSequenceStream } from "json-seq-stream";

type Value = { value: number };

const sequenceStream = new InputJsonSequenceStream<Value>();

const response = await fetch(url);
const readable = response.body!.pipeThrough(sequenceStream);

for await (const value of readable) {
  console.log(value);
}

JSON Lines / NDJSON

import { InputJsonLinesStream } from "json-seq-stream";

type Value = { value: number };

const linesStream = new InputJsonLinesStream<Value>();

const response = await fetch("https://example.com/data.jsonl");
const readable = response.body!.pipeThrough(linesStream);

for await (const value of readable) {
  console.log(value);
}

Output Stream Example

import { OutputJsonLinesStream } from "json-seq-stream";

type Value = { value: number };

const { readable, writable } = new OutputJsonLinesStream<Value>();

const writer = writable.getWriter();
void writer.write({ value: 1 });
void writer.write({ value: 2 });
void writer.close();

for await (const chunk of readable) {
  console.log(new TextDecoder().decode(chunk)); // NDJSON text lines
}

readable is a ReadableStream<Uint8Array>, so it can be passed directly as a Response or Request (fetch) body:

import { MIME_TYPE } from "json-seq-stream/jsonlines";

// Server-side streaming Response
const response = new Response(readable, {
  headers: { "content-type": MIME_TYPE },
});

// Client-side streaming Request (requires `duplex: "half"`)
await fetch("https://example.com/upload", {
  method: "POST",
  headers: { "content-type": MIME_TYPE },
  body: readable,
  duplex: "half",
});

Async Generator Helpers (AsyncIterable / Iterable)

import {
  parseJsonLines,
  parseJsonSequence,
  stringifyJsonLines,
  stringifyJsonSequence,
} from "json-seq-stream";

type Value = { value: number };

const items: Value[] = [{ value: 1 }, { value: 2 }];

// JSON Lines (NDJSON) round-trip with AsyncIterable
const jsonlBytes = stringifyJsonLines(items);
for await (const item of parseJsonLines<Value>(jsonlBytes)) {
  console.log(item);
}

// JSON Text Sequences (RFC 7464) round-trip with AsyncIterable
const jsonSeqBytes = stringifyJsonSequence(items);
for await (const item of parseJsonSequence<Value>(jsonSeqBytes)) {
  console.log(item);
}

Fetch Response with Content-Type Detection

import { parseJsonResponse } from "json-seq-stream";

type Value = { value: number };

const response = await fetch("https://example.com/data");
// application/json-seq -> RFC 7464, application/jsonl or application/x-ndjson -> JSON Lines
for await (const value of parseJsonResponse<Value>(response)) {
  console.log(value);
}

Error Handling

By default, input streams skip records that fail to parse and report them with a console.warn (RFC 7464 §2.1), while output streams error the stream when stringify fails. Use the errorFallback option to customize this:

import { InputJsonLinesStream } from "json-seq-stream";

const stream = new InputJsonLinesStream<MyType>({
  // false: skip silently / function: decide per record
  errorFallback: (error, { chunk }) => {
    console.error("broken line:", chunk, error);
  },
});

// strict mode: abort the stream on the first broken record
const strictStream = new InputJsonLinesStream<MyType>({
  errorFallback: (e, { error }) => error(e),
});

Format Constants (subpath exports)

The format constants are intentionally kept out of the main entry and exposed via dedicated subpaths:

import { MIME_TYPE, RS, LF } from "json-seq-stream/rfc7464";
import { NDJSON_MIME_TYPE, EXTENSION } from "json-seq-stream/jsonlines";

const response = await fetch(url, { headers: { accept: MIME_TYPE } });
  • json-seq-stream/rfc7464: RS, LF, MIME_TYPE (application/json-seq)
  • json-seq-stream/jsonlines: LF, CRLF, MIME_TYPE (application/jsonl), NDJSON_MIME_TYPE (application/x-ndjson), EXTENSION (.jsonl)

See also

About

Web Streams (TransformStream) and async iterator library for RFC 7464 (json-seq) and NDJSON/jsonl with zero dependencies.

Topics

Resources

Stars

1 star

Watchers

0 watching

Forks

Releases

Contributors

Languages