feat: Json data type

[matthewerwin]

Tested that this is working with mssql against Azure Sql Server.

Notes:

• validates JSON if parameter value is a string
• encodes as JSON if parameter value is an object
• passes 'null' to the server if parameter value is null
• The inclusion of this new data type does not create any breaking changes to the codebase

I used the following logic to deduce the implementation:

1. Referenced 0xF4 and PLP based on SqlClient
2. That no parameter collation bytes are sent because Microsoft is very clear that JSON collation is fixed to Latin1_General_100_BIN2_UTF8 to match JSON Spec
3. That the protocol only needed 0xf4 id prefix and no length bytes since its a newer data type which doesn't have a legacy <= 8000 implementation like varchar did (and the 'json' type has no length setting in SQL)
4. That due to the fixed collation that UTF-8 encoding in validate() is sufficient

I did not use the iconv-lite library b/c it seemed like unnecessary overhead vs the direct Buffer.from( ) support in Node.

This resolves this Feature Request I created earlier before deciding to address with a PR after a little digging.

[matthewerwin]

@arthurschreiber @dhensby -- is there a plan to roll any updates out to Tedious in the near future? Noticed there has been no movement on the repo in 6+ months.

I had a couple in-flight items including this PR and would like to do what I can to help if needed. Another one was

tediousjs/node-mssql#1727

and it seems a deprecation on Microsoft's side related to that as well:
#1684

Generally it seems some maintenance things are accumulating without any regular interval releases from the Tedious team. Happy to help organize and move some things through for a release including reviewing other PR's if it's a matter of limited time to organize/test/review.

[arthurschreiber]

👋🏻 Heya! You're right, I didn't have a lot of time to spend on tedious recently.

If you don't mind, can you mention me on whatever issues are the most urgent in your eyes, and I'll promise to carve out some time to get them reviewed and merged. 🙇

[matthewerwin]

@arthurschreiber you bet -- I'll make a pass through the open PR's & tickets over the next week and see what stands out.

[Copilot]

Pull Request Overview

Adds support for a JSON data type by implementing its PLP encoding and registering it in the type registry.

• Introduce src/data-types/json.ts to handle JSON validation, length resolution, and PLP data streaming.
• Register the new Json type in src/data-type.ts, update the type lookup tables, and augment the documentation.

Reviewed Changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 3 comments.

File	Description
src/data-types/json.ts	New module defining the JSON type’s declaration, validation, and PLP encoding.
src/data-type.ts	Imports and registers Json in TYPE/TYPES and updates the docs table.

[Copilot]

Using .length on a string gives UTF-16 code-unit count, not the byte length for UTF-8. Switch to Buffer.byteLength(value, 'utf8') to get the correct byte length.

Suggested change

	return parameter.value.length;
	else return JSON.stringify(parameter.value).length;
	return Buffer.byteLength(parameter.value, 'utf8');
	else return Buffer.byteLength(JSON.stringify(parameter.value), 'utf8');

[matthewerwin]

This also isn't accurate. To my knowledge this is sent by Tedious as nvarchar at serialization time -- the correct length is as indicated. @arthurschreiber thoughts?

[peterbud]

Is there a plan to merge this PR? It would be highly appreciated my many users I believe

[arthurschreiber]

Thanks for the work on this @matthewerwin, and apologies again for the long silence — I've gone through this in depth now. The core wire encoding for sending a json parameter looks right and matches what SqlClient does (single-byte 0xF4 TYPE_INFO, no collation, PLP-encoded UTF-8, 0xFF…FF for PLP NULL). A few things need to happen before this can ship, and since I have push access to your branch I'm happy to push the fixes myself rather than round-trip through review comments — just say if you'd rather take any of them yourself.

Issues found, roughly by severity:

1. No JSONSUPPORT feature-extension negotiation. Per MS-TDS, JSON support is enabled via a LOGIN7 FeatureExt handshake (feature id 0x0D) that the server acknowledges in FEATUREEXTACK — that's how SqlClient and mssql-jdbc gate the type. As written we send 0xF4 unconditionally: against SQL Server ≤ 2022 that surfaces as a cryptic msg 8016 (Data type 0xF4 is unknown) protocol error, and against SQL 2025/Azure it works only because the server currently tolerates unnegotiated writes. I'd like to add the FeatureExt (the structure from fix: rework FeatureExt generation #1718 makes this straightforward) and fail client-side with a clear error when the server doesn't ack it.

2. No read path. metadata-parser.ts / value-parser.ts don't handle 0xF4, so registering Json in the TYPE map only changes the error message when a server sends a json column (Unrecognised type Json instead of Unrecognised data type 0xF4). Today that's masked because an un-negotiating client gets json columns downcast to varchar — but the moment we negotiate JSONSUPPORT (point 1), the server will send 0xF4 in result metadata and every SELECT of a json column would kill the token stream. So 1 and 2 have to land together. The values are plain PLP UTF-8, so the read side is close to the existing max-varchar/Xml path.

3. Malformed PLP stream for empty values. generateParameterData returns without yielding anything for a zero-length value, but generateParameterLength has already emitted the unknown-length PLP header, which obligates the 00 00 00 00 terminator. It's currently unreachable (empty string fails JSON.parse in validate), but it's a latent protocol bug — the terminator should always be yielded, matching the nvarchar implementation.

4. validate() edge cases. A Buffer value silently takes the JSON.stringify branch and inserts {"type":"Buffer","data":[…]}; a function/symbol makes JSON.stringify return undefined and produces a confusing Buffer.from(undefined) TypeError. These should be explicit TypeErrors like our other types throw. Related: resolveLength runs after validate() has replaced the value with a Buffer, so it returns JSON.stringify(buffer).length — nothing consumes it for this type, so I'd just remove it rather than keep a wrong implementation.

5. Tests. We'll need unit coverage in test/unit/data-type.ts (buffer-level assertions for generateTypeInfo / generateParameterLength / generateParameterData, null and multi-byte UTF-8 cases) plus integration coverage once the negotiation and read path exist.

Plus some minor style alignment (braced conditionals, ===, dropping the boxed-String acceptance — no other type accepts new String(…)).

I'll start pushing to this branch shortly. Thanks again for doing the protocol digging — the SqlClient/MS-TDS references in the description made verifying this much easier.

[arthurschreiber]

All of the above is now resolved on this branch:

• 5793724 — merged latest master (clean, no conflicts; picks up the Node ≥ 22 requirement, the TypeScript test conversion, and the FeatureExt rework from fix: rework FeatureExt generation #1718).
• 0630b26 — issues 1 + 2: JSONSUPPORT is requested at login, the FEATUREEXTACK is tracked on the connection, and a TYPES.JSON parameter sent to a server that didn't ack now fails client-side with a descriptive RequestError (EJSONNOTSUPPORTED). The read path handles json COLMETADATA (no length/collation bytes, verified against SqlClient's implementation) and PLP UTF-8 values in ROW, NBCROW and RETURNVALUE tokens, returned as strings.
• 127dd31 — issue 3: the PLP terminator is now always emitted for non-null values.
• d53f039 — issue 4: explicit TypeErrors for Buffers and non-serializable values; resolveLength removed. The style nits are folded into these commits as well.
• 527cea2 — issue 5: unit coverage for the wire encoding, validation, ack parsing and token parsing (incl. multi-byte UTF-8), plus a new test/integration/json-test.ts that exercises both server capabilities — round-trips on servers that ack JSONSUPPORT, and the EJSONNOTSUPPORTED error on servers that don't.
• 091ed6f — one thing not in the list above: the type is exposed as TYPES.JSON rather than TYPES.Json, matching how JavaScript itself names it (the global JSON object). Since the type has never been released, this isn't a breaking change.

CI status: lint, CodeQL and the unit suite pass. The claude-review check fails for infrastructure reasons only (fork PRs get no OIDC token / API key), and the Azure SQL jobs skip on fork PRs for the same secrets reason.

One remaining caveat: the integration tests lint and typecheck, but I haven't been able to run them against a real SQL Server 2025 / Azure SQL instance. @matthewerwin since you have an Azure SQL setup that supports json, it would be a great help if you could run test/integration/json-test.ts against it (or re-test your original mssql scenario on this branch) to confirm the negotiation works end-to-end.

[codecov]

Codecov Report

❌ Patch coverage is 94.36620% with 4 lines in your changes missing coverage. Please review.
✅ Project coverage is 80.21%. Comparing base (650843a) to head (937e6c9).
⚠️ Report is 1 commits behind head on master.

Files with missing lines	Patch %	Lines
src/token/handler.ts	80.00%	0 Missing and 1 partial ⚠️
src/token/nbcrow-token-parser.ts	50.00%	0 Missing and 1 partial ⚠️
src/token/returnvalue-token-parser.ts	50.00%	0 Missing and 1 partial ⚠️
src/token/row-token-parser.ts	50.00%	0 Missing and 1 partial ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #1683      +/-   ##
==========================================
- Coverage   80.84%   80.21%   -0.63%     
==========================================
  Files          90       91       +1     
  Lines        4887     4954      +67     
  Branches      924      939      +15     
==========================================
+ Hits         3951     3974      +23     
- Misses        640      688      +48     
+ Partials      296      292       -4     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[matthewerwin]

@arthurschreiber thanks for the teamwork effort on this! I’ll have a closer look (and run all tests) when I return from Miami (Tues 7/21) but looks solid based on your write up.

[arthurschreiber]

A few more things landed on this branch since the last summary, all verified against SQL Server 2025 RTM-CU7 (17.0.4065.4):

• 2ccb056 — json columns in bulk loads. The server rejects the json data type (0xF4) in bulk load column metadata (Invalid column type from bcp client), even when JSONSUPPORT was negotiated — on RC1 and RTM-CU7 alike. Instead, varchar(max) type info is substituted in the COLMETADATA and the server converts the PLP-encoded UTF-8 values based on the json column type in the insert bulk statement. This matches SqlClient's SqlBulkCopy, which performs the same substitution (and whose issue tracker notes that SQL Server 2025 releases before CU5 have server-side bugs in this conversion path — dropped rows, broken NULL handling: SqlBulkCopy copies only first row when performing bulk copy for JSON datatype dotnet/SqlClient#3000, SqlBulkCopy throws an exception or ignores rows when inserting null values into JSON columns dotnet/SqlClient#3737).
• 38fb742 — json columns in TVPs. Sending 0xF4 in TVP column metadata is worse than in bulk loads: the server kills the session. The same varchar(max) substitution is applied here, with one important difference: the collation matters. With a zeroed collation the server decodes the column data using its default codepage, garbling multi-byte UTF-8. The substituted type info therefore carries Latin1_General_100_BIN2_UTF8 — the collation the json type is fixed to. (SqlClient has no json-in-TVP support at all; its TVP type info writer has no case for it.)
• c4aea72 — the bulk load substitution now sends the same collation instead of zeroed bytes, keeping the two substitutions identical. The server ignores the collation there (verified in a database with a non-UTF-8 default collation), but sending the truthful value keeps things consistent. This is a deliberate, documented deviation from SqlClient, which zeroes these bytes.
• 8f19878 — stricter JSONSUPPORT ack validation. A FEATUREEXTACK with a version other than 1 (or a malformed data length) previously degraded silently to "server does not support json". That's unsound: the server considers the negotiation complete and will use the acknowledged version's wire format, which this client would misparse as v1 UTF-8 text. The raw ack data is now carried on the token (like fedAuth) and the Login7TokenHandler fails the login with a ConnectionError for anything other than a clean version-1 ack — matching SqlClient.
• f715b4b — the EJSONNOTSUPPORTED error now distinguishes its two root causes: a server that didn't ack JSONSUPPORT vs. a configured tdsVersion below 7_4 (in which case the negotiation was never attempted and the message points at the config rather than blaming the server).
• 2a8b989, 937e6c9 — unit coverage for the RETURNVALUE read path (the last uncovered json path, incl. json output parameters against a live server) and style alignment of json.ts with the other data type files.

Test status: the full unit suite, lint and tsc pass; the integration suite in test/integration/json-test.ts now covers result sets, parameters, output parameters, bulk loads and TVPs, and passes against SQL Server 2025 RTM-CU7 on TDS 7.4 (plus the EJSONNOTSUPPORTED path on TDS 7.3).