Migrate from Jackson 2.x to Jackson 3 (tools.jackson) (#430)

.gitignore

@@ -9,3 +9,4 @@
 !/.idea/workspace.xml
 /JSONata4Java.iml
 /setupenv.cmd
+/.superpowers/

README.md

@@ -14,7 +14,7 @@ The easiest way to use this library is to include it as a dependency in your Mav
 <dependency>
   <groupId>com.ibm.jsonata4java</groupId>
   <artifactId>JSONata4Java</artifactId>
-  <version>2.6.4</version>
+  <version>3.0.0</version>
 </dependency>
 ```
 
@@ -51,8 +51,8 @@ Note: to build and deploy the jars to Maven Central you need to use a command li
 `mvn clean install deploy -Prelease`
 
 Once you have run the launcher, you can find the jar files in the /target directory. There are two&colon;
-* **JSONata4Java-2.6.4-jar-with-dependencies.jar** (thinks includes dependent jar files)
-* **JSONata4Java-2.6.4.jar** (only the JSONata4Java code)
+* **JSONata4Java-3.0.0-jar-with-dependencies.jar** (thinks includes dependent jar files)
+* **JSONata4Java-3.0.0.jar** (only the JSONata4Java code)
 
 The com.api.jsonata4java.Tester program enables you to enter an expression and run it 
 against the same JSON as is used at the https://try.jsonata.org site. You can also 

docs/specs/Jackson-3-Migration-Design.md

@@ -0,0 +1,207 @@
+# Migrate to Jackson 3 (`tools.jackson`) — JSONata4Java 3.0.0
+
+**Issue:** [#430](https://github.com/IBM/JSONata4Java/issues/430) — Migrate code from
+`com.fasterxml.jackson.core` to `tools.jackson.core`.
+
+## Summary
+
+Jackson 3.0 renamed both its Maven group-ids and Java packages from `com.fasterxml.jackson`
+to `tools.jackson`, made the core exception hierarchy unchecked, and moved mapper
+configuration to an immutable builder pattern. JSONata4Java exposes
+`com.fasterxml.jackson.databind.JsonNode` throughout its public API, so migrating to Jackson 3
+is a breaking change and warrants a new major version.
+
+The project's `pom.xml` already stages the transition by listing both the Jackson 2.x and
+Jackson 3.x dependencies. This work completes the cutover: remove the 2.x stack, migrate all
+source and test code to `tools.jackson`, and release as **3.0.0**.
+
+## Decisions
+
+- **Strategy: hard cutover.** Replace all `com.fasterxml.jackson.*` usage with `tools.jackson.*`
+  and drop the 2.x dependencies entirely. No dual-support/abstraction layer (JsonNode is woven
+  too deeply into the public API to make that worthwhile — YAGNI).
+- **Version: 3.0.0.** Bump from `2.6.4`. The public-API break (JsonNode package change) requires
+  a major version.
+- **XML in scope.** Port the `XmlMapper`/`ToXmlGenerator` usage to Jackson 3's
+  `tools.jackson.dataformat.xml`.
+- **CLI/UI helpers in scope.** `Tester`, `TesterTimeBox`, `TesterUI`, and `Test` are migrated
+  along with the core library, even though they carry most of the API churn.
+- **Target dependency version: 3.2.0** (current Jackson 3 line; confirmed published on Maven
+  Central for both `jackson-databind` and `jackson-dataformat-xml`).
+
+## Scope
+
+103 Java files reference Jackson today: 87 under `src/main/java`, 16 under `src/test/java`.
+The overwhelming majority only use JSON node types whose class names are unchanged in Jackson 3
+— only the package prefix moves.
+
+## Dependency changes (`pom.xml`)
+
+| Action | Coordinates |
+| --- | --- |
+| Remove | `com.fasterxml.jackson.core:jackson-databind:2.22.0` |
+| Remove | `com.fasterxml.jackson.dataformat:jackson-dataformat-xml:2.22.0` |
+| Keep   | `tools.jackson.core:jackson-databind:3.2.0` (already present) |
+| Add    | `tools.jackson.dataformat:jackson-dataformat-xml:3.2.0` |
+| Keep   | `com.fasterxml.woodstox:woodstox-core:7.2.1` (Jackson 3 XML still uses Stax/woodstox) |
+| Keep   | `re2j` (test), `gson`, `commons-text`, `antlr4-runtime`, `spring-context` (test), `junit` (test) |
+| Change | `<version>` `2.6.4` → `3.0.0` |
+
+Notes:
+- **Version string lives in more than `pom.xml`.** Per `ReleaseProcedure.txt`, the release
+  version appears in six places across five files, all of which move `2.6.4` → `3.0.0`:
+  `pom.xml` (1), `README.md` (3 — lines 17, 54, 55), `tester.sh` (1), `testerui.sh` (1),
+  `testerui.cmd` (1). The script/README occurrences are the `JSONata4Java-<version>...jar` names.
+- `jackson-core` is pulled in transitively by `jackson-databind`; no direct entry needed.
+- **`jackson-annotations` was NOT renamed** — it stays under `com.fasterxml.jackson.annotation`.
+  The codebase has no annotation imports, so this is a non-issue here, but do not blindly
+  rewrite `com.fasterxml.jackson.annotation` should it appear.
+
+## Migration approach
+
+Execute in three passes so the risky hand-edits stay isolated from the bulk mechanical change.
+
+### Pass 1 — Mechanical package rename (~103 files)
+
+Scripted find-replace across `src/main/java` and `src/test/java`, longest-prefix first to avoid
+double rewriting:
+
+1. `com.fasterxml.jackson.dataformat.xml` → `tools.jackson.dataformat.xml`
+2. `com.fasterxml.jackson.databind`      → `tools.jackson.databind`
+3. `com.fasterxml.jackson.core`          → `tools.jackson.core`
+
+This covers all the node types (`JsonNode`, `JsonNodeFactory`, `ArrayNode`, `ObjectNode`,
+`TextNode`, `LongNode`, `IntNode`, `DoubleNode`, `FloatNode`, `BooleanNode`, `NullNode`,
+`POJONode`, `ValueNode`, `NumericNode`, `JsonNodeType`) and the streaming/databind entry points,
+all of which keep their simple class names in Jackson 3.
+
+### Pass 2 — Hand-fix API-change hotspots (< 10 files)
+
+These sites use APIs that changed shape in Jackson 3 and will not compile after a pure rename.
+
+**Mapper configuration → immutable builder.** In Jackson 3, mappers are immutable; the mutating
+`enable`/`configure`/`getFactory().configure(...)` methods are gone. All affected sites are in
+CLI/UI helpers, not the core library:
+
+- `Tester.java:124` — `mapper.getFactory().configure(JsonWriteFeature.ESCAPE_NON_ASCII.mappedFeature(), true)`
+  → construct via `JsonMapper.builder().enable(JsonWriteFeature.ESCAPE_NON_ASCII).build()`.
+  `JsonWriteFeature` moves to `tools.jackson.core`.
+- `TesterUI.java:108` — `xmlMapper.enable(SerializationFeature.INDENT_OUTPUT)`
+  → set in the `XmlMapper.builder()...build()` chain.
+- `TesterUI.java:488–491` — `xmlMapper.configure(SerializationFeature.INDENT_OUTPUT, true)`,
+  `xmlMapper.configure(ToXmlGenerator.Feature.WRITE_XML_DECLARATION, true)`,
+  `xmlMapper.configure(ToXmlGenerator.Feature.WRITE_XML_1_1, true)`
+  → fold into the `XmlMapper.builder()` chain (verify the `ToXmlGenerator.Feature` enum location
+  under `tools.jackson.dataformat.xml`).
+
+**Plain mapper construction.** Sites that only call `readTree` / `writeValueAsString` on a
+`new ObjectMapper()` (in `Binding`, `ExpressionsVisitor`, `ContainsFunction`, `Expression`,
+`JSONataUtils`, `Issue180`, `Test`) need only the package rename; keep `new ObjectMapper()` (or
+switch to `JsonMapper.shared()` / `new JsonMapper()` if `new ObjectMapper()` proves unavailable).
+The exact constructor availability is confirmed at compile time in Pass 3.
+
+**Unchecked exceptions.** `JsonProcessingException`'s parent became `RuntimeException` and core
+introduced unchecked `JacksonException`. Catch blocks referencing `JsonProcessingException`
+generally still compile because it is now unchecked. Where a `catch (JsonProcessingException e)`
+becomes unreachable, or the type is no longer resolvable, replace with
+`tools.jackson.core.JacksonException`. Review each multi-catch
+(`EvaluateException | JsonProcessingException`) individually. Affected files: `TesterTimeBox`,
+`Test`, `Tester`, `TesterUI`, `ExpressionsVisitor`, `Issue180`.
+
+### Pass 3 — Compile, test, verify
+
+- `mvn clean compile` — resolve any remaining API breaks surfaced by the compiler; iterate until
+  clean.
+- `mvn test` — the existing unit tests plus `AgnosticTestSuite` are the correctness safety net;
+  the 16 migrated test files run here too.
+- Manually exercise the `Tester` CLI (JSON read/pretty-print path) and, where feasible, the Swing
+  `TesterUI` XML↔JSON conversion, since those hold the most API churn.
+
+## Build configuration
+
+- **OSGi (`bnd-maven-plugin`).** The `Export-Package` list references only `com.api.jsonata4java.*`
+  and needs no change. Confirm the generated `MANIFEST.MF` `Import-Package` header now imports
+  `tools.jackson.*` rather than `com.fasterxml.jackson.*`.
+- **JPMS.** No `module-info.java` exists; no module changes required.
+- No other plugin configuration references Jackson.
+
+### Maven build-warning cleanup (unrelated to Jackson, done on this branch)
+
+While migrating, three pre-existing Maven warnings surfaced on every build and were fixed in
+`pom.xml`:
+
+- **bnd private-reference warnings (×2).** The exported `com.api.jsonata4java` and
+  `com.api.jsonata4java.expressions` packages expose types from
+  `com.api.jsonata4java.expressions.regex` (`RegexEngine`, `RegexPattern`, …) in their public API,
+  but that sub-package was not in the `Export-Package` list, so bnd flagged it as a leaked private
+  reference. Added `com.api.jsonata4java.expressions.regex` to `Export-Package`.
+- **`additionalClasspathElements` unknown (×2).** Removed the empty
+  `<additionalClasspathElements/>` from `maven-compiler-plugin` — not a valid parameter for that
+  plugin (it did nothing).
+- **`excludes` unknown on assembly plugin.** Removed `<excludes>` from the `maven-assembly-plugin`
+  `single` goal — not a valid parameter there (exclusions belong in an assembly descriptor); it was
+  silently ignored and does not affect the produced `jar-with-dependencies`.
+
+A fourth warning — `gpg.passphrase` deprecated — originates from the developer's
+`~/.m2/settings.xml` (an `activeByDefault` profile setting the deprecated property), **not** from
+`pom.xml`, and is resolved locally by removing that property and relying on `gpg-agent`.
+
+## Out of scope
+
+- Any behavioral changes beyond what the package/API migration requires.
+- Adopting new Jackson 3 features (e.g., new default flips such as `FAIL_ON_UNKNOWN_PROPERTIES`)
+  beyond preserving existing behavior. If a Jackson 3 default change alters observed test
+  behavior, restore the prior behavior via builder configuration rather than accepting the new
+  default silently.
+- Unrelated refactoring of the touched files.
+
+## Risks / watch-items
+
+- **Default-value flips in Jackson 3** (e.g., `WRITE_DATES_AS_TIMESTAMPS`, `FAIL_ON_UNKNOWN_PROPERTIES`)
+  could shift test output. The test suite is the detector; restore prior behavior via builder
+  config where a test regresses.
+- **`ToXmlGenerator.Feature` and `JsonWriteFeature` relocations** — confirm exact package/enum
+  homes at compile time; the builder `enable(...)` overloads may differ from the 2.x
+  `configure(feature, boolean)` shape.
+- **`new ObjectMapper()` availability** — if the no-arg constructor is unavailable/discouraged in
+  3.x, fall back to `JsonMapper.shared()` or `new JsonMapper()`.
+
+## Retained Jackson 2 behaviors (as-built)
+
+Jackson 3 tightened several coercion/parse defaults that would otherwise change observable
+behavior. Each site below was fixed to **preserve the Jackson 2 behavior** — no test assertion or
+expected value was changed. These are the compatibility decisions a future maintainer should know
+about (and can revisit if the project decides to adopt the stricter Jackson 3 semantics):
+
+- **Container coercion — `ArrayUtils.compare` (main).** Jackson 3's no-arg `asText()`/`asString()`
+  throws for container nodes (`ObjectNode`/`ArrayNode`); Jackson 2 returned `""`. Fixed by passing
+  the default: `asText("")`. `$sort` on non-scalar operands therefore coerces to `""` exactly as
+  before instead of throwing.
+- **Regex `POJONode` emptiness checks — `MatchFunction`, `ReplaceFunction` (main).** A compiled
+  `RegularExpression` is stored in a `POJONode`. Jackson 2's `asText()` returned the POJO's
+  `toString()` (non-empty); Jackson 3 throws for a `POJONode`. The empty-pattern checks are now
+  guarded by `isTextual()` so the throwing call is never reached for a `POJONode`, while the
+  boolean outcome is identical to Jackson 2.
+- **Out-of-range `asLong()` — `AgnosticTestSuite` (test harness).** Jackson 3's `DoubleNode.asLong()`
+  throws when the value is outside `long` range (e.g. `1.0E46`); Jackson 2 performed a silent
+  narrowing cast. Expected-value normalization now uses `(long) d`, replicating Jackson 2 exactly.
+- **`FAIL_ON_TRAILING_TOKENS` — `Utils` test mapper (test harness).** Jackson 3 flips this default
+  to `true`; Jackson 2 silently ignored content after the first parsed value. The shared test
+  mapper disables the feature so existing test inputs parse identically. Note: this tolerates a
+  long-standing typo in a test input (`BasicExpressionsTests`, `"[\"h11\", \"h21\"]]"` — a stray
+  trailing `]`). The typo was intentionally **not** corrected, to keep the test data byte-for-byte
+  unchanged.
+- **Unchecked exceptions — several files.** Where Jackson 3 methods no longer throw checked
+  `IOException`, `catch (IOException)` blocks over Jackson-only bodies became unreachable and were
+  changed to `catch (JacksonException)`.
+
+### Other known items (not changed by this migration)
+
+- **Pre-existing `Tester` REPL NPE.** `Tester.main` (`Tester.java:156`) calls
+  `expression.length()` without a null check; `JSONataUtils.prompt()` returns `null` at stdin EOF,
+  causing a `NullPointerException` when input ends without a `q` line. This predates the migration
+  (authored 2022) and was left as-is.
+- **Transitive `jackson-annotations` 2.x.** `tools.jackson.core:jackson-databind:3.2.0` still pulls
+  in `com.fasterxml.jackson.core:jackson-annotations` — Jackson 3 continues to publish annotations
+  under the `com.fasterxml.jackson.annotation` namespace. This is expected and acceptable; it is not
+  a leftover Jackson 2 core/databind/dataformat dependency.

pom.xml

@@ -27,7 +27,7 @@
   <modelVersion>4.0.0</modelVersion>
   <groupId>com.ibm.jsonata4java</groupId>
   <artifactId>JSONata4Java</artifactId>
-  <version>2.6.4</version>
+  <version>3.0.0</version>
   <name>JSONata4Java</name>
   <description>Port of jsonata.js to Java to enable rules for JSON content</description>
   <url>https://github.com/IBM/JSONata4Java</url>
@@ -121,22 +121,19 @@
         <version>3.2.0</version>
         <scope>compile</scope>
     </dependency>
+    <!-- Source: https://mvnrepository.com/artifact/tools.jackson.dataformat/jackson-dataformat-xml -->
     <dependency>
-      <groupId>com.fasterxml.jackson.core</groupId>
-      <artifactId>jackson-databind</artifactId>
-      <version>2.22.0</version>
+        <groupId>tools.jackson.dataformat</groupId>
+        <artifactId>jackson-dataformat-xml</artifactId>
+        <version>3.2.0</version>
+        <scope>compile</scope>
     </dependency>
     <!-- https://mvnrepository.com/artifact/com.fasterxml.woodstox/woodstox-core -->
     <dependency>
       <groupId>com.fasterxml.woodstox</groupId>
       <artifactId>woodstox-core</artifactId>
       <version>7.2.1</version>
     </dependency>
-    <dependency>
-      <groupId>com.fasterxml.jackson.dataformat</groupId>
-      <artifactId>jackson-dataformat-xml</artifactId>
-      <version>2.22.0</version>
-    </dependency>
     <!-- https://mvnrepository.com/artifact/org.apache.commons/commons-text -->
     <dependency>
       <groupId>org.apache.commons</groupId>
@@ -239,7 +236,6 @@
         <configuration>
           <source>17</source>
           <target>17</target>
-          <additionalClasspathElements />
         </configuration>
       </plugin>
       <plugin>
@@ -306,9 +302,6 @@
           <descriptorRefs>
             <descriptorRef>jar-with-dependencies</descriptorRef>
           </descriptorRefs>
-          <excludes>
-            <exclude>src/test/resources/*</exclude>
-          </excludes>
         </configuration>
         <executions>
           <execution>
@@ -334,6 +327,7 @@
                         com.api.jsonata4java.expressions.generated, \
                         com.api.jsonata4java.expressions.path, \
                         com.api.jsonata4java.expressions.path.generated, \
+                        com.api.jsonata4java.expressions.regex, \
                         com.api.jsonata4java.expressions.utils
 					]]>
               Bundle-Developers: wnm3; email=wnm3@us.ibm.com;

src/main/java/com/api/jsonata4java/Binding.java

@@ -22,8 +22,8 @@
 import com.api.jsonata4java.expressions.generated.MappingExpressionParser.NullContext;
 import com.api.jsonata4java.expressions.generated.MappingExpressionParser.NumberContext;
 import com.api.jsonata4java.expressions.generated.MappingExpressionParser.StringContext;
-import com.fasterxml.jackson.databind.JsonNode;
-import com.fasterxml.jackson.databind.ObjectMapper;
+import tools.jackson.databind.JsonNode;
+import tools.jackson.databind.ObjectMapper;
 
 /**
  * Class mapping a variable name to a variable or function declaration

src/main/java/com/api/jsonata4java/Expression.java

@@ -20,9 +20,9 @@
 import com.api.jsonata4java.expressions.functions.DeclaredFunction;
 import com.api.jsonata4java.expressions.generated.MappingExpressionParser.ExprContext;
 import com.api.jsonata4java.expressions.regex.RegexEngine;
-import com.fasterxml.jackson.databind.JsonNode;
-import com.fasterxml.jackson.databind.ObjectMapper;
-import com.fasterxml.jackson.databind.node.JsonNodeFactory;
+import tools.jackson.databind.JsonNode;
+import tools.jackson.databind.ObjectMapper;
+import tools.jackson.databind.node.JsonNodeFactory;
 
 /**
  * Class to provide embedding and extending JSONata features
@@ -42,7 +42,7 @@ public class Expression implements Serializable {
 	public static List<Binding> createBindings(JsonNode bindingObj) throws ParseException {
 		ObjectMapper objectMapper = new ObjectMapper();
 		List<Binding> bindings = new ArrayList<Binding>();
-		for (Iterator<String> it = bindingObj.fieldNames(); it.hasNext();) {
+		for (Iterator<String> it = bindingObj.propertyNames().iterator(); it.hasNext();) {
 			String key = it.next();
 			JsonNode testObj = bindingObj.get(key);
 			String expression = "";

src/main/java/com/api/jsonata4java/JSONataUtils.java

@@ -53,10 +53,10 @@
 //import java.rmi.dgc.VMID;
 import java.util.UUID;
 import javax.management.modelmbean.InvalidTargetObjectTypeException;
-import com.fasterxml.jackson.databind.JsonNode;
-import com.fasterxml.jackson.databind.ObjectMapper;
-import com.fasterxml.jackson.databind.node.JsonNodeFactory;
-import com.fasterxml.jackson.databind.node.NullNode;
+import tools.jackson.databind.JsonNode;
+import tools.jackson.databind.ObjectMapper;
+import tools.jackson.databind.node.JsonNodeFactory;
+import tools.jackson.databind.node.NullNode;
 
 public class JSONataUtils implements Serializable {
 

src/main/java/com/api/jsonata4java/Sequence.java

@@ -24,10 +24,10 @@
 
 import java.io.Serializable;
 import java.util.List;
-import com.fasterxml.jackson.databind.JsonNode;
-import com.fasterxml.jackson.databind.node.ArrayNode;
-import com.fasterxml.jackson.databind.node.JsonNodeFactory;
-import com.fasterxml.jackson.databind.node.ObjectNode;
+import tools.jackson.databind.JsonNode;
+import tools.jackson.databind.node.ArrayNode;
+import tools.jackson.databind.node.JsonNodeFactory;
+import tools.jackson.databind.node.ObjectNode;
 
 public class Sequence implements Serializable {
 

src/main/java/com/api/jsonata4java/Signature.java

@@ -33,10 +33,10 @@
 import com.api.jsonata4java.expressions.functions.DeclaredFunction;
 import com.api.jsonata4java.expressions.generated.MappingExpressionParser.ExprContext;
 import com.api.jsonata4java.expressions.generated.MappingExpressionParser.ExprListContext;
-import com.fasterxml.jackson.databind.JsonNode;
-import com.fasterxml.jackson.databind.node.ArrayNode;
-import com.fasterxml.jackson.databind.node.JsonNodeFactory;
-import com.fasterxml.jackson.databind.node.ObjectNode;
+import tools.jackson.databind.JsonNode;
+import tools.jackson.databind.node.ArrayNode;
+import tools.jackson.databind.node.JsonNodeFactory;
+import tools.jackson.databind.node.ObjectNode;
 
 /**
  * Manages signature related functions