Never vendor the parser: fetch it from Maven everywhere, including the .bat path

A fresh clone now contains no GSP parser jar at all. lib/ held two,
gsqlparser-3.1.1.0 and gudusoft.gsqlparser-3.0.2.5, and they were doing real
damage rather than just taking up space.

  pom_dlineage.xml      pinned lib/gsqlparser-3.1.1.0.jar on system scope. This
                        build has been documented as failing since before the
                        demo trees were merged, on two "cannot find symbol"
                        errors for setTraceTablePosition and
                        generateColumnLevelLineageCsvSimple. The cause was the
                        pinned jar, not the code: pointed at the same parser the
                        root build resolves, it compiles, packages, and produces
                        correct lineage XML. Verified by running the jar.
  connector/*/pom.xml   pointed at ../../lib/gudusoft.gsqlparser-2.1.2.x.jar,
                        files this repository has not contained for a long time.
                        Now resolved from Maven like everything else.
  the .bat scripts      setenv.bat put lib\ first on the classpath, so the 3.1.1.0
                        jar shadowed anything newer and compile_checksyntax.bat
                        died on EOBTenantMode -- a symbol a current parser has.
  gudusoft.gsqlparser-3.0.2.5.jar
                        referenced by nothing at all.

So: both jars deleted, and every consumer resolves com.gudusoft:gsqlparser from
Gudu's public repository.

For the .bat workflow, which has no POM to resolve from, setenv.bat now
bootstraps itself. When external_lib\ has no parser it calls the new
setenv/fetch-parser.bat, which reads gsp.core.version out of pom.xml and does a
mvn dependency:copy. Idempotent, and external_lib/ is gitignored. This does mean
that route needs Maven once, which the README states plainly rather than keeping
the old "Windows, no Maven" heading: one artifact resolved from one place beats
a vendored copy that silently rots, which is exactly what happened here.

windows-bat now covers four demos instead of one, chosen for the shapes the
scripts come in: checksyntax (/f file /t vendor), formatsql (bare filename),
listGSPInfo (no arguments), and modifysql, whose compile and run scripts are
named differently (compile_modifysql.bat builds the folder,
run_replaceTablename.bat runs one class). The job also asserts up front that no
parser jar is committed, so vendoring one again fails CI.

mvn clean test: 144 tests, the same 3 known analyzespTest failures.
mvn -f pom_dlineage.xml package: builds, for the first time in this repository's
recorded history.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012qPRpoD8exYRrUmbfXXWXj

Author: 2 people

.github/workflows/build.yml

@@ -112,39 +112,63 @@ jobs:
           java-version: "8"
           cache: maven
 
-      # These scripts predate the parser being resolved from Maven. lib\ only
-      # carries old parser jars now, so on its own compile_<demo>.bat fails on
-      # symbols those jars predate. Drop the current parser into external_lib\,
-      # which setenv.bat puts ahead of lib\ on the classpath for exactly this
-      # reason. Not vendored: fetched here, and gitignored.
-      - name: Fetch the parser into external_lib
-        shell: bash
+      # No parser jar is committed, so setenv.bat fetches one into external_lib\
+      # on first use. This step only proves that bootstrap works from a clean
+      # checkout; the demo steps below would trigger it anyway.
+      - name: setenv.bat bootstraps the parser from Maven
+        shell: cmd
         run: |
-          set -euo pipefail
-          ver=$(mvn -q help:evaluate -Dexpression=gsp.core.version -DforceStdout)
-          echo "parser version: $ver"
-          mvn -q dependency:copy \
-            -Dartifact=com.gudusoft:gsqlparser:"$ver" \
-            -DoutputDirectory=external_lib
-          ls external_lib
+          if exist external_lib\gsqlparser-*.jar (
+            echo ::error::a parser jar is committed to this repository; it must be fetched, not vendored
+            exit /b 1
+          )
+          call setenv\fetch-parser.bat
+          dir external_lib
+          for %%f in (external_lib\gsqlparser-*.jar) do exit /b 0
+          echo ::error::fetch-parser.bat produced no parser jar
+          exit /b 1
 
       # `pause` at the end of each script would block forever on a runner with
       # no keyboard, so stdin is fed from NUL.
-      - name: compile_checksyntax.bat then run_checksyntax.bat
+      #
+      # These four cover the shapes the scripts come in: a file+vendor demo, a
+      # bare-filename demo, one that takes no arguments at all, and one nested a
+      # directory deeper (so its cd depth differs). They are generated from one
+      # template and go stale as a set, which is exactly what happened when the
+      # demos moved directory, so this is a canary rather than full coverage.
+      - name: Run the .bat scripts
         shell: cmd
         run: |
-          cd src\main\java\gudusoft\gsqlparser\demos\checksyntax
-          call compile_checksyntax.bat < NUL
-          if not exist "%GITHUB_WORKSPACE%\build\gudusoft\gsqlparser\demos\checksyntax\checksyntax.class" (
-            echo ::error::compile_checksyntax.bat did not produce checksyntax.class
-            exit /b 1
+          echo SELECT a.id FROM ta a; > "%GITHUB_WORKSPACE%\q.sql"
+
+          rem  dir            compile script    run script         args                                      expected in output
+          call :demo checksyntax  checksyntax       checksyntax       "/f %GITHUB_WORKSPACE%\q.sql /t oracle"   "syntax errors: 0" || exit /b 1
+          call :demo formatsql    formatsql         formatsql         "%GITHUB_WORKSPACE%\q.sql"                "SELECT"           || exit /b 1
+          call :demo listGSPInfo  listGSPInfo       listGSPInfo       ""                                        "Supported DBs"    || exit /b 1
+          call :demo modifysql    modifysql         replaceTablename  ""                                        "output sql"       || exit /b 1
+          echo All .bat demos passed.
+          exit /b 0
+
+          :demo
+          setlocal
+          set DEMO=%~1
+          set CSCRIPT=%~2
+          set RSCRIPT=%~3
+          set ARGS=%~4
+          set EXPECT=%~5
+          echo.
+          echo ==== %DEMO% : compile_%CSCRIPT%.bat / run_%RSCRIPT%.bat ====
+          cd /d "%GITHUB_WORKSPACE%\src\main\java\gudusoft\gsqlparser\demos\%DEMO%"
+          call compile_%CSCRIPT%.bat < NUL > "%GITHUB_WORKSPACE%\c.txt" 2>&1
+          type "%GITHUB_WORKSPACE%\c.txt"
+          findstr /i /c:"error" "%GITHUB_WORKSPACE%\c.txt" >NUL && (
+            echo ::error::compile_%CSCRIPT%.bat reported an error
+            endlocal & exit /b 1
           )
-          cd /d "%GITHUB_WORKSPACE%"
-          echo SELECT a.id FROM ta a; > q.sql
-          cd src\main\java\gudusoft\gsqlparser\demos\checksyntax
-          call run_checksyntax.bat /f "%GITHUB_WORKSPACE%\q.sql" /t oracle < NUL > "%GITHUB_WORKSPACE%\out.txt" 2>&1
-          type "%GITHUB_WORKSPACE%\out.txt"
-          findstr /c:"syntax errors: 0" "%GITHUB_WORKSPACE%\out.txt" >NUL || (
-            echo ::error::run_checksyntax.bat did not report "syntax errors: 0"
-            exit /b 1
+          call run_%RSCRIPT%.bat %ARGS% < NUL > "%GITHUB_WORKSPACE%\r.txt" 2>&1
+          type "%GITHUB_WORKSPACE%\r.txt"
+          findstr /c:"%EXPECT%" "%GITHUB_WORKSPACE%\r.txt" >NUL || (
+            echo ::error::run_%RSCRIPT%.bat did not print "%EXPECT%"
+            endlocal & exit /b 1
           )
+          endlocal & exit /b 0

README.md

@@ -408,30 +408,36 @@ Everything else in those packages still builds. Reviving them needs an API
 migration against a build that includes the metadata layer, not just a
 recompile.
 
-## The .bat scripts (Windows, no Maven)
+## The .bat scripts (Windows)
 
 Each demo directory also ships `compile_<demo>.bat` and `run_<demo>.bat`, with
-`setenv/setenv.bat` holding the shared environment. The workflow:
-
-1. **Put the parser in `external_lib/`.** From the repository root:
-   ```
-   mvn dependency:copy -Dartifact=com.gudusoft:gsqlparser:4.1.6 -DoutputDirectory=external_lib
-   ```
-2. `cd` into a demo directory, e.g. `src\main\java\gudusoft\gsqlparser\demos\checksyntax`
-3. run `compile_checksyntax.bat`, then `run_checksyntax.bat`
-
-You no longer have to edit `setenv.bat` for step 1 of the old instructions:
-it keeps whatever `JAVA_HOME` is already set and only falls back to a fixed path
-when there is none.
-
-**`external_lib/` is not optional, and it must come before `lib/`.** These
-scripts predate the parser being resolved from Maven, and `lib/` now holds only
-*old* parser jars that other things still pin (`gsqlparser-3.1.1.0`,
-`gudusoft.gsqlparser-3.0.2.5`). With `lib/` first on the classpath those shadow
-the current parser and the demos fail to compile on symbols the old jars predate
-— `checksyntax` dies on `EOBTenantMode`. `setenv.bat` therefore puts
-`external_lib\*` ahead of `lib\*`, and `external_lib/` is gitignored so the
-parser is fetched rather than vendored.
+`setenv/setenv.bat` holding the shared environment.
+
+```
+cd src\main\java\gudusoft\gsqlparser\demos\checksyntax
+compile_checksyntax.bat
+run_checksyntax.bat /f ..\..\..\..\..\..\..\q.sql /t oracle
+```
+
+That is the whole workflow now. You no longer edit `setenv.bat` first: it keeps
+whatever `JAVA_HOME` is already set, and it fetches the parser for you.
+
+**No parser jar is committed to this repository**, so the first script you run
+pulls one down. `setenv.bat` calls `setenv/fetch-parser.bat`, which reads
+`gsp.core.version` out of `pom.xml` and does a `mvn dependency:copy` into
+`external_lib/`. It is a no-op once the jar is there, and `external_lib/` is
+gitignored.
+
+That means the `.bat` route needs Maven **once**, to fetch, and never again.
+That is a deliberate trade. Vendoring a parser under `lib/` is what let these
+scripts quietly compile against a build years older than their own source: the
+jar in `lib/` was 3.1.1.0 while the demos had moved on to APIs like
+`EOBTenantMode`, so `compile_checksyntax.bat` failed on a symbol that a current
+parser has. One artifact, resolved from one place, is worth a one-time Maven
+call.
+
+`external_lib/` also comes before `lib/` on the classpath, so the fetched parser
+wins over anything dropped into `lib/` later.
 
 ### These are now tested on Windows
 
@@ -440,15 +446,24 @@ They had been stale for years — compiling `src\main\java\demos\<demo>\` and
 `gudusoft/gsqlparser/demos/`. Nothing noticed, because nothing ran them.
 
 The `windows-bat` job in `.github/workflows/build.yml` now does, on
-`windows-latest`: it fetches the parser into `external_lib/`, runs
-`compile_checksyntax.bat`, asserts the `.class` file appears, runs
-`run_checksyntax.bat`, and asserts it reports `syntax errors: 0`. Each script
-ends with `pause`, so CI feeds their stdin from `NUL` to keep them from blocking
-on a runner with no keyboard.
+`windows-latest`. It first asserts that **no parser jar is committed** and that
+`fetch-parser.bat` can bootstrap one, then compiles and runs four demos, each
+checked against a string its output must contain:
+
+| demo | shape it covers |
+|------|-----------------|
+| `checksyntax` | `/f <file> /t <vendor>` |
+| `formatsql` | bare filename |
+| `listGSPInfo` | no arguments at all |
+| `modifysql` | compile and run scripts named differently (`compile_modifysql.bat` builds the folder, `run_replaceTablename.bat` runs one class) |
+
+Each script ends with `pause`, so CI feeds their stdin from `NUL` to keep them
+from blocking on a runner with no keyboard.
 
-That covers one demo, not all 45. It is enough to catch the failure mode that
-actually happened here — the whole family going stale together after a
-directory move — since they are generated from one template and break as a set.
+Four demos, not all 45, is a deliberate stopping point: they are generated from
+one template and go stale as a set, which is exactly what happened when the
+demos moved directory. The four cover the distinct argument shapes, so a change
+that breaks the template shows up here.
 
 
 ## Building the dlineage demo on its own
@@ -459,7 +474,7 @@ standalone lineage tool:
 ```bash
 mvn -f pom_dlineage.xml package
 
-java -cp "target-dlineage/gsp_demo_java_dlineage-1.0-SNAPSHOT.jar:lib/*" \
+java -cp "target-dlineage/gsp_demo_java_dlineage-1.0-SNAPSHOT.jar:external_lib/*:lib/*" \
      gudusoft.gsqlparser.demos.dlineage.DataFlowAnalyzer \
      /f demo.sql /o lineage.json /json /graph \
      /simpleShowRelationTypes fdd,fdr /filterRelationTypes fdd
@@ -474,15 +489,16 @@ rather than the root build's `target/`. It shares this repository's
 without a separate output directory the compiler plugin's incremental-build
 cleanup would delete every `.class` file the root build produced that isn't
 part of this smaller source set — wiping out a working root build before this
-one even reaches its own compile error below. See
+one even reaches its own build. See
 [#39](https://github.com/sqlparser/gsp_demo_java/issues/39).
 
-> **This build currently fails**, and did so before the trees were merged. It
-> pins `lib/gsqlparser-3.1.1.0.jar`, but `DataFlowAnalyzer` has moved on and now
-> calls `getOption().setTraceTablePosition(...)` and
-> `ProcessUtility.generateColumnLevelLineageCsvSimple(...)`, neither of which
-> that jar has. It needs a parser build carrying both those methods and the
-> metadata layer. Recorded here so the invocations above are not lost.
+**This build used to fail**, and had done since before the demo trees were
+merged. The cause was the pinned jar, not the code: `pom_dlineage.xml` declared
+`lib/gsqlparser-3.1.1.0.jar` on `system` scope, while `DataFlowAnalyzer` had
+moved on to `getOption().setTraceTablePosition(...)` and
+`ProcessUtility.generateColumnLevelLineageCsvSimple(...)`, neither of which a
+3.1.1.0 jar has. It now resolves the same parser the root build does, and
+builds and runs.
 
 ## master and dev branches
 

connector/oracleConnector/pom.xml

@@ -17,13 +17,21 @@
 		<maven.compiler.target>1.6</maven.compiler.target>
 	</properties>
 
+	<repositories>
+		<repository>
+			<id>gudu-public-releases</id>
+			<url>https://www.sqlparser.com/maven/</url>
+		</repository>
+	</repositories>
+
 	<dependencies>
+		<!-- The parser is resolved from Gudu's public Maven repository. It used to
+		     point at ../../lib/gudusoft.gsqlparser-2.1.2.x.jar, a file this
+		     repository has not contained for a long time. -->
 		<dependency>
-			<groupId>com.sqlparser</groupId>
-			<artifactId>gsp</artifactId>
-			<version>2.1.2.2</version>
-			<scope>system</scope>
-			<systemPath>${project.basedir}/../../lib/gudusoft.gsqlparser-2.1.2.2.jar</systemPath>
+			<groupId>com.gudusoft</groupId>
+			<artifactId>gsqlparser</artifactId>
+			<version>4.1.6</version>
 		</dependency>
 		<dependency>
 			<groupId>com</groupId>

connector/snowflakeConnector/pom.xml

@@ -17,13 +17,21 @@
 		<maven.compiler.target>1.6</maven.compiler.target>
 	</properties>
 
+	<repositories>
+		<repository>
+			<id>gudu-public-releases</id>
+			<url>https://www.sqlparser.com/maven/</url>
+		</repository>
+	</repositories>
+
 	<dependencies>
+		<!-- The parser is resolved from Gudu's public Maven repository. It used to
+		     point at ../../lib/gudusoft.gsqlparser-2.1.2.x.jar, a file this
+		     repository has not contained for a long time. -->
 		<dependency>
-			<groupId>com.sqlparser</groupId>
-			<artifactId>gsp</artifactId>
-			<version>2.1.2.8</version>
-			<scope>system</scope>
-			<systemPath>${project.basedir}/../../lib/gudusoft.gsqlparser-2.1.2.8.jar</systemPath>
+			<groupId>com.gudusoft</groupId>
+			<artifactId>gsqlparser</artifactId>
+			<version>4.1.6</version>
 		</dependency>
 		<!-- Resolved from Maven Central rather than a jar you drop into lib/.
 		     3.12.9 was named here before, which Dependabot flagged for six

connector/sqlServerConnector/pom.xml

@@ -17,13 +17,21 @@
 		<maven.compiler.target>1.6</maven.compiler.target>
 	</properties>
 
+	<repositories>
+		<repository>
+			<id>gudu-public-releases</id>
+			<url>https://www.sqlparser.com/maven/</url>
+		</repository>
+	</repositories>
+
 	<dependencies>
+		<!-- The parser is resolved from Gudu's public Maven repository. It used to
+		     point at ../../lib/gudusoft.gsqlparser-2.1.2.x.jar, a file this
+		     repository has not contained for a long time. -->
 		<dependency>
-			<groupId>com.sqlparser</groupId>
-			<artifactId>gsp</artifactId>
-			<version>2.1.2.2</version>
-			<scope>system</scope>
-			<systemPath>${project.basedir}/../../lib/gudusoft.gsqlparser-2.1.2.2.jar</systemPath>
+			<groupId>com.gudusoft</groupId>
+			<artifactId>gsqlparser</artifactId>
+			<version>4.1.6</version>
 		</dependency>
 		<dependency>
 			<groupId>com.microsoft.sqlserver</groupId>

lib/gsqlparser-3.1.1.0.jar

@@ -23,8 +23,18 @@
         <maven.compiler.source>1.8</maven.compiler.source>
         <maven.compiler.target>1.8</maven.compiler.target>
         <maven.test.skip>true</maven.test.skip>
+        <gsp.core.version>4.1.6</gsp.core.version>
     </properties>
 
+    <!-- The parser is not on Maven Central; it comes from Gudu's public
+         repository, same as the root pom.xml. Nothing is vendored under lib/. -->
+    <repositories>
+        <repository>
+            <id>gudu-public-releases</id>
+            <url>https://www.sqlparser.com/maven/</url>
+        </repository>
+    </repositories>
+
     <dependencies>
         <dependency>
             <groupId>junit</groupId>
@@ -34,11 +44,9 @@
         </dependency>
 
         <dependency>
-            <groupId>gudusoft</groupId>
+            <groupId>com.gudusoft</groupId>
             <artifactId>gsqlparser</artifactId>
-            <version>3.1.1.0</version>
-            <scope>system</scope>
-            <systemPath>${project.basedir}/lib/gsqlparser-3.1.1.0.jar</systemPath>
+            <version>${gsp.core.version}</version>
         </dependency>
 
         <dependency>

lib/gudusoft.gsqlparser-3.0.2.5.jar

@@ -0,0 +1,51 @@
+@echo off
+REM # Fetches the General SQL Parser jar into external_lib\.
+REM #
+REM # No parser jar is committed to this repository. A fresh clone has none, by
+REM # design: the parser is a released artifact resolved from Gudu's public Maven
+REM # repository, and vendoring a copy is how this repository previously ended up
+REM # compiling demos against a build years older than their source.
+REM #
+REM # Called automatically by setenv\setenv.bat when external_lib\ has no parser,
+REM # so the compile_<demo>.bat / run_<demo>.bat scripts work on a fresh clone.
+REM # Safe to run directly, and a no-op once the jar is there.
+REM #
+REM # Requires Maven on PATH, once. After that the .bat workflow needs nothing.
+
+setlocal
+
+REM # Run from the repository root regardless of where this was invoked from.
+cd /d "%~dp0.."
+
+for %%f in (external_lib\gsqlparser-*.jar) do (
+    echo Parser already present: %%f
+    endlocal
+    exit /b 0
+)
+
+REM # Single source of truth for the version: gsp.core.version in pom.xml.
+for /f "delims=" %%v in ('mvn -q help:evaluate -Dexpression^=gsp.core.version -DforceStdout 2^>nul') do set GSPVER=%%v
+
+if "%GSPVER%"=="" (
+    echo.
+    echo ***************************
+    echo Could not read gsp.core.version from pom.xml.
+    echo Is Maven on your PATH? The parser has to be fetched once:
+    echo    mvn dependency:copy -Dartifact=com.gudusoft:gsqlparser:VERSION -DoutputDirectory=external_lib
+    echo ***************************
+    echo.
+    endlocal
+    exit /b 1
+)
+
+echo Fetching parser %GSPVER% into external_lib\ ...
+call mvn -q dependency:copy -Dartifact=com.gudusoft:gsqlparser:%GSPVER% -DoutputDirectory=external_lib
+if errorlevel 1 (
+    echo Failed to fetch the parser. See the message above.
+    endlocal
+    exit /b 1
+)
+
+echo Done.
+endlocal
+exit /b 0