Merge pull request #77 from sqlcj-dev/feature/subscripted-array-columns-as-expressions

feat(analysis): treat subscripted array columns as expressions

Author: omer-cengel

docs/postgresql.md

@@ -368,6 +368,11 @@ generates `List<String> tags` and `List<StageSetting> past`.
   index.
 - A `LIKE`/`ILIKE` pattern placeholder still requires a scalar `VARCHAR` or
   `TEXT` column, so an array column is rejected there.
+- A reference written with a subscript or a slice, such as `tags[1]`,
+  `tags[1:2]`, or `s.tags[1]`, reads one element or one range rather than the
+  column, so it is not a direct column anywhere: it types no result column and
+  no placeholder, and a cast states the type of either. See
+  [Queries](queries.md#projections).
 - The `= ANY` list predicate is the one analyzed array operator: a placeholder
   written as `<column> = ANY($1)` binds a list of the compared column's type,
   which must be a non-array column of an element type above or of a declared

docs/queries.md

@@ -128,6 +128,11 @@ configuration entry.
 
 - A direct column of a source is selected by its own name or qualified with the
   exposed source name.
+- A reference written with an array subscript or slice, such as `labels[1]`,
+  `labels[1:2]`, or `s.labels[1]`, is not a direct column: it reads one element
+  or one range of the array column rather than the column, so it is rejected
+  like any other computed projection unless a cast states its type, as in
+  `labels[1]::text AS first` or `labels[1:2]::varchar[] AS pair`.
 - `*` expands across the sources in their declared order, each source in its
   schema column order.
 - `qualifier.*` expands one source in its schema column order.
@@ -228,6 +233,16 @@ A `WHERE` clause may combine:
 - `= ANY` between a direct column and one `$N` list placeholder, such as
   `id = ANY($1)`.
 
+A direct column is a column reference written without an array subscript or
+slice. A subscripted or sliced reference, such as `labels[1]`, `labels[1:2]`, or
+`u.labels[1]`, is an expression of the array column, so no placeholder beside
+one is typed from that column: `labels[1] = $1` is rejected as an unanalyzed
+placeholder location, in a comparison of either order, an `IN` list, a `LIKE`
+pattern, a `BETWEEN` bound, an `= ANY` list, a join `ON` condition, and a
+write's `WHERE` alike. A cast states the type of such a placeholder, as in
+`WHERE labels[1] = $1::text`, and the subscript itself reaches the database as
+written.
+
 A `LIKE` or `ILIKE` pattern placeholder requires a `VARCHAR` or `TEXT` column
 and takes that column's type, so it is a `String` method parameter. sqlcj passes
 the pattern through unchanged, so the caller supplies the `%` and `_` wildcards
@@ -408,6 +423,15 @@ FROM authors
 WHERE id = $1;
 ```
 
+A written column may carry an array subscript or slice, as in
+`SET labels[1] = $1::text` or `INSERT INTO stages (id, labels[1])`. Such a
+target is not a direct column: it writes one element or one range, so its value
+is not typed from the array column and a placeholder value needs a cast, while
+the array column itself must exist in the written table and the subscript
+reaches the database as written. An uncast placeholder written there is rejected
+as an unanalyzed placeholder location, in an `UPDATE` or `DO UPDATE` assignment
+and in an `INSERT` value alike.
+
 ### Values that bind no placeholder
 
 An `INSERT` value or an `UPDATE` assignment that contains no placeholder — such
@@ -565,7 +589,10 @@ is rejected as
 `Unsupported RETURNING item <expression>; return columns of the target table or *`,
 naming the expression as the query writes it, on one line. A cast does not make
 such an item supported, because a `RETURNING` clause returns columns of the
-target table.
+target table. A reference written with an array subscript or slice, such as
+`RETURNING labels[1]` or `RETURNING labels[1:2] AS pair`, is not a direct
+column of the target table either, so it is rejected as a computed item, with or
+without an alias.
 
 Multi-row `VALUES`, `INSERT ... SELECT`, `UPDATE ... FROM`, and
 `DELETE ... USING` are rejected in every write, and common table expressions in
@@ -616,6 +643,11 @@ string literal, a quoted identifier, or a comment is not a parameter.
   Every other indexed cast placeholder is named `param<N>`
   after its own index, as in `param1` for `$1`, and colliding names take the
   usual numeric suffix.
+- A column written with an array subscript or slice names no parameter, because
+  the parameter beside it is one element rather than the column's value, so an
+  indexed cast placeholder there is `param<N>` and a named one keeps its own
+  name: `labels[1] = $1::text` binds `param1`, where `labels = $1::text` would
+  bind `labels`.
 - Every placeholder occurrence binds in textual order across the clauses and
   nesting depths above, whatever order sqlcj analyzes those clauses in, so a
   projected placeholder binds before a `WHERE` one and
@@ -943,6 +975,18 @@ query name, and its header line.
   `SELECT` for `SELECT $1 AS x`. A cast states the type of such a placeholder in
   every one of those locations, and it does not accept a placeholder that is not
   the direct operand of its cast, such as `(:x)::int`.
+- A column reference written with an array subscript or slice wherever a direct
+  column is required, because it reads one element or one range rather than the
+  column: `SELECT labels[1] AS first` and `SELECT labels[1:2] AS pair` are
+  rejected as computed projections, `RETURNING labels[1]` as a computed returned
+  item, and an uncast placeholder beside one — `labels[1] = $1`,
+  `$1 = labels[1]`, `labels[1] IN ($1, $2)`, `labels[1] LIKE $1`,
+  `scores[1] BETWEEN $1 AND $2`, `labels[1] = ANY($1)`, a join
+  `ON u.labels[1] = $1`, a write's `WHERE labels[1] = $1`, a
+  `DO UPDATE ... WHERE stages.labels[1] = $1`, `SET labels[1] = $1`, and
+  `INSERT INTO stages (id, labels[1]) VALUES ($1, $2)` — as an unanalyzed
+  placeholder location. A cast states the type in each of those locations, and
+  the array column of a subscripted write target must still exist.
 - A cast type sqlcj does not map, such as `$1::interval` or a
   multi-dimensional `$1::int[][]`, which is rejected naming the placeholder and
   the declared type, or the result column and the declared type when the cast
@@ -1446,6 +1490,22 @@ Parameters:
   `PostgresIntegrationTest.shouldExecuteGeneratedCastTypedOptionalFilterAgainstPostgres`
   executes a named cast-typed optional filter against PostgreSQL 16 with a null
   and a non-null argument.
+- `QueryAnalyzerTest.shouldRejectASubscriptedProjection`,
+  `QueryAnalyzerTest.shouldAnalyzeACastSubscriptedProjection`,
+  `QueryAnalyzerTest.shouldRejectASubscriptedReturningItem`,
+  `QueryAnalyzerTest.shouldRejectAnUncastPlaceholderBesideASubscriptedColumn`,
+  `QueryAnalyzerTest.shouldNameACastPlaceholderBesideASubscriptedColumnAfterItself`,
+  `QueryAnalyzerTest.shouldNameACastPlaceholderOfASubscriptedWriteTargetAfterItself`,
+  `QueryAnalyzerTest.shouldRejectAnUnknownSubscriptedWriteTarget`, and
+  `QueryAnalyzerTest.shouldAnalyzeALiteralAssignedToASubscriptedColumn` cover a
+  subscripted and a sliced reference in qualified and unqualified spellings:
+  the projection and `RETURNING` rejections, the cast projection that states its
+  type, the untyped-placeholder diagnostic in each typing location, the name a
+  cast placeholder keeps beside one, the write target whose array column must
+  exist, and the non-binding value assigned to one, and
+  `PostgresIntegrationTest.shouldReadAndFilterASubscriptedArrayElement` executes
+  a cast subscript projection and a cast subscript filter against PostgreSQL 16
+  as one `String` component and one `String` method parameter.
 
 Generated Java:
 

sqlcj-cli/src/main/java/dev/sqlcj/analysis/QueryAnalyzer.java

@@ -669,8 +669,8 @@ private String resolveReturningRowTable(ReturningClause returningClause, Source
     /**
      * Resolves one returned item, which is either a bare {@code *} or a direct
      * column of the write target, whose alias names its result component. A
-     * computed item has no schema type to generate, so it is rejected instead
-     * of analyzed.
+     * computed item, a subscripted reference included, has no schema type to
+     * generate, so it is rejected instead of analyzed.
      */
     private List<QueryColumn> resolveReturningItem(Expression expression, Alias alias, Source source) {
         if (expression instanceof AllTableColumns) {
@@ -695,8 +695,11 @@ private List<QueryColumn> resolveReturningItem(Expression expression, Alias alia
             return resolveAllColumns(source);
         }
 
-        if (expression instanceof net.sf.jsqlparser.schema.Column column) {
-            dev.sqlcj.schema.Column schemaColumn = resolveColumn(column, List.of(source)).column();
+        net.sf.jsqlparser.schema.Column returnedColumn = directColumn(expression);
+
+        if (returnedColumn != null) {
+            dev.sqlcj.schema.Column schemaColumn = resolveColumn(returnedColumn, List.of(source))
+                .column();
 
             return List.of(
                 new QueryColumn(
@@ -808,6 +811,10 @@ private void requireNoCommonTableExpressions(List<?> withItems) {
      * list with the single values row, which is also their textual order. A
      * value that binds no placeholder reaches the database as written, so only
      * its target column must exist.
+     *
+     * <p>A target written with a subscript inserts into one element of its
+     * array column, so its value is not typed from that column, which must
+     * still exist.
      */
     private void resolveInsertParameters(
         Insert insert,
@@ -829,13 +836,12 @@ private void resolveInsertParameters(
         }
 
         for (int index = 0; index < columns.size(); index++) {
-            String columnName = columns.get(index).getUnquotedColumnName();
+            net.sf.jsqlparser.schema.Column target = columns.get(index);
+            dev.sqlcj.schema.Column column = findColumn(table, target.getUnquotedColumnName());
 
-            resolveColumnValue(
-                values.get(index),
-                findColumn(table, columnName),
-                placeholders
-            );
+            if (directColumn(target) != null) {
+                resolveColumnValue(values.get(index), column, placeholders);
+            }
         }
     }
 
@@ -857,6 +863,10 @@ private ParenthesedExpressionList<?> resolveInsertValues(Insert insert) {
      * parameter in the {@code WHERE} expression. An assigned value that binds
      * no placeholder reaches the database as written, so only its target
      * column must exist.
+     *
+     * <p>A target written with a subscript assigns one element of its array
+     * column, so its value is not typed from that column, which must still
+     * exist.
      */
     private void resolveUpdateSetParameters(
         List<UpdateSet> updateSets,
@@ -870,13 +880,12 @@ private void resolveUpdateSetParameters(
                 );
             }
 
-            String columnName = updateSet.getColumn(0).getUnquotedColumnName();
+            net.sf.jsqlparser.schema.Column target = updateSet.getColumn(0);
+            dev.sqlcj.schema.Column column = findColumn(table, target.getUnquotedColumnName());
 
-            resolveColumnValue(
-                updateSet.getValue(0),
-                findColumn(table, columnName),
-                placeholders
-            );
+            if (directColumn(target) != null) {
+                resolveColumnValue(updateSet.getValue(0), column, placeholders);
+            }
         }
     }
 
@@ -1724,7 +1733,9 @@ private void resolveParameters(
     }
 
     private void resolveInExpression(InExpression in, List<Source> sources, Placeholders placeholders) {
-        if (!(in.getLeftExpression() instanceof net.sf.jsqlparser.schema.Column column)) {
+        net.sf.jsqlparser.schema.Column column = directColumn(in.getLeftExpression());
+
+        if (column == null) {
             return;
         }
 
@@ -1854,11 +1865,13 @@ private void resolveLikeExpression(
         if (pattern != null) {
             requireSupportedLikePattern(like);
 
-            if (!(left instanceof net.sf.jsqlparser.schema.Column column)) {
+            net.sf.jsqlparser.schema.Column testedColumn = directColumn(left);
+
+            if (testedColumn == null) {
                 return;
             }
 
-            dev.sqlcj.schema.Column schemaColumn = resolveColumn(column, sources).column();
+            dev.sqlcj.schema.Column schemaColumn = resolveColumn(testedColumn, sources).column();
 
             requireSupportedType(schemaColumn);
 
@@ -1872,15 +1885,16 @@ private void resolveLikeExpression(
         }
 
         CastPlaceholder castPattern = placeholders.ofCast(right);
+        net.sf.jsqlparser.schema.Column testedColumn = directColumn(left);
 
         if (
             castPattern != null
                 && unsupportedLikePatternReason(like) == null
-                && left instanceof net.sf.jsqlparser.schema.Column column
+                && testedColumn != null
         ) {
             addCastParameter(
                 castPattern,
-                resolveColumn(column, sources).column().name(),
+                resolveColumn(testedColumn, sources).column().name(),
                 placeholders
             );
         }
@@ -1983,11 +1997,10 @@ private void resolveBetweenExpression(
             return;
         }
 
-        if (
-            left instanceof net.sf.jsqlparser.schema.Column column
-                && (bindsValue(start, placeholders) || bindsValue(end, placeholders))
-        ) {
-            dev.sqlcj.schema.Column schemaColumn = resolveColumn(column, sources).column();
+        net.sf.jsqlparser.schema.Column testedColumn = directColumn(left);
+
+        if (testedColumn != null && (bindsValue(start, placeholders) || bindsValue(end, placeholders))) {
+            dev.sqlcj.schema.Column schemaColumn = resolveColumn(testedColumn, sources).column();
 
             resolveColumnValue(start, schemaColumn, placeholders);
             resolveColumnValue(end, schemaColumn, placeholders);
@@ -2013,52 +2026,55 @@ private void resolveParameterComparison(
         Placeholder leftPlaceholder = placeholders.of(left);
         Placeholder rightPlaceholder = placeholders.of(right);
 
-        if (left instanceof net.sf.jsqlparser.schema.Column column && rightPlaceholder != null) {
+        net.sf.jsqlparser.schema.Column leftColumn = directColumn(left);
+        net.sf.jsqlparser.schema.Column rightColumn = directColumn(right);
+
+        if (leftColumn != null && rightPlaceholder != null) {
             addParameter(
                 rightPlaceholder,
-                resolveColumn(column, sources).column(),
+                resolveColumn(leftColumn, sources).column(),
                 placeholders
             );
             return;
         }
 
-        if (leftPlaceholder != null && right instanceof net.sf.jsqlparser.schema.Column column) {
+        if (leftPlaceholder != null && rightColumn != null) {
             addParameter(
                 leftPlaceholder,
-                resolveColumn(column, sources).column(),
+                resolveColumn(rightColumn, sources).column(),
                 placeholders
             );
             return;
         }
 
-        if (comparison instanceof EqualsTo && left instanceof net.sf.jsqlparser.schema.Column column) {
+        if (comparison instanceof EqualsTo && leftColumn != null) {
             Expression listArgument = anyListArgument(right, placeholders);
 
             if (listArgument != null) {
-                resolveAnyListParameter(listArgument, column, sources, placeholders);
+                resolveAnyListParameter(listArgument, leftColumn, sources, placeholders);
                 return;
             }
         }
 
         if (isColumnTypedComparison(comparison)) {
-            if (left instanceof net.sf.jsqlparser.schema.Column column) {
+            if (leftColumn != null) {
                 CastPlaceholder cast = placeholders.ofCast(right);
 
                 if (cast != null) {
                     addCastParameter(
                         cast,
-                        resolveColumn(column, sources).column().name(),
+                        resolveColumn(leftColumn, sources).column().name(),
                         placeholders
                     );
                     return;
                 }
-            } else if (right instanceof net.sf.jsqlparser.schema.Column column) {
+            } else if (rightColumn != null) {
                 CastPlaceholder cast = placeholders.ofCast(left);
 
                 if (cast != null) {
                     addCastParameter(
                         cast,
-                        resolveColumn(column, sources).column().name(),
+                        resolveColumn(rightColumn, sources).column().name(),
                         placeholders
                     );
                 }
@@ -2372,6 +2388,9 @@ private Placeholders toPlaceholders(ParsedSql parsedSql, Schema schema) {
      * <p>A bare placeholder projection is a placeholder a cast makes a
      * supported result column, so it is rejected as an untyped placeholder
      * rather than as an unsupported result expression.
+     *
+     * <p>A subscripted reference projects an element rather than the column, so
+     * it is rejected as any other computed item is, and a cast states its type.
      */
     private List<QueryColumn> resolveColumns(
         PlainSelect plainSelect,
@@ -2405,8 +2424,10 @@ private List<QueryColumn> resolveColumns(
                 continue;
             }
 
-            if (expression instanceof net.sf.jsqlparser.schema.Column column) {
-                ResolvedColumn resolved = resolveColumn(column, sources);
+            net.sf.jsqlparser.schema.Column selectedColumn = directColumn(expression);
+
+            if (selectedColumn != null) {
+                ResolvedColumn resolved = resolveColumn(selectedColumn, sources);
                 dev.sqlcj.schema.Column schemaColumn = resolved.column();
 
                 columns.add(
@@ -2551,6 +2572,22 @@ private String selectedColumnName(Alias alias, dev.sqlcj.schema.Column schemaCol
             : alias.getUnquotedName();
     }
 
+    /**
+     * Reports the column one expression reads directly, and {@code null} when
+     * the expression reads none. A reference written with a subscript or a
+     * slice, such as {@code labels[1]} or {@code labels[1:2]}, reads one
+     * element or one range of the column's array value rather than the column,
+     * so it states neither the type nor the name of the column it reads, and
+     * every rule that takes a type or a name from a column leaves it to the
+     * rules of any other expression.
+     */
+    private net.sf.jsqlparser.schema.Column directColumn(Expression expression) {
+        return expression instanceof net.sf.jsqlparser.schema.Column column
+            && column.getArrayConstructor() == null
+                ? column
+                : null;
+    }
+
     /**
      * Resolves one column reference against the ordered query sources. A
      * qualified reference resolves through the exposed source name, and an