fix: add reconnect support to JsWidget so deephaven.ui widgets revive

docs/groovy/conceptual/what-is-barrage.md

@@ -0,0 +1,128 @@
+---
+title: What is Barrage?
+---
+
+Barrage is Deephaven's streaming data protocol for efficient, real-time table transport between servers and clients. Built on top of [Apache Arrow Flight](https://arrow.apache.org/docs/format/Flight.html), Barrage extends the standard Flight protocol to support Deephaven's [incremental update model](./table-update-model.md)—enabling clients to receive only the data that has changed rather than entire table snapshots.
+
+## Why Barrage?
+
+When working with real-time data, traditional approaches have significant limitations:
+
+- **Polling**: Repeatedly fetching entire tables wastes bandwidth and CPU cycles, especially for large tables with small changes.
+- **Standard Flight**: Apache Arrow Flight provides efficient data transport, but lacks native support for incrementally updating datasets.
+
+Barrage solves these problems by:
+
+- Streaming only the rows that have been added, removed, or modified.
+- Supporting viewports so clients can subscribe to just the visible portion of a table.
+- Batching updates to balance latency against network efficiency.
+
+## When to use Barrage
+
+| Use Case                                                                        | Barrage Feature                          |
+| ------------------------------------------------------------------------------- | ---------------------------------------- |
+| **Real-time dashboards**: Display live-updating tables in a UI                  | Subscribe to streaming updates           |
+| **Server-to-server data sharing**: Move tables between Deephaven instances      | Subscribe or snapshot via shared tickets |
+| **Point-in-time analysis**: Capture table state for offline processing          | Snapshot to get a static copy            |
+| **Large table visualization**: Display a scrollable view of a million-row table | Viewports (via web UI or JS client)      |
+
+## Key concepts
+
+### Subscriptions vs. snapshots
+
+Barrage supports two primary modes of retrieving data:
+
+- **Subscription**: Opens a persistent connection that streams updates as the source table changes. The client receives an initial snapshot followed by incremental updates. Use this for ticking tables you want to monitor in real time.
+
+- **Snapshot**: Retrieves a one-time, static copy of the table. The connection closes after the data is delivered. Use this for static tables or when you need a point-in-time capture.
+
+```python skip-test
+from deephaven.barrage import barrage_session
+
+session = barrage_session(host="remote-server", port=10000)
+
+# Subscription: receives ongoing updates
+streaming_table = session.subscribe(ticket_bytes)
+
+# Snapshot: one-time static copy
+static_table = session.snapshot(ticket_bytes)
+```
+
+### Shared tickets
+
+Shared tickets are endpoints that allow tables to be published and consumed across different sessions. A client can publish a table to a shared ticket, and other clients (or servers) can subscribe to or snapshot that ticket.
+
+```python skip-test
+from pydeephaven import Session
+from pydeephaven.ticket import SharedTicket
+
+# Publish a table to a shared ticket
+client = Session(host="remote-server", port=10000)
+ticket = SharedTicket.random_ticket()
+client.publish_table(ticket, my_table_ref)
+```
+
+See [Capture Python client tables](../how-to-guides/capture-tables.md) for complete examples.
+
+### Viewports
+
+A viewport defines a window over a table — a range of row positions and a subset of columns. Viewports are essential for interactive applications where users scroll through large tables. Rather than streaming millions of rows, the server sends only the data visible in the current view.
+
+Viewports are automatically managed by Deephaven's web UI and JavaScript client. When a user scrolls or resizes a table view, the client updates its viewport subscription accordingly.
+
+> [!NOTE]
+> The Python `BarrageSession` API currently supports full-table subscriptions. Viewport functionality is available through the JavaScript client or by subscribing to pre-filtered tables.
+> The Python `BarrageSession` API currently supports full-table subscriptions. Viewport functionality is available through the JavaScript client.
+
+### Update intervals and batching
+
+Barrage aggregates table updates before sending them to subscribers. This batching reduces network overhead when tables update frequently. The update interval is configurable:
+
+- **Server default**: Set via `-Dbarrage.minUpdateInterval` (milliseconds). Default: 1000 (1 second).
+- **Per-subscription**: Configurable when initiating the subscription (advanced use).
+
+A shorter interval reduces latency but increases network traffic. A longer interval reduces traffic but introduces delay.
+
+## Architecture overview
+
+![Barrage architecture](../assets/conceptual/remote_and_local_server.png)
+
+1. **Remote server** hosts a table referenced by a ticket — the ticket is just a reference, not the data itself. Tickets can be scope tickets (variables in the global scope), export tickets, or shared tickets for cross-session access.
+2. **Barrage protocol** transports the actual data using Arrow Flight with incremental update metadata.
+3. **Local server** subscribes via `BarrageSession` and receives a full local copy of the data that stays synchronized with the source. This local table can participate in downstream queries (joins, filters, aggregations) that execute on the local server.
+
+> [!NOTE]
+> Java and Groovy clients are unique among Deephaven clients — they can perform downstream computation locally because they run the full Deephaven engine. Other clients (e.g., JavaScript, C++, and Python) receive data but rely on the server for query execution.
+
+## Barrage vs. Arrow Flight
+
+| Feature                  | Standard Arrow Flight | Barrage               |
+| ------------------------ | --------------------- | --------------------- |
+| Data format              | Arrow columnar format | Arrow columnar format |
+| Static table transfer    | ✅ Supported          | ✅ Supported          |
+| Incremental updates      | ❌ Not supported      | ✅ Native support     |
+| Viewports                | ❌ Not supported      | ✅ Supported          |
+| Update batching          | ❌ N/A                | ✅ Configurable       |
+| Row shifts/modifications | ❌ N/A                | ✅ Efficient encoding |
+
+Barrage is fully compatible with Arrow Flight — you can use a standard Flight client to fetch static snapshots via `DoGet`. The incremental update features require a Barrage-aware client.
+
+## Performance considerations
+
+- **Large initial snapshots**: When subscribing to a large table, the initial snapshot can be memory-intensive. Use [subscription growth controls](../how-to-guides/performance/barrage-performance.md#control-subscription-snapshot-size) to break large snapshots into smaller chunks.
+
+- **High-frequency updates**: Tables that tick rapidly can generate significant network traffic. Consider increasing `barrage.minUpdateInterval` or filtering data before subscription.
+
+- **Column selection**: Subscribe only to the columns you need. Fewer columns means less data to transfer.
+
+- **Monitoring**: Use the [Barrage performance tables](../how-to-guides/performance/barrage-performance.md) to track subscription health and identify bottlenecks.
+
+## Related documentation
+
+- [Capture Python client tables](../how-to-guides/capture-tables.md) - Complete tutorial for using Barrage with the Python client
+- [Barrage metrics](../how-to-guides/performance/barrage-performance.md) - Monitor Barrage performance
+- [Interpret Barrage metrics](./barrage-metrics.md) - Understand what the metrics mean
+- [Barrage schema annotation](../how-to-guides/data-import-export/barrage-schema.md) - Annotate schemas for complex types
+- [Incremental update model](./table-update-model.md) - How Deephaven represents table changes
+- [Core API design](./deephaven-core-api.md) - Technical details on the Deephaven API
+- [Barrage protocol documentation](/barrage/docs) - Low-level wire format reference

docs/groovy/how-to-guides/capture-tables.md

@@ -119,6 +119,76 @@ local_t_static = my_barrage_session.snapshot(ticket.bytes)
 
 Voila! You now have _real_ Deephaven server tables called `local_t_streaming` and `local_t_static`. These are not just references to Deephaven tables - they are _real_ Deephaven server tables that can be used in any Deephaven query.
 
+## Subscription lifecycle management
+
+Understanding when and how to manage Barrage subscriptions helps you build efficient, reliable applications.
+
+### Choose between subscribe and snapshot
+
+Use **subscribe** when:
+
+- You need real-time updates as the source table changes.
+- You're building a live dashboard or monitoring system.
+- The source table is ticking (updating periodically).
+
+Use **snapshot** when:
+
+- You need a one-time, static copy of the data.
+- The source table is static and won't change.
+- You want to capture a point-in-time state for analysis.
+- You need to reduce ongoing resource consumption.
+
+### Subscription resource usage
+
+Each active subscription consumes resources on both the server and client:
+
+| Resource | Server Impact                                    | Client Impact                                     |
+| -------- | ------------------------------------------------ | ------------------------------------------------- |
+| Memory   | Maintains subscriber state and pending updates   | Stores table data and applies incremental updates |
+| CPU      | Aggregates and serializes updates per subscriber | Deserializes and processes incoming updates       |
+| Network  | Sends periodic update batches to each subscriber | Receives and buffers incoming data                |
+
+For tables with frequent updates or many subscribers, these costs can add up. Monitor subscription health using the [Barrage performance tables](./performance/barrage-performance.md).
+
+### Close subscriptions when finished
+
+When you no longer need real-time updates, close the Barrage session to release resources:
+
+```python skip-test
+# Close the session when done
+my_barrage_session.close()
+```
+
+If you need to keep the session open for other subscriptions but want to release a specific table, you can drop the reference to the subscribed table. However, the underlying subscription may remain active until the session is closed.
+
+### Handle connection issues
+
+Barrage subscriptions can be affected by network interruptions. Consider these patterns for production applications:
+
+- **Reconnection**: If the session disconnects, you'll need to create a new `barrage_session` and resubscribe. The remote table must still be published to the same shared ticket.
+
+- **Ticket lifetime**: Shared tickets remain valid as long as the publishing session is active. If the publishing session closes, the ticket becomes invalid and subscribers will lose their connection.
+
+- **Authentication expiry**: If using authenticated connections, ensure tokens or credentials remain valid for the duration of long-running subscriptions.
+
+### Memory considerations for large tables
+
+When subscribing to large ticking tables:
+
+- **Initial snapshot size**: The first update contains a complete snapshot of the table. For very large tables, this can consume significant memory. The server breaks large snapshots into chunks by default (see [snapshot size control](./performance/barrage-performance.md#control-subscription-snapshot-size)).
+
+- **Incremental updates**: After the initial snapshot, only changed rows are transmitted. This is typically much smaller than the full table.
+
+- **Server-side filtering**: If you only need a subset of the data, consider filtering the table on the remote server before subscribing. This reduces both network and memory usage. (Note: this is distinct from viewports, which define a scrollable window over row positions.)
+
+```python skip-test
+# On the remote server: filter before publishing
+filtered_ref = client_session.open_table("large_table").where("Region = `EAST`")
+client_session.publish_table(ticket, filtered_ref)
+
+# The subscriber now receives only the filtered data
+```
+
 ## Related documentation
 
 <!--- TODO: link https://github.com/deephaven/deephaven.io/issues/3918 when complete.-->

docs/groovy/how-to-guides/data-import-export/barrage-schema.md

@@ -1,6 +1,6 @@
 ---
-title: Barrage Schema Annotation
-sidebar_label: Barrage Schema Annotation
+title: Barrage schema annotation
+sidebar_label: Barrage schema annotation
 ---
 
 Deephaven tables support Object-typed columns that can hold arbitrary Java objects. When exporting these tables over Flight using the Barrage format, Deephaven uses Apache Arrow schemas to describe the data. By default, if a column is typed as `Object`, the Arrow schema may not capture the intended structure of the data, which can lead to inefficient serialization or loss of type information. Use the `Table.BARRAGE_SCHEMA_ATTRIBUTE` to inject explicit Arrow schema information, which ensures that the Flight export uses the correct wire format.
@@ -82,7 +82,7 @@ table_w_attributes = table.withAttributes(java.util.Map.of(Table.BARRAGE_SCHEMA_
 
 ## Example: Annotate `Map<String, String>` Columns
 
-The following example creates a table with a column of `Map<String, Double>`. The Arrow schema annotates the column as an Arrow `Map` with the correct types for key and values. The final table can be exported over Flight / Barrage without error.
+The following example creates a table with a column of `Map<String, String>`. The Arrow schema annotates the column as an Arrow `Map` with the correct types for key and values. The final table can be exported over Flight / Barrage without error.
 
 ```groovy order=table,table_w_attributes
 // Table creation
@@ -369,3 +369,9 @@ def new_schema = new Schema(fields)
 // Apply attributes, creating a new table reference which can be used for export; the original table is unchanged
 table_w_attributes = table.withAttributes(java.util.Map.of(Table.BARRAGE_SCHEMA_ATTRIBUTE, new_schema))
 ```
+
+## Related documentation
+
+- [What is Barrage?](../../conceptual/what-is-barrage.md)
+- [withAttributes](../../reference/table-operations/select/withAttributes.md)
+- [Arrow Flight integration](./arrow-flight.md)

docs/groovy/how-to-guides/performance/barrage-performance.md

@@ -82,6 +82,93 @@ Here are server-side flags that change the behavior of Barrage metrics.
 - `-DBarragePerformanceLog.enableAll`: record metrics for tables that do not have an explicit `TableKey` (default: `true`).
 - `-DBarragePerformanceLog.cycleDurationMillis`: the interval to flush aggregated statistics (default: `60000` - once per minute).
 
+## Control subscription snapshot size
+
+When a client subscribes to a ticking table, the server sends an initial snapshot of the table data. For large tables, constructing this snapshot requires holding the data in memory on the server side, which can lead to out-of-memory (OOM) errors.
+
+To address this, Deephaven can break the initial snapshot into smaller chunks spread across multiple update cycles. This behavior is controlled by the following properties:
+
+- `-DBarrageMessageProducer.subscriptionGrowthEnabled`: When `true` (the default), the server limits the size of each snapshot chunk. When `false`, the server sends the entire snapshot at once (unlimited size).
+
+When subscription growth is enabled, these additional properties control the chunk size:
+
+- `-DBarrageUtil.minSnapshotCellCount`: The minimum number of cells (rows × columns) per snapshot chunk. Default: `8192`.
+- `-DBarrageUtil.maxSnapshotCellCount`: The maximum number of cells per snapshot chunk. Default: `16777216` (approximately 16 million).
+
+The server adaptively adjusts the chunk size between these bounds based on how long each snapshot takes to generate, targeting a percentage of the update graph cycle time.
+
+### Reducing publisher memory usage
+
+For systems serving very large tables to many subscribers, you can reduce publisher-side memory usage by lowering `maxSnapshotCellCount`. Setting `minSnapshotCellCount` equal to `maxSnapshotCellCount` fixes the chunk size and disables adaptive sizing:
+
+```bash
+-DBarrageMessageProducer.subscriptionGrowthEnabled=true
+-DBarrageUtil.minSnapshotCellCount=1000000
+-DBarrageUtil.maxSnapshotCellCount=1000000
+```
+
+This configuration limits each snapshot chunk to exactly 1 million cells — well below the 16 million default maximum. Subscribers receive the full table data incrementally over multiple update cycles rather than all at once.
+
+> [!NOTE]
+> Setting smaller snapshot sizes increases the time required for subscribers to receive the initial table state but reduces peak memory usage on the server. These settings only affect initial snapshots — incremental updates are unaffected and must still be maintained in memory.
+
+## Additional Barrage configuration
+
+The following properties control other aspects of Barrage behavior:
+
+### Update interval
+
+- `-Dbarrage.minUpdateInterval`: The minimum interval (in milliseconds) between update batches sent to subscribers. Default: `1000` (1 second). Lower values reduce latency but increase CPU and network usage.
+
+### Message batching
+
+- `-DBarrageMessageWriterImpl.batchSize`: Maximum rows per Arrow record batch. Default: `Integer.MAX_VALUE`. Reduce this if clients have trouble processing very large batches.
+- `-DBarrageMessageWriterImpl.initialBatchSize`: Initial batch size for the first message. Default: `4096`. A smaller initial batch ensures clients receive data quickly while the server calibrates optimal batch sizes.
+- `-DBarrageMessageWriterImpl.maxOutboundMessageSize`: Maximum size (in bytes) for outbound messages. Default: `104857600` (100 MB). This matches the default incoming message limit for Java clients.
+
+## Troubleshooting
+
+Use the metrics tables described above to diagnose common Barrage issues.
+
+### High `SnapshotMillis`
+
+If `SnapshotMillis` is consistently high:
+
+- The source table may be very large. Consider using viewports or filtering data before subscription.
+- The update graph may be holding a lock. Check for long-running operations blocking the cycle.
+- Consider enabling subscription growth with smaller chunk sizes (see [Control subscription snapshot size](#control-subscription-snapshot-size)).
+
+### High `WriteMillis` or `WriteMegabits`
+
+If `WriteMillis` is high or `WriteMegabits` is large:
+
+- Network bandwidth may be saturated. Check network utilization.
+- Consider subscribing to fewer columns or using viewports to reduce data volume.
+- Increase `barrage.minUpdateInterval` to batch more updates together.
+
+### High `PropagateMillis`
+
+If `PropagateMillis` is consistently high:
+
+- Many subscribers may be connected to the same table. Consider load balancing across multiple server instances.
+- The server may be under memory pressure. Check JVM heap usage and garbage collection metrics.
+
+### Subscription errors
+
+Common subscription issues:
+
+- **"Ticket not found"**: The table was released or the session that published it closed. Ensure the publishing session remains active.
+- **Authentication failures**: Verify that the `auth_type` and `auth_token` match the server configuration.
+- **Connection refused**: Ensure the server is running and the host/port are correct. Check firewall rules.
+
+### Memory issues
+
+If the server experiences out-of-memory errors during subscriptions:
+
+- Enable subscription growth: `-DBarrageMessageProducer.subscriptionGrowthEnabled=true`
+- Lower snapshot cell counts to reduce peak memory usage.
+- Monitor the metrics tables to identify which tables consume the most resources.
+
 ## Related documentation
 
 - [Interpret Barrage metrics](../../conceptual/barrage-metrics.md)

docs/groovy/sidebar.json

@@ -132,6 +132,10 @@
               {
                 "label": "Servers & clients",
                 "path": "conceptual/client-server-model.md"
+              },
+              {
+                "label": "What is Barrage?",
+                "path": "conceptual/what-is-barrage.md"
               }
             ]
           },