Paginate Data Overview for wide datasets

[Berserker-GM]

Related Issues / Pull Requests

Closes #139

Description

Data Overview currently computes and renders summary statistics and a distribution chart for every feature, which is slow and heavy on wide datasets. This PR paginates both the summary table and Feature Distributions, and computes only the requested page on the server (it does not compute everything and hide the rest).

Behavior

• /summary-statistics accepts optional page and page_size (default 1 and 25, max 100). Statistics and histograms are computed only for that page's features.
• The UI shows a pager with a label like "Page 1 of 8 · Features 1-25 of 180", above the table and under the histograms. It is hidden when there is only one page.
• Dataset-wide totals (records, features, numerical/categorical counts and lists) always come from the full dataset and do not change between pages.
• Out-of-range pages return an empty page with the totals intact. Invalid page/page_size values are rejected.
• The Globus/remote path forwards page/page_size, caches results per page, and uses the same shared helper (summary_pagination.py).
• The intent profile now reads the page-1 cache and no longer under-counts nulls.

Before / after (local measurements with the Flask test client; approximate, not a benchmark)

• 200 columns x 500 rows: about 17.8 s and 5.07 MB / 180 histograms before; about 2.7 s and 0.71 MB / 25 histograms for page 1 after.
• 2,500 columns x 2,000 rows: did not finish within 90 s before; about 2.3 s and 0.76 MB for page 1 after, with correct totals.

Notes for reviewers

• Globus: the remote path requires a worker that reports the summary_pagination_v1 capability. An older worker gets a 409 "upgrade" response instead of falling back to computing everything. If you would prefer a (slow) fallback, I'm happy to change it.
• Globus was not tested against a live endpoint (no credentials). It is covered by integration tests that mock the remote path.
• Known edge case: for a dataset with only datetime columns, features_count is 1 while pagination.total_features is 0. The pager is hidden there, so nothing visible is wrong, and I left the existing meaning of features_count unchanged.
• The new optional query parameters on /summary-statistics are not documented in the Sphinx docs. Let me know if you want that added in this PR.
• Not included, per the issue's optional follow-ups: a page-size selector, jump to a feature by name, and sorting.

How to test

1. PYTHONPATH=. pytest tests/integration/test_summary_statistics.py tests/integration/test_globus.py tests/unit/test_summary_pagination.py -v
2. Run the app (flask --app 'web:create_app()' run), upload a CSV with 40 or more columns, and page through. Totals stay fixed while the features change.
3. Upload a small file (25 features or fewer): no pager appears.

Screenshots

What changes are proposed in this pull request?

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality not to work as expected; for instance, examples in this repository must be updated too)
• This change requires a documentation update

Checklist:

• My code modifies existing public API, or introduces new public API, and I updated or wrote documentation
• I have commented my code
• My code requires documentation updates, and I have