Integrate neural hydrology to showcase AI integration (inference and execution as of now)

[aashish24]

Google Neural Hydrology is an open source project for modeling watershed behavior (rain-runoff) using neural networks.

[johnkit]

The following is a summary of a more detailed outline at this google slides link.

Basic Structure

Google provides an open source library with code to produce an LSTM network modeling watershed discharge based on the history of meteorology inputs.

The LSTM structure is used because it has internal storage (hidden cells) to model temporal sequences of input data. Each input to the network is a sequence of 365 daily records.

The network structure is very small: 2 layer, 7 input cells, 80 hidden cells. Training is very light: 50 epochs

Published work to date is focused on accurately modeling historical records.

Data Source

The library uses CAMEL_US dataset which has daily aggregate records for 671 watersheds in US (8 digit HUC level) for the period 1980-2014

Possible DIVERS-H Use Case

For DIVERS-H, we should be able to:

• Build LSTM network to model 8-digit HUC that contains Fort Loudoun Dam (or some other site)
• Extract future meteorology predictions from CMIP6/ESM dataset(s)
• Use LSTM network to generate discharge predictions from the meteorology predictions
• Compare predicted results with present-day observations.

References

Item	Link
Repo	https://github.com/neuralhydrology/neuralhydrology
Docs	https://neuralhydrology.readthedocs.io
Paper	https://hess.copernicus.org/articles/22/6005/2018/hess-22-6005-2018.pdf

[johnkit]

Fyi according to Google, Fort Loudoun Dam is in HUC 06010201. This HUC is not in the CAMELS-US dataset. In fact, no 0601 HUCs are in CAMELS-US.

The CAMELS-US docs indicate it has data from 671 8-digit HUCs. Good tells me there are ~2400 in the US.

So we don't have data for the watershed containing Fort Loudoun Dam. We'll need to pick/find some other location.

[johnkit]

Outlined development plan in Google Docs: https://docs.google.com/document/d/1JMvs_wZUDlsDrM8wyiXao7ayKzFMyDciYzpgkKRrxAA/edit?usp=sharing

[johnkit]

Initial milestone: standalone docker prototype

• New software (prototype) to train/test model for an input basin (id).
• Scripts to build container image encapsulating neuralhydrology package
• New software to run container and do train/test
• Make container image with neural hydrology plus CAMEL-US data
• Add github workflow to build docker image
• Make installer package
• Add basic plotting script for results

[johnkit]

A note about container image size

1. Build from python:3.12-slim => 9.45 GB

The python base image is just 124 MB. Installing the neuralhyrdology pulled in pytorch and cuda packages that produced the large image.

2. Build from pytorch/pytorch => 6.58 GB

On the advice of Jason Parnham, I switched to a pytorch base image, specifically pytorch/pytorch:2.5.1-cuda12.1-cudnn9-runtime which is 5.96 GB. Adding neuralhdrology increased the size to 6.58 GB

I used this image with an external (mounted) CAMELS-US dataset to generate training and test runs for single basins.

3. Add CAMELS-US dataset => 20+ GB

That's not a typo. Adding the CAMELS-US dataset increases the size by 14 GB.

I didn't verify actually verify this. Using brute force, I created an image over 30GB in size. I'd need to make a multi-stage build in order to take out the original zip file downloads.

For our Phase I demonstration, I recommend we stick with the approach 2 and mount CAMELS-US externally.