Skip to content
Merged
Show file tree
Hide file tree
Changes from 5 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -32,3 +32,5 @@ scripts/download/data/*
# Version
_version.py

# agent research and plans
.agents/
28 changes: 3 additions & 25 deletions README.rst
Original file line number Diff line number Diff line change
Expand Up @@ -24,8 +24,6 @@ simply a client for accessing the database using python

.. _SnowEx database: https://www.github.com/SnowEx/snowex_db

WARNING - This is under active development in preparation for SnowEx Hackweek. Use at your own risk. Data will change as it is QA/QC'd and the end goal is for all data in this database to be pulled from NSIDC. The goal is for this to become a community database open to all.


Features
--------
Expand All @@ -35,7 +33,7 @@ Features
* Useful conversions to pandas and geopandas
* Lots of examples_

.. _examples: https://snowexsql.readthedocs.io/en/latest/gallery/index.html
.. _examples: https://projectpythia.org/snow-observations-cookbook/


Setup
Expand Down Expand Up @@ -156,31 +154,11 @@ Getting help
Jump over to `our discussion forum <https://github.com/SnowEx/snowexsql/discussions>`_
and get help from our community.


Documentation
-------------

Comment thread
aaarendt marked this conversation as resolved.
There is a whole host of resources for users in the documentation. It has been
setup for you to preview in your browser.

In there you will find:

* Examples of database use
* Database structure
* API to the python package snowexsql
* Links to other resources
* Notes about the data uploaded
* And more!

To see the documentation in your browser:

**Warning**: To see the examples/gallery, the snowex db needs to be up. Otherwise they will be left with the
last image submitted to GitHub.

.. code-block:: bash

make docs

Our read the docs pages include `documentation of the API structure <https://snowexsql.readthedocs.io/en/latest/api.html>`_
and `provides a detailed description of the database schema <https://snowexsql.readthedocs.io/en/latest/database_structure.html>`_.

I want to contribute
---------------------
Expand Down
1 change: 1 addition & 0 deletions docs/_config.yml
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,7 @@ sphinx:
- 'sphinx.ext.viewcode'
- 'sphinx_gallery.load_style'
- 'sphinx.ext.autosectionlabel'
- 'sphinxcontrib.mermaid'
config:
html_js_files:
- https://cdnjs.cloudflare.com/ajax/libs/require.js/2.3.4/require.min.js
18 changes: 0 additions & 18 deletions docs/_toc.yml
Original file line number Diff line number Diff line change
Expand Up @@ -8,23 +8,5 @@ chapters:
- file: community/index
- file: database_structure
- file: data_notes
- file: gallery/index
sections:
- file: gallery/what_is_in_the_db_example
- file: gallery/raster_union_and_more_example
- file: gallery/plot_raster_example
- file: gallery/plot_pit_swe_example
- file: gallery/overview_example
- file: gallery/graupel_smp_example
- file: gallery/graupel_pits_example
- file: gallery/getting_started_example
- file: gallery/get_spiral_example
- file: gallery/compare_UAVSAR_to_depths_example
- file: gallery/compare_SSA_instruments_example
- file: gallery/camera_derive_snow_depths_example
- file: gallery/api_intro_example
- file: gallery/api_plot_pit_density_example
- file: cheat_sheet
- file: qgis
- file: api
- file: history
139 changes: 52 additions & 87 deletions docs/api.rst
Original file line number Diff line number Diff line change
@@ -1,111 +1,76 @@
API Documentation
=================
.. role:: python(code)
:language: python
API Reference
=============

Background
----------
The API (not a rest API, more of an SDK) is a set of python classes
designed for easy and standardized access to the database data.
The SnowEx Python API provides two access paths to the database:

The classes can both describe what data is available, and return
data in a GeoPandas dataframe.
- **Direct database access** (:mod:`snowexsql.api`) — for users with
database credentials. Queries run locally against the PostgreSQL/PostGIS
database using SQLAlchemy.

Components
----------
There are two main API classes for data access.
- **Lambda client** (:mod:`snowexsql.lambda_client`) — for users without
credentials. Queries are sent to a public AWS Lambda Function URL that
proxies the request to the database server-side. No AWS account or
database credentials are required.

.. code-block:: python
Both paths expose the same high-level interface: ``from_filter()``,
``from_area()``, ``from_unique_entries()``, and ``all_*`` properties.

from snowexsql.api import PointMeasurements, LayerMeasurements
.. note::

:code:`PointMeasurements` gives access to the PointData (depths, GPR, etc), and
:code:`LayerMeasurements` gives access to the LayerData (pits, etc).
By default, queries are capped at 1000 records. If your query would
return more, a :class:`~snowexsql.api.LargeQueryCheckException` is raised
unless you explicitly pass ``limit=<n>`` with a value larger than 1000.

Both of the classes have the same methods, although they access different
tables in the database.

The primary methods for accessing data are :code:`.from_area` and
:code:`.from_filter`. Both of these methods return a GeoPandas dataframe.
Direct Database API
-------------------

.from_filter
------------
.. currentmodule:: snowexsql.api

The :code:`.from_filter` is the simpler of the two search methods. It takes in
a variety of key word args (kwargs) and returns a dataset that meets
all of the criteria.
BaseDataset
~~~~~~~~~~~

.. code-block:: python
.. autoclass:: BaseDataset
:members:
:undoc-members:
:show-inheritance:

df = LayerMeasurements.from_filter(
type="density",
site_name="Boise River Basin",
limit=1000
)
PointMeasurements
~~~~~~~~~~~~~~~~~

In this example, we filter to all the layer measurements of `density`
that were taken in the `Boise River Basin`, and we `limit` to the top
1000 measurements.
.. autoclass:: PointMeasurements
:members:
:undoc-members:
:show-inheritance:

Each kwarg (except date) **can take in a list or a single value** so you could change
this to :code:`site_name=["Boise River Basin", "Grand Mesa"]`
LayerMeasurements
~~~~~~~~~~~~~~~~~

To find what `kwargs` are allowed, we can check the class
.. autoclass:: LayerMeasurements
:members:
:undoc-members:
:show-inheritance:

.. code-block:: python
Exceptions
~~~~~~~~~~

LayerMeasurements.ALLOWED_QRY_KWARGS
.. autoexception:: LargeQueryCheckException

For :code:`LayerMeasurements` this will return
:code:`["site_name", "site_id", "date", "instrument", "observers", "type", "utm_zone", "pit_id", "date_greater_equal", "date_less_equal"]`

so we can filter by any of these as inputs to the function.
Lambda Client
-------------

**Notice `limit` is not specified here**. Limit is in the :code:`SPECIAL_KWARGS`
and gets handled at the end of the query.
.. currentmodule:: snowexsql.lambda_client

**Notice `date_greater_equal` and `date_less_equal`** for filtering the `date`
parameter using `>=` and `<=` logic.
SnowExLambdaClient
~~~~~~~~~~~~~~~~~~

To find what values are allowed for each, we can check the propeties of the
class. Both :code:`LayerMeasurements` and :code:`PointMeasurements` have
the following properties.
.. autoclass:: SnowExLambdaClient
:members:
:undoc-members:
:show-inheritance:

* all_site_names
* all_types
* all_dates
* all_observers
* all_instruments
create_client
~~~~~~~~~~~~~

So you can find all the instruments for filtering like :code:`LayerMeasurements().all_instruments`.
**Note** - these must be called from an instantiated class like shown earlier
in this line.

.from_area
----------

The signature for :code:`.from_area` looks like this

.. code-block:: python

def from_area(cls, shp=None, pt=None, buffer=None, crs=26912, **kwargs):

It is a class method, so it *does not need an instantiated class*.
The :code:`**kwargs` argument takes the same inputs as the :code:`from_filter`
function.

The big difference is that from area will filter to results either within
:code:`shp` (a `shapely` polygon) **or** within :code:`buffer` radius
around :code:`pt` (a `shapely` point).


Large Query Exception and Limit
-------------------------------

By default, if more than 1000 records will be returned, and **no limit**
is provided. The query will fail. This is intentional so that we are aware
of large queries. If you understand your query will be large and need
more than 1000 records returned, add a :code:`limit` kwarg to your query
with a value greater than the number you need returned.
**This will override the default behavior** and return as many records as
you requested.
.. autofunction:: create_client
Loading
Loading