Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 5 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,15 +22,15 @@ There are three ways to interact with DynaDojo and users can take on multiple "h

![toychallenges-chips](https://github.com/FlyingWorkshop/dynadojo/assets/56043296/dcae54df-ef98-48d4-ad2e-e37c4054e67b)

`DynaDojo` comes with three off-the-shelf challenges: [Fixed Error](demos/fixed_error_demo.ipynb), [Fixed Complexity](demos/fixed_complexity_demo.ipynb), and [Fixed Train Size](demos/fixed_train_size_demo.ipynb).
`DynaDojo` comes with three off-the-shelf challenges: [Fixed Error](demos/fixed_error_demo.ipynb), [Fixed Dimensionality](demos/fixed_dimensionality_demo.ipynb), and [Fixed Train Size](demos/fixed_train_size_demo.ipynb).

```python
from dynadojo.systems import LDSSystem
from dynadojo.baselines import LinearRegression, DNN
from dynadojo.challenges import FixedComplexity
from dynadojo.challenges import FixedDimensionality
import pandas as pd

challenge = FixedComplexity(
challenge = FixedDimensionality(
N=[10, 100, 1000], # number of training examples
l=3, # latent dimension
e=3, # embed dimension
Expand All @@ -48,11 +48,11 @@ data3 = challenge.evaluate(DNN, fit_kwargs={"epochs": 20}, id="linear network")
data = pd.concat((data1, data2, data3))
challenge.plot(data)
```
<b>Out (Fixed Complexity, C = latent_dim = 5):</b>
<b>Out (Fixed Dimensionality, C = latent_dim = 5):</b>
Note how for this LDSSystem, a linear network learns more from each added sample (larger decreases in error) than a nonlinear network, and how linear regression immediately saturates at very low error. These dynamics help contextualize model performance and comparision.

<p align="center">
<img src="https://github.com/FlyingWorkshop/dynadojo/blob/main/graphics/fixed_complexity.png" alt="DynaDojo FixedComplexity Challenge Graph" />
<img src="https://github.com/FlyingWorkshop/dynadojo/blob/main/graphics/fixed_dimensionality.png" alt="DynaDojo FixedDimensionality Challenge Graph" />
</p>


Expand Down
6 changes: 3 additions & 3 deletions demos/algorithms/dnn_vs_gru.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,7 @@
"from dynadojo.baselines.lr import LinearRegression\n",
"from dynadojo.baselines.dnn import DNN\n",
"from dynadojo.baselines.gru_rnn import GRU_RNN\n",
"from dynadojo.challenges import FixedComplexity\n",
"from dynadojo.challenges import FixedDimensionality\n",
"import pandas as pd"
]
},
Expand Down Expand Up @@ -75,7 +75,7 @@
}
],
"source": [
"challenge = FixedComplexity(\n",
"challenge = FixedDimensionality(\n",
" N=[10, 50, 100, 500, 1000],\n",
" l=5,\n",
" e=5,\n",
Expand Down Expand Up @@ -313,7 +313,7 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.10.14"
"version": "3.10.11"
}
},
"nbformat": 4,
Expand Down
38 changes: 22 additions & 16 deletions demos/fc_cnn_ca_demo.ipynb → demos/fd_cnn_ca_demo.ipynb

Large diffs are not rendered by default.

File renamed without changes.
34 changes: 21 additions & 13 deletions demos/fts_dmd_lds_demo.ipynb

Large diffs are not rendered by default.

Original file line number Diff line number Diff line change
Expand Up @@ -212,7 +212,7 @@
</ul>
</li>
<li class="toctree-l3 has-children"><a class="reference internal" href="dynadojo.challenges.html">dynadojo.challenges</a><input class="toctree-checkbox" id="toctree-checkbox-13" name="toctree-checkbox-13" role="switch" type="checkbox"/><label for="toctree-checkbox-13"><div class="visually-hidden">Toggle navigation of dynadojo.challenges</div><i class="icon"><svg><use href="#svg-arrow-right"></use></svg></i></label><ul>
<li class="toctree-l4"><a class="reference internal" href="dynadojo.challenges.FixedComplexity.html">dynadojo.challenges.FixedComplexity</a></li>
<li class="toctree-l4"><a class="reference internal" href="dynadojo.challenges.FixedDimensionality.html">dynadojo.challenges.FixedDimensionality</a></li>
<li class="toctree-l4"><a class="reference internal" href="dynadojo.challenges.FixedError.html">dynadojo.challenges.FixedError</a></li>
<li class="toctree-l4"><a class="reference internal" href="dynadojo.challenges.FixedTrainSize.html">dynadojo.challenges.FixedTrainSize</a></li>
<li class="toctree-l4"><a class="reference internal" href="dynadojo.challenges.ScalingChallenge.html">dynadojo.challenges.ScalingChallenge</a></li>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -212,7 +212,7 @@
</ul>
</li>
<li class="toctree-l3 has-children"><a class="reference internal" href="dynadojo.challenges.html">dynadojo.challenges</a><input class="toctree-checkbox" id="toctree-checkbox-13" name="toctree-checkbox-13" role="switch" type="checkbox"/><label for="toctree-checkbox-13"><div class="visually-hidden">Toggle navigation of dynadojo.challenges</div><i class="icon"><svg><use href="#svg-arrow-right"></use></svg></i></label><ul>
<li class="toctree-l4"><a class="reference internal" href="dynadojo.challenges.FixedComplexity.html">dynadojo.challenges.FixedComplexity</a></li>
<li class="toctree-l4"><a class="reference internal" href="dynadojo.challenges.FixedDimensionality.html">dynadojo.challenges.FixedDimensionality</a></li>
<li class="toctree-l4"><a class="reference internal" href="dynadojo.challenges.FixedError.html">dynadojo.challenges.FixedError</a></li>
<li class="toctree-l4"><a class="reference internal" href="dynadojo.challenges.FixedTrainSize.html">dynadojo.challenges.FixedTrainSize</a></li>
<li class="toctree-l4"><a class="reference internal" href="dynadojo.challenges.ScalingChallenge.html">dynadojo.challenges.ScalingChallenge</a></li>
Expand Down Expand Up @@ -576,7 +576,7 @@ <h1>dynadojo.abstractions.AbstractChallenge<a class="headerlink" href="#dynadojo
<p>This method should return a pandas DataFrame where each row is the result of an algorithm trained and evaluated on a single system for a given training set size.</p>
<div class="admonition note">
<p class="admonition-title">Note</p>
<p>A job is very abstractly defined, open an issue and tag &#64;carynbear or &#64;mkanwal for help. You can think of it as a trial for a single unit of computation that you wish to distribute. Sometimes we can do a single trial of multiple runs (see FixedComplexity) in special cases where the system is the same for different training set sizes.</p>
<p>A job is very abstractly defined, open an issue and tag &#64;carynbear or &#64;mkanwal for help. You can think of it as a trial for a single unit of computation that you wish to distribute. Sometimes we can do a single trial of multiple runs (see FixedDimensionality) in special cases where the system is the same for different training set sizes.</p>
</div>
<dl class="field-list simple">
<dt class="field-odd">Parameters<span class="colon">:</span></dt>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -212,7 +212,7 @@
</ul>
</li>
<li class="toctree-l3 has-children"><a class="reference internal" href="dynadojo.challenges.html">dynadojo.challenges</a><input class="toctree-checkbox" id="toctree-checkbox-13" name="toctree-checkbox-13" role="switch" type="checkbox"/><label for="toctree-checkbox-13"><div class="visually-hidden">Toggle navigation of dynadojo.challenges</div><i class="icon"><svg><use href="#svg-arrow-right"></use></svg></i></label><ul>
<li class="toctree-l4"><a class="reference internal" href="dynadojo.challenges.FixedComplexity.html">dynadojo.challenges.FixedComplexity</a></li>
<li class="toctree-l4"><a class="reference internal" href="dynadojo.challenges.FixedDimensionality.html">dynadojo.challenges.FixedDimensionality</a></li>
<li class="toctree-l4"><a class="reference internal" href="dynadojo.challenges.FixedError.html">dynadojo.challenges.FixedError</a></li>
<li class="toctree-l4"><a class="reference internal" href="dynadojo.challenges.FixedTrainSize.html">dynadojo.challenges.FixedTrainSize</a></li>
<li class="toctree-l4"><a class="reference internal" href="dynadojo.challenges.ScalingChallenge.html">dynadojo.challenges.ScalingChallenge</a></li>
Expand Down
2 changes: 1 addition & 1 deletion docs/build/html/_autosummary/dynadojo.abstractions.html
Original file line number Diff line number Diff line change
Expand Up @@ -212,7 +212,7 @@
</ul>
</li>
<li class="toctree-l3 has-children"><a class="reference internal" href="dynadojo.challenges.html">dynadojo.challenges</a><input class="toctree-checkbox" id="toctree-checkbox-13" name="toctree-checkbox-13" role="switch" type="checkbox"/><label for="toctree-checkbox-13"><div class="visually-hidden">Toggle navigation of dynadojo.challenges</div><i class="icon"><svg><use href="#svg-arrow-right"></use></svg></i></label><ul>
<li class="toctree-l4"><a class="reference internal" href="dynadojo.challenges.FixedComplexity.html">dynadojo.challenges.FixedComplexity</a></li>
<li class="toctree-l4"><a class="reference internal" href="dynadojo.challenges.FixedDimensionality.html">dynadojo.challenges.FixedDimensionality</a></li>
<li class="toctree-l4"><a class="reference internal" href="dynadojo.challenges.FixedError.html">dynadojo.challenges.FixedError</a></li>
<li class="toctree-l4"><a class="reference internal" href="dynadojo.challenges.FixedTrainSize.html">dynadojo.challenges.FixedTrainSize</a></li>
<li class="toctree-l4"><a class="reference internal" href="dynadojo.challenges.ScalingChallenge.html">dynadojo.challenges.ScalingChallenge</a></li>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -212,7 +212,7 @@
</ul>
</li>
<li class="toctree-l3 has-children"><a class="reference internal" href="dynadojo.challenges.html">dynadojo.challenges</a><input class="toctree-checkbox" id="toctree-checkbox-13" name="toctree-checkbox-13" role="switch" type="checkbox"/><label for="toctree-checkbox-13"><div class="visually-hidden">Toggle navigation of dynadojo.challenges</div><i class="icon"><svg><use href="#svg-arrow-right"></use></svg></i></label><ul>
<li class="toctree-l4"><a class="reference internal" href="dynadojo.challenges.FixedComplexity.html">dynadojo.challenges.FixedComplexity</a></li>
<li class="toctree-l4"><a class="reference internal" href="dynadojo.challenges.FixedDimensionality.html">dynadojo.challenges.FixedDimensionality</a></li>
<li class="toctree-l4"><a class="reference internal" href="dynadojo.challenges.FixedError.html">dynadojo.challenges.FixedError</a></li>
<li class="toctree-l4"><a class="reference internal" href="dynadojo.challenges.FixedTrainSize.html">dynadojo.challenges.FixedTrainSize</a></li>
<li class="toctree-l4"><a class="reference internal" href="dynadojo.challenges.ScalingChallenge.html">dynadojo.challenges.ScalingChallenge</a></li>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -212,7 +212,7 @@
</ul>
</li>
<li class="toctree-l3 has-children"><a class="reference internal" href="dynadojo.challenges.html">dynadojo.challenges</a><input class="toctree-checkbox" id="toctree-checkbox-13" name="toctree-checkbox-13" role="switch" type="checkbox"/><label for="toctree-checkbox-13"><div class="visually-hidden">Toggle navigation of dynadojo.challenges</div><i class="icon"><svg><use href="#svg-arrow-right"></use></svg></i></label><ul>
<li class="toctree-l4"><a class="reference internal" href="dynadojo.challenges.FixedComplexity.html">dynadojo.challenges.FixedComplexity</a></li>
<li class="toctree-l4"><a class="reference internal" href="dynadojo.challenges.FixedDimensionality.html">dynadojo.challenges.FixedDimensionality</a></li>
<li class="toctree-l4"><a class="reference internal" href="dynadojo.challenges.FixedError.html">dynadojo.challenges.FixedError</a></li>
<li class="toctree-l4"><a class="reference internal" href="dynadojo.challenges.FixedTrainSize.html">dynadojo.challenges.FixedTrainSize</a></li>
<li class="toctree-l4"><a class="reference internal" href="dynadojo.challenges.ScalingChallenge.html">dynadojo.challenges.ScalingChallenge</a></li>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -212,7 +212,7 @@
</ul>
</li>
<li class="toctree-l3 has-children"><a class="reference internal" href="dynadojo.challenges.html">dynadojo.challenges</a><input class="toctree-checkbox" id="toctree-checkbox-13" name="toctree-checkbox-13" role="switch" type="checkbox"/><label for="toctree-checkbox-13"><div class="visually-hidden">Toggle navigation of dynadojo.challenges</div><i class="icon"><svg><use href="#svg-arrow-right"></use></svg></i></label><ul>
<li class="toctree-l4"><a class="reference internal" href="dynadojo.challenges.FixedComplexity.html">dynadojo.challenges.FixedComplexity</a></li>
<li class="toctree-l4"><a class="reference internal" href="dynadojo.challenges.FixedDimensionality.html">dynadojo.challenges.FixedDimensionality</a></li>
<li class="toctree-l4"><a class="reference internal" href="dynadojo.challenges.FixedError.html">dynadojo.challenges.FixedError</a></li>
<li class="toctree-l4"><a class="reference internal" href="dynadojo.challenges.FixedTrainSize.html">dynadojo.challenges.FixedTrainSize</a></li>
<li class="toctree-l4"><a class="reference internal" href="dynadojo.challenges.ScalingChallenge.html">dynadojo.challenges.ScalingChallenge</a></li>
Expand Down
2 changes: 1 addition & 1 deletion docs/build/html/_autosummary/dynadojo.baselines.cnn.html
Original file line number Diff line number Diff line change
Expand Up @@ -212,7 +212,7 @@
</ul>
</li>
<li class="toctree-l3 has-children"><a class="reference internal" href="dynadojo.challenges.html">dynadojo.challenges</a><input class="toctree-checkbox" id="toctree-checkbox-13" name="toctree-checkbox-13" role="switch" type="checkbox"/><label for="toctree-checkbox-13"><div class="visually-hidden">Toggle navigation of dynadojo.challenges</div><i class="icon"><svg><use href="#svg-arrow-right"></use></svg></i></label><ul>
<li class="toctree-l4"><a class="reference internal" href="dynadojo.challenges.FixedComplexity.html">dynadojo.challenges.FixedComplexity</a></li>
<li class="toctree-l4"><a class="reference internal" href="dynadojo.challenges.FixedDimensionality.html">dynadojo.challenges.FixedDimensionality</a></li>
<li class="toctree-l4"><a class="reference internal" href="dynadojo.challenges.FixedError.html">dynadojo.challenges.FixedError</a></li>
<li class="toctree-l4"><a class="reference internal" href="dynadojo.challenges.FixedTrainSize.html">dynadojo.challenges.FixedTrainSize</a></li>
<li class="toctree-l4"><a class="reference internal" href="dynadojo.challenges.ScalingChallenge.html">dynadojo.challenges.ScalingChallenge</a></li>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -212,7 +212,7 @@
</ul>
</li>
<li class="toctree-l3 has-children"><a class="reference internal" href="dynadojo.challenges.html">dynadojo.challenges</a><input class="toctree-checkbox" id="toctree-checkbox-13" name="toctree-checkbox-13" role="switch" type="checkbox"/><label for="toctree-checkbox-13"><div class="visually-hidden">Toggle navigation of dynadojo.challenges</div><i class="icon"><svg><use href="#svg-arrow-right"></use></svg></i></label><ul>
<li class="toctree-l4"><a class="reference internal" href="dynadojo.challenges.FixedComplexity.html">dynadojo.challenges.FixedComplexity</a></li>
<li class="toctree-l4"><a class="reference internal" href="dynadojo.challenges.FixedDimensionality.html">dynadojo.challenges.FixedDimensionality</a></li>
<li class="toctree-l4"><a class="reference internal" href="dynadojo.challenges.FixedError.html">dynadojo.challenges.FixedError</a></li>
<li class="toctree-l4"><a class="reference internal" href="dynadojo.challenges.FixedTrainSize.html">dynadojo.challenges.FixedTrainSize</a></li>
<li class="toctree-l4"><a class="reference internal" href="dynadojo.challenges.ScalingChallenge.html">dynadojo.challenges.ScalingChallenge</a></li>
Expand Down
2 changes: 1 addition & 1 deletion docs/build/html/_autosummary/dynadojo.baselines.dmd.html
Original file line number Diff line number Diff line change
Expand Up @@ -212,7 +212,7 @@
</ul>
</li>
<li class="toctree-l3 has-children"><a class="reference internal" href="dynadojo.challenges.html">dynadojo.challenges</a><input class="toctree-checkbox" id="toctree-checkbox-13" name="toctree-checkbox-13" role="switch" type="checkbox"/><label for="toctree-checkbox-13"><div class="visually-hidden">Toggle navigation of dynadojo.challenges</div><i class="icon"><svg><use href="#svg-arrow-right"></use></svg></i></label><ul>
<li class="toctree-l4"><a class="reference internal" href="dynadojo.challenges.FixedComplexity.html">dynadojo.challenges.FixedComplexity</a></li>
<li class="toctree-l4"><a class="reference internal" href="dynadojo.challenges.FixedDimensionality.html">dynadojo.challenges.FixedDimensionality</a></li>
<li class="toctree-l4"><a class="reference internal" href="dynadojo.challenges.FixedError.html">dynadojo.challenges.FixedError</a></li>
<li class="toctree-l4"><a class="reference internal" href="dynadojo.challenges.FixedTrainSize.html">dynadojo.challenges.FixedTrainSize</a></li>
<li class="toctree-l4"><a class="reference internal" href="dynadojo.challenges.ScalingChallenge.html">dynadojo.challenges.ScalingChallenge</a></li>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -212,7 +212,7 @@
</ul>
</li>
<li class="toctree-l3 has-children"><a class="reference internal" href="dynadojo.challenges.html">dynadojo.challenges</a><input class="toctree-checkbox" id="toctree-checkbox-13" name="toctree-checkbox-13" role="switch" type="checkbox"/><label for="toctree-checkbox-13"><div class="visually-hidden">Toggle navigation of dynadojo.challenges</div><i class="icon"><svg><use href="#svg-arrow-right"></use></svg></i></label><ul>
<li class="toctree-l4"><a class="reference internal" href="dynadojo.challenges.FixedComplexity.html">dynadojo.challenges.FixedComplexity</a></li>
<li class="toctree-l4"><a class="reference internal" href="dynadojo.challenges.FixedDimensionality.html">dynadojo.challenges.FixedDimensionality</a></li>
<li class="toctree-l4"><a class="reference internal" href="dynadojo.challenges.FixedError.html">dynadojo.challenges.FixedError</a></li>
<li class="toctree-l4"><a class="reference internal" href="dynadojo.challenges.FixedTrainSize.html">dynadojo.challenges.FixedTrainSize</a></li>
<li class="toctree-l4"><a class="reference internal" href="dynadojo.challenges.ScalingChallenge.html">dynadojo.challenges.ScalingChallenge</a></li>
Expand Down
2 changes: 1 addition & 1 deletion docs/build/html/_autosummary/dynadojo.baselines.dnn.html
Original file line number Diff line number Diff line change
Expand Up @@ -212,7 +212,7 @@
</ul>
</li>
<li class="toctree-l3 has-children"><a class="reference internal" href="dynadojo.challenges.html">dynadojo.challenges</a><input class="toctree-checkbox" id="toctree-checkbox-13" name="toctree-checkbox-13" role="switch" type="checkbox"/><label for="toctree-checkbox-13"><div class="visually-hidden">Toggle navigation of dynadojo.challenges</div><i class="icon"><svg><use href="#svg-arrow-right"></use></svg></i></label><ul>
<li class="toctree-l4"><a class="reference internal" href="dynadojo.challenges.FixedComplexity.html">dynadojo.challenges.FixedComplexity</a></li>
<li class="toctree-l4"><a class="reference internal" href="dynadojo.challenges.FixedDimensionality.html">dynadojo.challenges.FixedDimensionality</a></li>
<li class="toctree-l4"><a class="reference internal" href="dynadojo.challenges.FixedError.html">dynadojo.challenges.FixedError</a></li>
<li class="toctree-l4"><a class="reference internal" href="dynadojo.challenges.FixedTrainSize.html">dynadojo.challenges.FixedTrainSize</a></li>
<li class="toctree-l4"><a class="reference internal" href="dynadojo.challenges.ScalingChallenge.html">dynadojo.challenges.ScalingChallenge</a></li>
Expand Down
2 changes: 1 addition & 1 deletion docs/build/html/_autosummary/dynadojo.baselines.html
Original file line number Diff line number Diff line change
Expand Up @@ -212,7 +212,7 @@
</ul>
</li>
<li class="toctree-l3 has-children"><a class="reference internal" href="dynadojo.challenges.html">dynadojo.challenges</a><input class="toctree-checkbox" id="toctree-checkbox-13" name="toctree-checkbox-13" role="switch" type="checkbox"/><label for="toctree-checkbox-13"><div class="visually-hidden">Toggle navigation of dynadojo.challenges</div><i class="icon"><svg><use href="#svg-arrow-right"></use></svg></i></label><ul>
<li class="toctree-l4"><a class="reference internal" href="dynadojo.challenges.FixedComplexity.html">dynadojo.challenges.FixedComplexity</a></li>
<li class="toctree-l4"><a class="reference internal" href="dynadojo.challenges.FixedDimensionality.html">dynadojo.challenges.FixedDimensionality</a></li>
<li class="toctree-l4"><a class="reference internal" href="dynadojo.challenges.FixedError.html">dynadojo.challenges.FixedError</a></li>
<li class="toctree-l4"><a class="reference internal" href="dynadojo.challenges.FixedTrainSize.html">dynadojo.challenges.FixedTrainSize</a></li>
<li class="toctree-l4"><a class="reference internal" href="dynadojo.challenges.ScalingChallenge.html">dynadojo.challenges.ScalingChallenge</a></li>
Expand Down
Loading