Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
31 changes: 25 additions & 6 deletions .github/workflows/jenn-ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ jobs:
runs-on: ubuntu-latest
strategy:
matrix:
python-version: ["3.8", "3.9", "3.10", "3.11", "3.12", "3.13", "3.14"]
python-version: ["3.x"]
steps:
- uses: actions/checkout@v4

Expand All @@ -29,10 +29,29 @@ jobs:
environments: dev # Specify the environment to install and activate

- name: Check linting
run: pixi run -e dev --frozen lint
run: pixi run -e dev lint

- name: Run tests
run: pixi run -e dev --frozen test

- name: Run tests (Python 3.9)
run: pixi run -e py39 test

- name: Run tests (Python 3.10)
run: pixi run -e py310 test

- name: Run tests (Python 3.11)
run: pixi run -e py311 test

- name: Run tests (Python 3.12)
run: pixi run -e py312 test

- name: Run tests (Python 3.13)
run: pixi run -e py313 test

- name: Run tests (Python 3.14)
run: pixi run -e py314 test

- name: Run tests (latest Python version)
run: pixi run -e dev test

- name: ⬆️ Upload build artifacts
uses: actions/upload-artifact@v4
Expand All @@ -53,12 +72,12 @@ jobs:
environments: dev # Specify the environment to install and activate

- name: Build docs using sphinx
run: pixi run -e dev --frozen docs
run: pixi run -e dev docs

- name: 💾 Build distribution files
shell: bash -el {0}
run: |
pixi run -e dev --frozen build
pixi run -e dev build

- name: ⬆️ Upload build artifacts
uses: actions/upload-artifact@v4
Expand Down
22 changes: 16 additions & 6 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,11 @@ build: Changes to the build process or tools.

# Changelog

## v1.0.9 (YYYY-MM-DD)
## v2.0.0 (YYYY-MM-DD)

### Feat

- Added prediction error histogram

### Fix

Expand All @@ -28,22 +32,27 @@ build: Changes to the build process or tools.

### Test

- Updated optimization tests to reflect line search changes
- Updated unit tests to reflect changes to synthetic data functions
- Updated unit tests to reflect changes to `Parameters.load()` method
- Updated unit tests to reflect refactoring changes
- Added testing for all supported Python versions (leveraging pixi)

### Build

- Added `pixi.toml` (rather than putting everything in `pyproject.toml`)
- Updated CI worflow
- Cleanup CI and updated worflow to test all supported python versions using pixi

### Docs

- Simplified CONTRIBUTING
- Updated docs to include API changes
- Updated README to reflect refactoring changes
- Updated docs to reflect refactoring changes

### Style

- Updated linting and annotations

### Refactor

- Dropped support for Python 3.8 (because SMT no longer supports it and it doesn't handle annotations as well)
- Changed API by moving module `model.py` into `core`
- Changed API by moving module `synthetic.py` into `synthetic_data` (synthetic functions are now modules not classes)
- Changed API by moving module `plot` into `post_processing`
Expand All @@ -55,6 +64,7 @@ build: Changes to the build process or tools.
- _Previous pattern: `reloaded = NeuralNet(layer_sizes=[1, 2, 3]).load("save_params.json")`_
- Converted `Parameters.load(...)` to a classmethod such that `reloaded = Parameters.load("save_params.json")`
- _Previous pattern: `reloaded = Parameters(layer_sizes=[1, 2, 3]).load("save_params.json")`_
- Replaced `NeuralNet.evaluate(x)` by `NeuralNet.__call__(x)`

## v1.0.8 (2024-06-26)

Expand Down
59 changes: 27 additions & 32 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

Jacobian-Enhanced Neural Networks (JENN) are fully connected multi-layer
perceptrons, whose training process is modified to predict partial
derivatives accurately. This is accomplished by minimizing a modified version
derivatives more accurately. This is accomplished by minimizing a modified version
of the Least Squares Estimator (LSE) that accounts for Jacobian prediction error (see [paper](https://doi.org/10.48550/arXiv.2406.09132)).
The main benefit of jacobian-enhancement is better accuracy with
fewer training points compared to standard fully connected neural nets, as illustrated below.
Expand Down Expand Up @@ -35,21 +35,16 @@ If you use JENN in a scientific publication, please consider citing it:
}
```

----
# Main Features

* Multi-Task Learning : predict more than one output with same model Y = f(X) where Y = [y1, y2, ...]
* Jacobian prediction : analytically compute the Jacobian (_i.e._ forward propagation of dY/dX)
* Gradient-Enhancement: minimize prediction error of partials (_i.e._ back-prop accounts for dY/dX)

----

# Installation

pip install jenn

----

# Example Usage

_See [demo](./docs/examples/) notebooks for more details_
Expand All @@ -60,23 +55,26 @@ Import library:

Generate example training and test data:

x_train, y_train, dydx_train = jenn.synthetic.Sinusoid.sample(
m_lhs=0,
x_train, y_train, dydx_train = jenn.utilities.sample(
f=jenn.synthetic_data.sinusoid.compute,
f_prime=jenn.synthetic_data.sinusoid.compute_partials,
m_random=0,
m_levels=4,
lb=-3.14,
ub=3.14,
)
x_test, y_test, dydx_test = jenn.synthetic.Sinusoid.sample(
m_lhs=30,
x_test, y_test, dydx_test = jenn.utilities.sample(
f=jenn.synthetic_data.sinusoid.compute,
f_prime=jenn.synthetic_data.sinusoid.compute_partials,
m_random=30,
m_levels=0,
lb=-3.14,
ub=3.14,
)


Train a model:

nn = jenn.model.NeuralNet(
nn = jenn.NeuralNet(
layer_sizes=[1, 12, 1],
).fit(
x=x_train,
Expand All @@ -88,7 +86,7 @@ Train a model:

Make predictions:

y, dydx = nn.evaluate(x)
y, dydx = nn(x)

# OR

Expand All @@ -102,36 +100,38 @@ Save model (parameters) for later use:

Reload saved parameters into new model:

reloaded = jenn.model.NeuralNet(layer_sizes=[1, 12, 1]).load('parameters.json')
reloaded = jenn.NeuralNet.load('parameters.json')

Optionally, if `matplotlib` is installed, import plotting utilities:
Check goodness of fit:

from jenn.utils import plot
jenn.plot_goodness_of_fit(
y_true=y_test,
y_pred=nn.predict(x_test),
title="y (JENN)"
)

Optionally, if `matplotlib` is installed, check goodness of fit:
Check goodness of fit of partials:

plot.goodness_of_fit(
y_true=dydx_test[0],
y_pred=nn.predict_partials(x_test)[0],
title="Partial Derivative: dy/dx (JENN)"
jenn.plot_goodness_of_fit(
y_true=dydx_test,
y_pred=nn.predict_partials(x_test),
title="dy/dx (JENN)"
)

Optionally, if `matplotlib` is installed, show sensitivity profiles:
Show sensitivity profiles:

plot.sensitivity_profiles(
f=[jenn.synthetic.Sinusoid.evaluate, nn.predict],
jenn.plot_sensitivity_profiles(
func=[jenn.synthetic_data.sinusoid.compute, nn.predict],
x_min=x_train.min(),
x_max=x_train.max(),
x_true=x_train,
y_true=y_train,
resolution=100,
legend=['true', 'pred'],
legend_label=['true', 'pred'],
xlabels=['x'],
ylabels=['y'],
)

----

# Use Case

JENN is intended for the field of computer aided design, where there is often
Expand All @@ -147,8 +147,6 @@ However, in the special case of gradient-enhanced methods, there is the addition
are accurate which is a critical property for one important use-case: **surrogate-based optimization**. The field of
aerospace engineering is rich in [applications](https://doi.org/10.1002/9780470686652.eae496) of such a use-case.

----

# Limitations

Gradient-enhanced methods require responses to be continuous and smooth,
Expand All @@ -157,12 +155,9 @@ is not excessive in the first place (e.g. adjoint methods), or if the need for a
computing the partials. Users should therefore carefully weigh the benefit of
gradient-enhanced methods relative to the needs of their application.

---
# License
Distributed under the terms of the MIT License.

----

# Acknowledgement

This code used the code by Prof. Andrew Ng in the
Expand Down
56 changes: 20 additions & 36 deletions docs/examples/.ipynb_checkpoints/demo_1_sinusoid-checkpoint.ipynb

Large diffs are not rendered by default.

27 changes: 10 additions & 17 deletions docs/examples/.ipynb_checkpoints/demo_2_rastrigin-checkpoint.ipynb

Large diffs are not rendered by default.

56 changes: 20 additions & 36 deletions docs/examples/demo_1_sinusoid.ipynb

Large diffs are not rendered by default.

27 changes: 10 additions & 17 deletions docs/examples/demo_2_rastrigin.ipynb

Large diffs are not rendered by default.

2 changes: 1 addition & 1 deletion docs/examples/demo_3_airfoil.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -316,7 +316,7 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.13.9"
"version": "3.11.14"
}
},
"nbformat": 4,
Expand Down
2 changes: 1 addition & 1 deletion docs/examples/demo_4_rosenbrock.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -636,7 +636,7 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.13.9"
"version": "3.11.14"
}
},
"nbformat": 4,
Expand Down
Binary file modified docs/pics/example_goodness_of_fit.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified docs/pics/example_sensitivity_profile.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
1 change: 1 addition & 0 deletions docs/source/conf.py
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,7 @@
"sphinx.ext.mathjax",
"sphinx.ext.viewcode",
"sphinx_rtd_theme",
"sphinx_copybutton",
]
templates_path = ["_templates"]
html_static_path = ["_static"]
Expand Down
3 changes: 3 additions & 0 deletions docs/source/sections/api.rst
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,9 @@ Core API

The core API implements all theory described in the `paper`_. This section is intended for developers.

.. automodule:: jenn.core.model
:members:

.. automodule:: jenn.core.activation
:members:

Expand Down
Loading