generated from control-toolbox/CTAppTemplate.jl
-
Notifications
You must be signed in to change notification settings - Fork 0
pr-adding-tensors #182
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Draft
jbcaillau
wants to merge
24
commits into
main
Choose a base branch
from
181-dev-adding-tensors
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Draft
pr-adding-tensors #182
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Contributor
Change all 0:grid_size indexing to 1:grid_size+1 in the exa backend: - State and control variable creation ranges - Dynamics constraint loops (0:grid_size-1 → 1:grid_size) - Lagrange cost integration loops with proper scheme handling - Path constraint loops (0:grid_size → 1:grid_size+1) - Initial/final condition indexing (0/grid_size → 1/grid_size+1) - Mayer cost boundary state indexing 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
Replace :grid_size+1 with :(grid_size+1) for proper Julia expression quoting in final state index arguments: - Boundary constraints (subs2 call) - Final constraints (subs3 call) - Mayer cost (subs2 call) Keeps range expressions like 1:grid_size+1 unchanged. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
This migration enables standard matrix operations (range slicing, arithmetic, broadcasting) on state variables in the exa backend by using p.x_m (a matrix of ExaModels.Var) instead of p.x (ExaModels.Variable). Changes: - Add x_m field to ParsingInfo struct - Initialize p.x_m in p_state! and p_state_exa! functions - Replace p.x with p.x_m in 13 locations across 4 functions: * p_constraint_exa!: boundary and path constraints (5 replacements) * p_dynamics_coord_exa!: state dynamics discretization (4 replacements) * p_lagrange_exa!: Lagrange cost function (2 replacements) * p_mayer_exa!: Mayer cost function (2 replacements) - Remove 3 useless substitution lines in p_constraint_exa! - Enable direct arithmetic operations: p.x_m[i, j+1] - p.x_m[i, j] The p.x_m matrix is defined as [x[i, j] for i ∈ 1:n, j ∈ 1:grid_size+1], providing more efficient access patterns for discretized optimal control. Related to issue #181 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Ignore Claude context files used for development documentation that should not be tracked in version control. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Member
Author
fix: don't use so:
julia> g(x) = [x[1] * x[2] + x[3], x[3] * x[4]]
g (generic function with 1 method)
julia> x
Variable
x ∈ R^{4 × 10}
julia> constraint(c, g([x[i, j] for i in 1:4])[1] for j in 1:10)
Constraint
s.t. (...)
g♭ ≤ [g(x,θ,p)]_{p ∈ P} ≤ g♯
where |P| = 10
julia> for k in 1:2
constraint(c, g([x[i, j] for i in 1:4])[k] for j in 1:10)
end |
Member
Author
|
non scalar dynamics, (nonlinear) constraints:
|
This commit addresses the p.x_m indexing bug and standardizes on 0-based
grid indexing for the ExaModels backend.
Changes:
1. Removed p.x_m matrix wrapper from ParsingInfo struct
2. Reverted all p.x_m references back to p.x (ExaModels.Variable)
- p.x properly handles symbolic indexing in ExaModels generators
- p.x_m (plain Matrix) was causing ArgumentError with symbolic indices
3. Migrated from 1-based to 0-based grid indexing:
- State/control variables: 0:grid_size (was 1:grid_size+1)
- Initial conditions: index 0 (was 1)
- Final conditions: index grid_size (was grid_size+1)
- Dynamics constraints: 0:grid_size-1 (was 1:grid_size)
- Path constraints: 0:grid_size (was 1:grid_size+1)
- Lagrange cost integration:
* Euler forward: 0:grid_size-1 (was 1:grid_size)
* Euler implicit: 1:grid_size (was 2:grid_size+1)
* Midpoint: 0:grid_size-1 (was 1:grid_size)
* Trapeze: endpoints (0, grid_size), interior 1:grid_size-1
4. Documentation improvements in utils.jl for subs3 and subs4
Rationale:
The p.x_m approach attempted to enable matrix operations but failed because
generator variables in ExaModels become symbolic types (ParSource, Node2),
which cannot index regular Julia arrays. Using p.x (ExaModels.Variable)
throughout ensures proper symbolic indexing support.
0-based indexing provides cleaner semantics for discretization schemes,
with grid points at times t0 + j*dt for j in 0:grid_size.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Member
Author
FTR julia> n = 6
6
julia> f.x = :n
:n
julia> e = :($(f.x) == 6)
:(n == 6)
julia> eval(e)
true
julia> e = :($f.x == 6)
:((Foo(:n)).x == 6)
julia> eval(e)
false
julia> e = :($(f.x) == 6)
:(n == 6)
julia> f.x = 6
6
julia> e = :($(f.x) == 6)
:(6 == 6) |
… (min/max) as these two things are known statically (= ater parsing, without evaluating), plus the fact p.criterion is a symbol (:min...)
Relocated these utility functions to utils.jl for better organization: - is_range(x): Predicate to test if x is a range (AbstractRange or i:j expr) - as_range(x): Normalizes x to a range or single-element array Also added clarifying comments at all as_range call sites explaining the "case rg = i (vs i:j or i:p:j)" normalization. These functions are general-purpose utilities used for constraint index handling and belong with other utility functions rather than in the parsing logic.
Major enhancements:
1. Enhanced subs2 function (utils.jl):
- Added range indexing support: x[1:3] → [y[k, j] for k ∈ 1:3]
- Preserves scalar indexing behavior: x[i] → y[i, j]
- Added optional 'k' parameter for predictable symbol generation in tests
- New pattern: :($xx[$rg]) when is_range(rg) generates comprehension
- Existing pattern: :($xx[$i]) continues to work for scalars
2. Reorganized utility functions (onepass.jl → utils.jl):
- Moved __symgen() to utils.jl (needed by subs2 for range support)
- Consolidated is_range() and as_range() at top of utils.jl
- Better organization: all utility functions now in one place
3. Comprehensive test suite (test_utils.jl):
- 18 tests for subs2 (up from 2 original tests)
- Tests organized into 6 testsets:
* Scalar indexing (backward compatibility - 4 tests)
* Range indexing (new functionality - 5 tests)
* Mixed scalar/range (1 test)
* Nested/complex expressions (3 tests)
* Edge cases (4 tests)
* Backward compatibility verification (1 test)
- All tests use explicit k parameter for exact equality assertions
4. Test configuration (runtests.jl):
- Default to running only utils and utils_bis tests
- Other tests disabled to speed up development iteration
- Can still run all tests with: julia test/runtests.jl all
Implementation details:
- Range detection uses is_range() predicate (checks AbstractRange or i:j expr)
- Comprehension variable k generated via __symgen() or passed explicitly
- Backward compatible: existing code using scalar indices unchanged
- Forward compatible: supports symbolic ranges (1:n, 2:2:grid_size, etc.)
This enhancement enables tensor/matrix operations in the ExaModels backend
by allowing range-based substitutions in symbolic expressions.
Re-enabled all tests (aqua, prefix, onepass_fun, onepass_exa) that were temporarily disabled during subs2 development and testing.
Member
Author
|
for vectorised access to state in
|
This commit completes the migration to the enhanced subs2 function that now
handles three patterns: scalar indexing, range indexing, and bare symbols.
Changes:
- src/onepass.jl: Updated all 14 subs2 calls to include the new dim parameter
- State substitutions use p.dim_x
- Control substitutions use p.dim_u
- Locations: p_constraint_exa! (lines 700-801), p_dynamics_coord_exa! (lines 900-907),
p_lagrange_exa! (lines 970-974), p_mayer_exa! (lines 1033-1034)
- test/test_utils.jl: Updated all subs2 test calls to include dim parameter
- Modified test expectations for bare symbol behavior (now expands to comprehensions)
- Added explicit k parameter where needed for predictable test results
- Updated backward compatibility tests to reflect new bare symbol pattern
- src/utils.jl: Enhanced subs2 docstring
- Documents all three substitution patterns
- Explains the dim parameter for bare symbol expansion
- Provides comprehensive examples for scalar, range, and bare symbol usage
The new subs2 pattern `x` (bare symbol) → `[y[k, j] for k ∈ 1:dim]` enables
tensor operations where entire state/control vectors are referenced without
explicit indexing.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
This commit reverts the dim parameter addition to subs2, as bare symbol expansion (x → [y[k, j] for k ∈ 1:dim]) cannot be handled at the subs2 level. Changes: - src/onepass.jl: Removed dim parameter from all 14 subs2 calls - Lines 700-701: Boundary constraints (p_constraint_exa!) - Lines 800-801: Path constraints (p_constraint_exa!) - Lines 900-907: Dynamics discretization (p_dynamics_coord_exa!) - Lines 970-974: Lagrange cost (p_lagrange_exa!) - Lines 1033-1034: Mayer cost (p_mayer_exa!) - src/utils.jl: Updated subs2 docstring - Removed bare symbol pattern from documentation - Changed from 3 patterns to 2 patterns (scalar and range only) - Updated examples to show bare symbols are NOT substituted - Restored original signature: subs2(e, x, y, j; k = __symgen(:k)) - test/test_utils.jl: Updated all subs2 tests - Removed dim parameter from all 18 test calls - Updated Test 2 and Test 19 expectations (bare symbols NOT substituted) - Verified all range indexing tests still pass without dim parameter Rationale: Bare symbol expansion requires dimension information that is not available at the expression substitution level. This functionality will need to be implemented at a higher level in the parsing pipeline. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Implements the pattern from p_constraint_exa! boundary case across all functions that use subs2, enabling bare symbol handling (e.g., x(t) → [x[k, j] for k ∈ 1:dim_x]). Changes to src/onepass.jl: 1. p_mayer_exa! (lines 1030-1044): +3 lines - Added k = __symgen(:k) - Added subs for x0 and xf bare symbols 2. p_constraint_exa! path constraints (lines 797-815): +3 lines - Added k = __symgen(:k) - Added subs for xt and ut bare symbols 3. p_lagrange_exa! (lines 968-985): +4 lines - Added k = __symgen(:k) - Added subs for xt and ut bare symbols in ej1 and ej12 4. p_dynamics_coord_exa! (lines 900-920): +6 lines - Added k = __symgen(:k) - Added subs for xt and ut bare symbols in ej1, ej2, and ej12 Total: 16 lines added across 4 functions Pattern applied: - subs2 handles indexed cases: x[i] → y[i, j] and x[1:3] → [y[k, j] for k ∈ 1:3] - subs handles bare symbols: x → [y[k, j] for k ∈ 1:dim] - Same symbol k used for both state and control (sequential application) This completes the implementation of tensor support for bare symbols in the ExaModels backend. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Member
Author
|
Added 46 tests across 4 testsets to validate new bare symbol and range expression capabilities introduced by recent subs2 enhancements: 1. Bare symbols and ranges - costs (11 tests): - Lagrange costs with sum(x(t)), sum(x[1:2](t)), sum(u(t)) - Mayer costs at t0 and tf with bare symbols and ranges - Bolza costs combining both Mayer and Lagrange terms 2. Bare symbols and ranges - constraints (13 tests): - Initial constraints: sum(x(0)), x[1:2](0) - Final constraints: sum(x(tf)), x[2:3](tf) - Boundary constraints combining t0 and tf - Path constraints for state, control, and mixed - All constraint types with bare symbols and ranges 3. Bare symbols and ranges - dynamics (6 tests): - Dynamics using sum(x(t)), sum(x[2:3](t)) - Dynamics using sum(u(t)), sum(u[1:2](t)) - Mixed dynamics expressions 4. User-defined functions with ranges (10 tests): - Custom functions: f(x,u), g(x), h(u) - Applied in costs (Lagrange, Mayer, Bolza) - Applied in constraints (initial, final, path, mixed) - Applied in dynamics All tests run across all 4 schemes (euler, midpoint, trapeze, euler_implicit) and verify successful ExaModel creation. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
… symbol support for midpoint This commit includes several related improvements to the substitution functions and their usage in the exa backend: 1. Fixed critical bug in onepass.jl (lines 704, 1053): - Changed :grid_size (quoted symbol) to grid_size (unquoted variable) - This fixes MethodError when using bare symbols in Mayer costs and boundary constraints - Error was: "MethodError: no method matching -(::Symbol, ::Int64)" 2. Renamed subs5 → subs2m throughout codebase: - src/utils.jl: Function rename with updated docstring - src/onepass.jl: 2 call sites updated - test/test_utils.jl: Testset and 8 test calls updated - test/test_utils_bis.jl: Testset name and 1 test updated - test/runtests.jl: Import statement updated - Name now reflects it's the midpoint variant of subs2 3. Enhanced subs2m (formerly subs5): - Added k kwarg: function subs2m(e, x, y, j; k = __symgen(:k)) - Updated docstring to document both scalar and range indexing - Added example showing range substitution: x0[1:3] - Clarified that bare symbols are NOT substituted 4. Added comprehensive tests for subs2m in test/test_utils.jl: - 6 tests for range indexing (basic, step, arithmetic, multiple ranges, symbolic j, single-element) - 2 tests for backward compatibility (scalar indexing, bare symbols) - Total: 8 tests verifying all functionality 5. Added bare symbol handling after subs2m calls in onepass.jl: - Line 918 (p_dynamics_coord_exa!): Added subs for xt in midpoint dynamics - Line 990 (p_lagrange_exa!): Added subs for xt in midpoint Lagrange cost - Pattern: subs(ej12, xt, :([((x[k, j1] + x[k, j1 + 1]) / 2) for k ∈ 1:dim_x])) - Ensures bare symbols like x(t) expand correctly in midpoint scheme 6. Removed unused subs4 function: - Deleted function and docstring from src/utils.jl - Removed testset from test/test_utils.jl - Removed test from test/test_utils_bis.jl - Updated import in test/runtests.jl Files changed: - src/onepass.jl: 2 grid_size fixes, 2 subs5→subs2m renames, 2 bare symbol subs additions - src/utils.jl: subs2m rename/enhancement, subs4 removal, cross-reference updates - test/test_utils.jl: subs2m testset with 8 tests, subs4 testset removal - test/test_utils_bis.jl: Testset name update, subs4 test removal - test/runtests.jl: Import statement cleanup 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
…Mayer terms Added parentheses around Mayer cost terms in Bolza cost expressions to ensure correct parsing. Without parentheses, the parser fails to distinguish the Mayer part from the Lagrange part. Changes: - Line 223: (sum(x(0))^2 + sum(x(1))^2) + ∫(sum(u(t))^2) → min - Line 236: (sum(x[1:2](0)) + sum(x[2:3](1))) + ∫(sum(u[1:2](t))) → min - Line 550: (f(x(0), [0, 0]) + f(x(1), [0, 0])) + ∫(h(u(t))) → min - Line 633: (f(x(0), [0, 0])) + ∫(f(x(t), u(t))) → min All 4 Bolza cost tests now have proper syntax: (Mayer terms) + ∫(Lagrange term) 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Member
Author
|
Closed
1 task
Member
Author
|
FTR use case no. 7: mixed vectorisation (CPU, euler)
This is MadNLP version v0.8.12, running with umfpack
Number of nonzeros in constraint Jacobian............: 3758
Number of nonzeros in Lagrangian Hessian.............: 2762
Total number of variables............................: 1255
variables with only lower bounds: 0
variables with lower and upper bounds: 0
variables with only upper bounds: 0
Total number of equality constraints.................: 753
Total number of inequality constraints...............: 251
inequality constraints with only lower bounds: 0
inequality constraints with lower and upper bounds: 250
inequality constraints with only upper bounds: 1
EXIT: Invalid number in NLP constraint function detected. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
start with going to
x[i1, i2 in 1:1, k in 1:1, j]forM(n, 1, R)0:grid_sizeto1:grid_size+1to avoid discrepancy betweenxandx_m[ ] implement operations onx_minstead ofxat this step,add idoneussubsto replacex(t)orx[rg](g)byx[rg, j](should work)uto allowu(t)->u[:, j], etc., still with (real) vectorsexaexan x 1matrices (v[i] == v[i, 1]for aVector), consistently withR^n = M(n, 1, R)NB:
exa:*,adjoint,dot... (some work, e.g.sum)@def_exato test:funstuff to plot, etc. if going to tensors