Bugfix: IPOPT log parser and no objective case #3738

mrmundt · 2025-09-30T15:57:29Z

Fixes #3736

Summary/Motivation:

The issue above mentions two bugs that are related to IPOPT/parsing both the output and the sol files. This PR fixes both of those bugs and adds tests.

Changes proposed in this PR:

Make IPOPT parser more robust
Set duals and RCs to 0 when no objective is supplied
Add tests!

Legal Acknowledgement

By contributing to this software project, I have read the contribution guide and agree to the following terms and conditions for my contribution:

I agree my contributions are submitted under the BSD license.
I represent I am authorized to make the contributions and grant the license. If my employer has rights to intellectual property that includes these contributions, I represent that I have received permission to make contributions and grant the required license on behalf of that employer.

dallan-keylogic · 2025-09-30T16:22:34Z

pyomo/contrib/solver/solvers/ipopt.py

+        # If the NL instance has no objectives, report zeros
+        if not getattr(self._nl_info, "objectives", None):
+            vars_ = (
+                vars_to_load if vars_to_load is not None else self._nl_info.variable_map
+            )
+            return {v: 0.0 for v in vars_}


This fix is suitable if IPOPT converges to a feasible point, but if it converges to a point of local infeasibility, the values of the dual variables convey useful information about which bounds may be active at a solution.

Per your suggestion, though, if there are no objectives, these should be set to 0. So I don't think we change this. We capture the raw logs anyway, so folks can inspect those if they want more details.

Would it be better / more appropriate to return {} here? Unspecified duals / reduced costs are assumed to be 0, right?

@mrmundt IPOPT switches objectives when it enters into Restoration mode, using the L1 norm of the constraint violation as an objective. If it converges to an infeasible point with the exit message EXIT: Converged to a point of local infeasibility. then it is at a local minimum of the constraint violation, and we expect nonzero dual variables and reduced costs for this objective. I don't know if IPOPT makes these values available to AMPL, though.

I am not sure what to do with this information because I only understand what half of it means.

If IPOPT converges to an infeasible point, the dual variables and reduced costs are nonzero and contain useful information. I'm not sure whether IPOPT makes that information available via the interface you're using.

blnicho

It might be worth adding some additional tests for the log parser with different Ipopt print_level. We might just want to detect that a non-default print_level was used and add logic to not try to parse it. This doesn't have to hold up this PR but I was thinking about it in the context of making the parser more robust.

jsiirola · 2025-09-30T17:29:27Z

pyomo/contrib/solver/solvers/ipopt.py

+        # If the NL instance has no objectives, report zeros
+        if not getattr(self._nl_info, "objectives", None):
+            vars_ = (
+                vars_to_load if vars_to_load is not None else self._nl_info.variable_map
+            )
+            return {v: 0.0 for v in vars_}


Would it be better / more appropriate to return {} here? Unspecified duals / reduced costs are assumed to be 0, right?

pyomo/contrib/solver/solvers/ipopt.py

jsiirola · 2025-09-30T17:47:12Z

pyomo/contrib/solver/solvers/ipopt.py

+            vars_ = (
+                vars_to_load if vars_to_load is not None else self._nl_info.variables
+            )
+            return ComponentMap((v, 0.0) for v in vars_)


Is it more correct to just return an empty ComponentMap? Aren't missing entries assumed to be 0?

So I did this (both here and for get_duals), and it actually goes against what we say is a norm, surprisingly enough:

________________________________________ TestSolvers.test_no_objective_3_ipopt _________________________________________ a = (<pyomo.contrib.solver.tests.solvers.test_solvers.TestSolvers testMethod=test_no_objective_3_ipopt>,), kw = {} @wraps(func) def standalone_func(*a, **kw): > return func(*(a + p.args), **p.kwargs, **kw) ../venv-pyomo/lib/python3.12/site-packages/parameterized/parameterized.py:620: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = <pyomo.contrib.solver.tests.solvers.test_solvers.TestSolvers testMethod=test_no_objective_3_ipopt> name = 'ipopt', opt_class = <class 'pyomo.contrib.solver.solvers.ipopt.Ipopt'>, use_presolve = False @parameterized.expand(input=_load_tests(all_solvers)) def test_no_objective( self, name: str, opt_class: Type[SolverBase], use_presolve: bool ): opt: SolverBase = opt_class() if not opt.available(): raise unittest.SkipTest(f'Solver {opt.name} not available.') check_duals = True if any(name.startswith(i) for i in nl_solvers_set): if use_presolve: check_duals = False opt.config.writer_config.linear_presolve = True else: opt.config.writer_config.linear_presolve = False m = pyo.ConcreteModel() m.x = pyo.Var() m.y = pyo.Var() m.a1 = pyo.Param(mutable=True) m.a2 = pyo.Param(mutable=True) m.b1 = pyo.Param(mutable=True) m.b2 = pyo.Param(mutable=True) m.c1 = pyo.Constraint(expr=m.y == m.a1 * m.x + m.b1) m.c2 = pyo.Constraint(expr=m.y == m.a2 * m.x + m.b2) params_to_test = [(1, -1, 2, 1), (1, -2, 2, 1), (1, -1, 3, 1)] for a1, a2, b1, b2 in params_to_test: m.a1.value = a1 m.a2.value = a2 m.b1.value = b1 m.b2.value = b2 res: Results = opt.solve(m) self.assertEqual(res.solution_status, SolutionStatus.optimal) self.assertAlmostEqual(m.x.value, (b2 - b1) / (a1 - a2)) self.assertAlmostEqual(m.y.value, a1 * (b2 - b1) / (a1 - a2) + b1) self.assertEqual(res.incumbent_objective, None) self.assertEqual(res.objective_bound, None) if check_duals: duals = res.solution_loader.get_duals() > self.assertAlmostEqual(duals[m.c1], 0) E KeyError: <pyomo.core.base.constraint.ScalarConstraint object at 0x11b9effc0> pyomo/contrib/solver/tests/solvers/test_solvers.py:994: KeyError

If I return an empty ComponentMap or an empty dict, then you can't actually access that item anymore.

Okay, empty ComponentMap is fine; it's empty dict for get_duals that causes these failures.

pyomo/contrib/solver/solvers/ipopt.py

pyomo/contrib/solver/solvers/sol_reader.py

jsiirola · 2025-09-30T18:11:24Z

pyomo/contrib/solver/solvers/sol_reader.py

+            cons = (
+                cons_to_load if cons_to_load is not None else self._nl_info.constraints
+            )
+            return {c: 0.0 for c in cons}


As with RC, is it more correct to return {}?

pyomo/contrib/solver/solvers/sol_reader.py

Co-authored-by: John Siirola <jsiirola@users.noreply.github.com>

mrmundt · 2025-10-02T01:40:31Z

@AlexGisi - you pointed out a different missed thing in the IPOPT parser, though! We did only have lowercase s in the accepted chars for alpha_pr. I added that in just now. Thanks for pointing it out!

pyomo/contrib/solver/solvers/ipopt.py

pyomo/contrib/solver/solvers/sol_reader.py

pyomo/contrib/solver/solvers/ipopt.py

codecov · 2025-10-07T00:06:18Z

Codecov Report

❌ Patch coverage is 92.85714% with 4 lines in your changes missing coverage. Please review.
✅ Project coverage is 89.31%. Comparing base (e8ed5dd) to head (31b6143).

Files with missing lines	Patch %	Lines
pyomo/contrib/solver/solvers/ipopt.py	90.90%	4 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #3738      +/-   ##
==========================================
- Coverage   89.31%   89.31%   -0.01%     
==========================================
  Files         896      896              
  Lines      103697   103728      +31     
==========================================
+ Hits        92619    92645      +26     
- Misses      11078    11083       +5

Flag	Coverage Δ
builders	`29.09% <3.57%> (+<0.01%)`	⬆️
default	`85.93% <92.85%> (?)`
expensive	`35.85% <3.57%> (?)`
linux	`86.96% <92.85%> (-2.09%)`	⬇️
linux_other	`86.96% <92.85%> (+<0.01%)`	⬆️
osx	`83.10% <92.85%> (+<0.01%)`	⬆️
win	`84.68% <92.85%> (-0.54%)`	⬇️
win_other	`?`

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Fix two parsing bugs

4d00144

mrmundt requested a review from jsiirola September 30, 2025 15:57

dallan-keylogic reviewed Sep 30, 2025

View reviewed changes

Remove usage of getattr

8032600

blnicho approved these changes Sep 30, 2025

View reviewed changes

jsiirola reviewed Sep 30, 2025

View reviewed changes

mrmundt and others added 5 commits September 30, 2025 12:41

Update pyomo/contrib/solver/solvers/ipopt.py

edf0528

Co-authored-by: John Siirola <jsiirola@users.noreply.github.com>

Update pyomo/contrib/solver/solvers/sol_reader.py

9e7c31b

Co-authored-by: John Siirola <jsiirola@users.noreply.github.com>

Address jsiirola's comments

0c7ffda

Return empty ComponentMap

b9b1453

Merge branch 'main' into ipopt-parser

00d9705

AlexGisi mentioned this pull request Oct 2, 2025

Fix two issues with ipopt output parsing #3741

Closed

S is in _ALPHA_PR_CHARS

cbc3293

mrmundt and others added 3 commits October 3, 2025 09:50

Merge branch 'main' into ipopt-parser

a9c3bf5

Add in tests for print levels (catching a new bug in the process)

b4fea1e

Residual debugging print stuff

89e021f

jsiirola reviewed Oct 6, 2025

View reviewed changes

Apply requested changes

393f61b

mrmundt requested a review from jsiirola October 6, 2025 19:23

mrmundt added 3 commits October 6, 2025 14:23

Add regex back in, fix incorrect assertion language

b5d8608

Not sure what's goiing on with windows

24ddea7

Try one more time

31b6143

Bugfix: IPOPT log parser and no objective case #3738

Are you sure you want to change the base?

Bugfix: IPOPT log parser and no objective case #3738

Conversation

mrmundt commented Sep 30, 2025

Fixes #3736

Summary/Motivation:

Changes proposed in this PR:

Legal Acknowledgement

Uh oh!

dallan-keylogic Sep 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

blnicho left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

mrmundt commented Oct 2, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

codecov bot commented Oct 7, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Uh oh!

dallan-keylogic Sep 30, 2025 •

edited

Loading

codecov bot commented Oct 7, 2025 •

edited

Loading