New GAMS solver interface, writer, and solution loader. #3683

AnhTran01 · 2025-08-07T18:07:31Z

Fixes NA

Summary/Motivation:

This PR introduced a new GAMS solver interface that uses the LinearRepnVisitor to enable coefficient gathering when writing out the scalar GAMS model in gms format.

Changes proposed in this PR:

New GAMS solver interface in pyomo.contrib.solver.solvers
New GAMS writer in pyomo.repn.plugins
New GAMS solution loader in pyomo.contrib.solver.solvers

Legal Acknowledgement

By contributing to this software project, I have read the contribution guide and agree to the following terms and conditions for my contribution:

I agree my contributions are submitted under the BSD license.
I represent I am authorized to make the contributions and grant the license. If my employer has rights to intellectual property that includes these contributions, I represent that I have received permission to make contributions and grant the required license on behalf of that employer.

…ittest and coverage test

AnhTran01 · 2025-08-07T18:07:57Z

@boxblox
@abhosekar

mrmundt · 2025-08-07T18:11:30Z

@AnhTran01 - Please make sure to run black on your changes!

mrmundt

Immediate piece of feedback, recognizing that I have not thoroughly looked over anything - there are no tests anywhere, and there absolutely will need to be some.

Thanks for this, though, I am so excited!!

…riting process. Added WIP unittest for the new solver interface

…ion error

… and writing to gms.

boxblox · 2025-08-14T11:30:35Z

pyomo/repn/plugins/gams_writer_v2.py

@@ -518,6 +518,11 @@ def write(self, model):
        con_list[label] = declaration + definition + bounds
        self.var_symbol_map.addSymbol(obj, label)

+        # Write the GAMS model
+        ostream.write("$offlisting\n")
+        # $offdigit ignores extra precise digits instead of erroring


what was the error that you were seeing that prompted this addition? maybe something was being set above 1e300?

in the previous commit test (b0c3450), the listing file output error message is "Too many digits in number $offdigit can be used to ignore trailing digits"

but do you know what triggered this? which test was causing this error?

I went back to check the failed test. The following tests that triggered the "Too many digits error" are:

TestSolvers.test_equality

TestSolvers.test_linear_expression

TestSolvers.test_mutable_param_with_range

TestSolvers.test_param_changes

TestSolvers.test_no_objective

TestSolvers.test_time_limit

TestLegacySolverInterface.test_param_updates

By testing locally, I see there are trailing decimal places and not the cases of 1e300 or large numbers.

… constraint). Fixed bug of load_solution that contains constraint id in addition to variable id

…e_gdx_results

…arsing solver_options

jsiirola

(Some initial comments, mostly on configuration)

pyomo/contrib/solver/solvers/gams.py

jsiirola · 2025-08-15T20:39:42Z

pyomo/contrib/solver/solvers/gams.py

+        self.writer_config: ConfigDict = self.declare(
+            'writer_config', GAMSWriter.CONFIG()
+        )
+        # Share the same config as the writer, passes at the function call write


Here and below: this should just insert a copy of the CONFIG from the writer. See Ipopt here for an example.

jsiirola · 2025-08-15T21:22:14Z

pyomo/contrib/solver/solvers/gams.py

+            n,
+        )
+
+    def version(self, config=None):


In our review of the new solver interfaces, we changed the API to:

def version(self, rehash: bool=False) -> Tuple:

Much like available(), this should leverage a cache (class dict attribute). You can look at the ipopt implementation for an idea (although that hasn't been updated to the new API).

Could you clarify the purpose of using rehash and whether the user can call rehash from the solve statement?

I found the documentation here: https://pyomo.readthedocs.io/en/latest/api/pyomo.common.fileutils.PathManager.html

Would the implementation be Executable(config.executable.path()).rehash()?

pyomo/contrib/solver/solvers/gams.py

jsiirola · 2025-08-15T21:42:22Z

pyomo/contrib/solver/solvers/gams.py

+        if config.logfile is not None:
+            config.logfile = os.path.abspath(config.logfile)
+
+        config.writer_config.put_results_format = 'gdx' if gdxcc_available else 'dat'


This should probably only overwrite the configuration if it is still None (that way users can set / override the interface)

…lable() and version()

…are no longer valid.

…etting default to gdx that caused failed test. Implemented raise infeasible

… rehash handling with Executable.rehash()

…age for infeasible model when calling load_var, dual, reduced_cost

… not none

…ded handling solvestat and modelstat when solver is interrupted

…imports

AnhTran01 · 2025-08-22T18:55:48Z

@mrmundt is the requested changes related to adding tests for the new solver interface/writer?

mrmundt · 2025-09-09T19:22:58Z

pyomo/contrib/solver/solvers/gams.py

+        # NOTE: Taken from the lp_writer
+        self.declare(
+            'row_order',
+            ConfigValue(
+                default=None,
+                description='Preferred constraint ordering',
+                doc="""
+                To use with ordered_active_constraints function.""",
+            ),
+        )


I don't quite understand why this config option has been added. Shouldn't that option live under the writer?

EDIT: Aha, I see it also lives under the writer! Why is it in both places? I suspect this was a mistake.

mrmundt · 2025-09-09T19:24:39Z

pyomo/contrib/solver/solvers/gams.py

+class GAMSResults(Results):
+    def __init__(self):
+        super().__init__()
+        self.return_code: ConfigDict = self.declare(
+            'return_code',
+            ConfigValue(default=None, description="Return code from the GAMS solver."),
+        )
+        self.gams_solver_termination_condition: ConfigDict = self.declare(
+            'gams_solver_termination_condition',
+            ConfigValue(
+                default=None,
+                description="Include additional TerminationCondition domain."
+                "Take precedence over model_termination_condition if interruption occur",
+            ),
+        )
+        self.gams_model_termination_condition: ConfigDict = self.declare(
+            'gams_model_termination_condition',
+            ConfigValue(
+                default=None,
+                description="Include additional TerminationCondition domain.",
+            ),
+        )
+        self.gams_solver_status: ConfigDict = self.declare(
+            'gams_solver_status',
+            ConfigValue(
+                default=None, description="Include additional SolverStatus domain."
+            ),
+        )


@jsiirola and @michaelbynum - I think I would prefer that all of these items live under Results.extra_info rather than as distinct items / having a "custom" GAMSResults object. Thoughts?

I have no preference.

mrmundt · 2025-09-09T19:45:08Z

pyomo/repn/plugins/gams_writer_v2.py

+    Keyword Arguments
+    -----------------
+    output_filename: str
+        Name of file to write GAMS model to. Optionally pass a file-like
+        stream and the model will be written to that instead.
+    io_options: str
+        - warmstart=True
+            Warmstart by initializing model's variables to their values.
+        - symbolic_solver_labels=False
+            Use full Pyomo component names rather than
+            shortened symbols (slower, but useful for debugging).
+        - labeler=None
+            Custom labeler. Incompatible with symbolic_solver_labels.
+        - solver=None
+            If None, GAMS will use default solver for model type.
+        - mtype=None
+            Model type. If None, will chose from lp, nlp, mip, and minlp.
+        - add_options=None
+            List of additional lines to write directly
+            into model file before the solve statement.
+            For model attributes, <model name> is GAMS_MODEL.
+        - skip_trivial_constraints=False
+            Skip writing constraints whose body section is fixed.
+        - output_fixed_variables=False
+            If True, output fixed variables as variables; otherwise,
+            output numeric value.
+        - file_determinism=1
+            | How much effort do we want to put into ensuring the
+            | GAMS file is written deterministically for a Pyomo model:
+                - NONE (0) : None
+                - ORDERED (10): rely on underlying component ordering (default)
+                - SORT_INDICES (20) : sort keys of indexed components
+                - SORT_SYMBOLS (30) : sort keys AND sort names (not declaration order)
+        - put_results='results'
+            Filename for optionally writing solution values and
+            marginals.  If put_results_format is 'gdx', then GAMS
+            will write solution values and marginals to
+            GAMS_MODEL_p.gdx and solver statuses to
+            {put_results}_s.gdx.  If put_results_format is 'dat',
+            then solution values and marginals are written to
+            (put_results).dat, and solver statuses to (put_results +
+            'stat').dat.
+        - put_results_format='gdx'
+            Format used for put_results, one of 'gdx', 'dat'.


This is going to sound weird coming from me but you do not need all of this extra documentation. The autodoc will handle it from the ConfigValue descriptions/docs. I would encourage you to delete this.

mrmundt · 2025-09-09T19:47:03Z

pyomo/repn/plugins/gams_writer_v2.py

+    def __init__(self):
+        self.config = self.CONFIG()


Suggested change

def __init__(self):

self.config = self.CONFIG()

def __init__(self):

#: Instance configuration;

#: see :ref:`pyomo.repn.plugins.gams_writer_v2.GAMSWriter::CONFIG`.

self.config = self.CONFIG()

mrmundt · 2025-09-09T19:48:03Z

pyomo/repn/plugins/gams_writer_v2.py

+
+    def __call__(self, model, filename, solver_capability, io_options):
+        if filename is None:
+            filename = 'GAMS_MODEL' + ".gms"


Suggested change

filename = 'GAMS_MODEL' + ".gms"

filename = model.name + ".gms"

This is more standard.

mrmundt · 2025-09-09T20:12:39Z

pyomo/contrib/solver/solvers/gms_sol_reader.py

+                'Solution loader does not currently have a valid solution. Please '
+                'check results.termination_condition and/or results.solution_status.'


You don't need the text: https://github.com/Pyomo/pyomo/blob/main/pyomo/contrib/solver/common/util.py#L34

mrmundt · 2025-09-09T20:13:20Z

pyomo/contrib/solver/solvers/gms_sol_reader.py

+            raise RuntimeError(
+                'Solution loader does not currently have a valid solution. Please '
+                'check results.termination_condition and/or results.solution_status.'
+            )


I suspect this is supposed to be a different error from those in here: github.com/Pyomo/pyomo/blob/main/pyomo/contrib/solver/common/util.py

mrmundt · 2025-09-09T20:14:06Z

pyomo/contrib/solver/solvers/gms_sol_reader.py

+                'Solution loader does not currently have valid duals. Please '
+                'check results.termination_condition and/or results.solution_status.'


You don't need the text: https://github.com/Pyomo/pyomo/blob/main/pyomo/contrib/solver/common/util.py#L41

mrmundt · 2025-09-09T20:14:15Z

pyomo/contrib/solver/solvers/gms_sol_reader.py

+                'Solution loader does not currently have valid duals. Please '
+                'check results.termination_condition and/or results.solution_status.'


You don't need the text: https://github.com/Pyomo/pyomo/blob/main/pyomo/contrib/solver/common/util.py#L41

mrmundt · 2025-09-09T20:14:34Z

pyomo/contrib/solver/solvers/gms_sol_reader.py

+            raise NoReducedCostsError(
+                'Solution loader does not currently have valid reduced costs. Please '
+                'check results.termination_condition and/or results.solution_status.'
+            )
+        if self._gdx_data is None:
+            raise NoReducedCostsError(
+                'Solution loader does not currently have valid reduced costs. Please '
+                'check results.termination_condition and/or results.solution_status.'
+            )


You don't need the text: https://github.com/Pyomo/pyomo/blob/main/pyomo/contrib/solver/common/util.py#L48

Added new GAMS solver interface, writer, and solution loader. WIP: un…

1177492

…ittest and coverage test

Norman_Tran and others added 2 commits August 7, 2025 14:14

Ran black -S -C on commited files

2385654

Typo!

b36086f

mrmundt requested changes Aug 7, 2025

View reviewed changes

Norman_Tran and others added 14 commits August 7, 2025 17:02

Added ignore checking for constant in the objective function during w…

e91e480

…riting process. Added WIP unittest for the new solver interface

Added support for add_options for multiple solver_options

b060096

typo

96ba646

ran black formatter

b0f16a3

added add_options in the solver_interface

dde656c

Fixed bug with unable to create tmpdir. Updated test_gams_v2.py

3476f72

Merge branch 'main' into GAMS_new_solver_interface

ddfd98c

Added handling single bounded variable case (None,#) or (#, None)

eb0a3db

Match solver interface and writer configs to prevent implicit definit…

c6d2d01

…ion error

Added new gams solver interface for test_all_solvers_list()

2a6dcb7

Fixed bug of handling valid ub when writing pyomo expression

b6ead27

Fixed bug when using ComponentMap to extract duals in gms_sol_reader.py

89f80e0

Corrected handling of solution_loader. Implemented bounded constraint…

b0c3450

… and writing to gms.

Added offdigit based on error log message suggestion

a9df1d7

boxblox reviewed Aug 14, 2025

View reviewed changes

mrmundt and others added 4 commits August 15, 2025 07:47

Merge branch 'main' into GAMS_new_solver_interface

e04c89f

Fixed bug of extracting constraint dual when alias exits (for bounded…

518758e

… constraint). Fixed bug of load_solution that contains constraint id in addition to variable id

Added aliases as part of parse_dat_results to be consistent with pars…

a2ef9cc

…e_gdx_results

Removed code duplication between the solver interface and writer by p…

c33095d

…arsing solver_options

jsiirola reviewed Aug 18, 2025

View reviewed changes

Norman_Tran and others added 4 commits August 19, 2025 14:31

Revised code based on John's review. WIP: Handling rehash within avai…

f609ca2

…lable() and version()

Re-added config into available and version, and deprecated test that …

ec5aa6a

…are no longer valid.

Merge branch 'main' into GAMS_new_solver_interface

e760a1c

Sanity check config.writer_config.put_results_format without setdefault

7bb817b

Norman_Tran and others added 10 commits August 19, 2025 22:53

Allow user to modify writer config via solver_options. Fixed bug of s…

72beeaf

…etting default to gdx that caused failed test. Implemented raise infeasible

Merge branch 'Pyomo:main' into GAMS_new_solver_interface

3dc83ca

Corrected exeception type raised when solver return infeasible. Added…

5c87b75

… rehash handling with Executable.rehash()

Corrected test_gams_v2.py to match error raised. Corrected error mess…

d6aa9ff

…age for infeasible model when calling load_var, dual, reduced_cost

Merge branch 'Pyomo:main' into GAMS_new_solver_interface

aba0eed

Prevent rewrite of termination condition in modelstat if the value is…

f85d217

… not none

Added an gams_solver_options (handled via solver_option in solve). Ad…

0e94742

…ded handling solvestat and modelstat when solver is interrupted

Divided solver and model termination condition into separate config.

9d5758e

Merge branch 'Pyomo:main' into GAMS_new_solver_interface

d1285e3

Added unittest skipif for linux_3.9_slim and linux_3.12_numpy failed …

2973d9c

…imports

AnhTran01 requested a review from mrmundt August 22, 2025 18:54

mrmundt requested changes Sep 9, 2025

View reviewed changes

	filename = 'GAMS_MODEL' + ".gms"
	filename = model.name + ".gms"

		'Solution loader does not currently have a valid solution. Please '
		'check results.termination_condition and/or results.solution_status.'

New GAMS solver interface, writer, and solution loader. #3683

Are you sure you want to change the base?

New GAMS solver interface, writer, and solution loader. #3683

Conversation

AnhTran01 commented Aug 7, 2025

Fixes NA

Summary/Motivation:

Changes proposed in this PR:

Legal Acknowledgement

Uh oh!

AnhTran01 commented Aug 7, 2025

Uh oh!

mrmundt commented Aug 7, 2025

Uh oh!

mrmundt left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jsiirola left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

AnhTran01 commented Aug 22, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!