Intermittent CI test failures in Matlab toolbox

[speth]

Problem description

Various recent CI jobs testing the Matlab toolbox have reported intermittent errors on some tests that are expected to raise exceptions, where retrieving the error message either returns "" or an incorrect error message.

The logic around CLib's error handling is pretty robust, but it is not thread safe, since it relies on a global cache of error messages stored in the Application object. However, I don't think that should be a problem here -- while Matlab has an option to run tests in parallel, that isn't explicitly enabled and I don't think it should be the case by default. So I'm a little confused as to what's hapening

Steps to reproduce

1. Trigger CI
2. View Logs

Behavior

A couple of examples:

  Verification failed in ctTestUndeclared/testRaiseUndeclaredSpecies.
      ---------------------
      Framework Diagnostic:
      ---------------------
      verifySubstring failed.
      --> The value does not contain the substring.
      
      Actual char:
          <no Cantera error>                                                    
      Expected Substring:
          contains undeclared species
      ------------------
      Stack Information:
      ------------------
      In /Users/runner/work/cantera/cantera/test/matlab/ctTestUndeclared.m (ctTestUndeclared.testRaiseUndeclaredSpecies) at 18

or

Verification failed in ctTestPureFluid/testWaterIPAWS.
      ---------------------
      Framework Diagnostic:
      ---------------------
      verifySubstring failed.
      --> The value does not contain the substring.
      
      Actual char:
          
          *******************************************************************************
          CanteraError thrown by Substance::Tsat:
          No convergence: p = 609.5464084855828
          *******************************************************************************
          
      Expected Substring:
          assumes liquid phase
      ------------------
      Stack Information:
      ------------------
      In /Users/runner/work/cantera/cantera/test/matlab/ctTestPureFluid.m (ctTestPureFluid.testWaterIPAWS) at 399

I've seen errors for several other test methods. So far, I have only seen this on macOS runners (macos-14 and macos-15) with both MATLAB R2024b and "latest".
Update: I've now seen this error on Windows for recent CI runs on the main branch.

I've run the test suite many times locally, on macOS using R2025b, and have not been able to replicate this error.

System information

• Cantera version: main branch and various PR branches
• OS: macOS
• MATLAB R2024b, MATLAB R2025b
• Also seen on "MATLAB R2024a on windows-2025"

[ischoegl]

I've noticed the same thing. Failures appear to happen at random, which is the frustrating part. The frustration is compounded by my inability to get any stdout etc. output to work even for inprocess mode. On macOS/Matlab R2025a myself. And yes, I've also not seen any local errors.

[speth]

The frustration is compounded by my inability to get any stdout etc. output to work even for inprocess mode. On macOS/Matlab R2025a myself.

Here at least there is some hope: If you run Matlab from the command line, all the normal output to stdout appears as expected. The problem just stems from the fact that the Matlab command window isn't the same as stdout. Just start with something like /Applications/MATLAB_R2025b.app/bin/matlab -nodisplay. For me, this works with Cantera loaded using both inprocess and outofprocess modes.