Learning Curve, and Applying Aviary to Low Speed, Low Weight Aircraft.

[will-whelan]

TL;DR: Struggling to determine where my poorly posed inputs are originating in the context of working toward a small, low-speed aircraft input deck that can be used to run multi-variable optimizations.

---

Hello!

I'm attempting to learn Aviary and apply it to smaller scale problems, think 4-seat general aviation with cruise speeds in the 120–200 KTAS range.
I've written simple optimization programs in the past, and I'm interested in picking up Aviary to decrease the amount of "plumbing" code in the future.

I appreciate any help, documentation, or critique anyone is able to offer.

This post serves two purposes:

1. A request for help. Looking for any pieces of documentation that I've overlooked, assumptions that are obviously wrong, critiques of my apparent illiteracy, etc.
2. Writing out the context and questions has already been helpful as a process. When I'm fully spun up, I hope to use some of this to contribute to the project documentation.

I've included a fair amount of context at the end, but questions first

---

Questions

1. Tracing outputs:
   I need to improve my ability to trace back solution outputs (drag, fuel burn, thrust, etc.) through their sequence to some initial set of variables.
   Is there a tool to do this? Maybe something as simple as finding the final object that holds the relevant data and using standard debug tools?

   • Example: Drag comes from some buildup off of FLOPS/GASP empirical methods. Is there a way to export CDo and CDi for the various components?
   • Fuel burn should come from the engine deck based on throttle/power/thrust requests, but I’ve seen negative fuel burn show up in a handful of runs if my inputs are poorly formed (e.g. design range exceeds possible range, drag exceeds possible thrust, etc.), and it’s been guess-and-check to try and fix that.

2. Meaning of "ref" value:
   What is the physical meaning of the ref value for the various mission objectives?

   • Docs: https://openmdao.github.io/Aviary/getting_started/onboarding_level2.html

     “Here, ref is the reference value. For different objectives, the range may vary significantly. We want to normalize the value. Ideally, users should choose ref such that the objective is in the range of (0,1). This is required by optimizer.”

   • But the default ref for Mission.Objectives.Fuel is 1e4?
     Is it a scaling factor? Some goal value?

3. Time Bounds for Phases
   I've modified the advanced_single_aisle_FLOPS deck to represent a smaller, lighter aircraft. (attached, at the end of the post)

• Currently seeing optimization failures. (See attached files)
• Objective is set to fuel, and my expectation is that it should optimize throttle setting and airspeed to minimize total mission fuel consumption.
• Dashboard > Results > Trajectory Results seem rational, but the optimization is failing.
• I think it’s because I’m hitting the upper bound of traj.descent.t_duration.
  I’m not following why Model > Driver Scaling is showing 300s, when phase_info has:

  'time_initial_bounds': ((200, 800), 'min'),
  'time_duration_bounds': ((5.0, 87.0), 'min'),

  • 
  • 

  • I'm also not following why these are needed in the first place. Time bounds seems to be over defining the problem given the power/energy requirements already outlined by the vehicle, engine, and mission definitions.

---

Context

My workflow/learning process has been as follows: start simple and slowly build up complexity.

---

1. Start with examples

I've started with run_level2_example.py as I'm comfortable with Python.
My goal is to understand the input files and dashboard/report outputs. I'm hoping that through understanding these outputs, I can hone in on section 3's optimization failure.

• Input data seems relatively straightforward.

• Looking for some sort of "Performance" summary; derived CDo/CDi from chosen method (if applicable), fully loaded climb rates, initial computed range/fuel (depending on input data).

• Basically: what does the optimizer start with? What's the most effective way to tell if the problem is poorly formed? (e.g. negative climb rate from insufficient power, not enough fuel volume to hit range, etc.)

  • Is this "Pre-Mission" data? Is it writable?
  • Results > Trajectory Results gets close.
  • https://openmdao.github.io/Aviary/user_guide/pre_mission_and_mission.html
  • https://openmdao.org/newdocs/versions/latest/other_useful_docs/om_command.html#om-command-scaling
  • Maybe digging through the Jacobian inputs?

• Optimization data is throwing me off:

  • Dashboard > Optimization > Summary:
    • mission:objective:fuel reports out as 1.3233458 with ref = 1.
      • Is this just a scaled version of mission:design:fuel_mass (13233.46 from Reports > Subsystems > core_mass.md)? fuel_mass * 10^-4?
    • mission:design:gross_mass is reading out as 0.662. Units are lbm, which is confusing.
    • Similar confusion around Dashboard > Model > Driver Scaling.
      • https://openmdao.github.io/Aviary/getting_started/onboarding_level1.html
      • Driver = Model * scaler
      • Is the scaler the response wrt design variables? IE: its a measure of the design variables impact on the objective we are optimizing for?
    • Are the values in Optimization > Summary expressed as Model * scaler? And is the scaler the same as in Driver Scaling?

---

2. Build a Recip EngineDeck

Represents something approximating a 180 HP recip engine with a fixed-pitch prop.

• This is complete. I will attempt to utilize the Hamilton Prop module when I'm feeling more confident in my knowledge base.

---

3. Modify an existing input deck

Modify one of the existing input decks to hit a "rational" set of weight and drag numbers that can use the EngineDeck.

• See Question 3.

4. Build an input deck from scratch

Represent a Cessna 172 and compare results to POH numbers, modifying the deck as necessary.

• The idea here is to understand how FLOPS and GASP handle smaller aircraft models (drag, weight, and systems).

---

Problems I've Encountered

• Exponential overflow from landing_mass.py (LandingTakeoffMassRatio class, line 27) when Mach numbers drop below ~0.2:

  cruise_factor = 5e-5 / (1.0 + np.exp(-1000 * (cruise_mach - 1))) + 4e-5

  Should just rapidly approach 4e-5, so I don’t expect this to introduce real problems.

• Time bounds for phases sometimes expected to be in 'ft'?
  I’ve seen discussion mentioning this relates to the 2DOF equations of motion solution, but it’s popped up in some height_energy tests I’ve run.

  • see twodof_phase.py
  • Will dig into this in more detail when it comes up again.

asaf_5000lbm_mod.csv
EngineDeck_180HP_NA.csv
ModifiedSingleIsle.py

[Kenneth-T-Moore]

Hey, will-whelan. Thank you for the detailed post; there are a lot of good suggestions here. We definitely have some work to do to get Aviary to more useful tool, and we appreciate feedback from users. I've got some answers for the main questions you asked, and below that, I have a couple of things I changed in your model to get a good converged solution.

1. tracing

This is an interesting idea for a graphical tool, where you select a variable and it traces back everything that contributed to it. It would get more involved once you have solver loops though.

When I want to trace something back, I find prob.model.list_vars(units=True, print_arrays=True) to be helpful. It provides a hierarchical listing of all inputs/outputs in component execution order.

2. ref value

The "ref" is a value used to scale the quantity for the optimizer. Ideally, you want the design variables and constraints to operate in a range of [0, 1] or [-1, 1]. The 'ref' value is the value that will be mapped to 1, so if you have a 125000 lbm aircraft, a ref of 1.25e5 would be appropriate. There are some default refs that aren't ideal, and there are still even some hard-coded refs (and upper/lower bounds) that haven't been added to the user-facing API, so don't be afraid to adjust those when needed.

Note that whether and how you scale is optimizer dependent. When using SNOPT, scaling makes a big difference. When using IPOPT, it isn't as important because IPOPT auto-scales the problem (and has some options to control that).

3. time bounds

I somewhat agree about the time_bounds not being "needed", but it will depend on the optimizer. IPOPT performs better if every design variable has a reasonable upper and lower bound, but other optimizers are fine without them. You can certainly set it much higher.

One thing about 'ft' as upper/lower bounds. There are some phases where distance is used as the integration variable for the differential equations rather than time. However, you should only see this if you are using "SOLVED_2DOF" for your settings:equations_of_motion, which would really only be needed for detailed take-off or detailed landing. You should probably be using either TWO_DEGREES_OF_FREEDOM or HEIGHT_ENERGY.

---

Thank you for providing all of the files to run your case. I download them and played around (though I did switch over to SNOPT) and noticed that the mass constraint was the one that was still violated when it terminated.

mass_constraint
  initial_mass                                                         [2766.51643915]           input   lbm          mission:summary:gross_mass
  operating_empty_mass                                                 [1785.86557737]           input   lbm          aircraft:design:operating_mass
  overall_fuel                                                         [142.58981996]            input   lbm          mission:summary:total_fuel_mass
  payload_mass                                                         [880.]                    input   lbm          aircraft:crew_and_payload:total_payload_mass
  mass_resid                                                           [41.93895818]             output  lbm          mission:constraints:mass_residual

The optimizer has control of the gross_mass, so it ought to be able to just increase it here, but something is preventing it. The time constraints you mentioned weren't the culprit (and I did relax them just in case.) One common problem is the engine isn't powerful enough to fly the mission, or alternatively, the drag is too high. You can experiment with the drag by adjusting the aircraft:design:subsonic_drag_coeff_factor in the csv file. Starting value was 0.4, so I dropped it to 0.35 and the optimization converged. Subsequently, I was able to bump it back up to 0.36. (0.38 did not converge).

Note, I also fixed the following, which looked like a mistake:

        'mach_initial': (0.3, 'unitless'),
        'mach_final': (0.4, 'unitless'),
        'mach_bounds': ((0.1, 0.3), 'unitless'),

I changed "mach_final" to 0.1, since I think that is what you meant.

So yeah, I think the drag here is the problem, which comes from some old correlations for large transport aircraft, and some of the tables are probably in extrapolation mode for these flight conditions/geometry. For your small plane, you might want to consider adding a drag polar that is more representative.

[will-whelan]

Hi Kenneth,

Thank you for the detailed response! I am deeply appreciative of the time you've taken to help and run the problem.

Was there anything in particular that tipped off drag/power imbalance as the culprit, or just prior experience? I was keeping an eye on the mission trajectory for any obvious errors, but nothing caught my eye. (Maybe a throttle setting of 1.04 at the beginning of the climb phase now that I'm looking closer at the failed runs?)

And thank you for the prob.model.list_vars() tip, that is helpful

[Kenneth-T-Moore]

Yeah, it was just prior experience. I wish I could find some telltale sign for when thrust is too low or drag too high, or maybe some kind of test we could do outside of the time-integration.

[EdoAlvarezR]

From my experience, 40% of the time that my optimization is not converging is because of scaling/tolerancing, 10% is a typo in input parameters, and the other 50% is because I have defined an impossible problem without realizing it (in your case, using an engine that can't propel the aircraft to flight speeds) ¯\_(ツ)_/¯

[jkirk5]

The thrust/drag convergence issue is pretty common in my experience. Legacy tools like FLOPS scale the engine up/down as needed by default, which Aviary currently does not. I've thought about making engine scaling (Aircraft.Engine.SCALE_FACTOR) a design variable by default to reduce how common this problem is, but I have been resisting adding too many design variables and constraints into Aviary by default because it is harder for new users to understand what is happening in an Aviary problem.

[jkirk5]

Engine scaling by default is very useful when designing a new configuration, but would be less helpful when you are trying to model an existing aircraft or at least one that is using a fixed/known powerplant

[EdoAlvarezR]

That's a great tip (adding Aircraft.Engine.SCALE_FACTOR as a design variable) for debugging an accidentally-infeasible problem.