Locating a library from an application build script

[t00sa]

When the class is in fab, this is not a problem. In LFRic apps, it needs to know where the classes are in core. This isn't a fab problem - it's a more general python problem related to library search paths.

[hiker]

Copying the discussion from MetOffice/lfric-baf#84

We agreed that apps should get its own base class (which is responsible for managing a command line option to indicate where core is). Trying to implement this, I ran into a few issues that I would want to clarify

[hiker]

We agreed in the telco on Thursdayto add a new base class to the apps repo, which is the base class for all lfric_apps scripts, and is responsible for modifying Python's sys.path to be able to import the base class of the core repository using a command line option. I have been trying to find various approaches, and tbh, all of them have disadvantages (including that @MatthewHambley might the solution still consider the solution to be using a wrapper script). Here what I have tried:

1. Adding an LFRicAppBase class is somewhat forward. Main issue is that the LFRicAppsBase class needs to import the LFRicBase class from core (which will be renamed to LFRicCoreBase of course). But as far as I can see it will trigger a flake8 error. Here my approach:

import sys
# Parse command line for --core command line option to find the location of lfric_core
lfric_core = args.core     # whatever
sys.path.append(lfric_core)

from lfric_base import LFRicBase    # requires the above modification of sys.path

class LFRicAppsBase(LFRicBase):

This triggers module level import not at top of file from flake8. I believe we could work around the linter warning by using importlib ... but that basically adds a lot of 'stuffing around' to work around a linter error (which contradicts the whole idea of a linter making the code more readable imho). Disable the linter warning?

Or am I missing a better solution here? Feedback really appreciated.

2. The next problem I have is: how can an application script find the LFRicAppsBase class? The apps base class would be in ../../build/lfric_apps_base ... but I can't use relative imports (ImportError: attempted relative import with no known parent package). I am not 100% familiar with pythonb package management, but adding __init__.py files doesn't help. We would have to turn lfric_apps into a proper python package in order to do that (then we could use from lfric_apps.build.lfric_apps_base import LFRicAppsBase)? I found some references to less known double-underscore methods (__module__), but that feels all just wrong.

Scott Wales had a different approach, using a top-level cli.py script, which would take the class to be called for building as paramter, e.g.:

../../cli.py --core path_to_lfric_core gungho ...

But I assume you would consider this as a wrapper script as well?

Could someone give me some details how you want this implemented? I.e. a set of dummy files, e,g.:

• lfric_core/lfric_core_base.py # containing LFRicCoreBase
• lfric_apps/build/lfric_apps_base.py # class inheriting from LFRicCoreBase
• lfric_apps/applications/gungho/fab_gungho.py # class inheriting from LFRicAppsBase

so that you can use ./fab_gungho.py to work. I think once someone explains to me how to do this, I can sort out the rest.

Thanks!