f32

[songololo]

#22

Makes use of f32 for reading, writing, internal arrays.

[biglimp]

Hi Gareth, can you please to clarify how these changes affect our code in UMEP for QGIS. As far as we can see, you have changed e.g. solweig_run.py. Have you tested this in QGIS environment? I think maybe we have missed this from a previous PR. Why are you making changes in dailyshading.py as this is not used in SOLWEIG? Please clarify?

[songololo]

Hi @biglimp - for this pull request I tried to change all instances of saving rasters to use f32 by default. I searched for all instances of common.save_raster lines where I added coerce_f64_to_f32=True which is a new setting to cast f64 arrays to f32 when saving. If preferred these can be set to False to leave as f64.

The changes to dailyshading.py were to the common.save_raster lines. When saving the file on my vscode it auto formats using the recommended ruff formatter, so the other changed lines are to do with formatting; e.g. ((vegdem2 * vegdem)) became (vegdem2 * vegdem) and int(0) became 0. It also automatically wraps or unwraps lines according to the formatting spec.

The changes to Solweig_run.py are similar (coerce_f64_to_f32=True).

What would be good is to find a way to test the common.py file using some automatic tests since this file needs to automatically detect whether it is running with access to rasterio or not, in which case we assume it is using gdal from QGIS. I'm not sure how to simulate a QGIS environment for testing but if anyone has ideas it would be nice to get some tests setup so that they automatically run when creating a pull request.

Another nice change would be to separate old code into a separate directory and to then auto format all current code using ruff which will help catch some potential errors like the f-string errors that pop up from time to time.

[biglimp]

Hi. Thanks for this info. I think you point towards the issue that we cannot control. There is no good way to test within a QGIS environment without doing it manually.

On another note, @nilswallenberg and I are discussing to create a separate pypi for the solweig model consisting of only the model itself, without any other workflow tools such as SVF-calculator. Then we can all use that pypi module to run the actual model. Do you see any issues with this?

[songololo]

@biglimp apologies for the slow response.

At the moment umep-core is designed to work either from Python or from QGIS (not tested at this point).

• SolweigRun - contains neutral logic
• SolweigRunCore - subclasses SolweigRun to implement vanilla Python logic
• SolweigRunQgis - subclasses SolweigRun to implement QGIS logic

For this to work, the common.py module is intended to check whether running in QGIS or not and to execute reading and writing operations accordingly (GDAL vs rasterio).

Personally I would recommend to keep SOLWEIG and other core functions for SVF and shadows together so that it isn't necessary to install and manage multiple packages. Though it would be easier to maintain if old or dead code is removed (or moved to a deprecated folder).

[biglimp]

@biglimp apologies for the slow response.

At the moment umep-core is designed to work either from Python or from QGIS (not tested at this point).

• SolweigRun - contains neutral logic
• SolweigRunCore - subclasses SolweigRun to implement vanilla Python logic
• SolweigRunQgis - subclasses SolweigRun to implement QGIS logic

For this to work, the common.py module is intended to check whether running in QGIS or not and to execute reading and writing operations accordingly (GDAL vs rasterio).

Personally I would recommend to keep SOLWEIG and other core functions for SVF and shadows together so that it isn't necessary to install and manage multiple packages. Though it would be easier to maintain if old or dead code is removed (or moved to a deprecated folder).

Sorry for late reply. I see your point but nevertheless, SVF and wall algorithm are separate tools that users make use of for many other applications a part from SOLWEIG-calculation. SOLWEIG is a separate model and therefore, it would be better to maintain this separately.

[biglimp]

Pulled the f32 branch and tried in the osgeo env. Got the following error:

C:\Users\xlinfr\Documents\PythonScripts\umep-core>C:/OSGeo4W/bin/python.exe c:/Users/xlinfr/Documents/PythonScripts/umep-core/demos/solweig_gbg_test.py
Traceback (most recent call last):
  File "C:\Users\xlinfr\Documents\PythonScripts\umep-core\umep\common.py", line 11, in <module>       
    import rasterio
  File "C:\OSGeo4W\apps\Python312\Lib\site-packages\rasterio\__init__.py", line 25, in <module>       
    from rasterio._base import DatasetBase
ImportError: DLL load failed while importing _base: The specified procedure could not be found.       

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\OSGeo4W\apps\Python312\Lib\site-packages\osgeo\__init__.py", line 30, in swig_import_helper
    return importlib.import_module(mname)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\OSGeo4W\apps\Python312\Lib\importlib\__init__.py", line 90, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen importlib._bootstrap>", line 1387, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1360, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1331, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 921, in _load_unlocked
  File "<frozen importlib._bootstrap>", line 813, in module_from_spec
  File "<frozen importlib._bootstrap_external>", line 1293, in create_module
  File "<frozen importlib._bootstrap>", line 488, in _call_with_frames_removed
ImportError: DLL load failed while importing _gdal: The specified procedure could not be found.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "c:\Users\xlinfr\Documents\PythonScripts\umep-core\demos\solweig_gbg_test.py", line 4, in <module>
    from umep import (
  File "C:\Users\xlinfr\Documents\PythonScripts\umep-core\umep\skyviewfactor_algorithm.py", line 53, in <module>
    from umep import class_configs, common
  File "C:\Users\xlinfr\Documents\PythonScripts\umep-core\umep\class_configs.py", line 19, in <module>
    from . import common
  File "C:\Users\xlinfr\Documents\PythonScripts\umep-core\umep\common.py", line 21, in <module>
    from osgeo import gdal
  File "C:\OSGeo4W\apps\Python312\Lib\site-packages\osgeo\__init__.py", line 35, in <module>
    _gdal = swig_import_helper()
            ^^^^^^^^^^^^^^^^^^^^
  File "C:\OSGeo4W\apps\Python312\Lib\site-packages\osgeo\__init__.py", line 32, in swig_import_helper
    return importlib.import_module('_gdal')
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\OSGeo4W\apps\Python312\Lib\importlib\__init__.py", line 90, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ModuleNotFoundError: No module named '_gdal'

C:\Users\xlinfr\Documents\PythonScripts\umep-core>

Here are the python packages installed (python 3.12.9):

C:\Users\xlinfr\Documents\PythonScripts\umep-core>pip list
Package                       Version
----------------------------- -------------
affine                        2.4.0
alabaster                     0.7.16
annotated-types               0.7.0
anyio                         4.3.0
appdirs                       1.4.4
argon2-cffi                   23.1.0
argon2-cffi-bindings          21.2.0
asteval                       1.0.0
asttokens                     2.4.1
async-lru                     2.0.4
atmosp                        0.2.9
attrs                         23.2.0
Babel                         2.14.0
beautifulsoup4                4.12.3
bleach                        6.1.0
cads-api-client               1.0.3
cdsapi                        0.7.6
certifi                       2024.2.2
cffi                          1.16.0
cftime                        1.6.4
chardet                       5.2.0
charset-normalizer            3.3.2
click                         8.1.7
click-plugins                 1.1.1
cligj                         0.7.2
cloudpickle                   3.0.0
colorama                      0.4.6
comm                          0.2.2
contourpy                     1.2.1
coverage                      7.4.4
crc32c                        2.7.1
cycler                        0.12.1
dask                          2024.6.2
datapi                        0.1.1
debugpy                       1.8.5
decorator                     5.1.1
defusedxml                    0.7.1
dill                          0.3.8
docutils                      0.20.1
donfig                        0.8.1.post1
ecmwf-datastores-client       0.1.0
et_xmlfile                    1.1.0
executing                     2.0.1
ExifRead                      3.0.0
f90nml                        1.4.4
f90wrap                       0.2.16
fastjsonschema                2.19.1
fiona                         1.10.1
flexcache                     0.3
flexparser                    0.3.1
fonttools                     4.51.0
fsspec                        2024.6.1
future                        1.0.0
GDAL                          3.10.2
geographiclib                 2.0
geopandas                     1.0.1
h11                           0.14.0
h3                            4.3.1
h5netcdf                      1.6.1
h5py                          3.11.0
httpcore                      1.0.5
httplib2                      0.22.0
httpx                         0.27.0
idna                          3.7
imagesize                     1.4.1
importlib_metadata            7.1.0
ipykernel                     6.29.5
ipython                       8.26.0
ipywidgets                    8.1.2
JayDeBeApi                    1.2.3
jedi                          0.19.1
Jinja2                        3.1.3
joblib                        1.4.0
JPype1                        1.5.0
json5                         0.9.25
jsonschema                    4.21.1
jsonschema-specifications     2023.12.1
jupyter                       1.1.1
jupyter_client                8.6.2
jupyter-console               6.6.3
jupyter_core                  5.7.2
jupyter-events                0.10.0
jupyter-lsp                   2.2.5
jupyter_server                2.14.0
jupyter_server_terminals      0.5.3
jupyterlab                    4.1.5
jupyterlab_pygments           0.3.0
jupyterlab_server             2.26.0
jupyterlab_widgets            3.0.10
kiwisolver                    1.4.5
linecache2                    1.0.0
llvmlite                      0.42.0
lmfit                         1.3.1
locket                        1.0.0
lxml                          5.3.0
markdown-it-py                4.0.0
MarkupSafe                    2.1.5
matplotlib                    3.10.0
matplotlib-inline             0.1.7
mdurl                         0.1.2
mistune                       3.0.2
mock                          5.1.0
multiprocess                  0.70.16
multiurl                      0.3.3
munch                         4.0.0
nbclient                      0.10.0
nbconvert                     7.16.3
nbformat                      5.10.4
nest-asyncio                  1.6.0
netCDF4                       1.7.1.post1
networkx                      3.3
nose                          1.3.7
nose2                         0.14.1
notebook                      7.1.2
notebook_shim                 0.2.4
numba                         0.59.0
numcodecs                     0.16.0
numdifftools                  0.9.41
numexpr                       2.10.1
numpy                         1.26.2
odfpy                         1.4.1
oldest-supported-numpy        2023.12.21
openpyxl                      3.1.2
overrides                     7.7.0
OWSLib                        0.32.0
packaging                     24.0
pandas                        2.2.2
pandocfilters                 1.5.1
parso                         0.8.4
partd                         1.4.2
pb-tool                       3.1.0
pillow                        11.1.0
Pint                          0.24.1
pip                           25.2
platformdirs                  4.2.2
Platypus-Opt                  1.0.4
plotly                        5.20.0
ply                           3.11
pooch                         1.8.2
prometheus_client             0.20.0
prompt_toolkit                3.0.47
psutil                        6.0.0
psycopg2                      2.9.10
pure_eval                     0.2.3
pvlib                         0.11.0
pyarrow                       22.0.0
pycparser                     2.22
pydantic                      2.11.7
pydantic_core                 2.33.2
Pygments                      2.17.2
pyodbc                        5.1.0
pyogrio                       0.9.0
PyOpenGL                      3.1.7
pyparsing                     3.1.2
PyPDF2                        3.0.1
pypiwin32                     223
pyproj                        3.7.0
PyQt5                         5.15.11
PyQt5_sip                     12.16.1
PyQtWebEngine                 5.15.7
pyserial                      3.5
python-dateutil               2.9.0.post0
python-json-logger            2.0.7
pytz                          2024.1
pywin32                       306
pywinpty                      2.0.13
PyYAML                        6.0.1
pyzmq                         26.2.0
qtconsole                     5.6.1
QtPy                          2.4.1
rasterio                      1.4.3
referencing                   0.34.0
remotior_sensus               0.4.4
reportlab                     4.2.5
requests                      2.31.0
retrying                      1.3.4
rfc3339_validator             0.1.4
rfc3986_validator             0.1.1
rich                          14.2.0
rioxarray                     0.18.2
rpds-py                       0.18.0
ruff                          0.12.8
scikit-learn                  1.4.2
scipy                         1.13.0
seaborn                       0.13.2
Send2Trash                    1.8.3
setuptools                    75.3.0
shapely                       2.0.6
simplejson                    3.19.2
sip                           6.9.1
six                           1.16.0
sniffio                       1.3.1
snowballstemmer               2.2.0
soupsieve                     2.5
Sphinx                        7.3.7
sphinx-rtd-theme              2.0.0
sphinxcontrib-applehelp       1.0.8
sphinxcontrib-devhelp         1.0.6
sphinxcontrib-htmlhelp        2.0.5
sphinxcontrib-jquery          4.1
sphinxcontrib-jsmath          1.0.1
sphinxcontrib-qthelp          1.0.7
sphinxcontrib-serializinghtml 1.1.10
stack-data                    0.6.3
supy                          2025.11.20rc1
target-py                     0.1.1
tenacity                      8.2.3
terminado                     0.18.1
threadpoolctl                 3.4.0
timezonefinder                6.5.9
tinycss2                      1.2.1
toml                          0.10.2
toolz                         0.12.1
tornado                       6.4.1
tqdm                          4.66.4
traceback2                    1.4.0
traitlets                     5.14.3
typing_extensions             4.14.0
typing-inspection             0.4.1
tzdata                        2025.2
tzfpy                         1.1.0
uncertainties                 3.2.1
unittest2                     1.1.0
urllib3                       2.2.1
uv                            0.6.8
uwg                           5.8.13
wcwidth                       0.2.13
webencodings                  0.5.1
websocket-client              1.7.0
widgetsnbextension            4.0.10
wxPython                      4.2.2
xarray                        2025.3.0
xlrd                          2.0.1
xlwt                          1.3.0
zarr                          3.0.8
zipp                          3.18.1

Am I missing something?

[biglimp]

Just made a simple test so that it was not my VSCode who did something strange. the code below:

import sys, os, rasterio
print("PYTHON:", sys.executable)
print("RASTERIO:", os.path.dirname(rasterio.__file__))
print("PATH:", os.environ["PATH"].split(";"))

gives below with no errors:

C:\Users\xlinfr\Documents\PythonScripts\umep-core>C:/OSGeo4W/bin/python.exe c:/Users/xlinfr/Documents/PythonScripts/umep-core/umep/importtest.py
PYTHON: C:\OSGeo4W\bin\python.exe
RASTERIO: C:\OSGeo4W\apps\Python312\Lib\site-packages\rasterio
PATH: ['C:\\OSGeo4W\\apps\\qt5\\bin', 'C:\\OSGeo4W\\apps\\Python312\\Scripts', 'C:\\OSGeo4W\\bin', 'C:\\windows\\system32', 'C:\\windows', 'C:\\windows\\system32\\WBem', 'C:\\OSGeo4W\\apps\\qgis\\bin', 'C:\\OSGeo4W\\apps\\Qt5\\bin', 'C:\\OSGeo4W\\apps\\Python312\\Scripts', 'c:\\Users\\xlinfr\\.vscode\\extensions\\ms-python.debugpy-2025.16.0-win32-x64\\bundled\\scripts\\noConfigScripts']

[songololo]

Thanks for the detailed report and for testing this. [EDIT - updated below in the later response.]

The suspected issue

OSGeo4W includes its own GDAL installation. When rasterio is pip-installed into that environment, it brings bundled GDAL DLLs that conflict with OSGeo4W's version. It might have been that the standalone rasterio test worked because it loaded before the conflict was triggered, but umep's fallback logic exposes the incompatibility.

Solution

Ideally, use a separate virtual environment instead of the OSGeo4W Python:

cd C:\Users\<path>\Documents\PythonScripts\umep-core

uv venv --python 3.12
.venv\Scripts\activate
uv pip install -e .

python demos/solweig_gbg_test.py

Or with standard venv:

python -m venv .venv
.venv\Scripts\activate
pip install -e .

Alternative: Conda

conda-forge maintains compatible versions of the geospatial stack:

conda create -n umep -c conda-forge python=3.12 rasterio geopandas pyproj shapely
conda activate umep
pip install -e .

Diagnostic tool

In the more recent tiler branch I've added umep-doctor to help identify environment issues. After pulling the latest:

umep-doctor

Let me know if you run into anything else.

[songololo]

Sorry for late reply. I see your point but nevertheless, SVF and wall algorithm are separate tools that users make use of for many other applications a part from SOLWEIG-calculation. SOLWEIG is a separate model and therefore, it would be better to maintain this separately.

Hi @biglimp - Happy New Year. Sorry I've been quiet.

Shall I go ahead and create a new separate repository to split out the Solweig logic?

This would make it easier to maintain if we can do a fresh, clean start.

I would ideally also like to simplify the function calls with a new logic, but would like to also keep it compatible with any other wrappers, e.g. QGIS that might rely on it. Please let me know if you have any thoughts.

[songololo]

Pulled the f32 branch and tried in the osgeo env. Got the following error:

Hi @biglimp - I took another look at this assuming you'd like to run it directly in the OSGEO4W environment: The common.py file in the tiler branch now has additional logic which will try to make an OSGEO4W environment work even if there are some conflicting package installs like rasterio (which has conflicting dlls).

Could you please give this a go and let me know if it works for you? (The latest changes are in the tiler branch which is ahead of the f32 branch.)

[biglimp]

Sorry for late reply. I see your point but nevertheless, SVF and wall algorithm are separate tools that users make use of for many other applications a part from SOLWEIG-calculation. SOLWEIG is a separate model and therefore, it would be better to maintain this separately.

Hi @biglimp - Happy New Year. Sorry I've been quiet.

Shall I go ahead and create a new separate repository to split out the Solweig logic?

This would make it easier to maintain if we can do a fresh, clean start.

I would ideally also like to simplify the function calls with a new logic, but would like to also keep it compatible with any other wrappers, e.g. QGIS that might rely on it. Please let me know if you have any thoughts.

Hi. Yes, that would be good. This will make it much easier to incorporate changes in the model, separate from e.g. Walls algorithms etc. @nilswallenberg , and me are also in contact with a group in the US who have made a GPU version of the code that we would like to include. Nils, could you add the link to the repo that has this code or is it non-public still?

[biglimp]

Pulled the f32 branch and tried in the osgeo env. Got the following error:

Hi @biglimp - I took another look at this assuming you'd like to run it directly in the OSGEO4W environment: The common.py file in the tiler branch now has additional logic which will try to make an OSGEO4W environment work even if there are some conflicting package installs like rasterio (which has conflicting dlls).

Could you please give this a go and let me know if it works for you? (The latest changes are in the tiler branch which is ahead of the f32 branch.)

Thanks for this. I will try this as soon as I find the time. @nilswallenberg , if you have time to try this before me, please let me know the outcome.

[nilswallenberg]

Hi,

This is their repo: https://github.com/nvnsudharsan/SOLWEIG-GPU
and it's available as a Python package: https://pypi.org/project/solweig-gpu/

Cheers,
Nils

[nilswallenberg]

Hi @songololo

I tried running in the OSGEO4W Python environment with success. I only ran into problems with "from pvlib.iotools import read_epw", but running without it worked without issues.

QGIS version: 3.44.3-Solothurn
Qt version: 5.15.13
Python version: 3.12.11
GDAL version: 3.11.3 — Eganville
OS version: Windows 11 Version 2009

Cheers,
Nils

[biglimp]

Hi @songololo

I tried running in the OSGEO4W Python environment with success. I only ran into problems with "from pvlib.iotools import read_epw", but running without it worked without issues.

QGIS version: 3.44.3-Solothurn Qt version: 5.15.13 Python version: 3.12.11 GDAL version: 3.11.3 — Eganville OS version: Windows 11 Version 2009

Cheers, Nils

Many thanks. If we fix that import we could finally merge this.