diff --git a/.readthedocs.yaml b/.readthedocs.yaml index 629eb1fa8..67b63551b 100644 --- a/.readthedocs.yaml +++ b/.readthedocs.yaml @@ -9,7 +9,7 @@ version: 2 build: os: ubuntu-22.04 tools: - python: "3.8" + python: "3.10" # Build documentation in the docs/ directory with Sphinx sphinx: diff --git a/doc/about.rst b/doc/about.rst index ad11ac9a7..cda77e87f 100644 --- a/doc/about.rst +++ b/doc/about.rst @@ -2,104 +2,79 @@ About eTraGo ============ -eTraGo stands for **e**\lectric **Tra**\nsmission **G**\rid **o**\ptimization. +*eTraGo* (**e**\lectric **Tra**\nsmission **G**\rid **o**\ptimization) has been developed as part of a comprehensive toolchain. +The following sections introduce key projects, data models and software tools within this context. +More detailed information can be found in the publications listed below or the following reports: [openeGo_report]_, [eGon_report]_ -The python package eTraGo provides optimization strategies of flexibility options -for transmission grids based on PyPSA. A peculiarity in this context is that -the German transmission grid is described by the 380, 220 and 110 kV voltage levels. -Conventionally the 110kV grid is part of the distribution grid. The integration of -the transmission and ‘upper’ distribution grid is part of eTraGo. -The focus of optimization are flexibility options with a special focus on -energy storage and grid expansion measures. +Research Projects and Publications +================================== - - -Research projects -==================== -This software project was initially developed in the research project -`open_eGo `_. +*eTraGo* was initially developed within the research projects +`open_eGo `_ and `eGon `_. It is constantly further developed in different reserach projects, -e.g. `eGon `_ and `PoWerD `_. - - -The OpenEnergy Platform -======================= -Within the open_eGo project we developed the OpenEnergy Platform which this software -is using in order to get and store the in- and output data. Before you start to -calculate a registration on the platform is needed. For more information see -`openenergy-platform `_ and login. - -The OpenEnergy platform mainly addresses students, researchers and scientists in -the field of energy modelling and analytics as well as interested persons in -those fields. The platform provides great tools to make your energy system -modelling process transparent. All data of the open_eGo project are stored at -this platform. -`Learn more about the database access `_. - +e.g. `PoWerD `_. +Publications: +* *The impact of redispatch on grid and storage expansion planning in the German energy system* [Buettner20242]_ +* *Avoiding False Inter-Zonal Meshing in the Clustering of a Large-Scale German Power Grid* [Esterl2024]_ +* *Influence of flexibility options on the German transmission grid — A sector-coupled mid-term scenario* [Buettner2024]_ +* *Integrated Techno-Economic Power System Planning of Transmission and Distribution Grids* [Mueller2019]_ +* *The eGo grid model: An open source approach towards a model of German high and extra-high voltage power grids* [Mueller20181]_ +* *The Role of the High Voltage Power Level in Future Power Systems and Their Modelling* [Mueller20182]_ -Tool overview -============= - - +eTraGo as part of the eGo-Toolchain +=================================== .. figure:: images/ego_tools.svg :align: center :scale: 75% + +The tools illustrated in the graph above have been developed for cross-grid level optimization of the sector-coupled energy system in Germany as part of the aforementioned research projects. The following section describes the individual elements of this toolchain. -eDisGo -====== -The python package eDisGo provides a toolbox for analysis and optimization -of distribution grids. It is closely related to the python project Ding0 as this -project is currently the single data source for eDisGo providing synthetic -grid data for whole Germany. `Learn more here `_. - +Open Energy Platform +-------------------- -eGo -=== +The `Open Energy Platform `_ has been developed within the *open_eGo* project. +It addresses students and researchers as well as interested people in the field of energy modelling and analytics and provides tools to make energy system modelling process transparent. -The python package eGo is a toolbox and application which connects the tool eTraGo -(optimization of flexibility options at transmission grid level) -and eDisGo (optimization of distribution grids). All those python -packages were initially developed in the research project -`open_eGo `_. -`Learn more here `_. +All data of the *open_eGo* and *eGon* project are stored at the *Open Energy Platform*. *eTraGo* retrieves the input data from the *Open Energ Platform* and enables to store back the results. +For access to the coresponding data models, registration and login are necessary. Learn more about the database access `here `_. +Data Model Creation +------------------- -Data model creation -=================== -For the eGon project the python-tool `eGon-data `_ was implemented, which creates input data for the optimization tools `eTraGo `_, `ding0 `_ and `eDisGo `_ and delivers for example data on grid topologies, demands/demand curves and generation capacities in a high spatial resolution. The outputs of egon-data are published under open source and open data licenses. +The corresponding data model is created using the Python tool `eGon-data `, which represents a further development of the `data processing ` tool. The model covers the coupling of electricity grid models on different voltage levels with a gas grid model, demands and flexibilities from the mobility, heat and hydrogen sectors as well as the integration of other electrical flexibilities such as demand-side management and dynamic line rating. It is characterised by a high spatial resolution within Germany, while other countries are considered in an aggregated form. Several future scenarios have been developed, each covering one year in hourly resolution and differing in terms of generation, demand and availability of some technologies. -eGon-data is a further development of the `Data processing `_ developed in the former research project `open_eGo `_. It aims for an extensions of the data models as well as for a better replicability and manageability of the data preparation and processing. -The resulting data set serves as an input for the optimization tools `eTraGo `_, `ding0 `_ and `eDisGo `_ and delivers for example data on grid topologies, demands/demand curves and generation capacities in a high spatial resolution. The outputs of egon-data are published under open source and open data licenses. +The developed data model provides data consistent on different aggregation levels and serves as input for the tools `eTraGo `_, `eDisGo `_ and `ding0 `_. The outputs of *eGon-data* are published under open source and open data licenses. +Distribution Grid Optimization +------------------------------ -Dingo -===== +`Ding0 `_ (**Di**\stribution **N**\etwork **G**\enerat**0**\r) is a tool to generate synthetic +medium and low voltage distribution grids based on open data. -The DIstribution Network GeneratOr (Ding0) is a tool to generate synthetic -medium and low voltage power distribution grids based on open -(or at least accessible) data. -`Learn more here `_. +`eDisGo `_ (**e**\lectric **Dis**\tributon **G**\rid **o**\ptimization) provides a toolbox for optimization and analysis +of medium and low voltage distribution grids. +Cross-Grid Level Optimization +----------------------------- +`eGo `_ combines the tools *eTraGo* and *eDisGo* for cross-grid level optimization of the sector-coupled energy system in Germany. -LICENSE +License ======= -© Copyright 2015-2023 +© Copyright 2015-2024 Flensburg University of Applied Sciences, Europa-Universität Flensburg, Centre for Sustainable Energy Systems and DLR-Institute for Networked Energy Systems - - This program is free software: you can redistribute it and/or modify it under the terms of the GNU Affero General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) diff --git a/doc/api.rst b/doc/api.rst index 4a49271d5..68e90ec22 100644 --- a/doc/api.rst +++ b/doc/api.rst @@ -11,5 +11,5 @@ API api/etrago.disaggregate api/etrago.execute api/etrago.tools - api/appl.rst - api/network.rst + api/appl + api/network diff --git a/doc/api/etrago.tools.rst b/doc/api/etrago.tools.rst index 2b5495d38..627436cd8 100644 --- a/doc/api/etrago.tools.rst +++ b/doc/api/etrago.tools.rst @@ -1,13 +1,6 @@ etrago.tools package ===================== -etrago.tools.calc\_results module ----------------------------------- - -.. automodule:: etrago.tools.calc_results - :members: - :undoc-members: - :show-inheritance: etrago.tools.constraints module ---------------------------------- @@ -17,14 +10,6 @@ etrago.tools.constraints module :undoc-members: :show-inheritance: -etrago.tools.execute module ----------------------------------- - -.. automodule:: etrago.tools.execute - :members: - :undoc-members: - :show-inheritance: - etrago.tools.extendable module ---------------------------------- @@ -41,22 +26,6 @@ etrago.tools.io module :undoc-members: :show-inheritance: -etrago.tools.network module ------------------------------ - -.. automodule:: etrago.tools.network - :members: - :undoc-members: - :show-inheritance: - -etrago.tools.plot module ---------------------------- - -.. automodule:: etrago.tools.plot - :members: - :undoc-members: - :show-inheritance: - etrago.tools.utilities module ------------------------------- diff --git a/doc/contributing.rst b/doc/contributing.rst new file mode 100644 index 000000000..713d620a2 --- /dev/null +++ b/doc/contributing.rst @@ -0,0 +1,251 @@ +.. _Contributing_ref: +============ +Contributing +============ + + +The package *eTraGo* is a collaborative projects with several people +contributing to it. The following section gives an overview of +applicable guidelines and rules to enable a prospering collaboration. +Any external contributions are welcome as well, and they are greatly +appreciated! Every little bit helps, and credit will always be given. + + +Bug reports and feature requests +================================ + +The best way to report bugs, inform about intended developments, send +feedback or propose a feature +is to file an issue at +https://github.com/openego/eTraGo/issues. + +Please tag your issue with one of the predefined labels as it helps +others to keep track of unsolved bugs, open tasks and questions. + +To inform others about intended developments please include: + +* a describtion of the purpose and the value it adds +* outline the required steps for implementation +* list open questions + +When reporting a bug please include all information needed to reproduce +the bug you found. +This may include information on + +* Your operating system name and version. +* Any details about your local setup that might be helpful in troubleshooting. +* Detailed steps to reproduce the bug. + +If you are proposing a feature: + +* Explain in detail how it would work. +* Keep the scope as narrow as possible, to make it easier to implement. + + +Contribution guidelines +======================= + + +Development +----------- + +Adding changes to the *eTraGo* repository should follow some guidelines: + +1. Create an `issue`_ in our `repository`_ to describe the intended + developments briefly + + .. _issue: https://github.com/openego/eTraGo/issues + .. _repository: https://github.com/openego/eTraGo + +2. Create a branch for your issue related development from the + dev-branch following our branch naming convention:: + + git checkout -b `/#-very-brief-description` + + where `issue-id` is the issue number on GitHub and `prefix` is one of + + - features + - fixes + - refactorings + + depending on which one is appropriate. This command creates a new + branch in your local repository, in which you can now make your + changes. Be sure to check out our `style conventions`_ so that your + code is in line with them. + If you don't have push rights to our `repository`_, you need to fork + it via the "Fork" button in the upper right of the `repository`_ + page and work on the fork. + + .. _style conventions: `Code and Commit Style`_ + +3. Make sure to update the documentation along with your code changes + +4. When you're done making changes run all the checks and docs builder + with `tox `_ one + command: + + .. code-block:: bash + + tox + +5. Commit your changes and push your branch to GitHub:: + + git add -p + git commit + git push origin features/#-very-brief-description + + Note that the :code:`-p` switch will make :code:`git add` iterate + through your changes and prompt for each one on whether you want to + include it in the upcoming commit. This is useful if you made multiple + changes which should conceptually be grouped into different commits, + like e.g. fixing the documentation of one function and changing the + implementation of an unrelated one in parallel, because it allows you + to still make separate commits for these changes. It has the drawback + of not picking up new files though, so if you added files and want to + put them under version control, you have to add them explicitly by + running :code:`git add FILE1 FILE2 ...` instead. + +6. Submit a pull request through the GitHub website. + + +Code and Commit Style +--------------------- + +We try the adhere to the `PEP 8 Style Guide `_ wherever possible. +In addition to that, we use `a code formatter` to have a consistent +style, even in cases where PEP 8 leaves multiple degrees of freedom. So +please run your code through :code:`black` before committing it. [#black]_ +PEP 8 also specifies a way to group imports, onto which we put the +additional constraint that the imports within each group are ordered +alphabetically. Once again, you don't have to keep track of this +manually, but you can use `isort`_ to have imports sorted automatically. + +Unfortunately these tools don't catch everything, so here's a short list +of things you have to keep track of manually: + + - :code:`Black` can't automatically break up overly long strings, so + make use of Python's automatic string concatenation feature by e.g. + converting + + .. code-block:: python + + something = "A really really long string" + + into the equivalent: + + .. code-block:: python + + something = ( + "A really really" + " long string" + ) + + - :code:`Black` also can't check whether you're using readable names + for your variables. So please don't use abbreviations. Use `readable + names`_. + + - :code:`Black` also can't reformat your comments. So please keep in + mind that PEP 8 specifies a line length of 72 for free flowing text + like comments and docstrings. This also extends to the documentation + in reStructuredText files. + +Last but not least, commit messages are a kind of documentation, too, +which should adhere to a certain style. There are quite a few documents +detailing this style, but the shortest and easiest to find is probably +https://commit.style. If you have 15 minutes instead of only five to +spare, there's also a very good and only `slightly longer article`_ on +this subject, containing references to other style guides, and also +explaining why commit messages are important. + +At the very least, try to only commit small, related changes. If you +have to use an "and" when trying to summarize your changes, they should +probably be grouped into separate commits. + +.. _a code formatter: https://pypi.org/project/black/ +.. _slightly longer article: https://chris.beams.io/posts/git-commit/ +.. _isort: https://pypi.org/project/isort/ +.. _pre-commit: https://pre-commit.com +.. _readable names: https://chrisdone.com/posts/german-naming-convention/ +.. [#black] + If you want to be really nice, run any file you touch through + :code:`black` before making changes, and commit the result + separately from other changes.. The repository may contain wrongly + formatted legacy code, and this way you commit eventually necessary + style fixes separated from your actually meaningful changes, which + makes the reviewers job a lot easier. + +Pull Request Guidelines +----------------------- + +We use pull requests (PR) to integrate code changes from branches. +PRs always need to be reviewed (exception proves the rule!). Therefore, ask +one of the other developers for reviewing your changes. Once approved, the PR +can be merged. Please delete the branch after merging. + +When requesting reviews, please keep in mind it might be a significant effort +to review the PR. Try to make it easier for them and keep the overall effort +as low as possible. Therefore, + +* asking for reviewing specific aspects helps reviewers a lot to focus on the + relevant parts +* when multiple people are asked for a review it should be avoided that they + check/test the same things. Be even more specific what you expect from + someone in particular. + + + +Documentation +============= + +*eTraGo* could always use more documentation, whether as part of the +official *eTraGo* docs, in docstrings, or even in articles, blog posts +or similar resources. Always keep in mind to update the documentation +along with your code changes though. + +The changes of the documentation in a feature branch get visible once a +pull request is opened. + +How to document Python scripts +------------------------------ + +Use docstrings to document your Python code. Note that PEP 8 also +contains a `section `_ on docstrings and that there is +a whole `PEP `_ dedicated to docstring conventions. Try to +adhere to both of them. +Additionally every Python script needs to contain a header describing +the general functionality and objective and including information on +copyright, license and authors. + +.. code-block:: python + + """ Provide an example of the first line of a module docstring. + + This is an example header describing the functionalities of a Python + script to give the user a general overview of what's happening here. + """ + + __copyright__ = "Example Institut" + __license__ = "GNU Affero General Public License Version 3 (AGPL-3.0)" + __url__ = "https://github.com/openego/eTraGo/blob/main/LICENSE" + __author__ = "github_alias1, github_alias2" + + + +You can build the documentation locally with (executed in the repos root +directory) + +.. code-block:: bash + + sphinx-build -E -a docs docs/_build/ + +Eventually, you might need to install additional dependencies for building the +documenmtation: + +.. code-block:: bash + + pip install -r docs/requirements.txt + + +.. _PEP8: https://www.python.org/dev/peps/pep-0008 +.. _PEP8-docstrings: https://www.python.org/dev/peps/pep-0008/#documentation-strings +.. _PEP257: https://www.python.org/dev/peps/pep-0257/ diff --git a/doc/developer_notes.rst b/doc/developer_notes.rst deleted file mode 100644 index cb85b1dfc..000000000 --- a/doc/developer_notes.rst +++ /dev/null @@ -1,37 +0,0 @@ - -=============== -Developer notes -=============== - - -Installation for Developers -=========================== - - -.. note:: - Installation is primarly tested on (Ubuntu like) linux OS. - -1. If you like, create a virtual environment (where you like it) and activate it (if you do not use venv start with 2.): - -.. code-block:: bash - - $ virtualenv --clear -p python3.10 etrago`` - $ cd etrago/ - $ source bin/activate - -2. Clone the source code from github: - -.. code-block:: bash - - $ git clone https://github.com/openego/eTraGo - -You can checkout to the dev branch and create new feature branches. -For the correct work-flow, please mind the -`Dreissen Branching Model `_ - -3. Use the pip -e to install eTraGo directly from the cloned repository: - -.. code-block:: bash - - $ pip3 install -e /path/to/eTraGo/ - diff --git a/doc/howToUse.rst b/doc/howToUse.rst index 20f97f2c3..da4ba0478 100644 --- a/doc/howToUse.rst +++ b/doc/howToUse.rst @@ -1,56 +1,33 @@ .. _HowToUse: ================== -How to use eTraGo? +How to use eTraGo ================== -After you installed eTraGo you would typically start optimization runs by -executing the ‘appl.py’ which is situated in -``./eTrago/etrago/`` (e.g by ``python3 appl.py`` from the terminal). - -eTraGo doesn't have a graphical user interface, -the ‘appl.py’ is used as a simple user interface which can be edited with -the preferred python-editor. -Here parameters, calculation methods and scenario settings are set in a python -dictionary called 'args'. -To run the desired calculation, it is crucial to understand these parameters. -In addition, some of them contradict the usage of others. -You find the documentation of all defined parameters from the 'args' here: -:func:`etrago.appl.run_etrago`. - -Alternatively, the 'args' dictionary can be edited in a json-file. -Then the path to the json-file has to be set in the initilization of the -Etrago-object (:class:`etrago.tools.network.Etrago`). Once a path is given -the 'args' dictionary within the 'appl.py' is ignored -and replaced by the 'args' of the json-file. - -The appl.py contains the :func:`etrago.appl.run_etrago` function which uses the -defined 'args' dictionary to start the desired calculation. - -To improve the performance of the optimization of the selected solver, -you might want to use solver options (part of 'args'). For gurobi -the most used ones are described -`here `_. - -For more specific or extensive changes you are highly invited -to write code and add new functionalities. - -Once the calculation has finished the PyPSA network of the Etrago-object will -contain all results. Some main results (e.g. anuual system costs) are calculated -by :meth:`etrago.calc_results` and can be accesed via 'etrago.results'. -You can use several plotting functions from the :meth:`etrago.tools.plot` in order -to visualize the results. For example -the :meth:`etrago.tools.plot.plot_grid` can be used to plot relative line loading -in % or the optimized expansion of all AC lines and DC links of the network. - -To save the results you can write them to csv files. These functionalites can be -specified also in the 'args' dictionary. +Once the *eTraGo* application has been installed, optimization runs can be initiated by executing the ``appl.py`` script, which is located in the ``./eTrago/etrago/`` directory. This may be done, for example, by entering the following command: + +.. code-block:: bash + + $ python3 appl.py + + +The ``appl.py`` file presents the user interface and may be edited with the preferred Python editor. Within the ``appl.py`` module, the scenario settings, parameters and calculation methods are defined within a Python dictionary, referred to as ``args``. It is important to comprehend the parameters in order to execute the desired calculation. It should be noted that some parameters are mutually exclusive, and thus, their usage must be carefully considered. Further information can be found in the subsequent section (Section ref:`Functionalities_ref`), or in the documentation of all defined parameters from the ``args`` dictionary, accessible in the function :func:`etrago.appl.run_etrago`. + +As an alternative approach, the ``args`` dictionary can be modified through the use of a JSON-file. The path to the JSON-file must be specified during the initialization of the ``Etrago object`` (of the class ``Etrago.network.Etrago``). Once a path is provided, the ``args`` dictionary in the ``appl.py`` is disregarded and substituted with the dictionary from the JSON-file. + +The ``appl.py`` contains the function :func:`etrago.appl.run_etrago` which uses the defined ``args`` dictionary to start the desired calculation. + +In order to enhance the efficacy of the optimization process using the selected solver, it may be beneficial to consider the utilisation of solver options (which form part of the ``args`` dictionary). For Gurobi, the some preferrable settings are outlined in Section ref:`Functionalities_ref`. + +For more specific or extensive changes you are kindly invited to write code and add new functionalities. Please see Section Section ref:`Contributing_ref`. + +Once the calculation has finished, the ``Etrago object`` will contain all of the resulting data. Some principal results (e.g. annual system costs) are calculated by the function :meth:`etrago.calc_results`. Additionally, several plotting functions are available (:meth:`etrago.analyze.plot`). To save the results, you can write them to csv files. You can specify this export within the ``args``. .. _Examples: -Examples and tutorial notebooks +Examples and Tutorial Notebooks =============================== - +The following links provide examples and tutorial notebooks: **eTraGo version 0.5.1:** `etrago_OpenMod_Zuerich18 `_. diff --git a/doc/images/complexity_spatial.png b/doc/images/complexity_spatial.png new file mode 100644 index 000000000..ad0e847ce Binary files /dev/null and b/doc/images/complexity_spatial.png differ diff --git a/doc/images/etrago_functionalities.png b/doc/images/etrago_functionalities.png new file mode 100644 index 000000000..e6af708e6 Binary files /dev/null and b/doc/images/etrago_functionalities.png differ diff --git a/doc/images/exemplary_results.png b/doc/images/exemplary_results.png new file mode 100644 index 000000000..8ffc863b4 Binary files /dev/null and b/doc/images/exemplary_results.png differ diff --git a/doc/images/input_data.png b/doc/images/input_data.png new file mode 100644 index 000000000..43b2529fd Binary files /dev/null and b/doc/images/input_data.png differ diff --git a/doc/images/modelling_concept.png b/doc/images/modelling_concept.png index d0f756e8b..1a8fd3c0f 100644 Binary files a/doc/images/modelling_concept.png and b/doc/images/modelling_concept.png differ diff --git a/doc/index.rst b/doc/index.rst index 6f190eecc..7c3d0e93a 100644 --- a/doc/index.rst +++ b/doc/index.rst @@ -3,32 +3,41 @@ You can adapt this file completely to your liking, but it should at least contain the root `toctree` directive. -Welcome to eTraGo's documentation! -================================== +eTraGo - **e**\lectric **Tra**\nsmission **G**\rid **o**\ptimization +==================================================================== + +*eTraGo* is a cross-sectoral grid planning tool focusing on the extra-high and high-voltage power grid level. In addition to the electricity sector, the gas (methane and hydrogen), heat and mobility sectors are considered. *eTraGo* optimizes grid and storage expansion as well as power plant deployment, taking into account various flexibility options. These include electrical flexibilities such as demand-side management or dynamic line rating, as well as flexibilities arising from sector coupling, such as heat stores, gas stores or shifting potentials from charging electric vehicles. *eTraGo* is an open-source Python package that is largely based on `PyPSA `_. .. figure:: images/etrago_logo.png :align: right :scale: 80% -.. warning:: Note, eTraGo and its documentation is in - continuous development. +Key Features +------------ + +* open, cross-sectoral grid planning tool on extra-high and high-voltage level +* includes a variety of functionalities for scenario variations and reduction of complexity in spatial and temporal dimension +* market optimization as well as linear-optimal power flow available +* analyses and plots the results + +.. note:: eTraGo and its documentation are in continuous development. .. toctree:: - :maxdepth: 2 + :maxdepth: 3 about installation howToUse theoretical_background - developer_notes + contributing whatsnew api + references -Indices and tables +Indices and Tables ================== * :ref:`genindex` * :ref:`modindex` -* :ref:`search` diff --git a/doc/installation.rst b/doc/installation.rst index eea87022f..435b43382 100644 --- a/doc/installation.rst +++ b/doc/installation.rst @@ -1,22 +1,24 @@ ============ Installation ============ -eTraGo is designed as a Python package therefore it is mandatory to have -`Python 3 `_ installed. If you have a -working Python3 environment, use pypi to install the latest eTraGo version. -We highly recommend you to use a virtual environment. Use following pip -command in order to install eTraGo. + +*eTraGo* is designed as a Python package therefore it is mandatory to have +`Python 3 `_ installed. The current +version of *eTraGo* is compatible to Python version 3.10 or higher. +If you have a working Python3 environment, use pypi to install the latest +*eTraGo* version. We highly recommend you to use a virtual environment. +Use following pip command in order to install *eTraGo*. .. code-block:: bash $ pip3 install eTraGo -Using a virtual environment -=========================== +Using a Virtual Environment +=========================== -Before installing eTraGo, -you create a virtual environment (where you like it) and activate it: +Before installing *eTraGo*, you create a virtual environment (where you like it) +and activate it: .. code-block:: bash @@ -25,19 +27,20 @@ you create a virtual environment (where you like it) and activate it: $ cd venv Inside your activated virtual environment you can -install eTraGo with the pip command, as previously explained. +install *eTraGo* with the pip command, as previously explained. + Linux and Ubuntu ================ -The Package eTraGo is tested with Ubuntu 16.04, 18.04, 20.04 and 22.04 inside the virtual +The package *eTraGo* is tested with Ubuntu 20.04 and 22.04 inside the virtual environments of `virtualenv `_. The installation is shown above. -Windows or Mac OSX users -======================== +Windows or Mac OSX +================== For Windows and/or Mac OSX user we highly recommend to install and use Anaconda for your Python3 installation. First install Conda including python 3.10 or @@ -51,33 +54,31 @@ prompt as administrator and run: $ conda activate etrago_env $ pip install eTraGo - The full Documentation can be found `on this page `_ . We use Anaconda with an own environment in order to reduze problems with Packages and different versions on our system. Learn more about -`Anacona `_ +`Anaconda `_ environments. - -Setup database connection +Setup Database Connection ========================= -The eTraGo module `db `_ +The *eTraGo* module `db `_ gives you a python SQL-Alchemy representations of -the `OpenEnergy-Database(oedb) `_ +the `OpenEnergy-Database (oedb) `_ and access to it by using the `oedialect `_, which is a SQL-Alchemy binding Python package for the REST-API used by the OpenEnergy Platform (OEP). -In order to connect eTraGo via the oedialect with the oedb you +In order to connect *eTraGo* via the oedialect with the oedb you have to create an account at `openenergy-platform.org/login `_. You can name the `'db' `_ argument of the 'args' of the :func:`etrago.appl.etrago` as you wish. Once the :func:`etrago.appl.etrago` is executed you will be asked to enter how you want to connect to which database. If you want to use -the oedialect enter the following connection parameter. For and +the oedialect enter the connection parameter specifed in the code block below. For and you have to take your credentials which you obtained by registering at `openenergy-platform.org/login `_. @@ -85,14 +86,13 @@ Your API access / login data will be saved in the folder ``.etrago_database`` in ``config.ini``. Consequently, in the config.ini you can also change your connection parameters or add new ones. In the following you can see how the config.ini looks like when you use the -oedialect, a local postgresql database or the old psycopg2 developer connection. +oedialect or a local postgresql database. Once you have created a connection (which is saved in the config.ini) you do not have to enter the connection parameter again. The software will take the connection parameter which corresponds to the entry at the `'db' `_ argument. - -oedialect connection +oedialect Connection -------------------- .. code-block:: desktop @@ -105,29 +105,45 @@ oedialect connection port = 80 password = - -Local database connection +Local Database Connection ------------------------- .. code-block:: desktop [local] - username = YourOEDBUserName + username = YourLocalUserName database = YourLocalDatabaseName host = localhost or 127.0.0.1 - port = 5433 + port = YourDatabasePort pw = YourLocalPassword + + +Installation for Developers +=========================== +.. note:: + Installation is primarly tested on (Ubuntu like) linux OS. +1. If you like, create a virtual environment (where you like it) and activate it (if you do not use venv start with 2.): -Old developer connection -------------------------- +.. code-block:: bash -.. code-block:: desktop + $ virtualenv --clear -p python3.10 etrago`` + $ cd etrago/ + $ source bin/activate + +2. Clone the source code from github: - [oedb] - username = YourOEDBUserName - database = oedb - host = oe2.iws.cs.ovgu.de - port = 5432 - pw = YourOEDBPassword +.. code-block:: bash + + $ git clone https://github.com/openego/eTraGo + +You can checkout to the dev branch and create new feature branches. +For the correct work-flow, please mind the +`Dreissen Branching Model `_ + +3. Use the pip -e to install *eTraGo* directly from the cloned repository: + +.. code-block:: bash + + $ pip3 install -e /path/to/eTraGo/ diff --git a/doc/references.rst b/doc/references.rst new file mode 100644 index 000000000..34e0c180e --- /dev/null +++ b/doc/references.rst @@ -0,0 +1,66 @@ +References +========== + +.. [PyPSA] T. Brown, J. Hörsch and D. Schlachtberger: + *PyPSA: Python for Power System Analysis*. 2018. + ``_ + +.. [Buettner20242] C. Buettner, U. P. Müller: + *The impact of redispatch on grid and storage expansion planning in the German energy system*. 2024. + +.. [Esterl2024] K. Esterl, C. A. Realpe Epia, U. P. Müller: + *Avoiding False Inter-Zonal Meshing in the Clustering of a Large-Scale German Power Grid*. 2024. + PREPRINT. ``_ + +.. [Buettner2024] C. Buettner, K. Esterl, I. Cussmann, C. A. Realpe Epia, J. Amme, A. Nadal: + *Influence of flexibility options on the German transmission grid — A sector-coupled mid-term scenario*. 2024. + ``_ + +.. [Mueller20181] U. P. Müller, L. Wienholt, D. Kleinhans, I. Cußmann, W.-D. Bunke, G. Pleßmann and J. Wendiggensen: + *The eGo grid model: An open source approach towards a model of German high and extra-high voltage power grids*. 2018. + ``_ + +.. [Mueller20182] U. P. Müller, L. Wienholt, I. Cußmann: + *The Role of the High Voltage Power Level in Future Power Systems and Their Modelling*. 2018. + ``_ + +.. [Mueller2019] U. P. Müller, B. Schachler, M. Scharf, W.-D. Bunke, S. Günther, J. Bartels, G. Pleßmann: + *Integrated Techno-Economic Power System Planning of Transmission and Distribution Grids*. 2019. + ``_ + +.. [eGon_report] I. Cußmann, B. Schachler, C. Büttner, H.-P. Tetens, K. Esterl, J. Amme, K. Helfenbein, M. Held, A. Nadal, S. Günther and C. A. Realpe Epia: + *Ein offenes netzebenen- und sektorenübergreifendes Planungsinstrument zur Bestimmung des optimalen Einsatzes und Ausbaus von Flexibilitätsoptionen in Deutschland - Projektabschlussbericht*. 2024. + ``_ + +.. [openeGo_report] U. P. Müller, B. Schachler, W. Bunke, J. Bartels, M. Glauer, C. Büttner, S. Günther, E. Kötter, I. Cußmann, L. Hülk, M. Scharf, T. Mossakowski and J. Wendiggensen: + *Netzebenenübergreifendes Planungsinstrument zur Bestimmung des optimalen Netz- und Speicherausbaus in Deutschland integriert in einer OpenEnergyPlatform - Projektabschlussbericht*. 2019. + ``_ + +.. [Esterl2022] K. Esterl: + *Vergleich von Methoden zur Reduktion der zeitlichen Komplexit\"at für die Optimierung sektorgekoppelter Energiesysteme*. 2022. + ``_ + +.. [NEP] Übertragungsnetzbetreiber Deutschland (2021): + *Netzentwicklungsplan Strom 2035*, Version 2021, 1. Entwurf. 2021. + +.. [BGR] Bundesanstalt fuer Geowissenschaften und Rohstoffe et al. (2020): + *nSpEE-DS - Teilprojekt Bewertungskriterien und Potenzialabschätzung*. 2020. + ``_ + +.. [Hoersch] J. Hörsch, T. Brown: + *The role of spatial scale in joint optimisations of generation and transmission for European highly renewable scenarios*. 2017. + ``_ + +.. [Pineda] S. Pineda, J. M. Morales: + *Chronological Time-Period Clustering for Optimal Capacity Expansion Planning With Storage*. 2018. + ``_ + +.. [Kotzur] L. Kotzur, P. Markewitz, M. Robinius, D. Stolten: + *Time series aggregation for energy system design: Modeling seasonal storage*. 2018. + ``_ + + + + + + diff --git a/doc/theoretical_background.rst b/doc/theoretical_background.rst index 62c8e2d9e..caab8211e 100644 --- a/doc/theoretical_background.rst +++ b/doc/theoretical_background.rst @@ -1,64 +1,81 @@ -====================== -Theoretical Background -====================== +.. _Functionalities_ref: +=============== +Functionalities +=============== + +.. figure:: images/etrago_functionalities.png + :align: center + :width: 800 + +*eTraGo* is based on the open source tool `PyPSA `_ and uses its definitions and units [PyPSA]_. -Definitions and Units -===================== +Data Model +========== -eTraGo is based on the open source tool `PyPSA `_ and uses its definitions and units. +The input data covers the coupling of electricity grid models on different voltage levels with a gas grid model, demands and flexibilities from the mobility, heat and hydrogen sectors as well as the integration of other electrical flexibilities such as demand-side management and dynamic line rating. It is characterised by a high spatial resolution within Germany, while other countries are considered in an aggregated form. Several future scenarios have been developed, each covering one year in hourly resolution and differing in terms of generation, demand and availability of some technologies. The data model is generated using the tool *eGon-data*. More details on the model can be found in the documentation of `eGon-data `_ or the following publications: [eGon_report]_ and [Buettner2024]_. The graphs below give some impressions [eGon_report]_: +.. figure:: images/input_data.png + :align: center + :width: 800 -Assumptions on Data -=================== -eTraGo fetches the input data from the `OpenEnergy Platform `_. The data includes electricity and gas grid topology as well as data on energy supply and load for the considered sectors (electricity, gas, heat and e-mobility) plus data on flexibility potential deriving from those sectors e.g. Dynamic Line Rating, Demand Side Management and flexibility potentials arising from e-mobility. More details on the data model can be found in the documentaton of `eGon-data `_. +*eTraGo* fetches the input data from the `Open Energy Platform `_. Alternatively, different scenarios of the data models are available through `zenodo `_. The data needs to be downloaded and locally stored as a PostgreSQL database to be accessable for *eTraGo*. More explanations can be found in the `zenodo upload `_. The following scenarios are available: -At the moment, there are two scenarios available basing on scenario C2035 of the network expansion plan ([NEP]_), version 2021. The base one is called eGon2035. To analyse the effect of flexibility options, there is an eGon2035_lowflex scenario available which depicts a lower penetration of flexibilities. More scenarios are being developed. The eGon100RE scenario is being implemented which is characterised by a 100% renewable generation. Analog to the scenario above, a eGon100RE_lowflex scenario will be available. +* `eGon2035 `_ basing on scenario C2035 of the network expansion plan ([NEP]_), version 2021 +* eGon2035_lowflex as scenario variant of eGon2035 with lower penetration of flexibilities +* eGon100RE (still under development) characterised by a 100% renewable generation +* `status2019 `_ depicting the status in 2019 You can see the modeling concepts of the scenarios in the figure below. The components marked green have exogenous capacity and endogenous dispatch whereas the components marked in red are optimised endogenously in capacity and dispatch. .. figure:: images/modelling_concept.png :align: center - :scale: 75% - - -Methods -======= + :width: 800 + +Scenario Variation +================== -Optimisation with PyPSA ------------------------ +Several features were developed to enhance the functionality of *eTraGo* and allow for adaptions within the scenarios introduced above. -Within eTraGo, the fetched data model is translated into a `PyPSA `_-network. The optimisation is performed with a linear approximation assuming eTraGo to fulfill the assumptions to perfom a LOPF (as those are small voltage angle differences, branch resistances negligible to their reactances, voltage magnitudes can be kept at nominal values) since it focuses on the extra-high and high voltage levels. As objective value of the optimisation, the overall system costs are considered. +* In ‚extendable‘ you can adapt the type of components you want to be optimised in capacity and set upper limits for grid expansion inside Germany and of lines to foreign countries. +* With ‘foreign_lines‘ you can adapt the foreign lines to be modeled as DC-links (e.g. to avoid loop flows). +* ‘branch_capacity_factor’ adds a factor to adapt all line capacities in order to consider (n-1) security. Because the average number of HV systems is much smaller than the one of eHV lines, you can choose factors for ‘HV’ and ‘eHV’ separately. +* The ‚extra_functionality‘-argument allows to consider extra constraints like limits for energy imort and export or minimal renewable shares in generation. +* The ‘load_shedding’-argument is used for debugging complex grids in order to avoid infeasibilities. It introduces a very expensive generator at each bus to meet the demand. When optimising storage units and grid expansion without limiting constraints, the need for load shedding should not be existent. -With the argument ‘pf_post_lopf’, after the LOPF a non-linear power flow simulation can be conducted. Complexity Reduction ---------------------- +==================== The data model is characterised by a high spatial (about 8,000 electrical and 600 gas nodes) and temporal resolution (8,760 timesteps). To reduce the complexity of the resulting optimisation problem, several methods can be applied. -Reduction in spatial dimension: -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +Reduction in Spatial Dimension: +------------------------------- -The **ehv clustering** maps all electrical nodes with a voltage level below the extra-high voltage level to their nearest neighboring node in the extra-high voltage level with the Dijkstra’s algorithm (110 kV —> 220 / 380 kV). +The **ehv clustering** maps all electrical nodes with a voltage level below the extra-high voltage level to their nearest neighboring node in the extra-high voltage level with the Dijkstra’s algorithm (110 kV —> 220 kV / 380 kV). The **k-means Clustering** reduces the electrical or gas network to an adjustable number of nodes by considering the geographical position of the respective nodes. This method has been implemented within PyPSA by [Hoersch]_. The **k-medoids Dijkstra Clustering** aggregates nodes considering the network topology. First, a k-medoids Clustering is used dividing the original nodes of the network into groups by their geographical positions while identifiying the geographical medoid nodes per cluster. Afterwards, the original nodes in the original network are assigned to the former identified medoids considering the original network’s topology applying a Dijkstra’s algorithm considering the line lengths. Afterall, the original nodes are represented by one aggregated node per cluster at the position of the former identified medoid node. +The procedures of the two methods are depicted in the following figure [Esterl2024]_: + +.. figure:: images/complexity_spatial.png + :align: center + :width: 800 + + In general, the clustering of the **sector-coupled system** is divided into two steps: First, the electrical and gas grid are clustered independently using one of the methods described above. Afterwards, nodes of the other sectors (hydrogen, heat, e-mobility and DSM nodes) are mapped according to their connection to electricity or gas buses and aggregated to one node per carrier. -After optimising the spatially reduced network, a **spatial disaggregation** can be conducted. - -Reduction in temporal dimension: -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +Reduction in Temporal Dimension: +-------------------------------- The method **Skip Snapshots** implies a downsampling to every nth time step. The considered snapshots are weighted respectively to account for the analysis of one whole year. @@ -66,53 +83,120 @@ By using the method called **Segmentation**, a hierarchical clustering of consec The **Snapshot Clustering on Typical Periods** implies a hierarchical clustering of time periods with a predefined length (e.g. days or weeks) to typical periods. Those typical periods are weighted according to the number of periods in their cluster. This method optionally includes the linkage of the typical periods in a second time layer to account for the intertemporal dependencies following [Kotzur]_. -By applying a 2-level-approach, a **temporal disaggregation** can be conducted. This means optimising dispatch using the fullcomplex time series in the second step after having optimised grid and storage expansion using the complexity-reduced time series in the first step. + +Calculation with PyPSA +====================== + +All optimization methods within *eTraGo* base on the Linear Optimal Power Flow (LOPF) implemented in `PyPSA `_. +The objective is the minimization of system costs, considering marginal cost of energy generation and investments in grid infrastructure, storage units and different flexibility options. The different options are specific for each scenario. + +Currently, two different optimization approaches are implemented considering different conficgurations of energy markets and optimization variables. The different options are described in the following sections. + +Integrated optimization with nodal pricing +------------------------------------------ + +The objective is to minimize marginal cost of energy generation and investments in grid infrastructure, storage units and different flexibility options in one optmization problem. +The following objective function is applied: + +.. math:: + :nowrap: + + \begin{gather*} + \sum_{n,s} c_{n,s} \bar{g}_{n,s} + \sum_{n,s} c_{n,s} \bar{h}_{n,s} + \sum_{l} c_{l} F_l \\ + + \sum_{t} w_t \left[\sum_{n,s} o_{n,s,t} g_{n,s,t} + \sum_{n,s} o_{n,s,t} h_{n,s,t} \right] + + \sum_{t} \left[suc_{n,s,t} + sdc_{n,s,t} \right] + \end{gather*} + + +This implies a nodal pricing approach and optimal dispatch and redispatch. Redispatch and curtailment of renewable energy is possible, without any additional redispatch costs. +Investments and expanded capacities of grid infrastructure, storage units and different flexibility options are a key result. It has to be noted that expansion is employed continiously. The expanded capacities not always refelect existing technical appliences. This allows to keep a linear optimization problem that is feasible for the described complex model. + +Various constrains are added to model the physical and electrical behavior. + +The integrated optimization results in highly cost-effective results, but does not reflect the acctual German energy market. +A more detailed description of this modelling apporoach as well as some results can be found in differnent studies and papers, e.g. in: [Mueller2019]_, [Buettner2024]_. + +Consecutive market and grid optimization +---------------------------------------- + +This methodology aims to represent an energy market that is close to the acctual current market design by separateing the market model from the grid model. The optimization method consists of three consecutive steps. +At first, seasonal storage behavior and market-based expansions of flexibility options are identified through a simplified yet annual calculation. The second step aims to come up with a realistic dispatch for each power plant and hour, taking into account non-linear Unit Commitment constraints and a short-term, rolling planning horizon. In both steps, the spatial resolution is defined by one node per current bidding zone. +In the last optimization step, the grid topology (potentially in a resuced spatial resolution as described in XX) is considered. This allows to optimize grid expansion. The market-based dispatch is defined by the previous step, redispatch and curtailment is possible, but results into additional system costs. + +A brief overview of the different optimization steps, their key characteristics and results is shown in the following figure. +A detailed description of the methodology is given in [Buettner20242]_, which also presents and analyses results. Grid and Storage / Store expansion ------------------------------------ +---------------------------------- The grid expansion is realized by extending the capacities of existing lines and substations. These capacities are considered as part of the optimisation problem whereby the possible extension is unlimited. With respect to the different voltage levels and lengths, MVA-specific costs are considered in the optimisation. As shown in the figure above, several options to store energy are part of the modeling concept. Extendable batteries (modeled as storage units) are assigned to every node in the electrical grid. A minimum installed capacity is being considered to account for home batteries ([NEP]_). The expansion and operation is part of the optimisation. Furthermore, two types of hydrogen stores (modeled as stores) are available. Overground stores are optimised in operation and dispatch without limitations whereas underground stores depicting saltcaverns are limited by geographical conditions ([BGR]_). Additionally, heat stores part of the optimisation in terms of power and energy without upper limits. -Miscellaneous Features ----------------------- +Non-linear power flow +--------------------- +With the argument ‘pf_post_lopf’, after the LOPF a non-linear power flow simulation can be conducted. + -Several features were developed to enhance the functionality of eTraGo. +Solver Options +-------------- To customize computation settings, ‘solver_options’ and ‘generator_noise’ should be adapted. The latter adds a reproducible small random noise to the marginal costs of each generator in order to prevent an optima plateau. The specific solver options depend on the applied solver (e.g. Gurobi, CPLEX or GLPK). -In ‚extendable‘ you can adapt the type of components you want to be optimised in capacity and set upper limits for gird expansion inside Germany and of lines to foreign countries. +**Insights on Solver Settings with Gurobi** -The ‚extra_functionality‘-argument allows to consider extra constraints like limits for energy imort and export or minimal renewable shares in generation. +* `threads `_: number of threads to apply to parallel algorithms (concurrent or barrier) -‘branch_capacity_factor’ adds a factor to adapt all line capacities in order to consider (n-1) security. Because the average number of HV systems is much smaller than the one of eHV lines, you can choose factors for ‘HV’ and ‘eHV’ separately. + * default: 0 (uses all cores in the machine) + * reduce if parallel calculations or tight memory +* `method `_: algorithm used to solve optimization of lopf -The ‘load_shedding’-argument is used for debugging complex grids in order to avoid infeasibilities. It introduces a very expensive generator at each bus to meet the demand. When optimising storage units and grid expansion without limiting constraints, the need for load shedding should not be existent. + * default (-1, concurrent): chooses between simplex or barrier method due to matrix range + * 1 (simplex): slower but less sensitive for numerical issues + * 2 (barrier): fastest method but sensitive for numerical issues +* `crossover `_ (barrier only): determines the crossover strategy used to transform the interior solution produced by barrier into a basic solution -With ‘foreign_lines‘ you can adapt the foreign lines to be modeled as DC-links (e.g. to avoid loop flows). + * default: -1 (chooses strategy automatically) + * preferred: 0 (disables crossover, solves fastest but may need 'NumericFocus' and 'BarHomogenus' to avoid numerical issues) +* `BarConvTol `_ (barrier only): the barrier solver terminates when the relative difference between the primal and dual objective values is less than the specified tolerance + * default: 1e-8 + * preferred: 1e-5 (little less accurate, but solves faster) +* `FeasibilityTol `_: tolerance of all constraints + + * default: 1e-6 + * preferred: 1e-5 (less accurate, but reduces number of iterations) +* logFile: destination and name of gurobi log file + + * default: None + * preferred: 'gurobi_eTraGo.log' +* `BarHomogeneous `_: (barrier only): determines whether to use the homogeneous barrier algorithm + + * default: -1 (homogeneous barrier algorithm turned off, faster but sensitive for numerical issues) + * preferred: 1 (homogeneous barrier algorithm turned on, bit slower but less sensitive for numerical issues, often used when crossover turned off) +* `NumericFocus `_: controls the degree to which the code attempts to detect and manage numerical issues + + * default: 0 (automatic choice with a slight preference for speed) + * 1 - 3 (shift the focus towards being slower but less sensitive for numerical issues) + +Disaggregation +============== + +By applying a 2-level-approach, a **temporal disaggregation** can be conducted. This means optimising dispatch using the fullcomplex time series in the second step after having optimised grid and storage expansion using the complexity-reduced time series in the first step. More information can be found in the master thesis by [Esterl2022]_. + +Afterterwards, a **spatial disaggregation** can be conducted distributing power plant and storage utilisation time series, the expansion of storage facilities and the use of flexibility options over the original number of nodes. The expansion of the transmission grid is not disaggregated and remains at the reduced spatial resolution. The methodology is explained in [eGon_report]_. + +Analysis +======== + +*eTraGo* contains various functions for evaluating the optimisation results in the form of graphics, maps and tables. Functions to quantify results can be found in :meth:`etrago.analyze.calc_results` and functions to plot results can be found in :meth:`etrago.analyze.plot`. +Some examplary graphs by [Buettner2024]_ are presented below: + +.. figure:: images/exemplary_results.png + :align: center + :width: 800 + -References -========== -.. [NEP] Übertragungsnetzbetreiber Deutschland (2021): - *Netzentwicklungsplan Strom 2035*, Version 2021, 1. Entwurf. 2021. - -.. [Hoersch] Jonas Hoersch et al. (2017): - *The role of spatial scale in joint optimisations of generation and transmission for European highly renewable scenarios*. 2017. - ``_ - -.. [Pineda] Salvador Pineda et al. (2018): - *Chronological Time-Period Clustering for Optimal Capacity Expansion Planning With Storage*. 2018. - ``_ - -.. [Kotzur] Leander Kotzur et al. (2018): - *Time series aggregation for energy system design: Modeling seasonal storage*. 2018. - ``_ - -.. [BGR] Bundesanstalt fuer Geowissenschaften und Rohstoffe et al. (2020): - *nSpEE-DS - Teilprojekt Bewertungskriterien und Potenzialabschätzung*. 2020. - ``_ diff --git a/doc/whatsnew.rst b/doc/whatsnew.rst index 87ed81be6..58c991bad 100644 --- a/doc/whatsnew.rst +++ b/doc/whatsnew.rst @@ -8,6 +8,7 @@ These are new features and improvements of note in each release. :local: :backlinks: top +.. include:: whatsnew/v0_10_0.rst .. include:: whatsnew/v0_9_0.rst .. include:: whatsnew/v0_8_0.rst .. include:: whatsnew/v0_7_2.rst diff --git a/doc/whatsnew/v0_10_0.rst b/doc/whatsnew/v0_10_0.rst new file mode 100644 index 000000000..485f2a7a1 --- /dev/null +++ b/doc/whatsnew/v0_10_0.rst @@ -0,0 +1,19 @@ +Release 0.10.0 (XXXX) ++++++++++++++++++++++ + +Added features +-------------- + +* eTraGo is now compatible with Python 3.9, 3.10 and 3.11 +* Updates to several packages, such as pypsa and pandas +* The module structure has been updated, modules are now grouped into analyze, cluster, disaggregate and tools packages. +* eTraGo is compatible to a status quo scenario representing the year 2019 (called: status2019) +* A new consecutive optimization method is implemented with the following key features +- Separate market from grid model +- Consider actual bidding zones in the market optimization +- Grid optimization considering network topology and cost-based redispatch +- Documentation on read the docs +* An iterative security constraint LOPF (SCLOPF) is now available. It is not selected by default because not all features are compatible yet. +* Analyzing and plotting functions have been updated to take into account the new optimization method. +* Various minor changes and bug fixes + diff --git a/doc/whatsnew/v0_9_0.rst b/doc/whatsnew/v0_9_0.rst index 469187656..df680453a 100644 --- a/doc/whatsnew/v0_9_0.rst +++ b/doc/whatsnew/v0_9_0.rst @@ -26,3 +26,4 @@ Added features * A temporal disaggregation is available through a 2-level-approach including a dispatch optimization on the temporally fullcomplex model. To limit the RAM usage, you can optionally divide the optimisation problem into a chosen number of slices. * New plotting functions to visualize the optimization results from all the included energy sectors were implemented * Functions to analyze results were updated to consider new sectors + diff --git a/etrago/analyze/plot.py b/etrago/analyze/plot.py index d8616acce..86182fd70 100644 --- a/etrago/analyze/plot.py +++ b/etrago/analyze/plot.py @@ -25,14 +25,19 @@ import logging import os +from geoalchemy2.shape import to_shape # noqa: F401 from matplotlib import pyplot as plt from matplotlib.legend_handler import HandlerPatch from matplotlib.patches import Circle, Ellipse +from pyproj import Proj, transform from pypsa.plot import draw_map_cartopy +from shapely.geometry import LineString, Point +import geopandas as gpd import matplotlib import matplotlib.patches as mpatches import numpy as np import pandas as pd +import tilemapbase cartopy_present = True try: @@ -44,11 +49,6 @@ logger = logging.getLogger(__name__) if "READTHEDOCS" not in os.environ: - from geoalchemy2.shape import to_shape # noqa: F401 - from pyproj import Proj, transform - from shapely.geometry import LineString, Point - import geopandas as gpd - import tilemapbase from etrago.execute import import_gen_from_links diff --git a/etrago/cluster/electrical.py b/etrago/cluster/electrical.py index 201f81ee1..042cb9bf7 100755 --- a/etrago/cluster/electrical.py +++ b/etrago/cluster/electrical.py @@ -21,10 +21,12 @@ """electrical.py defines the methods to cluster power grid networks spatially for applications within the tool eTraGo.""" +import logging import os +logger = logging.getLogger(__name__) + if "READTHEDOCS" not in os.environ: - import logging from pypsa import Network from pypsa.clustering.spatial import ( @@ -50,7 +52,6 @@ ) from etrago.tools.utilities import set_control_strategies - logger = logging.getLogger(__name__) __copyright__ = ( "Flensburg University of Applied Sciences, " diff --git a/etrago/cluster/gas.py b/etrago/cluster/gas.py index 8ff4a300c..47b2dc248 100644 --- a/etrago/cluster/gas.py +++ b/etrago/cluster/gas.py @@ -21,10 +21,10 @@ """gas.py defines the methods to cluster gas grid networks spatially for applications within the tool eTraGo.""" +import logging import os if "READTHEDOCS" not in os.environ: - import logging from pypsa.clustering.spatial import ( aggregatebuses, diff --git a/etrago/cluster/spatial.py b/etrago/cluster/spatial.py index 1a71551bf..7f83fd30f 100755 --- a/etrago/cluster/spatial.py +++ b/etrago/cluster/spatial.py @@ -20,12 +20,12 @@ # File description for read-the-docs """spatial.py defines the methods to run spatial clustering on networks.""" +import logging import os if "READTHEDOCS" not in os.environ: from itertools import product from math import ceil - import logging import multiprocessing as mp from networkx import NetworkXNoPath @@ -49,7 +49,7 @@ connected_transformer, ) - logger = logging.getLogger(__name__) +logger = logging.getLogger(__name__) __copyright__ = ( "Flensburg University of Applied Sciences, " diff --git a/etrago/disaggregate/spatial.py b/etrago/disaggregate/spatial.py index 70da2be5b..768a2d514 100644 --- a/etrago/disaggregate/spatial.py +++ b/etrago/disaggregate/spatial.py @@ -1,7 +1,32 @@ +# -*- coding: utf-8 -*- +# Copyright 2016-2023 Flensburg University of Applied Sciences, +# Europa-Universität Flensburg, +# Centre for Sustainable Energy Systems, +# DLR-Institute for Networked Energy Systems +# +# This program is free software; you can redistribute it and/or +# modify it under the terms of the GNU Affero General Public License as +# published by the Free Software Foundation; either version 3 of the +# License, or (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU Affero General Public License for more details. +# +# You should have received a copy of the GNU Affero General Public License +# along with this program. If not, see . + +# File description +""" +spatial.py defines the methods to run spatial disaggregation on networks. +""" + from functools import reduce from itertools import product from operator import methodcaller as mc, mul as multiply import cProfile +import os import time from loguru import logger as log @@ -9,8 +34,22 @@ from pypsa import Network import pandas as pd -from etrago.tools import noops -from etrago.tools.utilities import residual_load +if "READTHEDOCS" not in os.environ: + from etrago.tools import noops + from etrago.tools.utilities import residual_load + +__copyright__ = ( + "Flensburg University of Applied Sciences, " + "Europa-Universität Flensburg, " + "Centre for Sustainable Energy Systems, " + "DLR-Institute for Networked Energy Systems" +) +__license__ = "GNU Affero General Public License Version 3 (AGPL-3.0)" +__author__ = ( + "MGlauer, MarlonSchlemminger, mariusves, BartelsJ, gnn, lukasoldi, " + "ulfmueller, lukasol, ClaraBuettner, CarlosEpia, KathiEsterl, " + "pieterhexen, fwitte, AmeliaNadal, cjbernal071421" +) class Disaggregation: diff --git a/etrago/disaggregate/temporal.py b/etrago/disaggregate/temporal.py index 5e7b6edee..f30ebec98 100644 --- a/etrago/disaggregate/temporal.py +++ b/etrago/disaggregate/temporal.py @@ -19,7 +19,7 @@ # File description """ -execute.py defines optimization and simulation methods for the etrago object. +temporal.py defines the methods to run temporal disaggregation on networks. """ import logging import os diff --git a/etrago/tools/io.py b/etrago/tools/io.py index 579331aa9..992ea7817 100644 --- a/etrago/tools/io.py +++ b/etrago/tools/io.py @@ -49,19 +49,20 @@ __author__ = "ulfmueller, mariusves, pieterhexen, ClaraBuettner" from importlib import import_module +import logging import os import numpy as np import pandas as pd import pypsa +logger = logging.getLogger(__name__) + if "READTHEDOCS" not in os.environ: - import logging from sqlalchemy.orm.exc import NoResultFound import saio - logger = logging.getLogger(__name__) carr_ormclass = "Source" diff --git a/etrago/tools/utilities.py b/etrago/tools/utilities.py index 5ee967507..660dbc153 100755 --- a/etrago/tools/utilities.py +++ b/etrago/tools/utilities.py @@ -33,14 +33,14 @@ import zipfile from pyomo.environ import Constraint, PositiveReals, Var +from shapely.geometry import LineString, Point +import geopandas as gpd import numpy as np import pandas as pd import pypsa import sqlalchemy.exc if "READTHEDOCS" not in os.environ: - from shapely.geometry import LineString, Point - import geopandas as gpd from etrago.tools import db diff --git a/requirements-doc.txt b/requirements-doc.txt index afabde020..001580922 100644 --- a/requirements-doc.txt +++ b/requirements-doc.txt @@ -9,8 +9,9 @@ nbsphinx numpydoc pandas < 2 pyomo != 6.4.3 -pypsa == 0.20.1 +pypsa == 0.26.2 saio scikit-learn sphinx_rtd_theme > 1.2.2 -sqlalchemy \ No newline at end of file +shapely +sqlalchemy