diff --git a/.github/ISSUE_TEMPLATE/bug_report.md b/.github/ISSUE_TEMPLATE/bug_report.md index 4a251ea..b442187 100644 --- a/.github/ISSUE_TEMPLATE/bug_report.md +++ b/.github/ISSUE_TEMPLATE/bug_report.md @@ -1,41 +1,41 @@ ---- -name: Bug report -about: Create a report to help us improve -title: "[BUG]" -labels: bug -assignees: '' - ---- - -Describe the bug -================ -A clear and concise description of what the bug is. - -To Reproduce -============ - -Steps to reproduce the behavior -------------------------------- -1. Go to '...' -2. Click on '....' -3. Scroll down to '....' -4. See error - -System information ------------------- -Please complete the following information: - - OS: [Hal] - - Python version: [1.0] - - Other details about your setup that could be relevant: [aetherpy branch] - -Screenshots ------------ -If applicable, add screenshots to help explain your problem. - -Expected behavior -================= -A clear and concise description of what you expected to happen. - -Additional context -================== -Add any other context about the problem here. +--- +name: Bug report +about: Create a report to help us improve +title: "[BUG]" +labels: bug +assignees: '' + +--- + +Describe the bug +================ +A clear and concise description of what the bug is. + +To Reproduce +============ + +Steps to reproduce the behavior +------------------------------- +1. Go to '...' +2. Click on '....' +3. Scroll down to '....' +4. See error + +System information +------------------ +Please complete the following information: + - OS: [Hal] + - Python version: [1.0] + - Other details about your setup that could be relevant: [aetherpy branch] + +Screenshots +----------- +If applicable, add screenshots to help explain your problem. + +Expected behavior +================= +A clear and concise description of what you expected to happen. + +Additional context +================== +Add any other context about the problem here. diff --git a/.github/ISSUE_TEMPLATE/feature_request.md b/.github/ISSUE_TEMPLATE/feature_request.md index 6240974..b5496a9 100644 --- a/.github/ISSUE_TEMPLATE/feature_request.md +++ b/.github/ISSUE_TEMPLATE/feature_request.md @@ -1,26 +1,26 @@ ---- -name: Feature request -about: Suggest an idea for this project -title: "[FEAT]" -labels: enhancement -assignees: '' - ---- - -Is your feature request related to a problem? Please describe. -============================================================== -A clear and concise description of what the problem is. For example: -I'm always frustrated when [...] - -Describe the solution you'd like -================================ -A clear and concise description of what you want to happen. - -Describe alternatives you've considered -======================================= -A clear and concise description of any alternative solutions or features -you've considered. - -Additional context -================== -Add any other context or screenshots about the feature request here. +--- +name: Feature request +about: Suggest an idea for this project +title: "[FEAT]" +labels: enhancement +assignees: '' + +--- + +Is your feature request related to a problem? Please describe. +============================================================== +A clear and concise description of what the problem is. For example: +I'm always frustrated when [...] + +Describe the solution you'd like +================================ +A clear and concise description of what you want to happen. + +Describe alternatives you've considered +======================================= +A clear and concise description of any alternative solutions or features +you've considered. + +Additional context +================== +Add any other context or screenshots about the feature request here. diff --git a/.github/ISSUE_TEMPLATE/project-goal.md b/.github/ISSUE_TEMPLATE/project-goal.md index bd69d6c..21b7036 100644 --- a/.github/ISSUE_TEMPLATE/project-goal.md +++ b/.github/ISSUE_TEMPLATE/project-goal.md @@ -1,22 +1,22 @@ ---- -name: Project Goal -about: A general project goal that is not a feature or a bug -title: "[GOAL]" -labels: '' -assignees: '' - ---- - -Describe the goal -================== -Description of the goal, including it's aim and context. - -Provide a general outline -========================= -1. First step -2. Second step -3. Celebrate - -Benefits and downsides -====================== -What are the benefits and downsides of this goal? +--- +name: Project Goal +about: A general project goal that is not a feature or a bug +title: "[GOAL]" +labels: '' +assignees: '' + +--- + +Describe the goal +================== +Description of the goal, including it's aim and context. + +Provide a general outline +========================= +1. First step +2. Second step +3. Celebrate + +Benefits and downsides +====================== +What are the benefits and downsides of this goal? diff --git a/.github/PULL_REQUEST_TEMPLATE.md b/.github/PULL_REQUEST_TEMPLATE.md index d72af3e..3975f90 100644 --- a/.github/PULL_REQUEST_TEMPLATE.md +++ b/.github/PULL_REQUEST_TEMPLATE.md @@ -1,43 +1,43 @@ -# Description - -Addresses # (issue) - -Please include a summary of the change and which issue is fixed. Please also -include relevant motivation and context. List any dependencies that are required -for this change. Please see ``CONTRIBUTING.md`` for more guidelines. - -## Type of change - -Please delete options that are not relevant. - -- Bug fix (non-breaking change that fixes an issue) -- New feature (non-breaking change that adds functionality) -- Breaking change (fix or feature that would cause existing functionality - to not work as expected) -- This change requires a documentation update - - -# How Has This Been Tested? - -Please describe the tests that you ran to verify your changes. Provide -instructions so we can reproduce. - -## Test configuration - -* Operating system: [e.g., Hal] -* Compiler, version number: [e.g., Python 3.X, gcc version X.X] -* Any details about your local setup that are relevant - -# Checklist: - -- [ ] Make sure you are merging into the `develop` (not `main`) branch -- [ ] My code follows the style guidelines of this project -- [ ] I have performed a self-review of my own code -- [ ] I have commented my code, particularly in hard-to-understand areas -- [ ] I have made corresponding changes to the documentation -- [ ] My changes generate no new warnings -- [ ] I have added tests that prove my fix is effective or that my feature works -- [ ] New and existing unit tests pass locally with my changes -- [ ] Any dependent changes have been merged and published in downstream modules -- [ ] Add your name to the bottom of the creators list in the .zenodo.json file -- [N/A] Add a note to `CHANGELOG.md`, summarizing the changes +# Description + +Addresses # (issue) + +Please include a summary of the change and which issue is fixed. Please also +include relevant motivation and context. List any dependencies that are required +for this change. Please see ``CONTRIBUTING.md`` for more guidelines. + +## Type of change + +Please delete options that are not relevant. + +- Bug fix (non-breaking change that fixes an issue) +- New feature (non-breaking change that adds functionality) +- Breaking change (fix or feature that would cause existing functionality + to not work as expected) +- This change requires a documentation update + + +# How Has This Been Tested? + +Please describe the tests that you ran to verify your changes. Provide +instructions so we can reproduce. + +## Test configuration + +* Operating system: [e.g., Hal] +* Compiler, version number: [e.g., Python 3.X, gcc version X.X] +* Any details about your local setup that are relevant + +# Checklist: + +- [ ] Make sure you are merging into the `develop` (not `main`) branch +- [ ] My code follows the style guidelines of this project +- [ ] I have performed a self-review of my own code +- [ ] I have commented my code, particularly in hard-to-understand areas +- [ ] I have made corresponding changes to the documentation +- [ ] My changes generate no new warnings +- [ ] I have added tests that prove my fix is effective or that my feature works +- [ ] New and existing unit tests pass locally with my changes +- [ ] Any dependent changes have been merged and published in downstream modules +- [ ] Add your name to the bottom of the creators list in the .zenodo.json file +- [N/A] Add a note to `CHANGELOG.md`, summarizing the changes diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml index 5f62e01..c9de972 100644 --- a/.github/workflows/main.yml +++ b/.github/workflows/main.yml @@ -1,62 +1,62 @@ -# This workflow will install Python dependencies, run tests and lint with a -# variety of Python versions. For more information see: -# https://help.github.com/actions/language-and-framework-guides/using-python-with-github-actions - -name: Pytest with Flake8 - -on: [push, pull_request] - -jobs: - build: - strategy: - fail-fast: false - matrix: - os: [ubuntu-latest, windows-latest, macos-latest] - python-version: ["3.8", "3.9", "3.10"] - numpy_ver: ["latest"] - include: - - python-version: "3.7" - numpy_ver: "1.18" - os: ubuntu-latest - - name: Python ${{ matrix.python-version }} on ${{ matrix.os }} with numpy ${{ matrix.numpy_ver }} - runs-on: ${{ matrix.os }} - steps: - - uses: actions/checkout@v2 - - name: Set up ffmpeg - uses: FedericoCarboni/setup-ffmpeg@v1 - id: setup-ffmpeg - with: - token: ${{ secrets.GITHUB_TOKEN }} - - name: Set up Python ${{ matrix.python-version }} - uses: actions/setup-python@v2 - with: - python-version: ${{ matrix.python-version }} - - - name: Install standard dependencies and aetherpy - run: | - python -m pip install --upgrade pip - pip install -r requirements.txt - python setup.py install - - - name: Install requirements for testing setup - run: pip install -r test_requirements.txt - - - name: Install NEP29 dependencies - if: ${{ matrix.numpy_ver != 'latest'}} - run: | - pip install --no-binary :numpy: numpy==${{ matrix.numpy_ver }} - - - name: Test PEP8 compliance - run: flake8 . --count --select=D,E,F,H,W --show-source --statistics - - - name: Evaluate complexity - run: flake8 . --count --exit-zero --max-complexity=10 --statistics - - - name: Test with pytest - run: pytest --cov=aetherpy/ - - - name: Publish results to coveralls - env: - GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} - run: coveralls --rcfile=setup.cfg --service=github +# This workflow will install Python dependencies, run tests and lint with a +# variety of Python versions. For more information see: +# https://help.github.com/actions/language-and-framework-guides/using-python-with-github-actions + +name: Pytest with Flake8 + +on: [push, pull_request] + +jobs: + build: + strategy: + fail-fast: false + matrix: + os: [ubuntu-latest, windows-latest, macos-latest] + python-version: ["3.8", "3.9", "3.10"] + numpy_ver: ["latest"] + include: + - python-version: "3.7" + numpy_ver: "1.18" + os: ubuntu-latest + + name: Python ${{ matrix.python-version }} on ${{ matrix.os }} with numpy ${{ matrix.numpy_ver }} + runs-on: ${{ matrix.os }} + steps: + - uses: actions/checkout@v2 + - name: Set up ffmpeg + uses: FedericoCarboni/setup-ffmpeg@v1 + id: setup-ffmpeg + with: + token: ${{ secrets.GITHUB_TOKEN }} + - name: Set up Python ${{ matrix.python-version }} + uses: actions/setup-python@v2 + with: + python-version: ${{ matrix.python-version }} + + - name: Install standard dependencies and aetherpy + run: | + python -m pip install --upgrade pip + pip install -r requirements.txt + python setup.py install + + - name: Install requirements for testing setup + run: pip install -r test_requirements.txt + + - name: Install NEP29 dependencies + if: ${{ matrix.numpy_ver != 'latest'}} + run: | + pip install --no-binary :numpy: numpy==${{ matrix.numpy_ver }} + + - name: Test PEP8 compliance + run: flake8 . --count --select=D,E,F,H,W --show-source --statistics + + - name: Evaluate complexity + run: flake8 . --count --exit-zero --max-complexity=10 --statistics + + - name: Test with pytest + run: pytest --cov=aetherpy/ + + - name: Publish results to coveralls + env: + GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} + run: coveralls --rcfile=setup.cfg --service=github diff --git a/.gitignore b/.gitignore index 699f394..abd79f0 100644 --- a/.gitignore +++ b/.gitignore @@ -49,6 +49,9 @@ coverage.xml .hypothesis/ .pytest_cache/ +# Sample data directories +_data/ + # Translations *.mo *.pot @@ -108,6 +111,9 @@ ENV/ env.bak/ venv.bak/ +# VSCode stuff +.vscode/ + # Spyder project settings .spyderproject .spyproject @@ -125,3 +131,4 @@ dmypy.json # Pyre type checker .pyre/ +*.png diff --git a/.zenodo.json b/.zenodo.json index 1187e54..e60441d 100644 --- a/.zenodo.json +++ b/.zenodo.json @@ -1,30 +1,30 @@ -{ - "keywords": [ - "Aether", - "AetherModel", - "space-physics", - "space-weather", - "ionosphere", - "thermosphere", - "forecasting", - "modeling", - "modelling" - ], - "creators": [ - { - "orcid": "0000-0001-8875-9326", - "affiliation": "Naval Research Laboratory", - "name": "Angeline G. Burrell" - }, - { - "affiliation": "University of Michigan", - "name": "Ridley, Aaron", - "orcid": "0000-0001-6933-8534" - }, - { - "affiliation": "University of Michigan", - "name": "Qusai Al Shidi", - "orcid": "0000-0003-0426-038X" - } - ] -} +{ + "keywords": [ + "Aether", + "AetherModel", + "space-physics", + "space-weather", + "ionosphere", + "thermosphere", + "forecasting", + "modeling", + "modelling" + ], + "creators": [ + { + "orcid": "0000-0001-8875-9326", + "affiliation": "Naval Research Laboratory", + "name": "Angeline G. Burrell" + }, + { + "affiliation": "University of Michigan", + "name": "Ridley, Aaron", + "orcid": "0000-0001-6933-8534" + }, + { + "affiliation": "University of Michigan", + "name": "Qusai Al Shidi", + "orcid": "0000-0003-0426-038X" + } + ] +} diff --git a/CHANGELOG.md b/CHANGELOG.md index 12ac382..4d96041 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,8 +1,8 @@ -Change Log -========== -All notable changes to this project will be documented in this file. -This project adheres to [Semantic Versioning](https://semver.org/). - -[0.0.1a] - 2021-XX-XX ---------------------- -* Alpha release +Change Log +========== +All notable changes to this project will be documented in this file. +This project adheres to [Semantic Versioning](https://semver.org/). + +[0.0.1a] - 2021-XX-XX +--------------------- +* Alpha release diff --git a/CODE_OF_CONDUCT.md b/CODE_OF_CONDUCT.md index c993c3a..027e9e8 100644 --- a/CODE_OF_CONDUCT.md +++ b/CODE_OF_CONDUCT.md @@ -1,200 +1,200 @@ -# Citizen Code of Conduct - -## 1. Purpose - -A primary goal of Aether is to be inclusive to the largest number of -contributors, with the most varied and diverse backgrounds possible. As such, -we are committed to providing a friendly, safe and welcoming environment for -all, regardless of gender, sexual orientation, ability, ethnicity, -socioeconomic status, and religion (or lack thereof). - -This code of conduct outlines our expectations for all those who participate in -our community, as well as the consequences for unacceptable behavior. - -We invite all those who participate in Aether to help us create safe and -positive experiences for everyone. - -## 2. Open [Source/Culture/Tech] Citizenship - -A supplemental goal of this Code of Conduct is to increase open -[source/culture/tech] citizenship by encouraging participants to recognize and -strengthen the relationships between our actions and their effects on our -community. - -Communities mirror the societies in which they exist and positive action is -essential to counteract the many forms of inequality and abuses of power that -exist in society. - -If you see someone who is making an extra effort to ensure our community is -welcoming, friendly, and encourages all participants to contribute to the -fullest extent, we want to know. - -## 3. Expected Behavior - -The following behaviors are expected and requested of all community members: - - * Participate in an authentic and active way. In doing so, you contribute to - the health and longevity of this community. - * Exercise consideration and respect in your speech and actions. - * Attempt collaboration before conflict. - * Refrain from demeaning, discriminatory, or harassing behavior and speech. - * Be mindful of your surroundings and of your fellow participants. Alert - community leaders if you notice a dangerous situation, someone in distress, - or violations of this Code of Conduct, even if they seem inconsequential. - * Remember that community event venues may be shared with members of the - public; please be respectful to all patrons of these locations. - -## 4. Unacceptable Behavior - -The following behaviors are considered harassment and are unacceptable within -our community: - - * Violence, threats of violence or violent language directed against another - person. - * Sexist, racist, homophobic, transphobic, ableist or otherwise discriminatory - jokes and language. - * Posting or displaying sexually explicit or violent material. - * Posting or threatening to post other people's personally identifying - information ("doxing"). - * Personal insults, particularly those related to gender, sexual orientation, - race, religion, or disability. - * Inappropriate photography or recording. - * Inappropriate physical contact. You should have someone's consent before - touching them. - * Unwelcome sexual attention. This includes, sexualized comments or jokes; - inappropriate touching, groping, and unwelcomed sexual advances. - * Deliberate intimidation, stalking or following (online or in person). - * Advocating for, or encouraging, any of the above behavior. - * Sustained disruption of community events, including talks and presentations. - -## 5. Weapons Policy - -No weapons will be allowed at Aether events, community spaces, or in other -spaces covered by the scope of this Code of Conduct. Weapons include but are -not limited to guns, explosives (including fireworks), and large knives such as -those used for hunting or display, as well as any other item used for the -purpose of causing injury or harm to others. Anyone seen in possession of one -of these items will be asked to leave immediately, and will only be allowed to -return without the weapon. Community members are further expected to comply -with all state and local laws on this matter. Aether events will be governed -by the strictest of the policies (Code of Conduct, intitutional rules, -local laws, state laws, federal laws) that apply to a given event. - -## 6. Consequences of Unacceptable Behavior - -Unacceptable behavior from any community member, including sponsors and those -with decision-making authority, will not be tolerated. - -Anyone asked to stop unacceptable behavior is expected to comply immediately. - -If a community member engages in unacceptable behavior, the community -organizers may take any action they deem appropriate, up to and including a -temporary ban or permanent expulsion from the community without warning (and -without refund in the case of a paid event). - -## 7. Enforecement Guidelines - -Community leaders will follow these Community Impact Guidelines in determining -the consequences for any action they deem in violation of this Code of Conduct: - -I. Correction - -Community Impact: Use of inappropriate language or other behavior deemed -unprofessional or unwelcome in the community. - -Consequence: A private, written warning from community leaders, providing -clarity around the nature of the violation and an explanation of why the -behavior was inappropriate. A public apology may be requested. - -II. Warning - -Community Impact: A violation through a single incident or series of actions. - -Consequence: A warning with consequences for continued behavior. No interaction -with the people involved, including unsolicited interaction with those -enforcing the Code of Conduct, for a specified period of time. This includes -avoiding interactions in community spaces as well as external channels like -social media. Violating these terms may lead to a temporary or permanent ban. - -III. Temporary Ban - -Community Impact: A serious violation of community standards, including -sustained inappropriate behavior. - -Consequence: A temporary ban from any sort of interaction or public -communication with the community for a specified period of time. No public or -private interaction with the people involved, including unsolicited interaction -with those enforcing the Code of Conduct, is allowed during this period. -Violating these terms may lead to a permanent ban. - -IV. Permanent Ban - -Community Impact: Demonstrating a pattern of violation of community standards, -including sustained inappropriate behavior, harassment of an individual, or -aggression toward or disparagement of classes of individuals. - -Consequence: A permanent ban from any sort of public interaction within the -community. - - -## 8. Reporting Guidelines - -If you are subject to or witness unacceptable behavior, or have any other -concerns, please notify a community organizer as soon as possible. Currently, -this can be done by emailing the Aether Core Team at aether-core@umich.edu. In -the future, we intend to set up an anonymous method of reporting instances of -unacceptable behavior. - -[Reporting Guidelines](REPORTING_GUIDELINES.md) - -Additionally, community organizers are available to help community members -engage with local law enforcement or to otherwise help those experiencing -unacceptable behavior feel safe. In the context of in-person events, organizers -will also provide escorts as desired by the person experiencing distress. - -## 9. Addressing Grievances - -If you feel you have been falsely or unfairly accused of violating this Code of -Conduct, you should notify Aether Core Team with a concise description of your -grievance, as detailed in the [Reporting Guidelines](REPORTING_GUIDELINES.md). -Your grievance will be handled using the same guidelines as an incident, in -accordance with this document. - -## 10. Scope - -We expect all community participants (contributors, paid or otherwise; -sponsors; and other guests) to abide by this Code of Conduct in all community -venues--online and in-person--as well as in all one-on-one communications -pertaining to community business. - -This code of conduct and its related procedures also applies to unacceptable -behavior occurring outside the scope of community activities when such behavior -has the potential to adversely affect the safety and well-being of community -members. - -## 11. Contact info - -aether-core@umich.edu - -## 12. License and attribution - -The Citizen Code of Conduct is distributed by -[Stumptown Syndicate](http://stumptownsyndicate.org) under a -[Creative Commons Attribution-ShareAlike license](http://creativecommons.org/licenses/by-sa/3.0/). - -Portions of text derived from the -[Django Code of Conduct](https://www.djangoproject.com/conduct/) and the -[Geek Feminism Anti-Harassment Policy](http://geekfeminism.wikia.com/wiki/Conference_anti-harassment/Policy). - -Portions of the text have been taken from the -[Contributor Covenant v2.0](https://www.contributor-covenant.org/version/2/0/code_of_conduct/) - -The text has been further modified by the Aether Core Team. - -_Revision 2.3. Posted 6 March 2017._ - -_Revision 2.2. Posted 4 February 2016._ - -_Revision 2.1. Posted 23 June 2014._ - -_Revision 2.0, adopted by the [Stumptown Syndicate](http://stumptownsyndicate.org) board on 10 January 2013. Posted 17 March 2013._ +# Citizen Code of Conduct + +## 1. Purpose + +A primary goal of Aether is to be inclusive to the largest number of +contributors, with the most varied and diverse backgrounds possible. As such, +we are committed to providing a friendly, safe and welcoming environment for +all, regardless of gender, sexual orientation, ability, ethnicity, +socioeconomic status, and religion (or lack thereof). + +This code of conduct outlines our expectations for all those who participate in +our community, as well as the consequences for unacceptable behavior. + +We invite all those who participate in Aether to help us create safe and +positive experiences for everyone. + +## 2. Open [Source/Culture/Tech] Citizenship + +A supplemental goal of this Code of Conduct is to increase open +[source/culture/tech] citizenship by encouraging participants to recognize and +strengthen the relationships between our actions and their effects on our +community. + +Communities mirror the societies in which they exist and positive action is +essential to counteract the many forms of inequality and abuses of power that +exist in society. + +If you see someone who is making an extra effort to ensure our community is +welcoming, friendly, and encourages all participants to contribute to the +fullest extent, we want to know. + +## 3. Expected Behavior + +The following behaviors are expected and requested of all community members: + + * Participate in an authentic and active way. In doing so, you contribute to + the health and longevity of this community. + * Exercise consideration and respect in your speech and actions. + * Attempt collaboration before conflict. + * Refrain from demeaning, discriminatory, or harassing behavior and speech. + * Be mindful of your surroundings and of your fellow participants. Alert + community leaders if you notice a dangerous situation, someone in distress, + or violations of this Code of Conduct, even if they seem inconsequential. + * Remember that community event venues may be shared with members of the + public; please be respectful to all patrons of these locations. + +## 4. Unacceptable Behavior + +The following behaviors are considered harassment and are unacceptable within +our community: + + * Violence, threats of violence or violent language directed against another + person. + * Sexist, racist, homophobic, transphobic, ableist or otherwise discriminatory + jokes and language. + * Posting or displaying sexually explicit or violent material. + * Posting or threatening to post other people's personally identifying + information ("doxing"). + * Personal insults, particularly those related to gender, sexual orientation, + race, religion, or disability. + * Inappropriate photography or recording. + * Inappropriate physical contact. You should have someone's consent before + touching them. + * Unwelcome sexual attention. This includes, sexualized comments or jokes; + inappropriate touching, groping, and unwelcomed sexual advances. + * Deliberate intimidation, stalking or following (online or in person). + * Advocating for, or encouraging, any of the above behavior. + * Sustained disruption of community events, including talks and presentations. + +## 5. Weapons Policy + +No weapons will be allowed at Aether events, community spaces, or in other +spaces covered by the scope of this Code of Conduct. Weapons include but are +not limited to guns, explosives (including fireworks), and large knives such as +those used for hunting or display, as well as any other item used for the +purpose of causing injury or harm to others. Anyone seen in possession of one +of these items will be asked to leave immediately, and will only be allowed to +return without the weapon. Community members are further expected to comply +with all state and local laws on this matter. Aether events will be governed +by the strictest of the policies (Code of Conduct, intitutional rules, +local laws, state laws, federal laws) that apply to a given event. + +## 6. Consequences of Unacceptable Behavior + +Unacceptable behavior from any community member, including sponsors and those +with decision-making authority, will not be tolerated. + +Anyone asked to stop unacceptable behavior is expected to comply immediately. + +If a community member engages in unacceptable behavior, the community +organizers may take any action they deem appropriate, up to and including a +temporary ban or permanent expulsion from the community without warning (and +without refund in the case of a paid event). + +## 7. Enforecement Guidelines + +Community leaders will follow these Community Impact Guidelines in determining +the consequences for any action they deem in violation of this Code of Conduct: + +I. Correction + +Community Impact: Use of inappropriate language or other behavior deemed +unprofessional or unwelcome in the community. + +Consequence: A private, written warning from community leaders, providing +clarity around the nature of the violation and an explanation of why the +behavior was inappropriate. A public apology may be requested. + +II. Warning + +Community Impact: A violation through a single incident or series of actions. + +Consequence: A warning with consequences for continued behavior. No interaction +with the people involved, including unsolicited interaction with those +enforcing the Code of Conduct, for a specified period of time. This includes +avoiding interactions in community spaces as well as external channels like +social media. Violating these terms may lead to a temporary or permanent ban. + +III. Temporary Ban + +Community Impact: A serious violation of community standards, including +sustained inappropriate behavior. + +Consequence: A temporary ban from any sort of interaction or public +communication with the community for a specified period of time. No public or +private interaction with the people involved, including unsolicited interaction +with those enforcing the Code of Conduct, is allowed during this period. +Violating these terms may lead to a permanent ban. + +IV. Permanent Ban + +Community Impact: Demonstrating a pattern of violation of community standards, +including sustained inappropriate behavior, harassment of an individual, or +aggression toward or disparagement of classes of individuals. + +Consequence: A permanent ban from any sort of public interaction within the +community. + + +## 8. Reporting Guidelines + +If you are subject to or witness unacceptable behavior, or have any other +concerns, please notify a community organizer as soon as possible. Currently, +this can be done by emailing the Aether Core Team at aether-core@umich.edu. In +the future, we intend to set up an anonymous method of reporting instances of +unacceptable behavior. + +[Reporting Guidelines](REPORTING_GUIDELINES.md) + +Additionally, community organizers are available to help community members +engage with local law enforcement or to otherwise help those experiencing +unacceptable behavior feel safe. In the context of in-person events, organizers +will also provide escorts as desired by the person experiencing distress. + +## 9. Addressing Grievances + +If you feel you have been falsely or unfairly accused of violating this Code of +Conduct, you should notify Aether Core Team with a concise description of your +grievance, as detailed in the [Reporting Guidelines](REPORTING_GUIDELINES.md). +Your grievance will be handled using the same guidelines as an incident, in +accordance with this document. + +## 10. Scope + +We expect all community participants (contributors, paid or otherwise; +sponsors; and other guests) to abide by this Code of Conduct in all community +venues--online and in-person--as well as in all one-on-one communications +pertaining to community business. + +This code of conduct and its related procedures also applies to unacceptable +behavior occurring outside the scope of community activities when such behavior +has the potential to adversely affect the safety and well-being of community +members. + +## 11. Contact info + +aether-core@umich.edu + +## 12. License and attribution + +The Citizen Code of Conduct is distributed by +[Stumptown Syndicate](http://stumptownsyndicate.org) under a +[Creative Commons Attribution-ShareAlike license](http://creativecommons.org/licenses/by-sa/3.0/). + +Portions of text derived from the +[Django Code of Conduct](https://www.djangoproject.com/conduct/) and the +[Geek Feminism Anti-Harassment Policy](http://geekfeminism.wikia.com/wiki/Conference_anti-harassment/Policy). + +Portions of the text have been taken from the +[Contributor Covenant v2.0](https://www.contributor-covenant.org/version/2/0/code_of_conduct/) + +The text has been further modified by the Aether Core Team. + +_Revision 2.3. Posted 6 March 2017._ + +_Revision 2.2. Posted 4 February 2016._ + +_Revision 2.1. Posted 23 June 2014._ + +_Revision 2.0, adopted by the [Stumptown Syndicate](http://stumptownsyndicate.org) board on 10 January 2013. Posted 17 March 2013._ diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 1db51e7..76bbbe9 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -1,108 +1,108 @@ -Contributing -============ - -Code ----- - -Please read the [standards document](doc/design/standards) before -contributing. - -### Setup - -If you wish to contribute please start with checking out the `develop` branch -from your fork (you only need a fork if you don't have developer access to this -repository). - -```sh -git clone -git checkout develop -``` - -Once you make your changes you can either test locally by importing `aetherpy` -into `ipython`: - -```python -# assuming you are in the root directory of aetherpy -import aetherpy -``` - -Or you may install it to test how it may work globally while you continue to develop the code: - -```python -python setup.py develop --user -``` - -### Development - -Make new branches for features `git checkout -b my_feature` and commit often -and push a little less often. Try to merge back to main branch as soon as -you have something that works. - -#### Linting - -We recommend ensuring your code is follows PEP8 by using a linter such as -[flake8](https://flake8.pycqa.org/en/latest/). This may by installed using pip. - -To install `cpplint` - -```sh -# depending on your system one of these lines applies -pip install --user flake8 -pip install flake8 -python3 -m pip flake8 -python3 -m pip --user flake8 -``` - -Using a linter in an editor is a good supplement, but not a replacement for -static linters. The linter in the 'atom' editor requires that you install the -`linter` and `gcc-linter` packages. Atom also has additional packages -`whitespaces` and `tabs-to-spaces` to automatically remove whitespaces at the -end of the lines, and convert tabs to spaces. - -### Commit Styling - -The first line of the commit must be *at most* ~50 characters long and -should start with either. - -- `FEAT:` For new feature. -- `BUG:` For bug fix. -- `MERGE:` For merging. -- `DOC:` For documentation update. -- `TEST:` For the addition or modification of tests. -- `STY:` For a style update (e.g., linting). -- `DEP:` Deprecate something, or remove a deprecated object. -- `REVERT:` Revert an earlier commit. -- `MAINT:` For maintenance such as refactoring, typos, etc. - -The commit first line must be in *present* tense so that the commit log has -consistent formatting. For more information check out [conventional commit -messages](https://www.conventionalcommits.org/en/v1.0.0/). - -For example, - -*do:* - -``` -FEAT: Hydrostatic density implementation -``` - -*don't:* - -``` -Implemented hydrostatic density. (feature) -``` - -### Pull Requests - -Make sure you have linted and checked your code before asking for a pull -request. Before requesting a review, ensure the pull request check list has -been completed. Another member must check the code and approve it before merge. - -Issues ------- - -*Issues* are reporting bugs, feature requests, or goals for the project. In -order to submit an issue make sure it follows the [issue -template](.github/ISSUE_TEMPLATE). Please search through the existing issues -before submitting a new one, as someone else may have already come accross and -reported the problem you've encountered. +Contributing +============ + +Code +---- + +Please read the [standards document](doc/design/standards) before +contributing. + +### Setup + +If you wish to contribute please start with checking out the `develop` branch +from your fork (you only need a fork if you don't have developer access to this +repository). + +```sh +git clone +git checkout develop +``` + +Once you make your changes you can either test locally by importing `aetherpy` +into `ipython`: + +```python +# assuming you are in the root directory of aetherpy +import aetherpy +``` + +Or you may install it to test how it may work globally while you continue to develop the code: + +```python +python setup.py develop --user +``` + +### Development + +Make new branches for features `git checkout -b my_feature` and commit often +and push a little less often. Try to merge back to main branch as soon as +you have something that works. + +#### Linting + +We recommend ensuring your code is follows PEP8 by using a linter such as +[flake8](https://flake8.pycqa.org/en/latest/). This may by installed using pip. + +To install `cpplint` + +```sh +# depending on your system one of these lines applies +pip install --user flake8 +pip install flake8 +python3 -m pip flake8 +python3 -m pip --user flake8 +``` + +Using a linter in an editor is a good supplement, but not a replacement for +static linters. The linter in the 'atom' editor requires that you install the +`linter` and `gcc-linter` packages. Atom also has additional packages +`whitespaces` and `tabs-to-spaces` to automatically remove whitespaces at the +end of the lines, and convert tabs to spaces. + +### Commit Styling + +The first line of the commit must be *at most* ~50 characters long and +should start with either. + +- `FEAT:` For new feature. +- `BUG:` For bug fix. +- `MERGE:` For merging. +- `DOC:` For documentation update. +- `TEST:` For the addition or modification of tests. +- `STY:` For a style update (e.g., linting). +- `DEP:` Deprecate something, or remove a deprecated object. +- `REVERT:` Revert an earlier commit. +- `MAINT:` For maintenance such as refactoring, typos, etc. + +The commit first line must be in *present* tense so that the commit log has +consistent formatting. For more information check out [conventional commit +messages](https://www.conventionalcommits.org/en/v1.0.0/). + +For example, + +*do:* + +``` +FEAT: Hydrostatic density implementation +``` + +*don't:* + +``` +Implemented hydrostatic density. (feature) +``` + +### Pull Requests + +Make sure you have linted and checked your code before asking for a pull +request. Before requesting a review, ensure the pull request check list has +been completed. Another member must check the code and approve it before merge. + +Issues +------ + +*Issues* are reporting bugs, feature requests, or goals for the project. In +order to submit an issue make sure it follows the [issue +template](.github/ISSUE_TEMPLATE). Please search through the existing issues +before submitting a new one, as someone else may have already come accross and +reported the problem you've encountered. diff --git a/MANIFEST.in b/MANIFEST.in index bc03c0c..d6da5ec 100644 --- a/MANIFEST.in +++ b/MANIFEST.in @@ -1,7 +1,7 @@ -recursive-include aetherpy/*.py -include aetherpy/version.txt -include *.txt -include *.py -include *.md -include *.rst -include LICENSE +recursive-include aetherpy/*.py +include aetherpy/version.txt +include *.txt +include *.py +include *.md +include *.rst +include LICENSE diff --git a/README.md b/README.md index aa02a2c..707824e 100644 --- a/README.md +++ b/README.md @@ -1,56 +1,71 @@ -# aetherpy -[![Coverage Status](https://coveralls.io/repos/github/AetherModel/aetherpy/badge.svg?branch=main)](https://coveralls.io/github/AetherModel/aetherpy?branch=main) - -The Python package that supports Aether model data management and analysis. - -# Installation - -## Starting from scratch -* Python and the packages aetherpy depends on are freely available. Python may - be obtained through various package managers or from https://www.python.org/. - A common OS-independent Python package manager to obtain the aetherpy - dependencies is [PyPi](https://pypi.org/). - -## Installation from GitHub - -``` -git clone https://github.com/AetherModel/aetherpy -cd aetherpy -python setup.py install -``` - -If you want to install aetherpy for the entire system, you will need to add -`sudo` in front of the final command in the above block. To install for -yourself without root priveledges, append `--user` to the end of the final -command. - -## Installation from PyPi - -Pip installation will soon be available - -## Installation for development - -If you intend to contribute to the testing or development of aetherpy, you will -need to install in develop mode: - -``` -python setup.py develop -``` - -This allows you to change branches to the desired test brach or alter files -without needing to re-install aetherpy. - -# Getting Started - -aetherpy contains a test script to plot model results. It may be called via: - -``` -python aetherpy/run_plot_model_results.py var=3 -alt=120 [file_dir]/3DALL*.bin -``` - -where [file_dir] is the directory where you have Aether model output files. You -can see the help by typing: - -``` -python aetherpy/run_plot_model_results.py -h -``` +# aetherpy +[![Coverage Status](https://coveralls.io/repos/github/AetherModel/aetherpy/badge.svg?branch=main)](https://coveralls.io/github/AetherModel/aetherpy?branch=main) + +The Python package that supports Aether model data management and analysis. + +# Installation + +## Starting from scratch +* Python and the packages aetherpy depends on are freely available. Python may + be obtained through various package managers or from https://www.python.org/. + A common OS-independent Python package manager to obtain the aetherpy + dependencies is [PyPi](https://pypi.org/). + +## Installation from GitHub + +``` +git clone https://github.com/AetherModel/aetherpy +cd aetherpy +python setup.py install +``` + +If you want to install aetherpy for the entire system, you will need to add +`sudo` in front of the final command in the above block. To install for +yourself without root priveledges, append `--user` to the end of the final +command. + +## Installation from PyPi + +Pip installation will soon be available + +## Installation for development + +If you intend to contribute to the testing or development of aetherpy, you will +need to install in develop mode: + +``` +python setup.py develop +``` + +This allows you to change branches to the desired test brach or alter files +without needing to re-install aetherpy. + +## Installation for development using Conda + +For developers, aetherpy's dependencies may be installed using the conda package manager via the following steps: + +1. If not already installed, install conda through your preferred distribution of Anaconda / Miniconda. +2. Create a new conda environment with the desired dependencies by running the following command: +``` +conda create -n aetherpy_env matplotlib numpy pytest netCDF4 cartopy +``` +3. Activate the conda environment and install aetherpy in develop mode: +``` +conda activate aetherpy_env +python setup.py develop +``` + +# Getting Started + +aetherpy contains a test script to plot model results. It may be called via: + +``` +python aetherpy/run_plot_model_results.py var=3 -alt=120 [file_dir]/3DALL*.bin +``` + +where [file_dir] is the directory where you have Aether model output files. You +can see the help by typing: + +``` +python aetherpy/run_plot_model_results.py -h +``` diff --git a/aetherpy/__init__.py b/aetherpy/__init__.py index fb70530..26ee52b 100644 --- a/aetherpy/__init__.py +++ b/aetherpy/__init__.py @@ -1,24 +1,24 @@ -#!/usr/bin/env python -# Copyright 2020, the Aether Development Team (see doc/dev_team.md for members) -# Full license can be found in License.md -"""aetherpy package for ingesting and analysing model outputs.""" - -import logging -import os - -# Define a logger object to allow easier log handling -logging.raiseExceptions = False -logger = logging.getLogger('aetherpy_logger') - -# Import the sub-modules -from aetherpy import io -from aetherpy import plot -from aetherpy import utils - -# Define global variables -vfile = os.path.join(os.path.abspath(os.path.dirname(__file__)), 'version.txt') -with open(vfile, 'r') as fin: - __version__ = fin.read().strip() - -# Clean up -del vfile, fin +#!/usr/bin/env python +# Copyright 2020, the Aether Development Team (see doc/dev_team.md for members) +# Full license can be found in License.md +"""aetherpy package for ingesting and analysing model outputs.""" + +import logging +import os + +# Define a logger object to allow easier log handling +logging.raiseExceptions = False +logger = logging.getLogger('aetherpy_logger') + +# Import the sub-modules +from aetherpy import io +from aetherpy import plot +from aetherpy import utils + +# Define global variables +vfile = os.path.join(os.path.abspath(os.path.dirname(__file__)), 'version.txt') +with open(vfile, 'r') as fin: + __version__ = fin.read().strip() + +# Clean up +del vfile, fin diff --git a/aetherpy/io/__init__.py b/aetherpy/io/__init__.py index 3075f0b..af06db3 100644 --- a/aetherpy/io/__init__.py +++ b/aetherpy/io/__init__.py @@ -1,7 +1,7 @@ -#!/usr/bin/env python -# Copyright 2020, the Aether Development Team (see doc/dev_team.md for members) -# Full license can be found in License.md -"""Utilities supporting I/O operations.""" - -from aetherpy.io import fetch_routines -from aetherpy.io import read_routines +#!/usr/bin/env python +# Copyright 2020, the Aether Development Team (see doc/dev_team.md for members) +# Full license can be found in License.md +"""Utilities supporting I/O operations.""" + +from aetherpy.io import fetch_routines +from aetherpy.io import read_routines diff --git a/aetherpy/io/fetch_routines.py b/aetherpy/io/fetch_routines.py index 4f1b8cd..5c673b4 100644 --- a/aetherpy/io/fetch_routines.py +++ b/aetherpy/io/fetch_routines.py @@ -1,64 +1,64 @@ -#!/usr/bin/env python -# Copyright 2020, the Aether Development Team (see doc/dev_team.md for members) -# Full license can be found in License.md -"""Routines to find and retrieve files.""" - -from glob import glob -import os - -from aetherpy import logger - - -def get_filelist(file_dir, file_type='ALL', file_ext="bin"): - """Get a list of Aether-style model filenames from a specified directory. - - Parameters - ---------- - file_dir : str - File directory to search or a glob search string - file_type : str - Desired file type, accepts 'ALL', 'ION', and 'NEU' (default='ALL') - file_ext : str - File extenstion, without period (default='bin') - - Returns - ------- - filelist : list - List of Aether-style model filenames - - Notes - ----- - Only retrieves 1D files if 3D files are not present - - """ - - # Check the file type and warn user for untested use cases - file_type = file_type.upper() - if file_type not in ['ALL', 'ION', 'NEU']: - logger.warning(''.join(['unexpected file type [', file_type, - '], routine may not work'])) - - # Check to see if the directory exists - if os.path.isdir(file_dir): - # Get a list of 3D files or 1D files - filelist = glob(os.path.join( - file_dir, '3D{:s}*.{:s}'.format(file_type, file_ext))) - - if len(filelist) == 0: - logger.info("".join(["No 3D", file_type, " files found in ", - file_dir, ", checking for 1D", file_type])) - filelist = glob(os.path.join( - file_dir, '1D{:s}*.{:s}'.format(file_type, file_ext))) - - if len(filelist) == 0: - logger.warning('No 1D{:s} files found in {:s}'.format( - file_type, file_dir)) - else: - # This may be a glob search string - filelist = glob(file_dir) - - if len(filelist) == 0: - logger.warning('No files found using search string: {:s}'.format( - file_dir)) - - return filelist +#!/usr/bin/env python +# Copyright 2020, the Aether Development Team (see doc/dev_team.md for members) +# Full license can be found in License.md +"""Routines to find and retrieve files.""" + +from glob import glob +import os + +from aetherpy import logger + + +def get_filelist(file_dir, file_type='ALL', file_ext="bin"): + """Get a list of Aether-style model filenames from a specified directory. + + Parameters + ---------- + file_dir : str + File directory to search or a glob search string + file_type : str + Desired file type, accepts 'ALL', 'ION', and 'NEU' (default='ALL') + file_ext : str + File extenstion, without period (default='bin') + + Returns + ------- + filelist : list + List of Aether-style model filenames + + Notes + ----- + Only retrieves 1D files if 3D files are not present + + """ + + # Check the file type and warn user for untested use cases + file_type = file_type.upper() + if file_type not in ['ALL', 'ION', 'NEU']: + logger.warning(''.join(['unexpected file type [', file_type, + '], routine may not work'])) + + # Check to see if the directory exists + if os.path.isdir(file_dir): + # Get a list of 3D files or 1D files + filelist = glob(os.path.join( + file_dir, '3D{:s}*.{:s}'.format(file_type, file_ext))) + + if len(filelist) == 0: + logger.info("".join(["No 3D", file_type, " files found in ", + file_dir, ", checking for 1D", file_type])) + filelist = glob(os.path.join( + file_dir, '1D{:s}*.{:s}'.format(file_type, file_ext))) + + if len(filelist) == 0: + logger.warning('No 1D{:s} files found in {:s}'.format( + file_type, file_dir)) + else: + # This may be a glob search string + filelist = glob(file_dir) + + if len(filelist) == 0: + logger.warning('No files found using search string: {:s}'.format( + file_dir)) + + return filelist diff --git a/aetherpy/io/read_routines.py b/aetherpy/io/read_routines.py index f6f03ad..c14d2c8 100644 --- a/aetherpy/io/read_routines.py +++ b/aetherpy/io/read_routines.py @@ -3,6 +3,8 @@ # Full license can be found in License.md """Routines to read Aether files.""" +# TODO(#19): Most parts of this file will be changed when switching to xarray + import datetime as dt from netCDF4 import Dataset import numpy as np @@ -14,6 +16,32 @@ from aetherpy import logger +class DataArray(np.ndarray): + def __new__(cls, input_array, attrs={}): + obj = np.asarray(input_array).view(cls) + obj.attrs = attrs + return obj + + def __array_finalize__(self, obj): + if obj is None: + return + self.attrs = getattr(obj, 'attrs', { + 'units': None, + 'long_name': None + }) + + +def read_file(filename, file_vars=None): + file_ext = filename.rsplit('.', 1)[-1] + if (file_ext == 'nc'): + data = read_blocked_netcdf_file(filename, file_vars) + elif (file_ext == 'bin'): + data = read_gitm_file(filename, file_vars) + else: + raise ValueError(f"Invalid file extension {filename}") + return data + + def parse_line_into_int_and_string(line, parse_string=True): """Parse a data string into integer and string components. @@ -188,8 +216,8 @@ def read_aether_netcdf_header(filename, epoch_name='time'): header["nlons"] = nlons header["nlats"] = nlats header["nalts"] = nalts - elif(header['nlons'] != nlons or header['nlats'] != nlats - or header['nalts'] != nalts): + elif (header['nlons'] != nlons or header['nlats'] != nlats + or header['nalts'] != nalts): raise IOError(''.join(['unexpected dimensions for ', 'variable ', var.name, ' in file ', filename])) @@ -287,6 +315,164 @@ def read_aether_ascii_header(filename): return header +def read_blocked_netcdf_header(filename): + """Read header information from a blocked Aether netcdf file. + + Parameters + ---------- + filename : str + An Aether netCDF filename + + Returns + ------- + header : dict + A dictionary containing header information from the netCDF file, + including: + filename - filename of file containing header data + nlons - number of longitude grids per block + nlats - number of latitude grids per block + nalts - number of altitude grids per block + nblocks - number of blocks in file + vars - list of data variable names + time - datetime for time of file + + Raises + -------- + IOError + If the input file does not exist + KeyError + If any expected dimensions of the input netCDF file are not present + + Notes + ----- + This routine only works with blocked Aether netCDF files. + + """ + + # Checks for file existence + if not os.path.isfile(filename): + raise IOError(f"unknown aether netCDF blocked file: {filename}") + + header = {'filename': filename} # Included for compatibility + + with Dataset(filename, 'r') as ncfile: + # Process header information: nlons, nlats, nalts, nblocks + header['nlons'] = len(ncfile.dimensions['lon']) + header['nlats'] = len(ncfile.dimensions['lat']) + header['nalts'] = len(ncfile.dimensions['z']) + if 'block' in ncfile.dimensions: + header['nblocks'] = len(ncfile.dimensions['block']) + else: + header['nblocks'] = 1 + + # Included for compatibility ('vars' slices time out for some reason) + header['vars'] = list(ncfile.variables.keys())[1:] + header['time'] = epoch_to_datetime( + np.array(ncfile.variables['time'])[0]) + + return header + + +def read_blocked_netcdf_file(filename, file_vars=None): + """Read all data from a blocked Aether netcdf file. + + Parameters + ---------- + filename : str + An Aether netCDF filename + file_vars : list or NoneType + List of desired variable neames to read, or None to read all + (default=None) + + Returns + ------- + data : dict + A dictionary containing all data from the netCDF file, including: + filename - filename of file containing header data + nlons - number of longitude grids per block + nlats - number of latitude grids per block + nalts - number of altitude grids per block + nblocks - number of blocks in file + vars - list of data variable names + time - datetime for time of file + The dictionary also contains a read_routines.DataArray keyed to the + corresponding variable name. Each DataArray carries both the variable's + data from the netCDF file and the variable's corresponding attributes. + + Raises + -------- + IOError + If the input file does not exist + KeyError + If any expected dimensions of the input netCDF file are not present + + Notes + ----- + This routine only works with blocked Aether netCDF files. + + """ + + # Checks for file existence + if not os.path.isfile(filename): + raise IOError(f"unknown aether netCDF blocked file: {filename}") + + # NOTE: Includes header information for easy access until + # updated package structure is confirmed + # Initialize data dict with defaults (will remove these defaults later) + data = {'filename': filename, + 'units': '', + 'long_name': None} + + with Dataset(filename, 'r') as ncfile: + # Process header information: nlons, nlats, nalts, nblocks + data['nlons'] = len(ncfile.dimensions['lon']) + data['nlats'] = len(ncfile.dimensions['lat']) + data['nalts'] = len(ncfile.dimensions['z']) + if 'block' in ncfile.dimensions: + data['nblocks'] = len(ncfile.dimensions['block']) + + # Included for compatibility + data['vars'] = [var for var in ncfile.variables.keys() + if file_vars is None or var in file_vars] + + # Fetch requested variable data + for key in data['vars']: + var = ncfile.variables[key] # key is var name + data[key] = DataArray(np.array(var), var.__dict__) + + data['time'] = epoch_to_datetime(np.array(ncfile.variables['time'])[0]) + + return data + + +def standardize_data(data): + # Coerce to block-based (4-dimensional) in-place + if 'nblocks' not in data: + data['nblocks'] = 1 + for key in data.keys(): + if isinstance(data[key], np.ndarray) and len(data[key].shape) == 3: + data[key] = np.expand_dims(data[key], axis=0) + # Change to standardized coordinate variable names + standard_name_map = { + 'Longitude': 'lon', + 'Latitude': 'lat', + 'Altitude': 'z', + 'V!Dn!N(east)': 'Zonal Wind', + 'V!Dn!N(north)': 'Meridional Wind', + 'V!Dn!N(up)': 'Vertical Wind', + 'V!Di!N(east)': 'BulkIon Velocity (Zonal)', + 'V!Di!N(north)': 'BulkIon Velocity (Meridional)', + 'V!Di!N(up)': 'BulkIon Velocity (Vertical)' + } + for key in list(data.keys()): + if key in standard_name_map: + data[standard_name_map[key]] = data.pop(key) + for i, var in enumerate(data['vars']): + if var in standard_name_map: + data['vars'][i] = standard_name_map[var] + return data + + def read_aether_one_binary_file(header, ifile, vars_to_read): """Read in list of variables from a single netCDF file. @@ -423,8 +609,8 @@ def read_gitm_headers(filelist, finds=-1): header["nlons"] = nlons header["nlats"] = nlats header["nalts"] = nalts - elif(header['nlons'] != nlons or header['nlats'] != nlats - or header['nalts'] != nalts): + elif (header['nlons'] != nlons or header['nlats'] != nlats + or header['nalts'] != nalts): raise IOError(''.join(['unexpected dimensions in file ', filename])) @@ -596,11 +782,13 @@ def read_gitm_file(filename, file_vars=None): idata_length = ntotal * 8 + 8 # Save the data for the desired variables - for ivar in file_vars: + for ivar, var in enumerate(data['vars']): fin.seek(iheader_length + ivar * idata_length) sdata = unpack(end_char + 'l', fin.read(4))[0] - data[ivar] = np.array( + data[var] = np.array( unpack(end_char + '%id' % (ntotal), fin.read(sdata))).reshape( (data["nlons"], data["nlats"], data["nalts"]), order="F") + for var in ['Longitude', 'Latitude']: + data[var] = np.degrees(data[var]) return data diff --git a/aetherpy/plot/__init__.py b/aetherpy/plot/__init__.py index 6fb1f1d..bf0d100 100644 --- a/aetherpy/plot/__init__.py +++ b/aetherpy/plot/__init__.py @@ -1,7 +1,7 @@ -#!/usr/bin/env python -# Copyright 2020, the Aether Development Team (see doc/dev_team.md for members) -# Full license can be found in License.md -"""Utilities to plot model output as figures or movies.""" - -from aetherpy.plot import data_prep -from aetherpy.plot import movie_routines +#!/usr/bin/env python +# Copyright 2020, the Aether Development Team (see doc/dev_team.md for members) +# Full license can be found in License.md +"""Utilities to plot model output as figures or movies.""" + +from aetherpy.plot import data_prep +from aetherpy.plot import movie_routines diff --git a/aetherpy/plot/data_prep.py b/aetherpy/plot/data_prep.py index 731367d..476986a 100644 --- a/aetherpy/plot/data_prep.py +++ b/aetherpy/plot/data_prep.py @@ -1,165 +1,165 @@ -#!/usr/bin/env python -# Copyright 2020, the Aether Development Team (see doc/dev_team.md for members) -# Full license can be found in License.md -""" Utilities for slicing and preparing data for plotting -""" - -import numpy as np - -from aetherpy import logger - - -def get_cut_index(lons, lats, alts, cut_val, isgrid=False, cut_coord='alt'): - """Select indices needed to obtain a slice in the remaining two coords. - - Parameters - ---------- - lons : array-like - 1D array of longitudes in degrees - lats : array-like - 1D array of latitudes in degrees - alts : array-like - 1D array of altitudes in km - cut_val : int or float - Data value or grid number along that will be held constant - isgrid : bool - Flag that indicates `cut_val` is a grid index if True or that it is a - data value if False (default=False) - cut_coord : str - Expects one of 'lat', 'lon', or 'alt' and will return an index for - that data, allowing a 2D slice to be created along other two - coordinates (default='alt') - - Returns - ------- - icut : int - Cut index - cut_data : tuple - Tuple to select data of the expected dimensions along `icut` - x_coord : array-like - Array of data to include along the x-axis - y_coord : array-like - Array of data to include along the y-axis - z_val : float - Data value for cut index - - Notes - ----- - `lons`, `lats`, and `alts` do not need to be the same shape - - Raises - ------ - ValueError - If `cut_val` is outside the possible range of values - - """ - # Ensure inputs are array-like - lons = np.asarray(lons) - lats = np.asarray(lats) - alts = np.asarray(alts) - - # Initialize the output slices - out_slice = {"lon": slice(0, lons.shape[0], 1), - "lat": slice(0, lats.shape[0], 1), - "alt": slice(0, alts.shape[0], 1)} - - # Set the x, y, and z coordinates - x_coord = lons if cut_coord in ['alt', 'lat'] else lats - y_coord = alts if cut_coord in ['lat', 'lon'] else lats - - if cut_coord == 'alt': - z_coord = alts - else: - if cut_coord == 'lat': - z_coord = lats - else: - z_coord = lons - - # Find the desired index for the z-coordinate value - if isgrid: - icut = cut_val - else: - if cut_val < z_coord.min() or cut_val > z_coord.max(): - raise ValueError('Requested cut is outside the coordinate range') - - icut = abs(z_coord - cut_val).argmin() - - # Get the z-value if possible. - if icut < 0 or icut >= len(z_coord): - raise ValueError('Requested cut is outside the index range') - - # Warn the user if they selected a suspect index - if cut_coord == "alt": - if icut > len(z_coord) - 3: - logger.warning(''.join(['Requested altitude slice is above ', - 'the recommended upper limit'])) - else: - if icut == 0 or icut == len(z_coord) - 1: - logger.warning(''.join(['Requested ', cut_coord, ' slice is ', - 'beyond the recommended limits'])) - - z_val = z_coord[icut] - - # Finalize the slicing tuple - out_slice[cut_coord] = icut - cut_data = tuple([out_slice[coord] for coord in ['lon', 'lat', 'alt']]) - - return icut, cut_data, x_coord, y_coord, z_val - - -def calc_tec(alt, ne, ialt_min=2, ialt_max=-4): - """Calculate TEC for the specified altitude range from electron density. - - Parameters - ---------- - alt : array-like - 1D Altitude data in km - ne : array-like - Electron density in cubic meters, with altitude as the last axis - ialt_min : int - Lowest altitude index to use, may be negative (default=2) - ialt_max : int - Highest altitude index to use, may be negative (default=-4) - - Returns - ------- - tec : array-like - Array of TEC values in TECU - - Notes - ----- - TEC = Total Electron Content - TECU = 10^16 m^-2 (TEC Units) - - Raises - ------ - ValueError - If the altitude integration range is poorly defined. - - """ - alts = np.asarray(alt) - nes = np.asarray(ne) - - # Initialize the TEC - tec = np.zeros(shape=nes[..., 0].shape) - - # Get the range of altitudes to cycle over - if ialt_max < 0: - ialt_max += alt.shape[0] - - if ialt_min < 0: - ialt_min += alt.shape[0] - - if ialt_max <= ialt_min: - raise ValueError('`ialt_max` must be greater than `ialt_min`') - - # Cycle through each altitude bin, summing the contribution from the - # electron density - for ialt in np.arange(ialt_min, ialt_max, 1): - tec += 1000.0 * nes[..., ialt] * (alts[ialt + 1] - - alts[ialt - 1]) / 2.0 - - # Convert TEC from per squared meters to TECU - tec /= 1.0e16 - - return tec +#!/usr/bin/env python +# Copyright 2020, the Aether Development Team (see doc/dev_team.md for members) +# Full license can be found in License.md +""" Utilities for slicing and preparing data for plotting +""" + +import numpy as np + +from aetherpy import logger + + +def get_cut_index(lons, lats, alts, cut_val, isgrid=False, cut_coord='alt'): + """Select indices needed to obtain a slice in the remaining two coords. + + Parameters + ---------- + lons : array-like + 1D array of longitudes in degrees + lats : array-like + 1D array of latitudes in degrees + alts : array-like + 1D array of altitudes in km + cut_val : int or float + Data value or grid number along that will be held constant + isgrid : bool + Flag that indicates `cut_val` is a grid index if True or that it is a + data value if False (default=False) + cut_coord : str + Expects one of 'lat', 'lon', or 'alt' and will return an index for + that data, allowing a 2D slice to be created along other two + coordinates (default='alt') + + Returns + ------- + icut : int + Cut index + cut_data : tuple + Tuple to select data of the expected dimensions along `icut` + x_coord : array-like + Array of data to include along the x-axis + y_coord : array-like + Array of data to include along the y-axis + z_val : float + Data value for cut index + + Notes + ----- + `lons`, `lats`, and `alts` do not need to be the same shape + + Raises + ------ + ValueError + If `cut_val` is outside the possible range of values + + """ + # Ensure inputs are array-like + lons = np.asarray(lons) + lats = np.asarray(lats) + alts = np.asarray(alts) + + # Initialize the output slices + out_slice = {"lon": slice(0, lons.shape[0], 1), + "lat": slice(0, lats.shape[0], 1), + "alt": slice(0, alts.shape[0], 1)} + + # Set the x, y, and z coordinates + x_coord = lons if cut_coord in ['alt', 'lat'] else lats + y_coord = alts if cut_coord in ['lat', 'lon'] else lats + + if cut_coord == 'alt': + z_coord = alts + else: + if cut_coord == 'lat': + z_coord = lats + else: + z_coord = lons + + # Find the desired index for the z-coordinate value + if isgrid: + icut = cut_val + else: + if cut_val < z_coord.min() or cut_val > z_coord.max(): + raise ValueError('Requested cut is outside the coordinate range') + + icut = abs(z_coord - cut_val).argmin() + + # Get the z-value if possible. + if icut < 0 or icut >= len(z_coord): + raise ValueError('Requested cut is outside the index range') + + # Warn the user if they selected a suspect index + if cut_coord == "alt": + if icut > len(z_coord) - 3: + logger.warning(''.join(['Requested altitude slice is above ', + 'the recommended upper limit'])) + else: + if icut == 0 or icut == len(z_coord) - 1: + logger.warning(''.join(['Requested ', cut_coord, ' slice is ', + 'beyond the recommended limits'])) + + z_val = z_coord[icut] + + # Finalize the slicing tuple + out_slice[cut_coord] = icut + cut_data = tuple([out_slice[coord] for coord in ['lon', 'lat', 'alt']]) + + return icut, cut_data, x_coord, y_coord, z_val + + +def calc_tec(alt, ne, ialt_min=2, ialt_max=-4): + """Calculate TEC for the specified altitude range from electron density. + + Parameters + ---------- + alt : array-like + 1D Altitude data in km + ne : array-like + Electron density in cubic meters, with altitude as the last axis + ialt_min : int + Lowest altitude index to use, may be negative (default=2) + ialt_max : int + Highest altitude index to use, may be negative (default=-4) + + Returns + ------- + tec : array-like + Array of TEC values in TECU + + Notes + ----- + TEC = Total Electron Content + TECU = 10^16 m^-2 (TEC Units) + + Raises + ------ + ValueError + If the altitude integration range is poorly defined. + + """ + alts = np.asarray(alt) + nes = np.asarray(ne) + + # Initialize the TEC + tec = np.zeros(shape=nes[..., 0].shape) + + # Get the range of altitudes to cycle over + if ialt_max < 0: + ialt_max += alt.shape[0] + + if ialt_min < 0: + ialt_min += alt.shape[0] + + if ialt_max <= ialt_min: + raise ValueError('`ialt_max` must be greater than `ialt_min`') + + # Cycle through each altitude bin, summing the contribution from the + # electron density + for ialt in np.arange(ialt_min, ialt_max, 1): + tec += 1000.0 * nes[..., ialt] * (alts[ialt + 1] + - alts[ialt - 1]) / 2.0 + + # Convert TEC from per squared meters to TECU + tec /= 1.0e16 + + return tec diff --git a/aetherpy/plot/movie_routines.py b/aetherpy/plot/movie_routines.py index 92a1b24..2f627b1 100644 --- a/aetherpy/plot/movie_routines.py +++ b/aetherpy/plot/movie_routines.py @@ -1,114 +1,114 @@ -#!/usr/bin/env python -# Copyright 2020, the Aether Development Team (see doc/dev_team.md for members) -# Full license can be found in License.md -"""Utilities for outputing movies.""" - -from glob import glob -import os -import re - - -def setup_movie_dir(movie_dir, file_glob="image_????.png", overwrite=True): - """Set up a directory for movie files. - - Parameters - ---------- - movie_dir : str - Output filename with directory, but no extention - file_glob : str - Base filename without directory, using wildcards to identify potential - images that would be included in the movie (default='image_????.png') - overwrite : bool - Overwrite an existing movie of the same name. Windows OS frequently do - not allow Python to remove files, so Windows users must clean up these - files themselves. (default=True) - - Returns - ------- - img_names : str - Image name formatting string that may be used for `save_movie` - - Raises - ------ - IOError - If image files already exist and `overwrite` is False - - """ - - # Test the output directory for existence and existing image files - if os.path.isdir(movie_dir): - oldfiles = glob(os.path.join(movie_dir, file_glob)) - if len(oldfiles) > 0: - if overwrite: - for ofile in oldfiles: - os.remove(ofile) - else: - raise IOError('files present in movie directory: {:}'.format( - movie_dir)) - else: - os.makedirs(movie_dir) - - # Create the movie image naming string based on the `file_glob` variable - file_base, file_ext = os.path.splitext(file_glob) - dnum = len(file_base.split('?')) - 1 + 4 * (len(file_base.split("*")) - 1) - file_pre = re.split(r"\W", file_base)[0] - img_names = os.path.join(movie_dir, "".join([ - file_pre, "_%0", "{:d}".format(dnum), "d", file_ext])) - - return img_names - - -def save_movie(movie_dir, movie_name="movie.mp4", image_files="image_%04d.png", - rate=30, overwrite=True): - """Save the output as a movie. - - Parameters - ---------- - movie_dir : str - Output directory for the movie - move_name : str - Output movie name with extention (default='movie.mp4') - image_files : str - Full-path names for images to be used in the movie, using C-style - descriptors for numbers that indicate the order of file inclusion - within the movie. For example, 'image_0001.png' and 'image_0002.png' - would create a 2 frame movie with 'image_0001.png' appearing first - using the default value as long as the files exist in the directory - where the code is run (img_names='image_%04d.png') - rate : int - Movie frame rate (default=30) - overwrite : bool - Overwrite an existing movie of the same name. Windows OS frequently do - not allow Python to remove files, so Windows users must clean up these - files themselves. (default=True) - - Returns - ------- - outfile : str - Output movie name - - Raises - ------ - IOError - If movie file already exists and `overwrite` is False. - - Notes - ----- - Uses ffmpeg to create the movie, this must be installed for success - - """ - # Construct the output filenames - outfile = os.path.join(movie_dir, movie_name) - - # Test the output file - if os.path.isfile(outfile): - if overwrite: - os.remove(outfile) - else: - raise IOError('movie file {:} already exists'.format(outfile)) - - # Construct the movie commannd - command = "ffmpeg -r {:d} -i {:s} {:s}".format(rate, image_files, outfile) - os.system(command) - - return outfile +#!/usr/bin/env python +# Copyright 2020, the Aether Development Team (see doc/dev_team.md for members) +# Full license can be found in License.md +"""Utilities for outputing movies.""" + +from glob import glob +import os +import re + + +def setup_movie_dir(movie_dir, file_glob="image_????.png", overwrite=True): + """Set up a directory for movie files. + + Parameters + ---------- + movie_dir : str + Output filename with directory, but no extention + file_glob : str + Base filename without directory, using wildcards to identify potential + images that would be included in the movie (default='image_????.png') + overwrite : bool + Overwrite an existing movie of the same name. Windows OS frequently do + not allow Python to remove files, so Windows users must clean up these + files themselves. (default=True) + + Returns + ------- + img_names : str + Image name formatting string that may be used for `save_movie` + + Raises + ------ + IOError + If image files already exist and `overwrite` is False + + """ + + # Test the output directory for existence and existing image files + if os.path.isdir(movie_dir): + oldfiles = glob(os.path.join(movie_dir, file_glob)) + if len(oldfiles) > 0: + if overwrite: + for ofile in oldfiles: + os.remove(ofile) + else: + raise IOError('files present in movie directory: {:}'.format( + movie_dir)) + else: + os.makedirs(movie_dir) + + # Create the movie image naming string based on the `file_glob` variable + file_base, file_ext = os.path.splitext(file_glob) + dnum = len(file_base.split('?')) - 1 + 4 * (len(file_base.split("*")) - 1) + file_pre = re.split(r"\W", file_base)[0] + img_names = os.path.join(movie_dir, "".join([ + file_pre, "_%0", "{:d}".format(dnum), "d", file_ext])) + + return img_names + + +def save_movie(movie_dir, movie_name="movie.mp4", image_files="image_%04d.png", + rate=30, overwrite=True): + """Save the output as a movie. + + Parameters + ---------- + movie_dir : str + Output directory for the movie + move_name : str + Output movie name with extention (default='movie.mp4') + image_files : str + Full-path names for images to be used in the movie, using C-style + descriptors for numbers that indicate the order of file inclusion + within the movie. For example, 'image_0001.png' and 'image_0002.png' + would create a 2 frame movie with 'image_0001.png' appearing first + using the default value as long as the files exist in the directory + where the code is run (img_names='image_%04d.png') + rate : int + Movie frame rate (default=30) + overwrite : bool + Overwrite an existing movie of the same name. Windows OS frequently do + not allow Python to remove files, so Windows users must clean up these + files themselves. (default=True) + + Returns + ------- + outfile : str + Output movie name + + Raises + ------ + IOError + If movie file already exists and `overwrite` is False. + + Notes + ----- + Uses ffmpeg to create the movie, this must be installed for success + + """ + # Construct the output filenames + outfile = os.path.join(movie_dir, movie_name) + + # Test the output file + if os.path.isfile(outfile): + if overwrite: + os.remove(outfile) + else: + raise IOError('movie file {:} already exists'.format(outfile)) + + # Construct the movie commannd + command = "ffmpeg -r {:d} -i {:s} {:s}".format(rate, image_files, outfile) + os.system(command) + + return outfile diff --git a/aetherpy/run_plot_model_results.py b/aetherpy/run_plot_model_results.py index 805133c..a67ba17 100755 --- a/aetherpy/run_plot_model_results.py +++ b/aetherpy/run_plot_model_results.py @@ -190,7 +190,7 @@ def plot_model_results(): header = read_routines.read_aether_ascii_header(args["filelist"]) is_gitm = False else: - header = read_routines.read_aether_header(args["filelist"]) + header = read_routines.read_aether_headers(args["filelist"]) # If help is requested for a specific file, return it here if args['help']: @@ -443,7 +443,6 @@ def plot_model_results(): if args['movie'] > 0: movie_routines.save_movie(filename, ext=args['ext'], rate=args['movie']) - return diff --git a/aetherpy/tests/__init__.py b/aetherpy/tests/__init__.py index 79a95bc..37e584f 100644 --- a/aetherpy/tests/__init__.py +++ b/aetherpy/tests/__init__.py @@ -1,6 +1,6 @@ -#!/usr/bin/env python -# Copyright 2020, the Aether Development Team (see doc/dev_team.md for members) -# Full license can be found in License.md -"""aetherpy tests.""" - -from aetherpy.tests import utils +#!/usr/bin/env python +# Copyright 2020, the Aether Development Team (see doc/dev_team.md for members) +# Full license can be found in License.md +"""aetherpy tests.""" + +from aetherpy.tests import utils diff --git a/aetherpy/tests/test_data/3DALL_20110320_003000.nc b/aetherpy/tests/test_data/3DALL_20110320_003000.nc new file mode 100644 index 0000000..51335ad Binary files /dev/null and b/aetherpy/tests/test_data/3DALL_20110320_003000.nc differ diff --git a/aetherpy/tests/test_io_fetch.py b/aetherpy/tests/test_io_fetch.py index 52f14fc..f2fc56d 100644 --- a/aetherpy/tests/test_io_fetch.py +++ b/aetherpy/tests/test_io_fetch.py @@ -1,227 +1,227 @@ -#!/usr/bin/env python -# Copyright 2021, the Aether Development Team (see doc/dev_team.md for members) -# Full license can be found in License.md -"""Unit tests for I/O fetch utilities.""" - -import logging -import os -import pytest -import sys -import tempfile - -from aetherpy.io import fetch_routines -from aetherpy.tests.utils import sys_agnostic_remove - - -class TestLocalFetch(object): - """Unit tests for local file fetching routines.""" - - def setup(self): - """Initialize clean test environment.""" - ch = logging.StreamHandler() - ch.setLevel(logging.INFO) - - # TODO #9: remove if-statement when it is always triggered - tkwargs = {} - if sys.version_info.major >= 3 and sys.version_info.minor >= 10: - tkwargs = {"ignore_cleanup_errors": True} - self.tempdir = tempfile.TemporaryDirectory(**tkwargs) - self.tempfiles = [] - self.file_type = 'ALL' - self.file_ext = 'bin' - - return - - def teardown(self): - """Clean up the test environment.""" - # Remove the created files and directories - for filename in self.tempfiles: - sys_agnostic_remove(filename) - - # Remove the temporary directory - # TODO #9: Remove try/except when Python 3.10 is the lowest version - try: - self.tempdir.cleanup() - except Exception: - pass - - # Clear the test environment attributes - del self.file_type, self.file_ext, self.tempdir, self.tempfiles - return - - def make_files(self, dims=1): - """Create four temporary files in an existing directory. - - Parameters - ---------- - dims : int - Number of dimensions (expects 1 or 3) - - """ - - # Create the file base - filebase = "{:d}D{:s}".format(dims, self.file_type) - fileext = ".{:s}".format(self.file_ext) - - # Create an empty temporary file and save the filename - for i in range(4): - out = tempfile.mkstemp(suffix=fileext, prefix=filebase, - dir=self.tempdir.name) - self.tempfiles.append(out[1]) - - return - - @pytest.mark.parametrize("ftype", ["ALL", "ION", "NEU"]) - @pytest.mark.parametrize("ext", ["bin", "nc"]) - @pytest.mark.parametrize("dims", [1, 3]) - def test_get_filelist(self, ftype, ext, dims): - """Test successful retrieval of local files for standard types. - - Parameters - ---------- - ftype : str - Desired file type, accepts 'ALL', 'ION', and 'NEU' - ext : str - File extenstion, without period - dims : int - Number of dimensions, accepts 1 or 3 - - """ - - # Create temporary files - self.file_type = ftype - self.file_ext = ext - self.make_files(dims=dims) - - # Get the list of files - filelist = fetch_routines.get_filelist(self.tempdir.name, - file_type=ftype, file_ext=ext) - - # Evaluate the retrieved list - assert len(filelist) == len(self.tempfiles) - - for fname in filelist: - assert fname in self.tempfiles - - return - - def test_get_filelist_from_multidim_set(self): - """Test retrieval of 3D local files from dir with 1D and 3D.""" - - # Create temporary 1D and 3D files - self.make_files(dims=1) - self.make_files(dims=3) - - # Get the list of files - filelist = fetch_routines.get_filelist(self.tempdir.name, - file_type=self.file_type, - file_ext=self.file_ext) - - # Evaluate the retrieved list - assert len(filelist) == len(self.tempfiles) / 2 - - for fname in filelist: - assert fname in self.tempfiles - - return - - def test_get_filelist_from_unknown_dim_set(self, caplog): - """Test unsuccessful retrieval of local files with 2D and 4D files.""" - - # Create temporary 2D and 4D files - self.make_files(dims=2) - self.make_files(dims=4) - - # Get the list of files - with caplog.at_level(logging.INFO, logger="aetherpy_logger"): - filelist = fetch_routines.get_filelist(self.tempdir.name, - file_type=self.file_type, - file_ext=self.file_ext) - - # Evaluate the retrieved list - assert len(filelist) == 0 - - # Evaluate the logger output - ordered_msgs = ["No 3D", "No 1D"] - ordered_lvl = ['INFO', 'WARNING'] - assert len(caplog.records) == len(ordered_msgs) - - for i, record in enumerate(caplog.records): - # Evaluate the logging message - assert record.message.find(ordered_msgs[i]) >= 0, \ - "unexpected log output: {:s}".format(record.message) - - # Evaluate the logging output level - assert record.levelname == ordered_lvl[i] - - return - - def test_get_filelist_from_unknown_file_type(self, caplog): - """Test retrieval of local files with unknown file type.""" - - # Create temporary 3D files - self.file_type = "LOCAL" - self.make_files(dims=3) - - # Get the list of files - with caplog.at_level(logging.WARNING, logger="aetherpy"): - filelist = fetch_routines.get_filelist(self.tempdir.name, - file_type=self.file_type, - file_ext=self.file_ext) - - # Evaluate the retrieved list - assert len(filelist) == len(self.tempfiles) - - for fname in filelist: - assert fname in self.tempfiles - - # Evaluate the logger output - captured = caplog.text - assert captured.find("unexpected file type") >= 0, \ - "unexpected log output: {:s}".format(captured) - - return - - def test_get_filelist_from_bad_glob_str(self, caplog): - """Test unsuccessful retrieval of local files with bad glob string.""" - - # Create temporary 3D files - self.make_files(dims=3) - - # Create a bad glob string - glob_str = os.path.join(self.tempdir.name, "not_a_file*") - - # Get the list of files - with caplog.at_level(logging.WARNING, logger="aetherpy"): - filelist = fetch_routines.get_filelist(glob_str) - - # Evaluate the retrieved list - assert len(filelist) == 0 - - # Evaluate the logger output - captured = caplog.text - assert captured.find("No files found using search string") >= 0, \ - "unexpected log output: {:s}".format(captured) - - return - - def test_get_filelist_from_glob_str(self): - """Test successful retrieval of local files with a good glob string.""" - - # Create temporary 3D files - self.make_files(dims=3) - - # Create a good glob string - glob_str = os.path.join(self.tempdir.name, "3D{:s}*.{:s}".format( - self.file_type, self.file_ext)) - - # Get the list of files - filelist = fetch_routines.get_filelist(glob_str) - - # Evaluate the retrieved list - assert len(filelist) == len(self.tempfiles) - - for fname in filelist: - assert fname in self.tempfiles - - return +#!/usr/bin/env python +# Copyright 2021, the Aether Development Team (see doc/dev_team.md for members) +# Full license can be found in License.md +"""Unit tests for I/O fetch utilities.""" + +import logging +import os +import pytest +import sys +import tempfile + +from aetherpy.io import fetch_routines +from aetherpy.tests.utils import sys_agnostic_remove + + +class TestLocalFetch(object): + """Unit tests for local file fetching routines.""" + + def setup(self): + """Initialize clean test environment.""" + ch = logging.StreamHandler() + ch.setLevel(logging.INFO) + + # TODO #9: remove if-statement when it is always triggered + tkwargs = {} + if sys.version_info.major >= 3 and sys.version_info.minor >= 10: + tkwargs = {"ignore_cleanup_errors": True} + self.tempdir = tempfile.TemporaryDirectory(**tkwargs) + self.tempfiles = [] + self.file_type = 'ALL' + self.file_ext = 'bin' + + return + + def teardown(self): + """Clean up the test environment.""" + # Remove the created files and directories + for filename in self.tempfiles: + sys_agnostic_remove(filename) + + # Remove the temporary directory + # TODO #9: Remove try/except when Python 3.10 is the lowest version + try: + self.tempdir.cleanup() + except Exception: + pass + + # Clear the test environment attributes + del self.file_type, self.file_ext, self.tempdir, self.tempfiles + return + + def make_files(self, dims=1): + """Create four temporary files in an existing directory. + + Parameters + ---------- + dims : int + Number of dimensions (expects 1 or 3) + + """ + + # Create the file base + filebase = "{:d}D{:s}".format(dims, self.file_type) + fileext = ".{:s}".format(self.file_ext) + + # Create an empty temporary file and save the filename + for i in range(4): + out = tempfile.mkstemp(suffix=fileext, prefix=filebase, + dir=self.tempdir.name) + self.tempfiles.append(out[1]) + + return + + @pytest.mark.parametrize("ftype", ["ALL", "ION", "NEU"]) + @pytest.mark.parametrize("ext", ["bin", "nc"]) + @pytest.mark.parametrize("dims", [1, 3]) + def test_get_filelist(self, ftype, ext, dims): + """Test successful retrieval of local files for standard types. + + Parameters + ---------- + ftype : str + Desired file type, accepts 'ALL', 'ION', and 'NEU' + ext : str + File extenstion, without period + dims : int + Number of dimensions, accepts 1 or 3 + + """ + + # Create temporary files + self.file_type = ftype + self.file_ext = ext + self.make_files(dims=dims) + + # Get the list of files + filelist = fetch_routines.get_filelist(self.tempdir.name, + file_type=ftype, file_ext=ext) + + # Evaluate the retrieved list + assert len(filelist) == len(self.tempfiles) + + for fname in filelist: + assert fname in self.tempfiles + + return + + def test_get_filelist_from_multidim_set(self): + """Test retrieval of 3D local files from dir with 1D and 3D.""" + + # Create temporary 1D and 3D files + self.make_files(dims=1) + self.make_files(dims=3) + + # Get the list of files + filelist = fetch_routines.get_filelist(self.tempdir.name, + file_type=self.file_type, + file_ext=self.file_ext) + + # Evaluate the retrieved list + assert len(filelist) == len(self.tempfiles) / 2 + + for fname in filelist: + assert fname in self.tempfiles + + return + + def test_get_filelist_from_unknown_dim_set(self, caplog): + """Test unsuccessful retrieval of local files with 2D and 4D files.""" + + # Create temporary 2D and 4D files + self.make_files(dims=2) + self.make_files(dims=4) + + # Get the list of files + with caplog.at_level(logging.INFO, logger="aetherpy_logger"): + filelist = fetch_routines.get_filelist(self.tempdir.name, + file_type=self.file_type, + file_ext=self.file_ext) + + # Evaluate the retrieved list + assert len(filelist) == 0 + + # Evaluate the logger output + ordered_msgs = ["No 3D", "No 1D"] + ordered_lvl = ['INFO', 'WARNING'] + assert len(caplog.records) == len(ordered_msgs) + + for i, record in enumerate(caplog.records): + # Evaluate the logging message + assert record.message.find(ordered_msgs[i]) >= 0, \ + "unexpected log output: {:s}".format(record.message) + + # Evaluate the logging output level + assert record.levelname == ordered_lvl[i] + + return + + def test_get_filelist_from_unknown_file_type(self, caplog): + """Test retrieval of local files with unknown file type.""" + + # Create temporary 3D files + self.file_type = "LOCAL" + self.make_files(dims=3) + + # Get the list of files + with caplog.at_level(logging.WARNING, logger="aetherpy"): + filelist = fetch_routines.get_filelist(self.tempdir.name, + file_type=self.file_type, + file_ext=self.file_ext) + + # Evaluate the retrieved list + assert len(filelist) == len(self.tempfiles) + + for fname in filelist: + assert fname in self.tempfiles + + # Evaluate the logger output + captured = caplog.text + assert captured.find("unexpected file type") >= 0, \ + "unexpected log output: {:s}".format(captured) + + return + + def test_get_filelist_from_bad_glob_str(self, caplog): + """Test unsuccessful retrieval of local files with bad glob string.""" + + # Create temporary 3D files + self.make_files(dims=3) + + # Create a bad glob string + glob_str = os.path.join(self.tempdir.name, "not_a_file*") + + # Get the list of files + with caplog.at_level(logging.WARNING, logger="aetherpy"): + filelist = fetch_routines.get_filelist(glob_str) + + # Evaluate the retrieved list + assert len(filelist) == 0 + + # Evaluate the logger output + captured = caplog.text + assert captured.find("No files found using search string") >= 0, \ + "unexpected log output: {:s}".format(captured) + + return + + def test_get_filelist_from_glob_str(self): + """Test successful retrieval of local files with a good glob string.""" + + # Create temporary 3D files + self.make_files(dims=3) + + # Create a good glob string + glob_str = os.path.join(self.tempdir.name, "3D{:s}*.{:s}".format( + self.file_type, self.file_ext)) + + # Get the list of files + filelist = fetch_routines.get_filelist(glob_str) + + # Evaluate the retrieved list + assert len(filelist) == len(self.tempfiles) + + for fname in filelist: + assert fname in self.tempfiles + + return diff --git a/aetherpy/tests/test_io_read.py b/aetherpy/tests/test_io_read.py index 4c2960c..5043ed4 100644 --- a/aetherpy/tests/test_io_read.py +++ b/aetherpy/tests/test_io_read.py @@ -1,238 +1,316 @@ -#!/usr/bin/env python -# Copyright 2021, the Aether Development Team (see doc/dev_team.md for members) -# Full license can be found in License.md -"""Unit tests for I/O reading utilities.""" - -import datetime as dt -import logging -from glob import glob -import os -import numpy as np -import pytest -import sys -import tempfile - -import aetherpy -from aetherpy.io import read_routines - - -class TestIORead(object): - """Unit tests for file reading routines.""" - - def setup(self): - """Initialize clean test environment.""" - self.test_dir = os.path.join(os.path.split(aetherpy.__file__)[0], - "tests", "test_data") - self.header = {} - return - - def teardown(self): - """Clean up the test environment.""" - del self.test_dir, self.header - return - - def eval_header(self, file_list=True): - """Evaluate header output. - - Parameters - ---------- - file_list : bool - 'filename' key is a list if True and a string if False (default=True) - - """ - - # Ensure the base keys are present, more keys are allowed - base_keys = ['vars', 'time', 'filename', 'nlons', 'nlats', 'nalts'] - - assert np.all([bkey in self.header.keys() for bkey in base_keys]), \ - "missing required keys from header output." - - # Evalute the header output formats and types - for hkey in self.header.keys(): - if hkey in ['version']: - # Ensure floats are the correct type - assert isinstance(self.header[hkey], float) - elif hkey in base_keys[:3]: - # Ensure lists are the correct type, length, and have the - # expected value types. - if file_list or hkey == 'vars': - assert isinstance(self.header[hkey], list) - - if hkey == 'time': - if file_list: - assert len(self.header[hkey]) <= len( - self.header['filename']), \ - "header times cannot be more than number of files" - else: - self.header[hkey] = [self.header[hkey]] - - for htime in self.header[hkey]: - assert isinstance(htime, dt.datetime), \ - "unexpected time format for: {:}".format(htime) - elif hkey == 'filename': - if file_list: - assert len(self.header[hkey]) > 0 - else: - self.header[hkey] = [self.header[hkey]] - - for fname in self.header[hkey]: - assert os.path.isfile(fname), \ - "header filename {:} is not a file".format(fname) - else: - assert np.all(np.unique(self.header[hkey]) - == self.header[hkey]), \ - "duplicate variables in header list" - - for var in self.header[hkey]: - assert isinstance(var, str), \ - "variable name {:} is not a string".format(var) - elif hkey in ['nlons', 'nlats', 'nalts'] or hkey[0] == 'n': - # Ensure the counters are all integers - assert isinstance(self.header[hkey], int), \ - "counter {:} is not an int: {:}".format(hkey, - self.header[hkey]) - elif hkey == 'filename': - # The filename is a string and not a list - assert os.path.isfile(self.header[hkey]) - return - - @pytest.mark.parametrize("in_line, out_int", - [("1 2 3 4", 1), - ("42 NEU farm SpRiNg 45.0", 42)]) - @pytest.mark.parametrize("parse", [True, False]) - def test_parse_line_into_int_and_string(self, in_line, out_int, parse): - """Test successful int/string line parsing. - - Parameters - ---------- - in_line : str - Input line with correct formatting - out_int : int - Expected integer output - parse : bool - `parse_string` kwarg input - - """ - - # Parse the line - lnum, lstr = read_routines.parse_line_into_int_and_string(in_line, - parse) - - # Evaluate the retrieved values - assert lnum == out_int, "unexpected integer value for first column" - - if parse: - assert in_line.find(lstr) > 0, "unexpected line values" - else: - assert in_line == lstr, "unexpected line values" - return - - @pytest.mark.parametrize("in_line", - ["1.0 2 3 4", "NEU farm SpRiNg 45.0"]) - def test_parse_line_into_int_and_string_bad_format(self, in_line): - """Test successful int/string line parsing. - - Parameters - ---------- - in_line : str - Input line with incorrect formatting - - """ - - # Parse the line and check error output - with pytest.raises(ValueError) as verr: - read_routines.parse_line_into_int_and_string(in_line) - - # Evaluate the retrieved values - assert str(verr).find("invalid literal for int") >= 0 - return - - @pytest.mark.parametrize('fname', ['3DALL_20110320_003000_g0001.nc', - '3DBFI_20110320_000000_g0000.nc']) - def test_read_aether_netcdf_header(self, fname): - """Test successful Aether netCDF header reading. - - Parameters - ---------- - fname : str - File base name - - """ - filename = os.path.join(self.test_dir, fname) - assert os.path.isfile(filename), "missing test file: {:}".format( - filename) - - self.header = read_routines.read_aether_netcdf_header(filename) - self.eval_header(file_list=False) - return - - def test_read_aether_netcdf_header_bad_file(self): - """Test raises IOError with bad filename.""" - - with pytest.raises(IOError) as verr: - read_routines.read_aether_netcdf_header("not_a_file") - - assert str(verr).find("unknown aether netCDF file") >= 0 - return - - @pytest.mark.parametrize('fbase', ['3DALL_*.nc', '3DBFI_*.nc']) - @pytest.mark.parametrize('ftype', ['netcdf']) - @pytest.mark.parametrize('finds', [(-1), (None), ([0, 1]), (slice(1))]) - def test_read_aether_headers(self, fbase, ftype, finds): - """Test successful Aether header reading. - - Parameters - ---------- - fbase : str - File base name, with glob support - ftype : str - File type - finds : int, NoneType, list, slice - File indexers - - """ - filenames = glob(os.path.join(self.test_dir, fbase)) - - self.header = read_routines.read_aether_headers(filenames, finds, - ftype) - self.eval_header(file_list=True) - return - - @pytest.mark.parametrize('fname', ['3DALL_20110320_003000_g0001.nc', - '3DBFI_20110320_000000_g0000.nc']) - @pytest.mark.parametrize('fvars', [None, (['z'])]) - def test_read_aether_file_dict(self, fname, fvars): - """Test successful Aether NetCDF data reading into dict output. - - Parameters - ---------- - fname : str - File base name - fvars : NoneType or list - List of variable names to read in or NoneType to read all - - """ - filename = os.path.join(self.test_dir, fname) - assert os.path.isfile(filename), "missing test file: {:}".format( - filename) - - data = read_routines.read_aether_file(filename, file_vars=fvars) - - # Evaluate the output keys - assert isinstance(data, dict) - assert 'time' in data.keys(), "'time' missing from output" - assert 'units' in data.keys(), "'units' missing from output" - assert 'long_name' in data.keys(), "'long_name' missing from output" - assert 'vars' in data.keys(), "'vars' missing from output" - - # Evaluate the data variables and attributes - for i in range(len(data['vars'])): - assert i in data.keys(), \ - 'missing data index {:d} in output'.format(i) - assert data[i].dtype in [np.float16, np.float32, np.float64] - assert i < len(data['units']) - assert i < len(data['long_name']) - assert isinstance(data['units'][i], str) - assert isinstance(data['long_name'][i], str) - - return +#!/usr/bin/env python +# Copyright 2021, the Aether Development Team (see doc/dev_team.md for members) +# Full license can be found in License.md +"""Unit tests for I/O reading utilities.""" + +import datetime as dt +import logging +from glob import glob +import os +import numpy as np +import pytest +import sys +import tempfile + +import aetherpy +from aetherpy.io import read_routines + + +class TestIORead(object): + """Unit tests for file reading routines.""" + + def setup(self): + """Initialize clean test environment.""" + self.test_dir = os.path.join(os.path.split(aetherpy.__file__)[0], + "tests", "test_data") + self.header = {} + return + + def teardown(self): + """Clean up the test environment.""" + del self.test_dir, self.header + return + + def eval_header(self, file_list=True): + """Evaluate header output. + + Parameters + ---------- + file_list : bool + 'filename' key is a list if True and a string if False (default=True) + + """ + + # Ensure the base keys are present, more keys are allowed + base_keys = ['vars', 'time', 'filename', 'nlons', 'nlats', 'nalts'] + + assert np.all([bkey in self.header.keys() for bkey in base_keys]), \ + "missing required keys from header output." + + # Evalute the header output formats and types + for hkey in self.header.keys(): + if hkey in ['version']: + # Ensure floats are the correct type + assert isinstance(self.header[hkey], float) + elif hkey in base_keys[:3]: + # Ensure lists are the correct type, length, and have the + # expected value types. + if file_list or hkey == 'vars': + assert isinstance(self.header[hkey], list) + + if hkey == 'time': + if file_list: + assert len(self.header[hkey]) <= len( + self.header['filename']), \ + "header times cannot be more than number of files" + else: + self.header[hkey] = [self.header[hkey]] + + for htime in self.header[hkey]: + assert isinstance(htime, dt.datetime), \ + "unexpected time format for: {:}".format(htime) + elif hkey == 'filename': + if file_list: + assert len(self.header[hkey]) > 0 + else: + self.header[hkey] = [self.header[hkey]] + + for fname in self.header[hkey]: + assert os.path.isfile(fname), \ + "header filename {:} is not a file".format(fname) + else: + assert (len(set(self.header[hkey])) + == len(self.header[hkey])), \ + "duplicate variables in header list" + + for var in self.header[hkey]: + assert isinstance(var, str), \ + "variable name {:} is not a string".format(var) + elif hkey in ['nlons', 'nlats', 'nalts'] or hkey[0] == 'n': + # Ensure the counters are all integers + assert isinstance(self.header[hkey], int), \ + "counter {:} is not an int: {:}".format(hkey, + self.header[hkey]) + elif hkey == 'filename': + # The filename is a string and not a list + assert os.path.isfile(self.header[hkey]) + return + + @pytest.mark.parametrize("in_line, out_int", + [("1 2 3 4", 1), + ("42 NEU farm SpRiNg 45.0", 42)]) + @pytest.mark.parametrize("parse", [True, False]) + def test_parse_line_into_int_and_string(self, in_line, out_int, parse): + """Test successful int/string line parsing. + + Parameters + ---------- + in_line : str + Input line with correct formatting + out_int : int + Expected integer output + parse : bool + `parse_string` kwarg input + + """ + + # Parse the line + lnum, lstr = read_routines.parse_line_into_int_and_string(in_line, + parse) + + # Evaluate the retrieved values + assert lnum == out_int, "unexpected integer value for first column" + + if parse: + assert in_line.find(lstr) > 0, "unexpected line values" + else: + assert in_line == lstr, "unexpected line values" + return + + @pytest.mark.parametrize("in_line", + ["1.0 2 3 4", "NEU farm SpRiNg 45.0"]) + def test_parse_line_into_int_and_string_bad_format(self, in_line): + """Test successful int/string line parsing. + + Parameters + ---------- + in_line : str + Input line with incorrect formatting + + """ + + # Parse the line and check error output + with pytest.raises(ValueError) as verr: + read_routines.parse_line_into_int_and_string(in_line) + + # Evaluate the retrieved values + assert str(verr).find("invalid literal for int") >= 0 + return + + @pytest.mark.parametrize('fname', ['3DALL_20110320_003000_g0001.nc', + '3DBFI_20110320_000000_g0000.nc']) + def test_read_aether_netcdf_header(self, fname): + """Test successful Aether netCDF header reading. + + Parameters + ---------- + fname : str + File base name + + """ + filename = os.path.join(self.test_dir, fname) + assert os.path.isfile(filename), "missing test file: {:}".format( + filename) + + self.header = read_routines.read_aether_netcdf_header(filename) + self.eval_header(file_list=False) + return + + def test_read_aether_netcdf_header_bad_file(self): + """Test raises IOError with bad filename.""" + + with pytest.raises(IOError) as verr: + read_routines.read_aether_netcdf_header("not_a_file") + + assert str(verr).find("unknown aether netCDF file") >= 0 + return + + @pytest.mark.parametrize('fbase', ['3DALL_*g*.nc', '3DBFI_*g*.nc']) + @pytest.mark.parametrize('ftype', ['netcdf']) + @pytest.mark.parametrize('finds', [(-1), (None), ([0, 1]), (slice(1))]) + def test_read_aether_headers(self, fbase, ftype, finds): + """Test successful Aether header reading. + + Parameters + ---------- + fbase : str + File base name, with glob support + ftype : str + File type + finds : int, NoneType, list, slice + File indexers + + """ + filenames = glob(os.path.join(self.test_dir, fbase)) + + self.header = read_routines.read_aether_headers(filenames, finds, + ftype) + + self.eval_header(file_list=True) + return + + @pytest.mark.parametrize('fname', ['3DALL_20110320_003000_g0001.nc', + '3DBFI_20110320_000000_g0000.nc']) + @pytest.mark.parametrize('fvars', [None, (['z'])]) + def test_read_aether_file_dict(self, fname, fvars): + """Test successful Aether NetCDF data reading into dict output. + + Parameters + ---------- + fname : str + File base name + fvars : NoneType or list + List of variable names to read in or NoneType to read all + + """ + filename = os.path.join(self.test_dir, fname) + assert os.path.isfile(filename), "missing test file: {:}".format( + filename) + + data = read_routines.read_aether_file(filename, file_vars=fvars) + + # Evaluate the output keys + assert isinstance(data, dict) + assert 'time' in data.keys(), "'time' missing from output" + assert 'units' in data.keys(), "'units' missing from output" + assert 'long_name' in data.keys(), "'long_name' missing from output" + assert 'vars' in data.keys(), "'vars' missing from output" + + # Evaluate the data variables and attributes + for i in range(len(data['vars'])): + assert i in data.keys(), \ + 'missing data index {:d} in output'.format(i) + assert data[i].dtype in [np.float16, np.float32, np.float64] + assert i < len(data['units']) + assert i < len(data['long_name']) + assert isinstance(data['units'][i], str) + assert isinstance(data['long_name'][i], str) + + return + + @pytest.mark.parametrize('fname', ['3DALL_20110320_003000.nc']) + @pytest.mark.parametrize('fvars', [None, ['O']]) + def test_read_blocked_netcdf_header(self, fname, fvars): + """Test successful block-based Aether NetCDF header + + Parameters + ---------- + fname : str + File base name + fvars : NoneType or list + List of variable names to read in or NoneType to read all + + """ + filename = os.path.join(self.test_dir, fname) + assert os.path.isfile(filename), "missing test file: {:}".format( + filename) + + self.header = read_routines.read_blocked_netcdf_header( + filename) + self.eval_header(file_list=False) + return + + def test_read_blocked_netcdf_header_bad_file(self): + """Test raises IOError with bad filename.""" + + with pytest.raises(IOError) as verr: + read_routines.read_blocked_netcdf_header("not_a_file") + + assert str(verr).find("unknown aether netCDF blocked file") >= 0 + return + + @pytest.mark.parametrize('fname', ['3DALL_20110320_003000.nc']) + @pytest.mark.parametrize('fvars', [None, ['O']]) + def test_read_blocked_netcdf_file(self, fname, fvars): + """Test successful block-based Aether NetCDF data reading into dict. + + Parameters + ---------- + fname : str + File base name + fvars : NoneType or list + List of variable names to read in or NoneType to read all + + """ + filename = os.path.join(self.test_dir, fname) + assert os.path.isfile(filename), "missing test file: {:}".format( + filename) + + data = read_routines.read_blocked_netcdf_file(filename, file_vars=fvars) + + # Evaluate the output keys + # TODO(#18): 'units' and 'long_name' are here for compatibility + # will be removed when library is refactored + # They are dummy keys containing no data right now + assert isinstance(data, dict) + assert 'time' in data.keys(), "'time' missing from output" + assert 'units' in data.keys(), "'units' missing from output" + assert 'long_name' in data.keys(), "'long_name' missing from output" + assert 'vars' in data.keys(), "'vars' missing from output" + + # Evaluate the data variables -- no attributes yet + for var in data['vars']: + assert var in data.keys(), \ + 'missing variable {:} in output'.format(var) + # TODO: add attribute checking once implemented + + return + + def test_read_blocked_netcdf_file_bad_file(self): + """Test raises IOError with bad filename.""" + + with pytest.raises(IOError) as verr: + read_routines.read_blocked_netcdf_file("not_a_file") + + assert str(verr).find("unknown aether netCDF blocked file") >= 0 + return diff --git a/aetherpy/tests/test_plot_movie.py b/aetherpy/tests/test_plot_movie.py index 4eac719..fcd7eed 100644 --- a/aetherpy/tests/test_plot_movie.py +++ b/aetherpy/tests/test_plot_movie.py @@ -1,219 +1,219 @@ -#!/usr/bin/env python -# Copyright 2021, the Aether Development Team (see doc/dev_team.md for members) -# Full license can be found in License.md -"""Unit tests for plot-to-movie utilities.""" - -import logging -import numpy as np -import os -import pytest -import sys -import tempfile - -from aetherpy.plot import movie_routines as mr -from aetherpy.tests.utils import sys_agnostic_remove -from aetherpy.tests.utils import sys_agnostic_rename - - -class TestMovie(object): - """Unit tests for plot-to-movie functions.""" - - def setup(self): - """Initialize clean test environment.""" - - # Create a temporary directory - tkwargs = {} - # TODO #9: remove if-statement when it is always triggered - if sys.version_info.major >= 3 and sys.version_info.minor >= 10: - tkwargs = {"ignore_cleanup_errors": True} - self.tempdir = tempfile.TemporaryDirectory(**tkwargs) - self.movie_dir = os.path.join(self.tempdir.name, "movie_dir") - self.fileext = ".png" - self.filebase = "test_" - self.tempfiles = [] - self.moviename = "test.mp4" - - return - - def teardown(self): - """Clean up the test environment.""" - # Remove the created files and directories - for filename in self.tempfiles: - sys_agnostic_remove(filename) - - # TODO #9: Remove try/except when Python 3.10 is the lowest version - if os.path.isdir(self.movie_dir): - try: - os.rmdir(self.movie_dir) - except Exception: - pass - - # Remove the temporary directory - # TODO #9: Remove try/except when Python 3.10 is the lowest version - try: - self.tempdir.cleanup() - except Exception: - pass - - # Clear the test environment attributes - del self.movie_dir, self.tempdir, self.tempfiles, self.moviename - del self.fileext, self.filebase - return - - def make_files(self, data=False): - """Create a directory with temporary files. - - Parameters - ---------- - data : bool - Add data to the temporary file - """ - os.makedirs(self.movie_dir) - - for i in range(4): - # Create a temporary file that must be removed - out = tempfile.mkstemp(suffix=self.fileext, prefix=self.filebase, - dir=self.movie_dir) - - # Rename the temporary file to match the necessary format - goodname = os.path.join(self.movie_dir, "{:s}{:04d}{:s}".format( - self.filebase, i, self.fileext)) - - # Windows OS sometimes needs time to allow a rename - sys_agnostic_rename(out[1], goodname) - - # Add data to the temporary file - if data: - with open(goodname, "wb") as fout: - fout.write(b"AAAn") - - # Save the good filename - self.tempfiles.append(goodname) - - return - - def test_setup_movie_dir_newdir(self): - """Test sucessful creation of a new directory for movie files.""" - assert not os.path.isdir(self.movie_dir) - mr.setup_movie_dir(self.movie_dir) - assert os.path.isdir(self.movie_dir) - - return - - @pytest.mark.skipif(sys.platform in ["win32", "cygwin", "windows"], - reason="Windows can't remove directories") - @pytest.mark.parametrize("wcard", ["*", "????"]) - def test_setup_movie_dir_olddir(self, wcard): - """Test sucessful creation of a new directory for movie files. - - Parameters - ---------- - wcard : str - Accepted wildcard strings for the test files - - """ - self.make_files() - file_glob = "".join([self.filebase, wcard, self.fileext]) - - img_names = mr.setup_movie_dir(self.movie_dir, file_glob=file_glob) - assert os.path.isdir(self.movie_dir) - for filename in self.tempfiles: - assert not os.path.isfile(filename), "old file not removed" - - assert img_names.find(self.movie_dir) >= 0, "unexpected dest directory" - assert img_names.find(self.filebase) >= 0, "unexpected file prefix" - assert img_names.find(self.fileext) >= 0, "unexpected file extension" - return - - def test_setup_movie_dir_no_overwrite(self): - """Test raises IOError when conflicting files are present.""" - self.make_files() - file_glob = "*".join([self.filebase, self.fileext]) - - with pytest.raises(IOError) as ierr: - mr.setup_movie_dir(self.movie_dir, file_glob=file_glob, - overwrite=False) - - assert str(ierr).find("files present in movie directory") >= 0 - return - - @pytest.mark.parametrize("rate", [30, 60]) - def test_save_movie_nooverwrite_success(self, rate): - """Test the creation of a movie file in a clean directory. - - Parameters - ---------- - rate : int - Frame rate - - """ - # Set up the movie file directory - self.make_files(data=True) - image_files = os.path.join(self.movie_dir, "".join([ - self.filebase, "%04d", self.fileext])) - - # Create the movie file - outfile = mr.save_movie(self.movie_dir, movie_name=self.moviename, - image_files=image_files, rate=rate) - - # Test the output - assert os.path.isfile(outfile), "movie file not created" - - # Prepare for cleanup - self.tempfiles.append(outfile) - return - - @pytest.mark.skipif(sys.platform in ['win32', 'cygwin', 'windows'], - reason="Windows can't remove directories") - def test_save_movie_overwrite_success(self): - """Test the creation of a movie file when overwriting the old movie.""" - # Set up the movie file directory - self.make_files(data=True) - image_files = os.path.join(self.movie_dir, "".join([ - self.filebase, "%04d", self.fileext])) - - # Make an output file with the same name as the movie file - out = tempfile.mkstemp(suffix=self.fileext, prefix=self.filebase, - dir=self.movie_dir) - sys_agnostic_rename(out[1], - os.path.join(self.movie_dir, self.moviename)) - - # Create the move file - outfile = mr.save_movie(self.movie_dir, movie_name=self.moviename, - image_files=image_files, overwrite=True) - - # Test the output - assert os.path.isfile(outfile), "movie file not created" - - # Prepare for cleanup - self.tempfiles.append(outfile) - return - - @pytest.mark.skipif(sys.platform in ['win32', 'cygwin', 'windows'], - reason="Windows create the test file") - def test_save_movie_overwrite_blocking(self): - """Test raises IOError when a movie exists and overwrite is blocked.""" - # Set up the movie file directory - self.make_files(data=True) - image_files = os.path.join(self.movie_dir, "".join([ - self.filebase, "%04d", self.fileext])) - - # Make an output file with the same name as the movie file - out = tempfile.mkstemp(suffix=self.fileext, prefix=self.filebase, - dir=self.movie_dir) - outfile = os.path.join(self.movie_dir, self.moviename) - sys_agnostic_rename(out[1], outfile) - assert os.path.isfile(outfile), \ - "Unable to create file {:} on {:}".format(outfile, sys.platform) - - # Create the move file - with pytest.raises(IOError) as ierr: - mr.save_movie(self.movie_dir, movie_name=self.moviename, - image_files=image_files, overwrite=False) - - # Test the output - assert str(ierr).find('already exists') >= 0, "unexpected IOError" - - # Prepare for cleanup - self.tempfiles.append(outfile) - return +#!/usr/bin/env python +# Copyright 2021, the Aether Development Team (see doc/dev_team.md for members) +# Full license can be found in License.md +"""Unit tests for plot-to-movie utilities.""" + +import logging +import numpy as np +import os +import pytest +import sys +import tempfile + +from aetherpy.plot import movie_routines as mr +from aetherpy.tests.utils import sys_agnostic_remove +from aetherpy.tests.utils import sys_agnostic_rename + + +class TestMovie(object): + """Unit tests for plot-to-movie functions.""" + + def setup(self): + """Initialize clean test environment.""" + + # Create a temporary directory + tkwargs = {} + # TODO #9: remove if-statement when it is always triggered + if sys.version_info.major >= 3 and sys.version_info.minor >= 10: + tkwargs = {"ignore_cleanup_errors": True} + self.tempdir = tempfile.TemporaryDirectory(**tkwargs) + self.movie_dir = os.path.join(self.tempdir.name, "movie_dir") + self.fileext = ".png" + self.filebase = "test_" + self.tempfiles = [] + self.moviename = "test.mp4" + + return + + def teardown(self): + """Clean up the test environment.""" + # Remove the created files and directories + for filename in self.tempfiles: + sys_agnostic_remove(filename) + + # TODO #9: Remove try/except when Python 3.10 is the lowest version + if os.path.isdir(self.movie_dir): + try: + os.rmdir(self.movie_dir) + except Exception: + pass + + # Remove the temporary directory + # TODO #9: Remove try/except when Python 3.10 is the lowest version + try: + self.tempdir.cleanup() + except Exception: + pass + + # Clear the test environment attributes + del self.movie_dir, self.tempdir, self.tempfiles, self.moviename + del self.fileext, self.filebase + return + + def make_files(self, data=False): + """Create a directory with temporary files. + + Parameters + ---------- + data : bool + Add data to the temporary file + """ + os.makedirs(self.movie_dir) + + for i in range(4): + # Create a temporary file that must be removed + out = tempfile.mkstemp(suffix=self.fileext, prefix=self.filebase, + dir=self.movie_dir) + + # Rename the temporary file to match the necessary format + goodname = os.path.join(self.movie_dir, "{:s}{:04d}{:s}".format( + self.filebase, i, self.fileext)) + + # Windows OS sometimes needs time to allow a rename + sys_agnostic_rename(out[1], goodname) + + # Add data to the temporary file + if data: + with open(goodname, "wb") as fout: + fout.write(b"AAAn") + + # Save the good filename + self.tempfiles.append(goodname) + + return + + def test_setup_movie_dir_newdir(self): + """Test sucessful creation of a new directory for movie files.""" + assert not os.path.isdir(self.movie_dir) + mr.setup_movie_dir(self.movie_dir) + assert os.path.isdir(self.movie_dir) + + return + + @pytest.mark.skipif(sys.platform in ["win32", "cygwin", "windows"], + reason="Windows can't remove directories") + @pytest.mark.parametrize("wcard", ["*", "????"]) + def test_setup_movie_dir_olddir(self, wcard): + """Test sucessful creation of a new directory for movie files. + + Parameters + ---------- + wcard : str + Accepted wildcard strings for the test files + + """ + self.make_files() + file_glob = "".join([self.filebase, wcard, self.fileext]) + + img_names = mr.setup_movie_dir(self.movie_dir, file_glob=file_glob) + assert os.path.isdir(self.movie_dir) + for filename in self.tempfiles: + assert not os.path.isfile(filename), "old file not removed" + + assert img_names.find(self.movie_dir) >= 0, "unexpected dest directory" + assert img_names.find(self.filebase) >= 0, "unexpected file prefix" + assert img_names.find(self.fileext) >= 0, "unexpected file extension" + return + + def test_setup_movie_dir_no_overwrite(self): + """Test raises IOError when conflicting files are present.""" + self.make_files() + file_glob = "*".join([self.filebase, self.fileext]) + + with pytest.raises(IOError) as ierr: + mr.setup_movie_dir(self.movie_dir, file_glob=file_glob, + overwrite=False) + + assert str(ierr).find("files present in movie directory") >= 0 + return + + @pytest.mark.parametrize("rate", [30, 60]) + def test_save_movie_nooverwrite_success(self, rate): + """Test the creation of a movie file in a clean directory. + + Parameters + ---------- + rate : int + Frame rate + + """ + # Set up the movie file directory + self.make_files(data=True) + image_files = os.path.join(self.movie_dir, "".join([ + self.filebase, "%04d", self.fileext])) + + # Create the movie file + outfile = mr.save_movie(self.movie_dir, movie_name=self.moviename, + image_files=image_files, rate=rate) + + # Test the output + assert os.path.isfile(outfile), "movie file not created" + + # Prepare for cleanup + self.tempfiles.append(outfile) + return + + @pytest.mark.skipif(sys.platform in ['win32', 'cygwin', 'windows'], + reason="Windows can't remove directories") + def test_save_movie_overwrite_success(self): + """Test the creation of a movie file when overwriting the old movie.""" + # Set up the movie file directory + self.make_files(data=True) + image_files = os.path.join(self.movie_dir, "".join([ + self.filebase, "%04d", self.fileext])) + + # Make an output file with the same name as the movie file + out = tempfile.mkstemp(suffix=self.fileext, prefix=self.filebase, + dir=self.movie_dir) + sys_agnostic_rename(out[1], + os.path.join(self.movie_dir, self.moviename)) + + # Create the move file + outfile = mr.save_movie(self.movie_dir, movie_name=self.moviename, + image_files=image_files, overwrite=True) + + # Test the output + assert os.path.isfile(outfile), "movie file not created" + + # Prepare for cleanup + self.tempfiles.append(outfile) + return + + @pytest.mark.skipif(sys.platform in ['win32', 'cygwin', 'windows'], + reason="Windows create the test file") + def test_save_movie_overwrite_blocking(self): + """Test raises IOError when a movie exists and overwrite is blocked.""" + # Set up the movie file directory + self.make_files(data=True) + image_files = os.path.join(self.movie_dir, "".join([ + self.filebase, "%04d", self.fileext])) + + # Make an output file with the same name as the movie file + out = tempfile.mkstemp(suffix=self.fileext, prefix=self.filebase, + dir=self.movie_dir) + outfile = os.path.join(self.movie_dir, self.moviename) + sys_agnostic_rename(out[1], outfile) + assert os.path.isfile(outfile), \ + "Unable to create file {:} on {:}".format(outfile, sys.platform) + + # Create the move file + with pytest.raises(IOError) as ierr: + mr.save_movie(self.movie_dir, movie_name=self.moviename, + image_files=image_files, overwrite=False) + + # Test the output + assert str(ierr).find('already exists') >= 0, "unexpected IOError" + + # Prepare for cleanup + self.tempfiles.append(outfile) + return diff --git a/aetherpy/tests/test_plot_prep.py b/aetherpy/tests/test_plot_prep.py index e166de6..db9467c 100644 --- a/aetherpy/tests/test_plot_prep.py +++ b/aetherpy/tests/test_plot_prep.py @@ -1,229 +1,229 @@ -#!/usr/bin/env python -# Copyright 2021, the Aether Development Team (see doc/dev_team.md for members) -# Full license can be found in License.md -"""Unit tests for plot data preparation utilities.""" - -import logging -import numpy as np -import pytest - -from aetherpy.plot import data_prep - - -class TestDataPrep(object): - """Unit tests for data preparation functions.""" - - def setup(self): - """Initialize clean test environment.""" - - # Set the testing latitude and longitude range to - # include all locations where model can uniquely - # define both latitude and longitude - self.in_coords = {"lon": np.arange(-180, 360, 1.0), - "lat": np.arange(-89.5, 90, 0.5), - "alt": np.arange(100, 10000, 100.0)} - - return - - def teardown(self): - """Clean up the test environment.""" - del self.in_coords - - @pytest.mark.parametrize("cut_val, isgrid, cut_coord", [ - (0, True, "lon"), (0, True, "lat"), (0, True, "alt"), - (50.0, False, "lon"), (50.0, False, "lat"), (600, False, "alt")]) - def test_get_cut_index(self, cut_val, isgrid, cut_coord): - """Test sucessful construction of slicing indices. - - Parameters - ---------- - cut_val : int or float - Data value or grid number along which icut will be set - isgrid : bool - Flag that indicates `cut_val` is a grid index if True or that it is - a data value if False - cut_coord : str - Expects one of 'lat', 'lon', or 'alt' and will return an index for - that data, allowing a 2D slice to be created along other two - coordinates - - """ - - # Get the desired slice - out = data_prep.get_cut_index( - self.in_coords["lon"], self.in_coords["lat"], - self.in_coords["alt"], cut_val, isgrid, cut_coord) - - # Test the output - assert len(out) == 5, "unexpected number of returned variables" - assert len(out[1]) == 3, "unexpected number of coordinates" - - if isgrid: - assert out[0] == cut_val, "unexpected grid slice intersection" - else: - assert self.in_coords[cut_coord][out[0]] == cut_val, \ - "unexpected value slice intersection" - - iout_coords = 2 - for i, coord in enumerate(["lon", "lat", "alt"]): - if isinstance(out[1][i], slice): - assert np.all(out[iout_coords] - == self.in_coords[coord][out[1][i]]), \ - "unexpected {:s} slice".format(coord) - iout_coords += 1 - else: - assert out[-1] == self.in_coords[coord][out[1][i]], \ - "unexpected {:s} value".format(coord) - - return - - @pytest.mark.parametrize("cut_val", [-1, 600]) - @pytest.mark.parametrize("cut_coord", ["lon", "lat", "alt"]) - def test_bad_index_get_cut_index(self, cut_val, cut_coord): - """Test raises ValueError when requested index is out of range. - - Parameters - ---------- - cut_val : int - Grid number along which icut will be set - cut_coord : str - Expects one of 'lat', 'lon', or 'alt' and will return an index for - that data, allowing a 2D slice to be created along other two - coordinates - - """ - - with pytest.raises(ValueError) as verr: - data_prep.get_cut_index( - self.in_coords["lon"], self.in_coords["lat"], - self.in_coords["alt"], cut_val, True, cut_coord) - - assert str(verr).find("Requested cut is outside the index range") >= 0 - return - - @pytest.mark.parametrize("cut_val", [-400, 400000]) - @pytest.mark.parametrize("cut_coord", ["lon", "lat", "alt"]) - def test_bad_value_get_cut_index(self, cut_val, cut_coord): - """Test raises ValueError when requested index is out of range. - - Parameters - ---------- - cut_val : float - Data value along which icut will be set - cut_coord : str - Expects one of 'lat', 'lon', or 'alt' and will return an index for - that data, allowing a 2D slice to be created along other two - coordinates - - """ - - with pytest.raises(ValueError) as verr: - data_prep.get_cut_index( - self.in_coords["lon"], self.in_coords["lat"], - self.in_coords["alt"], cut_val, False, cut_coord) - - assert str(verr).find("Requested cut is outside the coordinate") >= 0 - return - - @pytest.mark.parametrize("lowlim", [True, False]) - @pytest.mark.parametrize("isgrid", [True, False]) - @pytest.mark.parametrize("cut_coord", ["lon", "lat", "alt"]) - def test_suspect_get_cut_index(self, caplog, lowlim, isgrid, cut_coord): - """Test raises log warning when requested cut is not recommended. - - Parameters - ---------- - lowlim : bool - Data will be cut at the lower not-recommended limit if True, or - along the higher not-recommended limit if False - isgrid : bool - Flag that indicates `cut_val` is a grid index if True or that it is - a data value if False - cut_coord : str - Expects one of 'lat', 'lon', or 'alt' and will return an index for - that data, allowing a 2D slice to be created along other two - coordinates - - """ - - # Get the desired cut value - if lowlim: - if cut_coord == "alt": - # There are no problems at the lower altitude limit - return - else: - if isgrid: - cut_val = 0 - else: - cut_val = self.in_coords[cut_coord][0] - else: - if isgrid: - cut_val = len(self.in_coords[cut_coord]) - 1 - else: - cut_val = self.in_coords[cut_coord][-1] - - # Raise the expected warning - with caplog.at_level(logging.WARNING, logger="aetherpy"): - data_prep.get_cut_index( - self.in_coords["lon"], self.in_coords["lat"], - self.in_coords["alt"], cut_val, isgrid, cut_coord) - - # Test the logger warning message - captured = caplog.text - if cut_coord == "alt": - assert captured.find("Requested altitude slice is above the") >= 0 - else: - assert captured.find("beyond the recommended limits") >= 0 - return - - @pytest.mark.parametrize("in_kwargs", [{"ialt_min": 10}, {}, - {"ialt_max": 10}, - {"ialt_min": 0, "ialt_max": -1}]) - def test_calc_tec(self, in_kwargs): - """Test successful TEC calculation. - - Parameters - ---------- - in_kwargs : dict - Function kwargs - - """ - - # Initialize local data - ne = np.ones(shape=(self.in_coords['lon'].shape[0], - self.in_coords['lat'].shape[0], - self.in_coords['alt'].shape[0]), dtype=float) - - # Calculate the TEC - tec = data_prep.calc_tec(self.in_coords['alt'], ne, **in_kwargs) - - # Test the output - assert len(np.unique(tec)) == 1 - assert tec.shape == (self.in_coords['lon'].shape[0], - self.in_coords['lat'].shape[0]) - assert tec.min() >= 0.0 - return - - @pytest.mark.parametrize("in_kwargs", [{"ialt_min": -1}, {"ialt_max": 0}, - {"ialt_min": 10, "ialt_max": 10}]) - def test_calc_tec_bad_index(self, in_kwargs): - """Test TEC calculation raises ValueError with bad alt index range. - - Parameters - ---------- - in_kwargs : dict - Function kwargs - - """ - - # Initialize local data - ne = np.ones(shape=(self.in_coords['lon'].shape[0], - self.in_coords['lat'].shape[0], - self.in_coords['alt'].shape[0]), dtype=float) - - # Calculate the TEC - with pytest.raises(ValueError) as verr: - data_prep.calc_tec(self.in_coords['alt'], ne, **in_kwargs) - - assert str(verr).find("ialt_max` must be greater than `ialt_min") >= 0 - return +#!/usr/bin/env python +# Copyright 2021, the Aether Development Team (see doc/dev_team.md for members) +# Full license can be found in License.md +"""Unit tests for plot data preparation utilities.""" + +import logging +import numpy as np +import pytest + +from aetherpy.plot import data_prep + + +class TestDataPrep(object): + """Unit tests for data preparation functions.""" + + def setup(self): + """Initialize clean test environment.""" + + # Set the testing latitude and longitude range to + # include all locations where model can uniquely + # define both latitude and longitude + self.in_coords = {"lon": np.arange(-180, 360, 1.0), + "lat": np.arange(-89.5, 90, 0.5), + "alt": np.arange(100, 10000, 100.0)} + + return + + def teardown(self): + """Clean up the test environment.""" + del self.in_coords + + @pytest.mark.parametrize("cut_val, isgrid, cut_coord", [ + (0, True, "lon"), (0, True, "lat"), (0, True, "alt"), + (50.0, False, "lon"), (50.0, False, "lat"), (600, False, "alt")]) + def test_get_cut_index(self, cut_val, isgrid, cut_coord): + """Test sucessful construction of slicing indices. + + Parameters + ---------- + cut_val : int or float + Data value or grid number along which icut will be set + isgrid : bool + Flag that indicates `cut_val` is a grid index if True or that it is + a data value if False + cut_coord : str + Expects one of 'lat', 'lon', or 'alt' and will return an index for + that data, allowing a 2D slice to be created along other two + coordinates + + """ + + # Get the desired slice + out = data_prep.get_cut_index( + self.in_coords["lon"], self.in_coords["lat"], + self.in_coords["alt"], cut_val, isgrid, cut_coord) + + # Test the output + assert len(out) == 5, "unexpected number of returned variables" + assert len(out[1]) == 3, "unexpected number of coordinates" + + if isgrid: + assert out[0] == cut_val, "unexpected grid slice intersection" + else: + assert self.in_coords[cut_coord][out[0]] == cut_val, \ + "unexpected value slice intersection" + + iout_coords = 2 + for i, coord in enumerate(["lon", "lat", "alt"]): + if isinstance(out[1][i], slice): + assert np.all(out[iout_coords] + == self.in_coords[coord][out[1][i]]), \ + "unexpected {:s} slice".format(coord) + iout_coords += 1 + else: + assert out[-1] == self.in_coords[coord][out[1][i]], \ + "unexpected {:s} value".format(coord) + + return + + @pytest.mark.parametrize("cut_val", [-1, 600]) + @pytest.mark.parametrize("cut_coord", ["lon", "lat", "alt"]) + def test_bad_index_get_cut_index(self, cut_val, cut_coord): + """Test raises ValueError when requested index is out of range. + + Parameters + ---------- + cut_val : int + Grid number along which icut will be set + cut_coord : str + Expects one of 'lat', 'lon', or 'alt' and will return an index for + that data, allowing a 2D slice to be created along other two + coordinates + + """ + + with pytest.raises(ValueError) as verr: + data_prep.get_cut_index( + self.in_coords["lon"], self.in_coords["lat"], + self.in_coords["alt"], cut_val, True, cut_coord) + + assert str(verr).find("Requested cut is outside the index range") >= 0 + return + + @pytest.mark.parametrize("cut_val", [-400, 400000]) + @pytest.mark.parametrize("cut_coord", ["lon", "lat", "alt"]) + def test_bad_value_get_cut_index(self, cut_val, cut_coord): + """Test raises ValueError when requested index is out of range. + + Parameters + ---------- + cut_val : float + Data value along which icut will be set + cut_coord : str + Expects one of 'lat', 'lon', or 'alt' and will return an index for + that data, allowing a 2D slice to be created along other two + coordinates + + """ + + with pytest.raises(ValueError) as verr: + data_prep.get_cut_index( + self.in_coords["lon"], self.in_coords["lat"], + self.in_coords["alt"], cut_val, False, cut_coord) + + assert str(verr).find("Requested cut is outside the coordinate") >= 0 + return + + @pytest.mark.parametrize("lowlim", [True, False]) + @pytest.mark.parametrize("isgrid", [True, False]) + @pytest.mark.parametrize("cut_coord", ["lon", "lat", "alt"]) + def test_suspect_get_cut_index(self, caplog, lowlim, isgrid, cut_coord): + """Test raises log warning when requested cut is not recommended. + + Parameters + ---------- + lowlim : bool + Data will be cut at the lower not-recommended limit if True, or + along the higher not-recommended limit if False + isgrid : bool + Flag that indicates `cut_val` is a grid index if True or that it is + a data value if False + cut_coord : str + Expects one of 'lat', 'lon', or 'alt' and will return an index for + that data, allowing a 2D slice to be created along other two + coordinates + + """ + + # Get the desired cut value + if lowlim: + if cut_coord == "alt": + # There are no problems at the lower altitude limit + return + else: + if isgrid: + cut_val = 0 + else: + cut_val = self.in_coords[cut_coord][0] + else: + if isgrid: + cut_val = len(self.in_coords[cut_coord]) - 1 + else: + cut_val = self.in_coords[cut_coord][-1] + + # Raise the expected warning + with caplog.at_level(logging.WARNING, logger="aetherpy"): + data_prep.get_cut_index( + self.in_coords["lon"], self.in_coords["lat"], + self.in_coords["alt"], cut_val, isgrid, cut_coord) + + # Test the logger warning message + captured = caplog.text + if cut_coord == "alt": + assert captured.find("Requested altitude slice is above the") >= 0 + else: + assert captured.find("beyond the recommended limits") >= 0 + return + + @pytest.mark.parametrize("in_kwargs", [{"ialt_min": 10}, {}, + {"ialt_max": 10}, + {"ialt_min": 0, "ialt_max": -1}]) + def test_calc_tec(self, in_kwargs): + """Test successful TEC calculation. + + Parameters + ---------- + in_kwargs : dict + Function kwargs + + """ + + # Initialize local data + ne = np.ones(shape=(self.in_coords['lon'].shape[0], + self.in_coords['lat'].shape[0], + self.in_coords['alt'].shape[0]), dtype=float) + + # Calculate the TEC + tec = data_prep.calc_tec(self.in_coords['alt'], ne, **in_kwargs) + + # Test the output + assert len(np.unique(tec)) == 1 + assert tec.shape == (self.in_coords['lon'].shape[0], + self.in_coords['lat'].shape[0]) + assert tec.min() >= 0.0 + return + + @pytest.mark.parametrize("in_kwargs", [{"ialt_min": -1}, {"ialt_max": 0}, + {"ialt_min": 10, "ialt_max": 10}]) + def test_calc_tec_bad_index(self, in_kwargs): + """Test TEC calculation raises ValueError with bad alt index range. + + Parameters + ---------- + in_kwargs : dict + Function kwargs + + """ + + # Initialize local data + ne = np.ones(shape=(self.in_coords['lon'].shape[0], + self.in_coords['lat'].shape[0], + self.in_coords['alt'].shape[0]), dtype=float) + + # Calculate the TEC + with pytest.raises(ValueError) as verr: + data_prep.calc_tec(self.in_coords['alt'], ne, **in_kwargs) + + assert str(verr).find("ialt_max` must be greater than `ialt_min") >= 0 + return diff --git a/aetherpy/tests/test_utils_inputs.py b/aetherpy/tests/test_utils_inputs.py index 0ac7111..9bb5d33 100644 --- a/aetherpy/tests/test_utils_inputs.py +++ b/aetherpy/tests/test_utils_inputs.py @@ -1,97 +1,97 @@ -#!/usr/bin/env python -# Copyright 2021, the Aether Development Team (see doc/dev_team.md for members) -# Full license can be found in License.md -"""Unit tests for input utilities.""" - -import pytest -import sys - -from aetherpy.utils import inputs as input_utils - - -class TestInputs(object): - """Unit tests for input utilities.""" - - @pytest.mark.parametrize("in_str, out_val", [ - ("true", True), ("false", False), ("TRUE", True), ("FALSE", False), - ("T", True), ("F", False), ("t", True), ("f", False), ("1", True), - ("0", False), ("", True), ("True", True), ("False", False), - ("tRuE", True), ("fAlSe", False)]) - def test_bool_string_success(self, in_str, out_val): - """Test conversion of a string to a Boolean. - - Parameters - ---------- - in_str : str - Input string with acceptable boolean-interpretable values - out_val : bool - Expected Boolean output - - """ - - out = input_utils.bool_string(in_str) - - assert out == out_val - return - - @pytest.mark.parametrize("in_val", ["Tru", "Fals", "3", "1.0", "0.0"]) - def test_bool_string_failure(self, in_val): - """Test conversion from datetime to epoch seconds. - - Parameters - ---------- - in_val : str - Input value that will not be recognized as a string that may be - converted to a boolean - - """ - - with pytest.raises(ValueError) as verr: - input_utils.bool_string(in_val) - - assert str(verr).find("input not interpretable as a boolean") >= 0 - return - - @pytest.mark.parametrize("in_str", ["none", "None", "nOne", ""]) - def test_none_string_success(self, in_str): - """Test conversion of a string to NoneType. - - Parameters - ---------- - in_str : str - Input string with acceptable NoneType conversion values - - """ - - assert input_utils.none_string(in_str) is None - return - - @pytest.mark.parametrize("in_str", ["non", "Non", "nOn", " "]) - def test_none_string_failure(self, in_str): - """Test conversion of a string to NoneType. - - Parameters - ---------- - in_str : str - Input string with unacceptable NoneType conversion values - - """ - - assert input_utils.none_string(in_str) == in_str - return - - @pytest.mark.parametrize("in_args, num_args", [(["ipython", "one"], 0), - (["module_name", "one"], 1)]) - def test_process_command_line_input(self, in_args, num_args): - """Test `process_command_line_input` using the command line.""" - - # Set and process the system arguments - sys.argv = in_args - out_args = input_utils.process_command_line_input() - - # Test the processed output - assert len(out_args) == num_args - for i, in_val in enumerate(in_args[len(in_args) - num_args:]): - assert out_args[i] == in_val - - return +#!/usr/bin/env python +# Copyright 2021, the Aether Development Team (see doc/dev_team.md for members) +# Full license can be found in License.md +"""Unit tests for input utilities.""" + +import pytest +import sys + +from aetherpy.utils import inputs as input_utils + + +class TestInputs(object): + """Unit tests for input utilities.""" + + @pytest.mark.parametrize("in_str, out_val", [ + ("true", True), ("false", False), ("TRUE", True), ("FALSE", False), + ("T", True), ("F", False), ("t", True), ("f", False), ("1", True), + ("0", False), ("", True), ("True", True), ("False", False), + ("tRuE", True), ("fAlSe", False)]) + def test_bool_string_success(self, in_str, out_val): + """Test conversion of a string to a Boolean. + + Parameters + ---------- + in_str : str + Input string with acceptable boolean-interpretable values + out_val : bool + Expected Boolean output + + """ + + out = input_utils.bool_string(in_str) + + assert out == out_val + return + + @pytest.mark.parametrize("in_val", ["Tru", "Fals", "3", "1.0", "0.0"]) + def test_bool_string_failure(self, in_val): + """Test conversion from datetime to epoch seconds. + + Parameters + ---------- + in_val : str + Input value that will not be recognized as a string that may be + converted to a boolean + + """ + + with pytest.raises(ValueError) as verr: + input_utils.bool_string(in_val) + + assert str(verr).find("input not interpretable as a boolean") >= 0 + return + + @pytest.mark.parametrize("in_str", ["none", "None", "nOne", ""]) + def test_none_string_success(self, in_str): + """Test conversion of a string to NoneType. + + Parameters + ---------- + in_str : str + Input string with acceptable NoneType conversion values + + """ + + assert input_utils.none_string(in_str) is None + return + + @pytest.mark.parametrize("in_str", ["non", "Non", "nOn", " "]) + def test_none_string_failure(self, in_str): + """Test conversion of a string to NoneType. + + Parameters + ---------- + in_str : str + Input string with unacceptable NoneType conversion values + + """ + + assert input_utils.none_string(in_str) == in_str + return + + @pytest.mark.parametrize("in_args, num_args", [(["ipython", "one"], 0), + (["module_name", "one"], 1)]) + def test_process_command_line_input(self, in_args, num_args): + """Test `process_command_line_input` using the command line.""" + + # Set and process the system arguments + sys.argv = in_args + out_args = input_utils.process_command_line_input() + + # Test the processed output + assert len(out_args) == num_args + for i, in_val in enumerate(in_args[len(in_args) - num_args:]): + assert out_args[i] == in_val + + return diff --git a/aetherpy/tests/test_utils_time.py b/aetherpy/tests/test_utils_time.py index a3828c6..c9ffd28 100644 --- a/aetherpy/tests/test_utils_time.py +++ b/aetherpy/tests/test_utils_time.py @@ -1,165 +1,165 @@ -#!/usr/bin/env python -# Copyright 2021, the Aether Development Team (see doc/dev_team.md for members) -# Full license can be found in License.md -"""Unit tests for time conversion utilities.""" - -import datetime as dt -import numpy as np -import pytest - -from aetherpy.utils import time_conversion as tc - - -class TestEpochTime(object): - """Unit tests for epoch time conversions.""" - - @pytest.mark.parametrize("esec, dtime", [ - (-10, dt.datetime(1964, 12, 31, 23, 59, 50)), - (0, dt.datetime(1965, 1, 1)), - (1000.5, dt.datetime(1965, 1, 1, 0, 16, 40, 500000))]) - def test_epoch_to_datetime(self, esec, dtime): - """Test conversion from epoch seconds to datetime. - - Parameters - ---------- - esec : int or float - Input epoch seconds - dtime : dt.datetime - Output datetime object - - """ - - out_time = tc.epoch_to_datetime(esec) - - assert isinstance(out_time, dt.datetime) - assert out_time == dtime - return - - @pytest.mark.parametrize("dtime, esec", [ - (dt.datetime(1950, 1, 1), -473385600.0), - (dt.datetime(1965, 1, 1), 0.0), - (dt.datetime(2010, 2, 3, 4, 5, 6, 7), 1422936306.000007)]) - def test_datetime_to_epoch(self, dtime, esec): - """Test conversion from datetime to epoch seconds. - - Parameters - ---------- - dtime : dt.datetime - Input datetime object - esec : float - Output epoch seconds - - """ - - out_time = tc.datetime_to_epoch(dtime) - - assert isinstance(out_time, float) - assert out_time == esec - return - - -class TestUT(object): - """Unit tests for conversions based around UT.""" - - def setup(self): - """Create a clean testing environment.""" - - self.times = [dt.datetime(2001, 1, 1) + dt.timedelta(seconds=i * 900) - for i in range(96)] - self.lon = 180.0 - self.lt = [12.0, 12.25, 12.5, 12.75, 13.0, 13.25, 13.5, 13.75, 14.0, - 14.25, 14.5, 14.75, 15.0, 15.25, 15.5, 15.75, 16.0, 16.25, - 16.5, 16.75, 17.0, 17.25, 17.5, 17.75, 18.0, 18.25, 18.5, - 18.75, 19.0, 19.25, 19.5, 19.75, 20.0, 20.25, 20.5, 20.75, - 21.0, 21.25, 21.5, 21.75, 22.0, 22.25, 22.5, 22.75, 23.0, - 23.25, 23.5, 23.75, 0.0, 0.25, 0.5, 0.75, 1.0, 1.25, 1.5, - 1.75, 2.0, 2.25, 2.5, 2.75, 3.0, 3.25, 3.5, 3.75, 4.0, 4.25, - 4.5, 4.75, 5.0, 5.25, 5.5, 5.75, 6.0, 6.25, 6.5, 6.75, 7.0, - 7.25, 7.5, 7.75, 8.0, 8.25, 8.5, 8.75, 9.0, 9.25, 9.5, 9.75, - 10.0, 10.25, 10.5, 10.75, 11.0, 11.25, 11.5, 11.75] - return - - def teardown(self): - """Clean up the test environment.""" - - del self.times, self.lon, self.lt - - def update_lon(self, multi_lon): - """Update the `lon` attribute. - - Parameters - ---------- - multi_lon : bool - Include a single longitude value or an array-like object - - """ - - if multi_lon: - self.lon = np.full(shape=len(self.times), fill_value=self.lon) - return - - @pytest.mark.parametrize("lon_sign", [1, -1]) - @pytest.mark.parametrize("multi_lon", [True, False]) - def test_ut_to_lt(self, lon_sign, multi_lon): - """Test the datetime in UT to solar local time conversion. - - Parameters - ---------- - lon_sign : int - One or negative one, specifies whether longitude should be - positive or negative - multi_lon : bool - Include a single longitude value or an array-like object - - """ - - # Update the longitude - self.update_lon(multi_lon) - self.lon *= lon_sign - - # Get the solar local time - out = tc.ut_to_lt(self.times, self.lon) - - # Test the time output - assert out.shape[0] == len(self.times) - assert np.all(out == np.array(self.lt)) - return - - @pytest.mark.parametrize("multi_lon", [True, False]) - def test_lt_to_ut(self, multi_lon): - """Test the solar local time to UT in hours of day conversion. - - Parameters - ---------- - multi_lon : bool - Include a single longitude value or an array-like object - - """ - - self.update_lon(multi_lon) - out = tc.lt_to_ut(self.lt, self.lon) - - assert out.shape[0] == len(self.lt) - - for i, uth in enumerate(out): - test_uth = self.times[i].hour + self.times[i].minute / 60.0 \ - + (self.times[i].second - + self.times[i].microsecond * 1e-6) / 3600.0 - assert test_uth == uth, \ - "{:d} time element doesn't match {:f} != {:f}".format( - i, test_uth, uth) - return - - def test_calc_time_shift(self): - """Test the polar dial shift calculation.""" - - # The output will be 15 degrees times the UT hours of day - out = tc.lt_to_ut(self.lt, self.lon) * 15.0 - - for i, test_shift in enumerate(out): - shift = tc.calc_time_shift(self.times[i]) - - assert shift == test_shift, \ - "{:d} time element has unexpected offset {:f} != {:f}".format( - i, shift, test_shift) - return +#!/usr/bin/env python +# Copyright 2021, the Aether Development Team (see doc/dev_team.md for members) +# Full license can be found in License.md +"""Unit tests for time conversion utilities.""" + +import datetime as dt +import numpy as np +import pytest + +from aetherpy.utils import time_conversion as tc + + +class TestEpochTime(object): + """Unit tests for epoch time conversions.""" + + @pytest.mark.parametrize("esec, dtime", [ + (-10, dt.datetime(1964, 12, 31, 23, 59, 50)), + (0, dt.datetime(1965, 1, 1)), + (1000.5, dt.datetime(1965, 1, 1, 0, 16, 40, 500000))]) + def test_epoch_to_datetime(self, esec, dtime): + """Test conversion from epoch seconds to datetime. + + Parameters + ---------- + esec : int or float + Input epoch seconds + dtime : dt.datetime + Output datetime object + + """ + + out_time = tc.epoch_to_datetime(esec) + + assert isinstance(out_time, dt.datetime) + assert out_time == dtime + return + + @pytest.mark.parametrize("dtime, esec", [ + (dt.datetime(1950, 1, 1), -473385600.0), + (dt.datetime(1965, 1, 1), 0.0), + (dt.datetime(2010, 2, 3, 4, 5, 6, 7), 1422936306.000007)]) + def test_datetime_to_epoch(self, dtime, esec): + """Test conversion from datetime to epoch seconds. + + Parameters + ---------- + dtime : dt.datetime + Input datetime object + esec : float + Output epoch seconds + + """ + + out_time = tc.datetime_to_epoch(dtime) + + assert isinstance(out_time, float) + assert out_time == esec + return + + +class TestUT(object): + """Unit tests for conversions based around UT.""" + + def setup(self): + """Create a clean testing environment.""" + + self.times = [dt.datetime(2001, 1, 1) + dt.timedelta(seconds=i * 900) + for i in range(96)] + self.lon = 180.0 + self.lt = [12.0, 12.25, 12.5, 12.75, 13.0, 13.25, 13.5, 13.75, 14.0, + 14.25, 14.5, 14.75, 15.0, 15.25, 15.5, 15.75, 16.0, 16.25, + 16.5, 16.75, 17.0, 17.25, 17.5, 17.75, 18.0, 18.25, 18.5, + 18.75, 19.0, 19.25, 19.5, 19.75, 20.0, 20.25, 20.5, 20.75, + 21.0, 21.25, 21.5, 21.75, 22.0, 22.25, 22.5, 22.75, 23.0, + 23.25, 23.5, 23.75, 0.0, 0.25, 0.5, 0.75, 1.0, 1.25, 1.5, + 1.75, 2.0, 2.25, 2.5, 2.75, 3.0, 3.25, 3.5, 3.75, 4.0, 4.25, + 4.5, 4.75, 5.0, 5.25, 5.5, 5.75, 6.0, 6.25, 6.5, 6.75, 7.0, + 7.25, 7.5, 7.75, 8.0, 8.25, 8.5, 8.75, 9.0, 9.25, 9.5, 9.75, + 10.0, 10.25, 10.5, 10.75, 11.0, 11.25, 11.5, 11.75] + return + + def teardown(self): + """Clean up the test environment.""" + + del self.times, self.lon, self.lt + + def update_lon(self, multi_lon): + """Update the `lon` attribute. + + Parameters + ---------- + multi_lon : bool + Include a single longitude value or an array-like object + + """ + + if multi_lon: + self.lon = np.full(shape=len(self.times), fill_value=self.lon) + return + + @pytest.mark.parametrize("lon_sign", [1, -1]) + @pytest.mark.parametrize("multi_lon", [True, False]) + def test_ut_to_lt(self, lon_sign, multi_lon): + """Test the datetime in UT to solar local time conversion. + + Parameters + ---------- + lon_sign : int + One or negative one, specifies whether longitude should be + positive or negative + multi_lon : bool + Include a single longitude value or an array-like object + + """ + + # Update the longitude + self.update_lon(multi_lon) + self.lon *= lon_sign + + # Get the solar local time + out = tc.ut_to_lt(self.times, self.lon) + + # Test the time output + assert out.shape[0] == len(self.times) + assert np.all(out == np.array(self.lt)) + return + + @pytest.mark.parametrize("multi_lon", [True, False]) + def test_lt_to_ut(self, multi_lon): + """Test the solar local time to UT in hours of day conversion. + + Parameters + ---------- + multi_lon : bool + Include a single longitude value or an array-like object + + """ + + self.update_lon(multi_lon) + out = tc.lt_to_ut(self.lt, self.lon) + + assert out.shape[0] == len(self.lt) + + for i, uth in enumerate(out): + test_uth = self.times[i].hour + self.times[i].minute / 60.0 \ + + (self.times[i].second + + self.times[i].microsecond * 1e-6) / 3600.0 + assert test_uth == uth, \ + "{:d} time element doesn't match {:f} != {:f}".format( + i, test_uth, uth) + return + + def test_calc_time_shift(self): + """Test the polar dial shift calculation.""" + + # The output will be 15 degrees times the UT hours of day + out = tc.lt_to_ut(self.lt, self.lon) * 15.0 + + for i, test_shift in enumerate(out): + shift = tc.calc_time_shift(self.times[i]) + + assert shift == test_shift, \ + "{:d} time element has unexpected offset {:f} != {:f}".format( + i, shift, test_shift) + return diff --git a/aetherpy/tests/utils.py b/aetherpy/tests/utils.py index f58b755..5851c27 100644 --- a/aetherpy/tests/utils.py +++ b/aetherpy/tests/utils.py @@ -1,51 +1,51 @@ -#!/usr/bin/env python -# Copyright 2021, the Aether Development Team (see doc/dev_team.md for members) -# Full license can be found in License.md -"""Utilities for test classes.""" - -import os - - -def sys_agnostic_rename(inname, outname, max_attemps=100): - """Wrap os.rename, Windows OS sometimes needs time to allow rename to work. - - Parameters - ---------- - inname : str - Input filename or directory - outname : str - Output filename or directory - max_attemps : int - Maximum rename attemps (default=100) - - """ - for retry in range(max_attemps): - try: - os.rename(inname, outname) - break - except Exception: - pass - - return - - -def sys_agnostic_remove(fname, max_attemps=100): - """Wrap os.remove, Windows OS sometimes needs time to allow remove to work. - - Parameters - ---------- - fname : str - Filename to be removed - max_attemps : int - Maximum rename attemps (default=100) - - """ - for retry in range(max_attemps): - if os.path.isfile(fname): - try: - os.remove(fname) - break - except Exception: - pass - - return +#!/usr/bin/env python +# Copyright 2021, the Aether Development Team (see doc/dev_team.md for members) +# Full license can be found in License.md +"""Utilities for test classes.""" + +import os + + +def sys_agnostic_rename(inname, outname, max_attemps=100): + """Wrap os.rename, Windows OS sometimes needs time to allow rename to work. + + Parameters + ---------- + inname : str + Input filename or directory + outname : str + Output filename or directory + max_attemps : int + Maximum rename attemps (default=100) + + """ + for retry in range(max_attemps): + try: + os.rename(inname, outname) + break + except Exception: + pass + + return + + +def sys_agnostic_remove(fname, max_attemps=100): + """Wrap os.remove, Windows OS sometimes needs time to allow remove to work. + + Parameters + ---------- + fname : str + Filename to be removed + max_attemps : int + Maximum rename attemps (default=100) + + """ + for retry in range(max_attemps): + if os.path.isfile(fname): + try: + os.remove(fname) + break + except Exception: + pass + + return diff --git a/aetherpy/thermo_plot.py b/aetherpy/thermo_plot.py new file mode 100644 index 0000000..b67c0f1 --- /dev/null +++ b/aetherpy/thermo_plot.py @@ -0,0 +1,608 @@ +#!/usr/bin/env python +# Copyright 2020, the Aether Development Team (see doc/dev_team.md for members) +# Full license can be found in License.md +"""Block-based model visualization routines.""" + +from glob import glob +import os +import re +import argparse +from xml.dom import minicompat +import matplotlib as mpl +import matplotlib.pyplot as plt +import matplotlib.cm as cm +import matplotlib.path as mpath +import matplotlib.patches as mpatches +import matplotlib.colors as colors +import numpy as np +import cartopy.crs as ccrs +from scipy.spatial import KDTree + +from aetherpy import logger +from aetherpy.io import read_routines +from aetherpy.plot import data_prep +from aetherpy.plot import movie_routines +from aetherpy.utils import inputs +from aetherpy.utils import time_conversion + + +# ---------------------------------------------------------------------------- +# Define the support routines + +def get_help(file_vars=None): + """Provide string explaining how to run the command line interface. + + Parameters + ---------- + file_vars : list or NoneType + List of file variables or None to exclude this output (default=None) + + Returns + ------- + help_str : str + String with formatted help statement + + """ + + mname = os.path.join( + os.path.commonpath([inputs.__file__, data_prep.__file__]), + 'plot_block_model_results.py') if __name__ == '__main__' else __name__ + + # TODO: Update help string + help_str = ''.join(['Usage:\n{:s} -[flags] [filenames]\n'.format(mname), + 'Flags:\n', + ' -help : print this message, include filename ', + 'for variable names and indices\n', + ' -var=number : index of variable to plot\n', + ' -cut=alt, lat, or lon : which cut you would ', + 'like\n', + ' -alt=number : alt in km or grid number ', + '(closest)\n', + ' -lat=number : latitude in degrees (closest)\n', + ' -lon=number: longitude in degrees (closest)\n', + ' -log : plot the log of the variable\n', + ' -winds : overplot winds\n', + ' -tec : plot the TEC variable\n', + ' -movie=number : provide a positive frame rate', + ' to create a movie\n', + ' -ext=str : figure or movie extension\n', + 'At end, list the files you want to plot. This code ', + 'should work with either GITM files (*.bin) or Aether', + ' netCDF files (*.nc)']) + + if file_vars is not None: + help_str += "File Variables (index, name):\n" + for ivar, var in enumerate(file_vars): + help_str += " ({:d}, {:s})\n".format(ivar, var) + + return help_str + + +def argparse_command_line_args(): + parser = argparse.ArgumentParser( + description='Plotting script for Aether model output files', + epilog='This code should work with either GITM files (*.bin) \ + or Aether netcdf files (*.nc).', + add_help=False) + parser.add_argument('filelist', nargs='+', help='file(s) to plot') + parser.add_argument('-h', '--help', action='store_true', + help='show help message and list of variables, exit') + parser.add_argument('-v', '--var', nargs='*', default=[], + help='name of variable(s) to plot') + parser.add_argument('-n', '--varn', nargs='*', type=int, default=[], + help='index of variable(s) to plot') + parser.add_argument('-c', '--cut', default='alt', + choices=['alt', 'lat', 'lon'], + help='which cut you would like') + parser.add_argument('-a', '--alt', default=np.nan, type=float, + help='alt in km (closest)') + parser.add_argument('--lat', default=np.nan, type=float, + help='latitude in degrees (closest)') + parser.add_argument('--lon', default=np.nan, type=float, + help='longitude in degrees (closest)') + parser.add_argument('-l', '--log', action='store_true', + help='plot the log of the variable') + parser.add_argument('-t', '--tec', action='store_true', + help='plot the TEC variable') + parser.add_argument('-w', '--winds', action='store_true', + help='overplot neutral winds (only if --ions is off)') + parser.add_argument('-i', '--ions', action='store_true', + help='overplot ion winds') + parser.add_argument('-d', '--diff', action='store_true', + help='flag for difference with other plots') + parser.add_argument('--is_gitm', action='store_true', + help='flag for plotting gitm files') + parser.add_argument('--has_header', action='store_true', + help='flag for if file headers exist') + parser.add_argument('--movie', default=0, type=int, + help='provide a positive framerate to create a movie') + parser.add_argument('--ext', default='png', + help='figure or movie extension') + parser.add_argument('--original', action='store_true', + help='create exact plots (warning: very slow)') + args = parser.parse_args() + # Output generic help if no files + if (len(args.filelist) == 0): + parser.print_help() + return False + # Process var and varn into one list, output indices and variables if bad + header = read_routines.read_file(args.filelist[0]) + if (args.help): + parser.print_help() + list_header_variables(header) + return False + varnames = args.var + for varn in args.varn: + try: + varnames.append(header['vars'][varn]) + except IndexError: + print(f"{varn} is an invalid variable index") + list_header_variables(header) + return False + for name in varnames: + if name not in header['vars']: + print(f"{name} is an invalid variable name") + list_header_variables(header) + return False + args.var = varnames + return args.__dict__ + + +def list_header_variables(header): + print("File Variables: ") + print("\tIndex\tName") + for i, var in enumerate(header['vars']): + print(f"\t{i}\t{var}") + + +def determine_min_max_within_range(data, var, alt, + min_lon=-np.inf, max_lon=np.inf, + min_lat=-np.inf, max_lat=np.inf): + """Determines the minimum and maximum values of var at a given altitude + within a rectangular latitude / longitude range. + + Parameters + ---------- + data : dict + Dictionary containing each block's data + var : str + Name of the variable to find the min/max of + alt : int + Index of the altitude to find the min/max at + min_lon : float + Minimum longitude of desired range (inclusive), defaults to -inf + max_lon : float + Maximum longitude of desired range (inclusive), defaults to +inf + min_lat : float + Minimum latitude of desired range (inclusive), defaults to -inf + max_lat : float + Maximum latitude of desired range (inclusive), defaults to +inf + + Returns + ---------- + mini, maxi : tuple + Tuple containing min and max + """ + all_lons = data['lon'][:, 2:-2, 2:-2, alt] + all_lats = data['lat'][:, 2:-2, 2:-2, alt] + all_v = data[var][:, 2:-2, 2:-2, alt] + cond = (all_lons >= min_lon) & (all_lons <= max_lon) \ + & (all_lats >= min_lat) & (all_lats <= max_lat) + mini = np.nanmin(all_v[cond], initial=np.inf) + maxi = np.nanmax(all_v[cond], initial=-np.inf) + return mini, maxi + + +def determine_min_max(data, var, alt): + """Determines the minimum and maximum values of var at a given altitude; + convenience function for determine_min_max_within_range. + + Parameters + ---------- + data : dict + Dictionary containing each block's data + var : str + Name of the variable to find the min/max of + alt : int + Index of the altitude to find the min/max at + + Returns + ---------- + mini, maxi : tuple + Tuple containing min and max + """ + return determine_min_max_within_range( + data, var, alt + ) + + +def get_plotting_bounds(data, var, alt): + mini, maxi = determine_min_max(data, var, alt) + if mini < 0: + maxi = max(np.abs(mini), np.abs(maxi)) + mini = -maxi + return mini, maxi + + +# ---------------------------------------------------------------------------- +# Define the main plotting routines + + +def lon_lat_to_cartesian(lon, lat, R=1): + """ + calculates lon, lat coordinates of a point on a sphere with + radius R + """ + lon_r = np.radians(lon) + lat_r = np.radians(lat) + + x = R * np.cos(lat_r) * np.cos(lon_r) + y = R * np.cos(lat_r) * np.sin(lon_r) + z = R * np.sin(lat_r) + return x, y, z + + +def generate_spherical_mesh(lon_cells, lat_cells): + lon_halfdim = 360 / (2 * lon_cells) + lat_halfdim = 180 / (2 * lat_cells) + x = np.linspace(0 + lon_halfdim, 360 - lon_halfdim, lon_cells) + y = np.linspace(-90 + lat_halfdim, 90 - lat_halfdim, lat_cells) + X, Y = np.meshgrid(x, y) + return X, Y + + +def generate_circular_mesh(lon_cells, lat_cells, reference_lat): + lons = [] + lats = [] + lat_halfdim = 180 / (2 * lat_cells) + lat_centers = np.linspace(-90 + lat_halfdim, 90 - lat_halfdim, lat_cells) + reference_cos = np.cos(np.radians(reference_lat)) + ring_lon_cells = np.int64( + lon_cells * np.cos(np.radians(lat_centers)) / reference_cos) + for lat, cells in zip(lat_centers, ring_lon_cells): + ring_lons = np.linspace(0, 360, cells + 1)[:-1] + ring_lats = [lat] * cells + lons.extend(ring_lons) + lats.extend(ring_lats) + lons = np.asarray(lons) + lats = np.asarray(lats) + return lons, lats + + +def plot_winds(ax, data, alt_idx_to_plot, args, target_mesh): + if not (args['winds'] or args['ions']): + return + desire_neutral_winds = not args['ions'] + neutral_vars = { + 'east': 'Zonal Wind', + 'north': 'Meridional Wind', + 'up': 'Vertical Wind' + } + ion_vars = { + 'east': 'BulkIon Velocity (Zonal)', + 'north': 'BulkIon Velocity (Meridional)', + 'up': 'BulkIon Velocity (Vertical)' + } + wind_vars = neutral_vars if desire_neutral_winds else ion_vars + vars_to_plot = list(wind_vars.values()) + # TODO: Create adjusted wind mesh to fix density around poles + + target_lon, target_lat, target_winds = generate_interpolated_data( + data, vars_to_plot, alt_idx_to_plot, target_mesh=target_mesh + ) + # TODO: Choose x_winds and y_winds based on cut from args + x_winds = target_winds[wind_vars['east']] + y_winds = target_winds[wind_vars['north']] + + # Example uniform winds for testing + # x_winds = np.ones(target_lon.shape) + # y_winds = np.ones(target_lon.shape) * 0.5 + + ax.quiver(target_lon, target_lat, x_winds, y_winds, + transform=ccrs.PlateCarree()) + + +def generate_interpolated_data(data, vars_to_plot, alt_idx_to_plot, + lon_cells=1800, lat_cells=900, target_mesh=None, + neighbors=1): + # Validate vars_to_plot input type + if not isinstance(vars_to_plot, list): + vars_to_plot = [vars_to_plot] + # Load input data for all variables + source_lons = [] + source_lats = [] + source_vals = {var: [] for var in vars_to_plot} + for i in range(data['nblocks']): + lon = data['lon'][i, 2:-2, 2:-2, alt_idx_to_plot] + lat = data['lat'][i, 2:-2, 2:-2, alt_idx_to_plot] + source_lons.extend(lon.flatten()) + source_lats.extend(lat.flatten()) + for var in vars_to_plot: + v = data[var][i, 2:-2, 2:-2, alt_idx_to_plot] + source_vals[var].extend(v.flatten()) + + # Generate KDTree of source point cloud + x_source, y_source, z_source = lon_lat_to_cartesian(source_lons, + source_lats) + tree = KDTree(list(zip(x_source, y_source, z_source))) + + # Generate target mesh + if target_mesh is None: + target_lon, target_lat = generate_spherical_mesh(lon_cells, lat_cells) + else: + target_lon, target_lat = target_mesh + + # Compute IDW interpolation weights + x_target, y_target, z_target = lon_lat_to_cartesian(target_lon.flatten(), + target_lat.flatten()) + d, inds = tree.query(list(zip(x_target, y_target, z_target)), k=neighbors) + if len(d.shape) == 1: + d = np.expand_dims(d, axis=1) + if len(inds.shape) == 1: + inds = np.expand_dims(inds, axis=1) + d += 0.00001 + w = 1.0 / (d * d) + + print(d.shape, inds.shape) + + # Apply weights to get interpolated variable data + target_vs = {} + for var, values in source_vals.items(): + values = np.asarray(values) + target_v = np.sum(w * values[inds], axis=1) / np.sum(w, axis=1) + target_v.shape = target_lon.shape + target_vs[var] = target_v + + # Simplify output if only one variable was given + target_output = target_v if len(vars_to_plot) == 1 else target_vs + + return target_lon, target_lat, target_output + + +def plot_all_blocks(data, var_to_plot, alt_idx_to_plot, plot_filename, args, + mini=None, maxi=None): + print(f" Plotting variable: {var_to_plot}") + + # Initialize figure to plot on + fig = plt.figure(figsize=(13, 13)) + altitude = round( + data['z'][0, 0, 0, alt_idx_to_plot] / 1000.0, 2) + time = data['time'] + title = f"{time}; var: {var_to_plot}; alt: {altitude} km" + + # Initialize colorbar information + if mini is None or maxi is None: + mini, maxi = get_plotting_bounds( + data, var_to_plot, alt_idx_to_plot) + norm = colors.Normalize(vmin=mini, vmax=maxi) + cmap = cm.plasma if mini >= 0 else cm.bwr + col = 'white' if mini >= 0 else 'black' + + # Calculate circle plot rotations to place sun at top + hours = time.hour + time.minute / 60 + time.second / 3600 + north_longitude = -15 * hours + south_longitude = 180 - 15 * hours + + # Define subplot projections and gridspecs, create subplots + north_proj = ccrs.Stereographic( + central_latitude=90, central_longitude=north_longitude) + south_proj = ccrs.Stereographic( + central_latitude=-90, central_longitude=south_longitude) + world_proj = ccrs.PlateCarree(central_longitude=0) + + # Create subplots + gs = fig.add_gridspec(nrows=2, ncols=2, hspace=-0.2) + + north_ax = fig.add_subplot(gs[0, 0], projection=north_proj) + south_ax = fig.add_subplot(gs[0, 1], projection=south_proj) + world_ax = fig.add_subplot(gs[1, :], projection=world_proj) + world_ax.title.set_text(title) + + # Set subplot extents + world_ax.set_global() + + # Limit latitudes of circle plots to >45 degrees N/S + border_latitude = 45 + set_circle_plot_bounds([north_ax, south_ax], north_proj, border_latitude) + + # Plot interpolated var_to_plot data onto subplots + target_lon, target_lat, target_v = generate_interpolated_data( + data, var_to_plot, alt_idx_to_plot) + + # Apply log to data if desired + if args['log']: + target_v = np.log10(target_v) + mini = np.log10(mini) + maxi = np.log10(maxi) + + # Plot interpolated data + ax_list = [north_ax, south_ax, world_ax] + plot_kwargs = { + 'vmin': mini, + 'vmax': maxi, + 'cmap': cmap, + 'transform': ccrs.PlateCarree() + } + for ax in ax_list: + ax.pcolormesh(target_lon, target_lat, target_v, **plot_kwargs) + + # Plot interpolated winds + if args['winds'] or args['ions']: + world_ax_mesh = generate_spherical_mesh(36, 18) + circle_ax_mesh = generate_circular_mesh(36, 18, 45) + plot_winds(world_ax, data, alt_idx_to_plot, args, world_ax_mesh) + for ax in [north_ax, south_ax]: + plot_winds(ax, data, alt_idx_to_plot, args, circle_ax_mesh) + + # Add elements affecting all subplots + for ax in ax_list: + ax.coastlines(color=col) + + # Configure colorbar + power = int(np.log10(max(maxi, 1))) + create_colorbar(fig, norm, cmap, ax_list, var_to_plot, power) + + # Add labels to circle plots + north_minmax = determine_min_max_within_range( + data, var_to_plot, alt_idx_to_plot, + min_lat=45, max_lat=90 + ) + south_minmax = determine_min_max_within_range( + data, var_to_plot, alt_idx_to_plot, + min_lat=-90, max_lat=-45 + ) + label_circle_plots(north_ax, south_ax, *north_minmax, *south_minmax) + + # Save plot + print(f" Saving plot to: {plot_filename}.png") + fig.savefig(plot_filename, bbox_inches='tight') + plt.close(fig) + + +def set_circle_plot_bounds(circle_ax_list, circle_proj, border_latitude): + border_latitude = abs(border_latitude) + r_limit = abs(circle_proj.transform_point( + 90 + circle_proj.proj4_params['lon_0'], + border_latitude, ccrs.PlateCarree())[0] + ) + r_extent = r_limit * 1.0001 + circle_bound = mpath.Path.unit_circle() + circle_bound = mpath.Path( + circle_bound.vertices.copy() * r_limit, circle_bound.codes.copy()) + for ax in circle_ax_list: + ax.set_xlim(-r_extent, r_extent) + ax.set_ylim(-r_extent, r_extent) + ax.set_boundary(circle_bound) + + +def label_circle_plots(north_ax, south_ax, north_min, north_max, + south_min, south_max): + # Add local time labels to circle plots + north_kwargs = { + 'horizontalalignment': 'center', + 'verticalalignment': 'center', + 'fontsize': 'small', + 'transform': north_ax.transAxes + } + south_kwargs = { + 'horizontalalignment': 'center', + 'verticalalignment': 'center', + 'fontsize': 'small', + 'transform': south_ax.transAxes + } + + north_ax.text(0.5, -0.03, '00', **north_kwargs) # Bottom + north_ax.text(1.03, 0.5, '06', **north_kwargs) # Right + north_ax.text(0.5, 1.03, '12', **north_kwargs) # Top + north_ax.text(-0.03, 0.5, '18', **north_kwargs) # Left + + south_ax.text(0.5, -0.03, '00', **south_kwargs) # Bottom + south_ax.text(-0.03, 0.5, '06', **south_kwargs) # Left + south_ax.text(0.5, 1.03, '12', **south_kwargs) # Top + south_ax.text(1.03, 0.5, '18', **south_kwargs) # Right + + # Add min/max labels to circle plots + north_mintext = f"Min: {north_min:.3e}".replace('+', '') + north_maxtext = f"Max: {north_max:.3e}".replace('+', '') + south_mintext = f"Min: {south_min:.3e}".replace('+', '') + south_maxtext = f"Max: {south_max:.3e}".replace('+', '') + north_ax.text(0.125, 0.125, north_mintext, **north_kwargs, rotation=-45) + north_ax.text(0.875, 0.125, north_maxtext, **north_kwargs, rotation=45) + south_ax.text(0.125, 0.125, south_mintext, **south_kwargs, rotation=-45) + south_ax.text(0.875, 0.125, south_maxtext, **south_kwargs, rotation=45) + + +def create_colorbar(fig, norm, cmap, ax_list, var_to_plot, power): + cbar = fig.colorbar(cm.ScalarMappable(norm=norm, cmap=cmap), + ax=ax_list, shrink=0.5, pad=0.03) + cbar.formatter.set_useMathText(True) + cbar.ax.yaxis.get_offset_text().set_rotation('vertical') + cbar_label = f"{var_to_plot} (x$10^{{{power}}} / $m$^3$)" + cbar.set_label(cbar_label, rotation='vertical') + cbar.formatter.set_powerlimits((0, 0)) + + +def read_all_files(filelist, file_vars=None): + files_data = {} + common_vars = None + for filename in filelist: + # Retrieve data + data = read_routines.read_file(filename) + data = read_routines.standardize_data(data) + # Save data to dict, update set of common variables + files_data[filename] = data + if common_vars is None: + common_vars = set(data['vars']) + else: + common_vars &= set(data['vars']) + common_vars = [var for var in common_vars if var != 'time'] + return files_data, common_vars + + +def plot_model_block_results(): + # Get the input arguments + # args = get_command_line_args(inputs.process_command_line_input()) + args = argparse_command_line_args() + if (args is False): + return + + # Read headers for input files (assumes all files have same header) + # header = read_routines.read_blocked_netcdf_header(args['filelist'][0]) + header = read_routines.read_file(args['filelist'][0]) + + # Output help + if (len(args['filelist']) == 0 or args['help']): + help_str = get_help(header['vars'] if args['help'] else None) + print(help_str) + return + + # Determine variables to plot (currently hardcoded) + # TODO: handle winds correctly + desired_alt = args['alt'] * 1000 + file_vars = ['lon', 'lat', 'z', *args['var']] if args['var'] else None + + # Process all file data + files_data, common_vars = read_all_files(args['filelist'], file_vars) + + # Calculate min and max for all common vars over all files + var_min = { + var: np.inf + for var in common_vars + } + var_max = { + var: -np.inf + for var in common_vars + } + for filename, data in files_data.items(): + available_alts = data['z'][0, 0, 0, :] + alt_idx_to_plot = np.argmin(np.abs(available_alts - desired_alt)) + for var in common_vars: + data_min, data_max = determine_min_max(data, var, alt_idx_to_plot) + var_min[var] = min(var_min[var], data_min) + var_max[var] = max(var_max[var], data_max) + + # Generate plots for each file + for filename, data in files_data.items(): + print(f"Currently plotting: {filename}") + + # Plot desired variable if given, plot all variables if not + all_vars = [v for v in data['vars'] + if v not in ['time', 'lon', 'lat', 'z']] + plot_vars = args['var'] if args['var'] else all_vars + available_alts = data['z'][0, 0, 0, :] + alt_idx_to_plot = np.argmin(np.abs(available_alts - desired_alt)) + + # Generate plots for each variable requested + for var_to_plot in plot_vars: + var_name_stripped = var_to_plot.replace(" ", "") + plot_filename = f"{filename.split('.')[0]}_{var_name_stripped}" + mini = var_min[var_to_plot] if var_to_plot in var_min else None + maxi = var_max[var_to_plot] if var_to_plot in var_max else None + plot_all_blocks( + data, var_to_plot, alt_idx_to_plot, + plot_filename, args, mini, maxi) + + +# Needed to run main script as the default executable from the command line +if __name__ == '__main__': + plot_model_block_results() diff --git a/aetherpy/utils/__init__.py b/aetherpy/utils/__init__.py index 8c1db2a..62f5ab9 100644 --- a/aetherpy/utils/__init__.py +++ b/aetherpy/utils/__init__.py @@ -1,8 +1,8 @@ -#!/usr/bin/env python -# Copyright 2020, the Aether Development Team (see doc/dev_team.md for members) -# Full license can be found in License.md -"""General model utilities.""" - - -from aetherpy.utils import inputs -from aetherpy.utils import time_conversion +#!/usr/bin/env python +# Copyright 2020, the Aether Development Team (see doc/dev_team.md for members) +# Full license can be found in License.md +"""General model utilities.""" + + +from aetherpy.utils import inputs +from aetherpy.utils import time_conversion diff --git a/aetherpy/utils/inputs.py b/aetherpy/utils/inputs.py index 50e1cf3..7090de0 100644 --- a/aetherpy/utils/inputs.py +++ b/aetherpy/utils/inputs.py @@ -1,83 +1,83 @@ -#!/usr/bin/env python -# Copyright 2020, the Aether Development Team (see doc/dev_team.md for members) -# Full license can be found in License.md -"""Utilities for handling command line inputs.""" - -import sys - - -def bool_string(line): - """Determine whether a string should be True or False. - - Parameters - ---------- - line : string - Line to be tested - - Returns - ------- - bout : bool - Boolean output (True/False) - - Raises - ------ - ValueError - If the value cannot be interpreted as True or False - - Notes - ----- - Accepts empty, true, false, t, f, 1, and 0 in any capitalization combo - - """ - - line = line.lower() - - bout = None - if line in ['true', 't', '1', '']: - bout = True - elif line in ['false', 'f', '0']: - bout = False - - if bout is None: - raise ValueError('input not interpretable as a boolean') - - return bout - - -def none_string(line): - """Determine whether a string should be None. - - Parameters - ---------- - line : str - Line to be tested - - Returns - ------- - out_line : str or NoneType - None if all-lowercase version of line is "none" or line is zero length. - Otherwise returns original value of line. - - """ - - out_line = None if line.lower() == "none" or len(line) == 0 else line - return out_line - - -def process_command_line_input(): - """Process command line input, needed to possible ipython use. - - Returns - ------- - input_args : list - List of input arguments - - """ - - input_args = sys.argv - if input_args[0].find('ipython') >= 0: - input_args = list() - else: - input_args.pop(0) - - return input_args +#!/usr/bin/env python +# Copyright 2020, the Aether Development Team (see doc/dev_team.md for members) +# Full license can be found in License.md +"""Utilities for handling command line inputs.""" + +import sys + + +def bool_string(line): + """Determine whether a string should be True or False. + + Parameters + ---------- + line : string + Line to be tested + + Returns + ------- + bout : bool + Boolean output (True/False) + + Raises + ------ + ValueError + If the value cannot be interpreted as True or False + + Notes + ----- + Accepts empty, true, false, t, f, 1, and 0 in any capitalization combo + + """ + + line = line.lower() + + bout = None + if line in ['true', 't', '1', '']: + bout = True + elif line in ['false', 'f', '0']: + bout = False + + if bout is None: + raise ValueError('input not interpretable as a boolean') + + return bout + + +def none_string(line): + """Determine whether a string should be None. + + Parameters + ---------- + line : str + Line to be tested + + Returns + ------- + out_line : str or NoneType + None if all-lowercase version of line is "none" or line is zero length. + Otherwise returns original value of line. + + """ + + out_line = None if line.lower() == "none" or len(line) == 0 else line + return out_line + + +def process_command_line_input(): + """Process command line input, needed to possible ipython use. + + Returns + ------- + input_args : list + List of input arguments + + """ + + input_args = sys.argv + if input_args[0].find('ipython') >= 0: + input_args = list() + else: + input_args.pop(0) + + return input_args diff --git a/aetherpy/utils/time_conversion.py b/aetherpy/utils/time_conversion.py index 13d9800..46d4b8e 100644 --- a/aetherpy/utils/time_conversion.py +++ b/aetherpy/utils/time_conversion.py @@ -1,151 +1,151 @@ -#!/usr/bin/env python -# Copyright 2020, the Aether Development Team (see doc/dev_team.md for members) -# Full license can be found in License.md -"""Routines to perform temporal calculations.""" - -import datetime as dt -import numpy as np - - -def epoch_to_datetime(epoch_time): - """Convert from epoch seconds to datetime. - - Parameters - ---------- - epoch_time : int - Seconds since 1 Jan 1965 - - Returns - ------- - dtime : dt.datetime - Datetime object corresponding to `epoch_time` - - Notes - ----- - Epoch starts at 1 Jan 1965. - - """ - - dtime = dt.datetime(1965, 1, 1) + dt.timedelta(seconds=epoch_time) - - return dtime - - -def datetime_to_epoch(dtime): - """Convert datetime to epoch seconds. - - Parameters - ---------- - dtime : dt.datetime or dt.date - Datetime object - - Returns - ------- - epoch_time : float - Seconds since 1 Jan 1965 - - """ - - epoch_time = (dtime - dt.datetime(1965, 1, 1)).total_seconds() - - return epoch_time - - -def ut_to_lt(time_array, glon): - """Compute local time from date and longitude. - - Parameters - ---------- - time_array : array-like - Array-like of datetime objects in universal time - glon : array-like or float - Float or array-like of floats containing geographic longitude in - degrees. If single value or array of a different shape, all longitudes - are applied to all times. If the shape is the same as `time_array`, - the values are paired in the SLT calculation. - - Returns - ------- - lt : array of floats - List of local times in hours - - Raises - ------ - TypeError - For badly formatted input - - """ - - time_array = np.asarray(time_array) - glon = np.asarray(glon) - - # Get UT seconds of day - utsec = [(ut.hour * 3600.0 + ut.minute * 60.0 + ut.second - + ut.microsecond * 1.0e-6) / 3600.0 for ut in time_array] - - # Determine if the calculation is paired or broadcasted - if glon.shape == time_array.shape: - lt = np.array([utime + glon[i] / 15.0 for i, utime in enumerate(utsec)]) - else: - lt = np.array([utime + glon / 15.0 for utime in utsec]) - - # Adjust to ensure that 0.0 <= lt < 24.0 - while np.any(lt < 0.0): - lt[lt < 0.0] += 24.0 - - while np.any(lt >= 24.0): - lt[lt >= 24.0] -= 24.0 - - return lt - - -def lt_to_ut(lt, glon): - """Compute universal time in hours from local time and longitude. - - Parameters - ---------- - lt : float or array-like - Local time(s) in hours - glon : float or array-like - Geographic longitude(s) in degrees. - - Returns - ------- - uth : float - Universal time in hours - - Notes - ----- - If both `lt` and `glon` are array-like, they must be broadcastable. - - """ - - uth = np.asarray(lt) - np.asarray(glon) / 15.0 - - # Ensure the values range from 0-24 h - uth[uth <= 0] += 24.0 - uth[uth >= 24.0] -= 24.0 - - return uth - - -def calc_time_shift(utime): - """Calculate the time shift needed to orient a polar dial. - - Parameters - ---------- - utime : dt.datetime - Datetime object - - Returns - ------- - shift : float - Time shift in degrees - - """ - - uth = utime.hour + (utime.minute + ( - utime.second + utime.microsecond * 1.0e-6) / 60.0) / 60.0 - shift = uth * 15.0 - - return shift +#!/usr/bin/env python +# Copyright 2020, the Aether Development Team (see doc/dev_team.md for members) +# Full license can be found in License.md +"""Routines to perform temporal calculations.""" + +import datetime as dt +import numpy as np + + +def epoch_to_datetime(epoch_time): + """Convert from epoch seconds to datetime. + + Parameters + ---------- + epoch_time : int + Seconds since 1 Jan 1965 + + Returns + ------- + dtime : dt.datetime + Datetime object corresponding to `epoch_time` + + Notes + ----- + Epoch starts at 1 Jan 1965. + + """ + + dtime = dt.datetime(1965, 1, 1) + dt.timedelta(seconds=epoch_time) + + return dtime + + +def datetime_to_epoch(dtime): + """Convert datetime to epoch seconds. + + Parameters + ---------- + dtime : dt.datetime or dt.date + Datetime object + + Returns + ------- + epoch_time : float + Seconds since 1 Jan 1965 + + """ + + epoch_time = (dtime - dt.datetime(1965, 1, 1)).total_seconds() + + return epoch_time + + +def ut_to_lt(time_array, glon): + """Compute local time from date and longitude. + + Parameters + ---------- + time_array : array-like + Array-like of datetime objects in universal time + glon : array-like or float + Float or array-like of floats containing geographic longitude in + degrees. If single value or array of a different shape, all longitudes + are applied to all times. If the shape is the same as `time_array`, + the values are paired in the SLT calculation. + + Returns + ------- + lt : array of floats + List of local times in hours + + Raises + ------ + TypeError + For badly formatted input + + """ + + time_array = np.asarray(time_array) + glon = np.asarray(glon) + + # Get UT seconds of day + utsec = [(ut.hour * 3600.0 + ut.minute * 60.0 + ut.second + + ut.microsecond * 1.0e-6) / 3600.0 for ut in time_array] + + # Determine if the calculation is paired or broadcasted + if glon.shape == time_array.shape: + lt = np.array([utime + glon[i] / 15.0 for i, utime in enumerate(utsec)]) + else: + lt = np.array([utime + glon / 15.0 for utime in utsec]) + + # Adjust to ensure that 0.0 <= lt < 24.0 + while np.any(lt < 0.0): + lt[lt < 0.0] += 24.0 + + while np.any(lt >= 24.0): + lt[lt >= 24.0] -= 24.0 + + return lt + + +def lt_to_ut(lt, glon): + """Compute universal time in hours from local time and longitude. + + Parameters + ---------- + lt : float or array-like + Local time(s) in hours + glon : float or array-like + Geographic longitude(s) in degrees. + + Returns + ------- + uth : float + Universal time in hours + + Notes + ----- + If both `lt` and `glon` are array-like, they must be broadcastable. + + """ + + uth = np.asarray(lt) - np.asarray(glon) / 15.0 + + # Ensure the values range from 0-24 h + uth[uth <= 0] += 24.0 + uth[uth >= 24.0] -= 24.0 + + return uth + + +def calc_time_shift(utime): + """Calculate the time shift needed to orient a polar dial. + + Parameters + ---------- + utime : dt.datetime + Datetime object + + Returns + ------- + shift : float + Time shift in degrees + + """ + + uth = utime.hour + (utime.minute + ( + utime.second + utime.microsecond * 1.0e-6) / 60.0) / 60.0 + shift = uth * 15.0 + + return shift diff --git a/aetherpy/version.txt b/aetherpy/version.txt index 6f72174..84c56c7 100644 --- a/aetherpy/version.txt +++ b/aetherpy/version.txt @@ -1 +1 @@ -0.0.1a +0.0.1a diff --git a/requirements.txt b/requirements.txt index f1d25ee..2aa7161 100644 --- a/requirements.txt +++ b/requirements.txt @@ -1,3 +1,3 @@ -matplotlib -netCDF4 -numpy +matplotlib +netCDF4 +numpy diff --git a/setup.cfg b/setup.cfg index 3d912ce..62e106f 100644 --- a/setup.cfg +++ b/setup.cfg @@ -1,54 +1,54 @@ -[metadata] -name = aetherpy -version = file: aetherpy/version.txt -url = https://github.com/AetherModel/aetherpy -author = aetherpy development team -author_email = aether-core@umich.edu -description = 'Python support for the Aether model' -keywords = - Aether - AetherModel - ionosphere - thermosphere - space-weather - space-physics - forecasting - modelling - modeling -classifiers = - Development Status :: 3 - Alpha - Topic :: Scientific/Engineering :: Physics - Topic :: Scientific/Engineering :: Atmospheric Science - Intended Audience :: Science/Research - License :: OSI Approved :: BSD License - Natural Language :: English - Programming Language :: Python :: 3.7 - Programming Language :: Python :: 3.8 - Programming Language :: Python :: 3.9 - Programming Language :: Python :: 3.10 - Operating System :: MacOS :: MacOS X - Operating System :: POSIX :: Linux -license_file = LICENSE -long_description = file: README.md -long_description_content_type = text/markdown - -[options] -python_requires = >= 3.7 -setup_requires = setuptools >= 38.6; pip >= 10 -include_package_data = True -zip_safe = False -packages = find: -install_requires = matplotlib - netCDF4 - numpy - -[flake8] -max-line-length = 80 -ignore = - D200 - D202 - W503 - aetherpy/__init__.py F401 E402 - aetherpy/io/__init__.py F401 - aetherpy/plot/__init__.py F401 - aetherpy/utils/__init__.py F401 +[metadata] +name = aetherpy +version = file: aetherpy/version.txt +url = https://github.com/AetherModel/aetherpy +author = aetherpy development team +author_email = aether-core@umich.edu +description = 'Python support for the Aether model' +keywords = + Aether + AetherModel + ionosphere + thermosphere + space-weather + space-physics + forecasting + modelling + modeling +classifiers = + Development Status :: 3 - Alpha + Topic :: Scientific/Engineering :: Physics + Topic :: Scientific/Engineering :: Atmospheric Science + Intended Audience :: Science/Research + License :: OSI Approved :: BSD License + Natural Language :: English + Programming Language :: Python :: 3.7 + Programming Language :: Python :: 3.8 + Programming Language :: Python :: 3.9 + Programming Language :: Python :: 3.10 + Operating System :: MacOS :: MacOS X + Operating System :: POSIX :: Linux +license_file = LICENSE +long_description = file: README.md +long_description_content_type = text/markdown + +[options] +python_requires = >= 3.7 +setup_requires = setuptools >= 38.6; pip >= 10 +include_package_data = True +zip_safe = False +packages = find: +install_requires = matplotlib + netCDF4 + numpy + +[flake8] +max-line-length = 80 +ignore = + D200 + D202 + W503 + aetherpy/__init__.py F401 E402 + aetherpy/io/__init__.py F401 + aetherpy/plot/__init__.py F401 + aetherpy/utils/__init__.py F401 diff --git a/setup.py b/setup.py index 7f5d4c7..b98d874 100644 --- a/setup.py +++ b/setup.py @@ -1,12 +1,12 @@ -#!/usr/bin/env python -# -*- coding: utf-8 -*- -# Copyright 2020, the Aether Development Team (see doc/dev_team.md for members) -# Full license can be found in LICENSE -# ----------------------------------------------------------------------------- - -from setuptools import setup - - -# Run setup. Setuptools will look for parameters in [metadata] section of -# setup.cfg -setup() +#!/usr/bin/env python +# -*- coding: utf-8 -*- +# Copyright 2020, the Aether Development Team (see doc/dev_team.md for members) +# Full license can be found in LICENSE +# ----------------------------------------------------------------------------- + +from setuptools import setup + + +# Run setup. Setuptools will look for parameters in [metadata] section of +# setup.cfg +setup() diff --git a/test_requirements.txt b/test_requirements.txt index 16add8f..4c08c8f 100644 --- a/test_requirements.txt +++ b/test_requirements.txt @@ -1,7 +1,7 @@ -coveralls -flake8 -numpydoc -pytest -pytest-cov -sphinx -sphinx_rtd_theme +coveralls +flake8 +numpydoc +pytest +pytest-cov +sphinx +sphinx_rtd_theme