Fix unit test warnings and compatibility issues (Python 3.12, Pandas, SQLAlchemy)#3722
Open
jkppr wants to merge 8 commits intogoogle:masterfrom
Open
Fix unit test warnings and compatibility issues (Python 3.12, Pandas, SQLAlchemy)#3722jkppr wants to merge 8 commits intogoogle:masterfrom
jkppr wants to merge 8 commits intogoogle:masterfrom
Conversation
- Upgraded `pytest` to 9.0.2 in `test_requirements.txt` to resolve compatibility warnings with Python 3.12 (`TestResult has no addDuration method`). - Fixed `SAWarning` in `timesketch/lib/testlib.py` by removing redundant engine recreation in `ModelBaseTest`, which conflicted with the existing session. - Moved `get_yaml_config` and `get_config_path` from `timesketch/lib/analyzers/interface.py` to `timesketch/lib/utils.py` to resolve circular import dependencies between `interface`, `ontology`, and `api/v1/utils`. - Updated imports in `timesketch/lib/analyzers/interface.py` and `timesketch/lib/ontology.py`. - Fixed `FutureWarning` in `timesketch/lib/analyzers/authentication/utils.py` by removing `inplace=True` from `fillna` and `sort_values` calls on Pandas DataFrames. - Fixed `DeprecationWarning` in `timesketch/lib/llms/features/nl2q.py` by using `include_groups=False` in `groupby().apply()` and updating the callback to access the group name correctly. Co-authored-by: jkppr <99879757+jkppr@users.noreply.github.com>
- Upgraded `pytest` to 9.0.2 in `test_requirements.txt` to fix compatibility with Python 3.12 (resolves `TestResult has no addDuration method`). - Upgraded `networkx` to 3.4.2 in `requirements.txt` to fix `datetime` deprecation warnings on Python 3.12. - Fixed `SAWarning` in `timesketch/lib/testlib.py` by removing redundant engine recreation in `ModelBaseTest`, which conflicted with the session from `BaseTest`. - Moved `get_yaml_config` and `get_config_path` from `timesketch/lib/analyzers/interface.py` to `timesketch/lib/utils.py` to resolve circular import dependencies between `interface` and `ontology`. - Updated `timesketch/lib/ontology.py` and `timesketch/lib/analyzers/interface.py` to import config functions from `timesketch.lib.utils`. - Fixed `FutureWarning` in `timesketch/lib/analyzers/authentication/utils.py` by removing `inplace=True` from `fillna` and `sort_values` calls on Pandas DataFrames. - Fixed `DeprecationWarning` in `timesketch/lib/llms/features/nl2q.py` by using `include_groups=False` in `groupby().apply()` and updating the callback to access the group name correctly. Co-authored-by: jkppr <99879757+jkppr@users.noreply.github.com>
- Upgraded `pytest` to 9.0.2 in `test_requirements.txt` to fix compatibility with Python 3.12 (resolves `TestResult has no addDuration method`). - Upgraded `networkx` to 3.4.2 in `requirements.txt` to fix `datetime` deprecation warnings on Python 3.12. - Fixed `SAWarning` in `timesketch/lib/testlib.py` by removing redundant engine recreation in `ModelBaseTest`, which conflicted with the session from `BaseTest`. - Moved `get_yaml_config` and `get_config_path` from `timesketch/lib/analyzers/interface.py` to `timesketch/lib/utils.py` to resolve circular import dependencies between `interface` and `ontology`. - Updated `timesketch/lib/ontology.py` and `timesketch/lib/analyzers/interface.py` to import config functions from `timesketch.lib.utils`. - Fixed `FutureWarning` in `timesketch/lib/analyzers/authentication/utils.py` by removing `inplace=True` from `fillna` and `sort_values` calls on Pandas DataFrames. - Fixed `DeprecationWarning` in `timesketch/lib/llms/features/nl2q.py` by using `include_groups=False` in `groupby().apply()` and updating the callback to access the group name correctly. Co-authored-by: jkppr <99879757+jkppr@users.noreply.github.com>
- Upgraded `pytest` to 9.0.2 in `test_requirements.txt` to fix compatibility with Python 3.12 (resolves `TestResult has no addDuration method`). - Upgraded `networkx` to 3.4.2 in `requirements.txt` to fix `datetime` deprecation warnings on Python 3.12. - Fixed `SAWarning` in `timesketch/lib/testlib.py` by removing redundant engine recreation in `ModelBaseTest`, which conflicted with the session from `BaseTest`. - Moved `get_yaml_config` and `get_config_path` from `timesketch/lib/analyzers/interface.py` to `timesketch/lib/utils.py` to resolve circular import dependencies between `interface` and `ontology`. - Updated `timesketch/lib/ontology.py` and `timesketch/lib/analyzers/interface.py` to import config functions from `timesketch.lib.utils`. - Fixed `FutureWarning` in `timesketch/lib/analyzers/authentication/utils.py` by removing `inplace=True` from `fillna` and `sort_values` calls on Pandas DataFrames. - Fixed `DeprecationWarning` in `timesketch/lib/llms/features/nl2q.py` by using `include_groups=False` in `groupby().apply()` and updating the callback to access the group name correctly. - Addressed code review feedback: removed unused imports in `interface.py` and fixed logger formatting in `utils.py`. Co-authored-by: jkppr <99879757+jkppr@users.noreply.github.com>
Collaborator
Author
|
/gemini review |
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Code Review
This pull request effectively addresses a range of compatibility issues and warnings, enhancing the project's robustness and maintainability. The dependency upgrades for pytest and networkx are necessary for Python 3.12 support. The refactoring of Pandas calls to remove inplace=True and update groupby().apply() usage aligns with modern library practices and resolves future warnings. Moving utility functions to lib/utils.py to break a circular dependency is a solid architectural improvement, and the opportunity was taken to fix a minor logging style issue. The fix in testlib.py to remove redundant SQLAlchemy engine creation correctly resolves a warning during test execution. Overall, these are all positive changes that improve code quality. I have no further comments.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR resolves multiple warnings and compatibility issues encountered during unit test execution, ensuring full compatibility with Python 3.12. It addresses deprecation warnings from core libraries (Pandas, NetworkX), fixes SQLAlchemy session management in tests, and resolves a circular import dependency.
Key Changes:
Test Dependencies:
pytestto9.0.2intest_requirements.txtto fixRuntimeWarning: TestResult has no addDurationmethod.networkxto3.4.2inrequirements.txtto resolveDeprecationWarning: datetime.datetime.utcfromtimestamp() on Python 3.12.Code Refactoring & Fixes:
timesketch/lib/testlib.pyby removing redundant engine recreation inModelBaseTest, ensuring correct session reuse fromBaseTest.get_yaml_configandget_config_pathfrom analyzers/interface.pytolib/utils.pyto break the dependency cycle betweeninterfaceandontology.authentication/utils.py, replacedinplace=Truewith explicit assignment forfillnaandsort_valuesto fixFutureWarning.llms/features/nl2q.py, updatedgroupby().apply()to useinclude_groups=False(Pandas 2.2+ requirement) and adjusted group name access.