-
Notifications
You must be signed in to change notification settings - Fork 0
Description
Describe the bug
This was found while debugging #6 because there was another bug where the split mode would not be cleared after a context copy. If the pipeline is set to split after a step completes, and the split key is not a list, a subset of the remaining contexts to be run will be deleted. This is an issue with downstream implementation, but it would be better if the developer was warned that they are trying to split on an invalid data type.
To Reproduce
Steps to reproduce the behavior:
- Build a pipeline (SA or HA)
- Create a step that sets a context value to a primitive value (e.g.
1). - Set the context to split on step completion.
- Note that subsequent steps will not run the correct number of times relative to previous splits/filters.
Expected behavior
Pipeline should throw an error when attempting to split on a key that is not a list, and this should halt the pipeline completely as this is most likely a code error and not a data error.
Watergrid specific
Watergrid version/commit: 77e5755
Python version: 3.10
Redis version (if using HAPipeline): N/A
Additional context
N/A