Conversation
metaflow/decorators.py
Outdated
| flow, graph, environment, flow_datastore, metadata, logger, echo, opts | ||
| ) | ||
| for rd in deco: | ||
| opts = {option: deco_options[option] for option in rd.options} |
There was a problem hiding this comment.
This should be hoisted out of the inner for loop because it makes it look like there are different options for each rd which is not the case (they are shared for a given type of decorator).
There was a problem hiding this comment.
Won't the decorator options be different per decorator instance. For example :
@airflow_sensor_a(myopt1="DEF")
@airflow_sensor_a(myopt1="ABC")
class MyFlow(FlowSpec):There was a problem hiding this comment.
Yes, but those are attributes and not options (I got confused too). The attributes are things extracted from the decorator itself and the optiosn are extracted from the command line (so for project, for example, you have name as an attribute and production and branch as options).
To remove this confusion, when we were talking with @savingoyal today, we thought that maybe we could assert that if allow_multiple is true, the options dict is empty. An even better way of doing it is ensuring this at runtime too but it's a bit more complicated (ie: ensuring that it can't be filled in later too). This will prevent some weird things and ppl getting surprised.
There was a problem hiding this comment.
Refactored it. Ready for review.
82f7640 to
f8337ed
Compare
romain-intel
left a comment
There was a problem hiding this comment.
LGTM. Gimme until monday to make my changes and test it out but should be good.
080513c to
24854af
Compare
24854af to
75c690b
Compare
PR Stack Merge Order (#1252 --> #1234 --> #1226 --> #1225 --> #1224)