Rethink Workflow options

[josephjclark]

this is a kinda big project, needs to be done with time and patience

Current a workflow.json serializes like this:

{ workflow, options }

Where 99% of the time options is just the start node

It's kind of an annoying data structure, and annoying to hand write. So workflow.yaml shuld just contain a workflow and nothing else.

The seralized workflow structure, in the v2 project yaml and and the serialized workflow.yaml, should have an options key.

Considerations here:

• For back compat, we need to support the { workflow, options } format and convert it at ingress time in the CLI
• Is start an option, or core to a workflow? When I pull from lightning, the trigger node is always the start, that's not an option really. It's how the workflow is designed. That's different to a CLII start option, where the workflow starts at trigger but I want to skip that and just run from step x. That's probably not even an option we want to serialize?
• So I think start is not an option

The other way of looking at this is:

• Yes, drop the { workflow, options } wrapper
• when converting a lightning project, make the trigger the first step, so that it is naturally executed first
• Support an optional start key on a workflow to set a custom start node. Not likely to be useful but cheap to support
• Don't serialize workflow options at all? Just ignore them. Reintroduce them only when needed.

[josephjclark]

The runtime depends on an Execution Plan, with { workflow, options }

I intend to keep that structure for now. This is just about the Project and CLI handling of options.