Allow the compilation of each spec file separately

[findinpath]

I was eager to try out the latest developments in dtspec 0.7.1 and I created a small spike project :

https://github.com/findinpath/dbt_infer_deletions_in_batched_staging_data/tree/spike/dtspec-0.7.1

I've used https://github.com/gnilrets/dbt-container-skeleton as an initial source of inspiration. That's actually a very good proof of concept project.

I got inspired from
https://github.com/inside-track/dtspec/blob/main/dtspec/cli.py#L313

and made a few modification in my small project so that I can be able to execute the dtspec tests via pytest.
The reasoning behind this choice (of not using dtspec test-dbt) is that the output of the dtspec tests can be exported as a JUnit file which is helpful for CI builds. Also with Snowflake database there are a few things that need adjustments (e.g. writing/reading VARIANT fields).

It is definitely very beneficial to make use of template values like:

- {{ TODAY }}
- {{ TOMORROW }}
- {{ dbt_source('dbt_shop', 'raw_products') }}

There appears though to exist an issue with the current dtspec version when working with
multiple spec files. By default, they all get compiled to one huge compiled_spec.yml
which can lead to clashes between the identifier / source /target definitions.

In case of my project I had to use only one spec file (before I had two spec files - see master branch) to get it running without clashes.

Also in case of not defining a global factory the "compilation" of the spec fails
because by default it is expected to have at least one global factory (minItems = 1).

It would be great to allow the compilation of each spec with dbt separately and if possible in memory (and not in the explicit file compiled_spec.yml)

Even though dtspec test-dbt is great, it would be even greater to easily allow the integration of dtspec in various other testing scenarios.

From a client point of view, it would be also very beneficial to allow easily the inclusion of multiple python functions like the ones already present by default in dtspec:

https://github.com/inside-track/dtspec/blob/main/dtspec/specs.py#L96

Can you maybe allow passing a set of functions/lambdas to the method exposed of compiling a spec?

[gnilrets]

Thanks for noticing that it fails if there is no global factory. That should be fixed.

I was thinking about adding a tag property to scenarios and cases and allow the user to only compile specs based on matching those tags. Might be more manageable than using regex matching on the names when you want to run distinct sets of scenarios.

I like the idea of being able to pass your own lambdas to the spec parser.

PRs welcome!