[READY] Flint cicd

[JunkyDeveloper]

my cicd version

[Joniii11]

yeah what will you do @wouf??? you don't know who i am

[JunkyDeveloper]

Let's try it @notwouf

[Joniii11]

ye @notwouf

[bensaru]

this will work? seems like there is a bug in at 32 line.

[JunkyDeveloper]

this will work? seems like there is a bug in at 32 line.

Yes as you can see in the job:

0 passed, 23 failed
Summary saved to log/flint_summary.json
test adapter::tests::test_run_all_flint_benchmarks ... FAILED

So the tests will be found, I personally use this already, to test with a little different setup, but this needs to be discussed then should this pipeline fail and then should it pass.
So the question is what is your use case? So I can help you to write a better job.
This is my job which I wrote for my branch: https://github.com/JunkyDeveloper/SteelMC/actions/runs/21965116144/job/63452789655

[coco875]

hmm it would be better to have the job fail only if a test who pass previously no longer pass, only for that case

[JunkyDeveloper]

I thought also of that, which is possible, because of the json export of all tests, so it maybe I the job can access that artifact from the master to check the current json with json from master and compare (I think a small flint tool should do that job, that means this tool can be also used for all cicds).

But didn't had much time, and I think this is also currently a bit on ice, because 4lve want's to have it on flint side there this isn't really possible for all PRs. maybe next week I find time for it

[JunkyDeveloper]

This should be now a working ci with flint-steel. It will fail then a test fails which was run successful in the past.
New or already failing tests, will have no influence. So I think at the end that's a good solution. for also the block implementations. so you can see if something breaks a block impl in a PR

[coco875]

It publish an artifact on test fail or succeed ?

[JunkyDeveloper]

Yes a full test result json + a baseline there all successful tests are added and that will be used to compare