latest Julia version + Onnx reader

[phK3]

The ONNX parser using NaiveNASflux and ONNXNaiveNASflux prevented updating to the latest Julia version.
So I deleted those packages and replaced them with an ONNX reader I implemented in my colleagues package VNNLib.jl
(see github repo here: https://github.com/samysweb/VNNLib.jl/tree/extension).
The new ONNX reader is currently still on the extension branch of VNNLib.jl, but should be on the main branch soon.

An example showing the necessary changes for running the introductory example in the Readme.md using the new ONNX reader from VNNLib.jl is implemented in develop/onnx_parser_adaptations.jl in my fork of ModelVerification.jl.

The modifications I've already done for the version update and a general description of the steps necessary to move to the new ONNX reader are described in the changes.md of my fork.

A main difference would be that the propagate_layer_batch() method would have to dispatch on the subtype of VNNLib.OnnxParser.Node instead of directly dispatching on the Flux operation.
But this is necessary, as many ONNX operations don't directly map to Flux (e.g. Gemm with transposed inputs).

How does this align with your plans for the future of ModelVerification.jl?

Feel free to suggest any requirements, that you need to be met.

All the best,
Philipp

[Wei-TianHao]

Hi Phillip! Thanks for your contribution! Could you revise the CI (continuous integration) file to remove the following lines? The current PR could not pass the continuous integration test because of the file changes.

          julia --project=. -e 'using Pkg; Pkg.develop(path="onnx_parser/NaiveNASflux")'
          julia --project=. -e 'using Pkg; Pkg.develop(path="onnx_parser/ONNXNaiveNASflux")'

https://github.com/phK3/ModelVerification.jl/blob/783e2cabd657eee2188c13b386c5d90c34fb14dd/.github/workflows/CI.yml#L50

My orignal plan was to wait until https://github.com/FluxML/ONNX.jl is ready to use because it is native to Flux. But it takes much longer than I expected. What's your future plan to maintain VNNlib.jl? It seems to be a better choice than ONNXNaiveNASflux because we don't really need the NAS part, but only a ONNX parser. If VNNlib.jl will be long term supported, it may be a better choice for MV.jl.

[phK3]

I deleted the lines you indicated from the CI.yml.

Note that most tests will probably fail. For the verifiers to work, one would have to implement the changes outlined in develop/onnx_parser_adaptations.jl.

My colleague (Samuel) and me plan to maintain VNNLib.jl for as long as we are working in research.

We also have some NN verifiers of our own written in Julia (e.g. VeryDiff, NNPoly) that we want to migrate to VNNLib.jl - so we also have some intrinsic motivation to maintain the repo.

When ONNX.jl gets to a state where it is usable and has wider support among the Julia community, it is probably the best choice, but I don't know when that will be the case.

[phK3]

VNNLib.jl currently uses Flux v0.16.4 which requires Julia 1.10.

I bumped the version in the CI.yml to 1.10, 1.11 and pre.

Installing the package should work in that case.

[phK3]

It looks like 'pre' is not a valid version specifier for julia-actions/setup-julia@v1, I have another project that uses

- uses: julia-actions/setup-julia@v2

where 'pre' seems to work.

Do you think upgrading the julia-actions/setup-julia version is safe? I don't know how to test that locally.

Otherwise, we could just not test for the latest prerelease and delete the 'pre' from CI.yml.

Let me know what you think is the best option.

[Wei-TianHao]

Hi Philipp,
Thanks for the update!

Note that most tests will probably fail. For the verifiers to work, one would have to implement the changes outlined in develop/onnx_parser_adaptations.jl.

Do you mean the change may break the current pipeline? Is there a particular reason that you didn't directly replace the original verify function? If my understanding is correct, the original onnx handling pipeline will be broken anyway after replacing the ONNXNaiveNASflux with you package.

Upgrading the julia-actions/setup-julia version should be safe. To test it, I would suggest setup the github workflow on your forked branch. It's free and should be quite intuitive.

Thanks!