pip installable version of validator and scorer?

[AngledLuffa]

Hey, I'd be up for making a pip installable package from the version of udtools in the latest UD 2.17. Any objections?

Any suitable name? udtools and ud2tools are both available

[dan-zeman]

OK, I will look into it.

[AngledLuffa]

I'd be happy to do it. Just takes a goahead (and a name)

[harisont]

Regarding the validator, just a reminder that #132 is (already) pip-installable, just not complete yet. This term has been brutally busy for me so far (not the least with the UD 2.17 release!), but I'm positive that, if we tried to collaborate a bit, progress would be significantly faster.

[dan-zeman]

I'd be happy to do it. Just takes a goahead (and a name)

It is not needed, thanks. I was planning to do it anyway, and to use the opportunity and learn the process. The name is udtools and the validator is already there (the scorer will come later), although it is not very stable yet. Some ideas from #132 have been implemented already.

[AngledLuffa]

Thank you, that is excellent!

Regarding the scorer, again, lmk if my help would be useful there.

[dan-zeman]

I have added eval.py to udtools 0.2.1. In contrast to the validator, I am less familiar with this code and feel slightly more hesitant to change it; but in the long run, I think I will want to look at it more closely and see how the two tools and their interfaces can be made more similar.

[martinpopel]

Thanks. I am one of the original authors, so I can help with this. Just let me know what should be changed.
BTW: I had implemented the evaluation script in Udapi, independently on the Milan Straka's implementation, and we compared the differences in outputs and thus fixed many edge cases in both implementations. My implementation using Udapi was much shorter, but we didn't want to have any dependencies at that time. Now, when udtools depends on udapi anyway, it may not be such an issue. It could be easily refactored from a block to a script (and perhaps joining all sentences into one instead of the util.ResegmentGold solution). That said, my implementation is missing the post-CoNLL18 edits (metrics ELAS and EULAS etc)

[AngledLuffa]

Thank you! This is excellent.

Since eval is a keyword, is there room to change that to something else, like ud_eval?

[AngledLuffa]

Also: what would you think about adding it to conda? There are some tools which automatically migrate pip to conda, so it wouldn't create extra work per release.

[AngledLuffa]

also: to what extent does it need to be 3.11 or higher? i wonder if 3.9 would still be enough, for example

[dan-zeman]

also: to what extent does it need to be 3.11 or higher? i wonder if 3.9 would still be enough, for example

3.11 is somewhat arbitrary. It seemed safe to indicate the Python version with which I am actually running the code myself. It is likely that it works with older versions but I don't know how old.

[AngledLuffa]

it would save me a non-zero amount of time if we YOLOd version 3.10, which is still considered supported, at least until proven otherwise