Fix CI

[MrAnno]

This PR fixes Cirrus CI build and test jobs on macOS, Windows, and FreeBSD.

I've also had a quick look at GitHub Actions, it seems to be a stable/native/fast CI alternative, which works really well with GitHub.
Unfortunately, FreeBSD is not officially available there.

Depends on Snaipe/BoxFort#31
Depends on Snaipe/BoxFort#32

Resolves the linking issue in #391
Fixes #377

[MrAnno]

The macOS job is failing because certain test binaries have no permissions to call mprotect(PROT_READ | PROT_WRITE | PROT_EXEC) for injecting Boxfort's trampoline.

[MrAnno]

Yep, it only works when libcriterion is dynamically linked to the test.
I hope it's not something similar to "Arbitrary Code Guard".

Update:
Unfortunately, that's a protection in macOS (maxSegProtection).
https://stackoverflow.com/questions/60654834/using-mprotect-to-make-text-segment-writable-on-macos

I was able to bypass this using vm_protect() with VM_PROT_COPY.

[MrAnno]

cram adds an environment variable called TESTSHELL to each test.
Unfortunately, the value is encoded with fsencode, which produces a bytes object on Windows, but Python on Windows does not support non-string environment:

'TESTSHELL': b'C:\\Program Files\\Git\\usr\\bin\\sh.EXE'

File "C:\Python\lib\subprocess.py", line 1435, in _execute_child
    hp, ht, pid, tid = _winapi.CreateProcess(executable, args,
TypeError: environment can only contain strings

I'm afraid I can not fix this without modifying the source of cram.

https://github.com/brodie/cram/blob/59c164dfa6cbe4845aad2c958e77695073d5e802/cram/_main.py#L22
https://github.com/brodie/cram/blob/59c164dfa6cbe4845aad2c958e77695073d5e802/cram/_test.py#L110

See also: aiiie/cram#24

[Snaipe]

I'm not opposed in keeping a couple of patches in the ci/ directory and applying them during the windows build. In fact, this used to be the case for some python2/3 compatibility changes on cram some time ago.

[MrAnno]

Okay, I will give it a try. Thank you, @Snaipe.

[MrAnno]

Yay, it works!

With Snaipe/BoxFort#32, all tests passed.

[kasperk81]

cram is abandadoned for 5+ years now https://github.com/brodie/cram (last commit from 2016).
maybe switch to exactly https://github.com/emilkarlen/exactly#id1 @emilkarlen can probably comment how it compares with cram.

[emilkarlen]

Hi,
thanks for interest in exactly! I will take a closer look at what your tests look like soon - tonight or tomorrow (and also at cram, which is new to me).

[Snaipe]

We can open a separate issue for evaluating what it'd take to switch from cram; for now, this overall looks good to me.

[MrAnno]

@Snaipe Thank you for reviewing my PR.

I've updated the BoxFort subproject revision after you merged Snaipe/BoxFort#32.

[emilkarlen]

I am looking at Exactly vs Cram.
It took some time to get Criterion building.

I've started to implement the cram tests using Exactly: https://github.com/emilkarlen/Criterion-exactly

[emilkarlen]

I have added Exactly counterparts of the Cram tests:
emilkarlen@8778dca

if is of interest.

[Snaipe]

@MrAnno Thanks for the changes, LGTM.

@emilkarlen Would you mind opening a PR with the changes on your fork? This will be a better place for discussing anything related to the topic of moving away from Cram.