Speed up Swift Build and Foreign Binding Tests, it's taking more than 10mins just to build tests

[thomas-waite]

Building these tests is taking more than 10mins and really slowing PR CI.

Some options:

1. Cache builds if nothing's changed
2. Use a more performant runner