[Feature Request] goroutines for k8s-bench task execution

[zvdy]

Feature

Implement a more efficient way to execute and run tasks that is not sequential, relying in goroutines when a TasksDir is set.

1. Improve the process of benchmarking LLMs and their respective models in a quicker manner.
2. Implement this in the future in CI/CD before main merging, adding a few github action secrets and spinning up a mock cluster should do the job as far as I know

Hope this makes sense.

Bottlenecks

One thing worth noting is that the "simple" solution which is already implemented executes tasks sequentially, which will take quite some time depending on the hardware and the amount of tasks the folder contains, also based on #145 , I agree It's missing some common troubleshooting/issues scenarios people happen to have while working with k8s that are worth benchmarking against. this will make the time increase even more, currently:

Total Time ~= sum(tasks)

The solution for the "bottleneck" of executing all tasks sequentially would be using goroutines and channels to increase the throughput, where:

Total Time ~= longest individual task time + overhead

I also wondered if an hybrid approach would be best but I leave that open to your recommendations.

Let me know if an hybrid approach allowing configurable concurrency, even down to sequential execution or a fully goroutine approach would be best, I would like to work on implementing this.