Create dagger function to run CLI tests

[adityachopra29]

Summary

• Creates dagger function to run tests using a dagger service running harbor
• Fixes Make tests not Dependent on Demo Server #249

Changes

• Creates harbor-test dagger function which runs tests after starting a service running harbor
• Current tests supported are only login_tests (since currently only call the demo server)
• Creates documenation for the new command

Implementation details

• Uses the same config for the harbor service as used in harbor-satellite see here
• Current harbor service starts the following component services :
  • Core (since it contains the main API)
  • Postgres (to strore container and credentials etc)
  • Redis server (to store sessions details and caches)
• Modifies the tests to initially call the container endpoints instead of the demo server

Testing

• Currently the postgres and redis images used in running the harbor service are not uploaded to harbor-registry . (They are the same ones used by harbor-satellite). I have raised issues regarding them in harbor-satellite repo as well see here and here.
• Due to this complete testing was not done. However, individual unit tests did not break with the new changes

Usage

dagger call harbor-test

[codecov]

Codecov Report

❌ Patch coverage is 0% with 12 lines in your changes missing coverage. Please review.
✅ Project coverage is 6.44%. Comparing base (60ad0bd) to head (a33d82e).
⚠️ Report is 81 commits behind head on main.

Files with missing lines	Patch %	Lines
test/helper/helpers.go	0.00%	12 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff            @@
##             main    #585      +/-   ##
=========================================
- Coverage   10.99%   6.44%   -4.55%     
=========================================
  Files         173     260      +87     
  Lines        8671   12854    +4183     
=========================================
- Hits          953     828     -125     
- Misses       7612   11920    +4308     
  Partials      106     106              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[adityachopra29]

Hi @bupd Please take a look at this pr.
I was not able to test the implementation completely, since the images for the postgres and redis images used as services to run harbor are not uploaded to the registry. This is also causing the dagger call test-end-to-end function of harbor-satellite repo as well. (I have added the corresponding links in the main pr itself) to fail as well.
So if you can guide me how I can add those images there or they can be added by some other means, then I will be able to test the dagger function properly
If anything else is required please let me know
Rest everything is complete from my side

[adityachopra29]

@bupd any update on this pr's review?

[adityachopra29]

Hi @bupd @Vad1mo . Kindly take a look at this PR. Thanks!

[bupd]

@adityachopra29 Thanks for your contribution,

given that multiple iterations have been done on running harbor inside its always been not a great outcome in terms setup speed and config options.

So, I believe this would be really simple if we just run harbor on docker compose / podman compose

https://github.com/bupd/harbor-podman - check this repo this would much simple and really fast to spin up harbor in ci.

we can setup harbor on ci using the above repo and update the tests to use that harbor instead of demo.goharbor.io

[adityachopra29]

@bupd
Okay, will do

[adityachopra29]

@bupd , the repository https://github.com/bupd/harbor-podman provides a Podman-based setup for running Harbor on Arch Linux, and several steps in the guide are specific to SELinux-enabled distributions. However, most of our CI workflows (for example, the default.yaml) run on ubuntu-latest.

I am able to adapt the setup to work specifically on Ubuntu. Given this, should we run the tests on Arch Linux to match the Podman setup in the repo, or should we instead use the Ubuntu-specific steps and continue running the tests on ubuntu-latest?

[adityachopra29]

Also, the repo uses podman-compose instead of Docker Compose. Given that Docker Compose is already available by default and significantly easier to set up and manage, should we prefer Docker Compose here rather than introducing podman-compose?

[adityachopra29]

Hi @bupd

the repository https://github.com/bupd/harbor-podman provides a Podman-based setup for running Harbor on Arch Linux, and several steps in the guide are specific to SELinux-enabled distributions. However, most of our CI workflows (for example, the default.yaml) run on ubuntu-latest.

I tried the first approach: ie used a privileged Fedora image(since it is SELinux based), to and run the repo https://github.com/bupd/harbor-podman inside it). However, the podman container required cgroups controllers to be available, which were not delegated properly by github actions runners. (See CI workflow
This made me come to the conclusion that this approach wont work.)

So I have adapted the installation steps for AppArmor based Linux(ubuntu), and run the tests on it, and it is working now. PTAL.

[adityachopra29]

Closing this PR in favour of #648