add source tarballs for releases

[phanirithvij]

I like the monorepo structure and reasoning, and also know about the xz (jia tan incident) where release tarballs become slightly untrusted compared to source builds.

I feel like there is a middle ground where we can leverage the github workflows to provide immutable and attested release assets per project.

eg. for goupile, we can pack the vendor/ src/ etc. which only goupile needs to build, (if possible maybe felix can be decoupled into it's own release asset).

I am not sure how much savings can be made by doing this but this would greatly reduce the requirement the current monorepo tarball size of around 360MB.

If this is technically feasible and you are open to this, please let me know where we can communicate synchronously maybe for eg. on matrix so that I can get some clarity on how to achieve this. I am willing to contribute to add the release workflows.

As of now I am in the process of contributing to ngi-nix https://ngi.nixos.org/ and planning to add a nixos module and package goupile for Nixpkgs, you can track the progress here ngi-nix/projects#134

[Koromix]

Hi!

Project-specific source tarballs would be good, indeed. I'll look into it soon, at least for the "make a source tarball with only the files needed for a specific project" part.

Here is how I see this working:

• There would be a script for each project for which tarballs are distributed, probably something like src/goupile/dist/source/package.sh.
• These scripts will use felix and leverage the info from FelixBuild.ini to get a list of source files used by Goupile. That covers > 95% of the file list/pattern part.
• The sources for felix and the bootstrap script will be included in each source tarball. It's easier and I make occasional breaking changes in felix, which is easy right now because I can change FelixBuild.ini in the same commit.

For the part about GitHub Actions, I'd prefer Codeberg/Forgejo CI/CD "stuff" (which I don't know anything about, just like GitHub Actions in fact), if possible. The same repository lives here: https://codeberg.org/Koromix/rygel/. It only lives here to keep some kind of network effect, but I don't want to become locked to GitHub-specific stuff.

It would also have to be either manually triggered (I really don't like stuff that runs for each commit), or automatically when a release tag (goupile/* for Goupile, rekkord/* for Rekkord, etc...) appears.

[phanirithvij]

I can read up on how to setup ci/cd for release tags on codeberg/forgejo, so far I have only had some experience with github actions for this, but I understand where you're coming from.

Agreed with including felix in each source tarball. I assumed felix which exists in the tree at the time of tagging some subproject, will be the best candidate to build the release over the last tagged felix version.

If you think you could do it soon, and see how useful it is by looking at source tarball sizes for all projects mainly.

on an unrelated note: I saw some references to Goupiel in the codebase, wanted to mention it now incase its a mistake. https://github.com/search?q=repo%3AKoromix%2Frygel%20Goupiel&type=code

[Koromix]

The script now exists and the source tarballs seem to be able to build Goupile correctly. Clone/pull the repository and run src/goupile/dist/source/source.sh. It will generate the source packages in bin/Packages.

I've changed my mind about GitHub Actions. As long as the CI/CD script remains simple enough (to avoid excessive vendor lock-in), let's use GitHub stuff. Might as well incur the costs on Microsoft instead of Codeberg.

[phanirithvij]

Thanks for the very quick implementation. I will test it out and then work on adding the release github action.