lmp/jobserv: prepare for kirkstone/scarthgap/main builds #371

quaresmajose · 2025-01-15T14:31:54Z

Our main release CI job called build-release is tracking the lmp-manifest/main branch which is already based on kirksotne branches for all the layers. So it is time to rename that job to build-kirkstone and also start tracking the lmp-manifest/kirksotne branch that we already have.

This will also remove the old build-release-stable job which has not been used for some time now and starts to be problematic because it tracks the kirkstone branch that we didn't have but we recently created.

After this PR we will have one job used for develpment:

build-main

And more two for the corresponding LTS branches:

build-kirkstone
build-scarthgap

After the final revision it was agreed to renamed jobs to:

build-eol kirkstone branch
build-lts scarthgap branch
build-main main branch

angolini · 2025-01-16T17:43:23Z

Please add a description to the PR

angolini · 2025-01-16T17:46:01Z

Can I understand we are going to perform 3 different kind of builds? main, kirkstone and scarthgap?

Also, can you, please, describe how the test builds is implemented in a section (or child page) in the confluence page for Release Process?

quaresmajose · 2025-01-17T11:10:26Z

Can I understand we are going to perform 3 different kind of builds? main, kirkstone and scarthgap?

right, we will have 3 jobs:

build-kirkstone
build-scarthgap
build-main

I think this naming is more appropriate as it makes clear what branch form the Yocto LTS we are using.

Also, can you, please, describe how the test builds is implemented in a section (or child page) in the confluence page for Release Process?

I can add something but I don't fully understand the tests.

quaresmajose · 2025-03-13T16:51:22Z

We have already the kirkstone branch created on lmp-manifest and meta-lmp but the scarthgap need to be created as well on the two repos.

ricardosalveti · 2025-03-13T20:45:52Z

lmp/jobserv.yml

        persistent-volumes:
          bitbake: /var/cache/bitbake

-  - name: build-release-stable


This was actually used even as part of our previous kirkstone merge (https://ci.foundries.io/projects/lmp/builds/2753/).

build-release-stable is currently basically a post merge run for kirkstone.

build-release-stable is currently basically a post merge run for kirkstone.

this build-release-stable is already there before we create the kirkstone branch and the last change of the job is from 3 year ago. before it is used for honister.

This was actually used even as part of our previous kirkstone merge (https://ci.foundries.io/projects/lmp/builds/2753/).

Right but please note this is a side effect of the kirkstone branch creation. When we created the kirkstone branch we should have changed the configuration of the build-release job that follows refs/heads/main to start following the refs/heads/kirkstone.

ricardosalveti · 2025-03-13T20:46:47Z

lmp/jobserv.yml

 timeout: 540  # a build with no cache is quite slow
 triggers:
-  - name: build-release
+  - name: build-kirkstone


The machines for kirkstone were actually aligned with build-release-stable, build-release is actually aligned with scarthgap.

This is what we should have done at the time of creating the branch as I mentioned in the previous comment.
When we created the kirkstone branch we should have changed the configuration of the build-release job that follows refs/heads/main to start following the refs/heads/kirkstone.

this is going to break test automation.

the reason of this remaining is because we will have more than one type of build-release, for example v94.x kirkstone based and v95+ scarthgap based.

lmp/jobserv.yml

ricardosalveti · 2025-03-13T20:48:52Z

lmp/jobserv.yml

    runs:
+      - name: lmp-sdk
+        host-tag: amd64
+        container: foundries/dind-ci:19.03.9_b38f166


Why are we using this old tag for lmp-sdk?

we only run this lmp-sdk in the build-kirkstone and I copy it also to build-scarthgap and build-main. the tag is the same

I have dropped the job to build the container from the build-kirkstone and build-main and keep it only on the build-scarthgap.

I will add in another PR381 the change to build it on build-main but for this I need @doanac help because we need to tag the container with a different name like next or something to only be used on the main branch.

mwasilew · 2025-03-18T10:13:40Z

This change should be aligned with the testing setup. At the moment the test scheduler is relying on the build names. So changing build-release to build-main will break automated merge/build/test setup: https://github.com/foundriesio/conductor/blob/master/conductor/api/views.py#L261

LTS testing is currently tracking kirkstone branch in lmp-manifest, but again the name of the build matters.

Alternatively there should be a patch to jobserve to allow for some other field to be added so it can be tracked easily not looking at the build name.

quaresmajose · 2025-03-18T11:52:18Z

This change should be aligned with the testing setup. At the moment the test scheduler is relying on the build names. So changing build-release to build-main will break automated merge/build/test setup: https://github.com/foundriesio/conductor/blob/master/conductor/api/views.py#L261

We can keep the names that are already in use but I think we need 3 different types of builds so we need to add one more at least.

LTS testing is currently tracking kirkstone branch in lmp-manifest, but again the name of the build matters.

I think it would be clearer if the names of the build were the same as the branches used and that was the main reason for naming them:

build-main: follow master branch
build-kirkstone: follow kirkstone branch
build-scarthgap: follow scarthgap branch

if it is possible to have a name build-release following two distinct branches kirkstone and scarthgap perhaps the process can be simplified.

angolini · 2025-03-19T19:14:40Z

I think the amount of builds we have should be related with the release support strategy we want. I'm not sure having 3 is what we want.
But we can decide it is. So let's think about the naming.
If naming affects the testing mechanism, I don't see why we are changing the name of each build every time a new LTS is out. Anyway, a new LTS is out by 2 years, so it's not that much of effort.

but I would keep the build name different from the yocto LTS version:
build-dev -> main
build-release -> scarthgap (when the time comes)
build-stable? build-stale? -> kirkstone (when the time comes)

quaresmajose · 2025-03-26T12:45:02Z

but I would keep the build name different from the yocto LTS version:
build-dev -> main
build-release -> scarthgap (when the time comes)
build-stable? build-stale? -> kirkstone (when the time comes)

We can have better names, another suggestion:

build-dev -> (main)
build-lts -> current LTS (scarthgap)
build-eol -> last LTS (kirkstone)

Although we don't have to change the names, we will always have to change the branch used, which changed with each new LTS.

quaresmajose · 2025-03-26T12:58:29Z

@mwasilew as the build names will have some impact on testing, can you please make any suggestions as to what names we should adopt to make 3 types of builds. Or any other solution so we can do 3 different builds with the corresponding tests.

mwasilew · 2025-03-26T13:41:21Z

I like what @angolini proposed the most. From the techincal pov it doesn't matter that much. Once we agree on the names I'll push proper changes to conductor.

angolini

ok

This is an old job that is not used in the last 3 years [1]. It is currently back in use as a result of the creation of the kirkstone branch but note that this ci job did not exist for this purpose. [1] foundriesio@f74174e Signed-off-by: Jose Quaresma <jose.quaresma@foundries.io>

ricardosalveti · 2025-05-05T19:00:04Z

Please just drop the patch instead of adding and reverting it as part of the same pr.

quaresmajose · 2025-05-06T09:23:46Z

Please just drop the patch instead of adding and reverting it as part of the same pr.

done

quaresmajose · 2025-05-06T09:39:35Z

but I would keep the build name different from the yocto LTS version:
build-dev -> main
build-release -> scarthgap (when the time comes)
build-stable? build-stale? -> kirkstone (when the time comes)

We can have better names, another suggestion:

build-dev -> (main)

build-lts -> current LTS (scarthgap)

build-eol -> last LTS (kirkstone)

Although we don't have to change the names, we will always have to change the branch used, which changed with each new LTS.

We still need to define and agree in choosing the CI build names?

angolini · 2025-05-07T13:10:57Z

for me any naming is good enough. I only want to have a better communication with the internal team on what is changed and why after the merge

mwasilew · 2025-05-07T13:36:54Z

I'd like to avoid using branch names in the build names. Something like "main", "lts", "eol" would be my preference.

ricardosalveti · 2025-05-07T21:28:18Z

I'd like to avoid using branch names in the build names. Something like "main", "lts", "eol" would be my preference.

Yeah, let's do main, lts and eol.

Signed-off-by: Jose Quaresma <jose.quaresma@foundries.io>

This job will stop following the main branch and will start following the kirkstone branch. Signed-off-by: Jose Quaresma <jose.quaresma@foundries.io>

…ed machines" This reverts commit 7be3f30. Given previous job name change and the coresponding ref to follow the kirkstone branch, we can enable this machines again since these machines are still available in v94 version. Signed-off-by: Jose Quaresma <jose.quaresma@foundries.io>

All the other jobs have the AKLITE_TAG so add them. Signed-off-by: Jose Quaresma <jose.quaresma@foundries.io>

We don't need anymore the build-main-next job because we have the main branch now free to track the upstream main/master. Let's also start with less machinery and then we add as needed. Signed-off-by: Jose Quaresma <jose.quaresma@foundries.io>

Let's start from oldest to newest: - build-eol - build-lts - build-main Signed-off-by: Jose Quaresma <jose.quaresma@foundries.io>

The run of the job was on the build-kirkstone that is now the previous LTS. Move it to the latest LTS stable which is the build-scarthgap. Signed-off-by: Jose Quaresma <jose.quaresma@foundries.io>

quaresmajose · 2025-05-08T14:56:29Z

Jobs renamed to:

build-eol
build-lts
build-main

quaresmajose requested a review from a team January 15, 2025 14:32

quaresmajose force-pushed the build-kirkstone branch from ea1b248 to 35358ac Compare January 15, 2025 16:04

quaresmajose force-pushed the build-kirkstone branch from 60c4a43 to f0bb2d7 Compare March 13, 2025 16:45

quaresmajose changed the title ~~lmp/jobserv: prepare the kirkstone build~~ lmp/jobserv: prepare for kirkstone/scarthgap/main builds Mar 13, 2025

quaresmajose requested review from a team and removed request for a team March 13, 2025 16:51

quaresmajose force-pushed the build-kirkstone branch from f0bb2d7 to 122b106 Compare March 13, 2025 16:55

ricardosalveti reviewed Mar 13, 2025

View reviewed changes

lmp/jobserv.yml Outdated Show resolved Hide resolved

ricardosalveti reviewed Mar 13, 2025

View reviewed changes

quaresmajose force-pushed the build-kirkstone branch from 122b106 to e34ae60 Compare March 14, 2025 10:17

quaresmajose force-pushed the build-kirkstone branch from f637aba to 94f701a Compare March 26, 2025 12:35

angolini approved these changes Mar 26, 2025

View reviewed changes

quaresmajose force-pushed the build-kirkstone branch from 94f701a to b43a3da Compare March 27, 2025 18:08

quaresmajose force-pushed the build-kirkstone branch from b43a3da to 4f18380 Compare March 31, 2025 17:34

quaresmajose force-pushed the build-kirkstone branch from b21427d to 5ddd566 Compare May 6, 2025 09:23

quaresmajose force-pushed the build-kirkstone branch from 5ddd566 to bc17ed1 Compare May 6, 2025 09:24

quaresmajose mentioned this pull request May 6, 2025

Build lmp-sdk container for main branch #381

Closed

quaresmajose added 7 commits May 8, 2025 15:47

lmp/jobserv: rename the build-scarthgap to build-lts

490b055

Signed-off-by: Jose Quaresma <jose.quaresma@foundries.io>

lmp/jobserv: rename the build-release to build-eol

b7ceb53

This job will stop following the main branch and will start following the kirkstone branch. Signed-off-by: Jose Quaresma <jose.quaresma@foundries.io>

lmp/jobserv: add AKLITE_TAG to build-eol job

77d8057

All the other jobs have the AKLITE_TAG so add them. Signed-off-by: Jose Quaresma <jose.quaresma@foundries.io>

lmp/jobserv: reorder the build jobs

1433c89

Let's start from oldest to newest: - build-eol - build-lts - build-main Signed-off-by: Jose Quaresma <jose.quaresma@foundries.io>

lmp/jobserv: build the lmp-sdk on the latest LTS build

305d281

The run of the job was on the build-kirkstone that is now the previous LTS. Move it to the latest LTS stable which is the build-scarthgap. Signed-off-by: Jose Quaresma <jose.quaresma@foundries.io>

quaresmajose force-pushed the build-kirkstone branch from bc17ed1 to 305d281 Compare May 8, 2025 14:52

ricardosalveti approved these changes May 8, 2025

View reviewed changes

ricardosalveti merged commit 0c7cb54 into foundriesio:master May 8, 2025
2 checks passed

quaresmajose deleted the build-kirkstone branch May 9, 2025 10:33

lmp/jobserv: prepare for kirkstone/scarthgap/main builds #371

lmp/jobserv: prepare for kirkstone/scarthgap/main builds #371

Uh oh!

Conversation

quaresmajose commented Jan 15, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

angolini commented Jan 16, 2025

Uh oh!

angolini commented Jan 16, 2025

Uh oh!

quaresmajose commented Jan 17, 2025

Uh oh!

quaresmajose commented Mar 13, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mwasilew commented Mar 18, 2025

Uh oh!

quaresmajose commented Mar 18, 2025

Uh oh!

angolini commented Mar 19, 2025

Uh oh!

quaresmajose commented Mar 26, 2025

Uh oh!

quaresmajose commented Mar 26, 2025

Uh oh!

mwasilew commented Mar 26, 2025

Uh oh!

angolini left a comment

Choose a reason for hiding this comment

Uh oh!

ricardosalveti commented May 5, 2025

Uh oh!

quaresmajose commented May 6, 2025

Uh oh!

quaresmajose commented May 6, 2025

Uh oh!

angolini commented May 7, 2025

Uh oh!

mwasilew commented May 7, 2025

Uh oh!

ricardosalveti commented May 7, 2025

Uh oh!

quaresmajose commented May 8, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

quaresmajose commented Jan 15, 2025 •

edited

Loading

quaresmajose commented Mar 13, 2025 •

edited

Loading