style: add builder for releaselist. #1273

stevenhua0320 · 2025-11-03T20:51:04Z

@sbillinge Ready for initial review, I should further add filter_software similar style as filter_presentation in tools.py and add btype into builder.py. But for now, we should first confirm whether I am on the right track.

codecov · 2025-11-03T20:57:24Z

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 79.86%. Comparing base (c1c5a62) to head (d7cb793).
⚠️ Report is 6 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #1273      +/-   ##
==========================================
+ Coverage   79.64%   79.86%   +0.22%     
==========================================
  Files          17       17              
  Lines        1366     1381      +15     
==========================================
+ Hits         1088     1103      +15     
  Misses        278      278

Files with missing lines	Coverage Δ
tests/test_builders.py	`84.28% <ø> (ø)`
tests/test_tools.py	`98.22% <100.00%> (+0.10%)`	⬆️

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

sbillinge

I will review the code shortly but we need to add this to builder.py so the tests fail

news/releasebuilder.rst

sbillinge

Looking good. See typo

src/regolith/builder.py

sbillinge

I just found this unsubmitted review....sorry about that. I mentioned some of the things at group meeting yesterday, but there are probably a few more. Please see below.

This is already good. I think it may not work out of the box because in the presentations, each presentation has a date, but here we are iterating over programs in softare, and we then have to further iterate over releases in program to get dates, so that may take more work, unless you already did that.

I don't see the output template here. The way to build that is to take the tex file from the test and replace the text with jinja template code and iterators that will build the lists.

Good progress!

sbillinge · 2025-11-04T12:38:46Z

src/regolith/builders/releaselistbuilder.py

+        gtx["grants"] = sorted(all_docs_from_collection(rc.client, "grants"), key=_id_key)
+        gtx["groups"] = sorted(all_docs_from_collection(rc.client, "groups"), key=_id_key)
+        gtx["software"] = sorted(all_docs_from_collection(rc.client, "software"), key=_id_key)
+        gtx["institutions"] = sorted(all_docs_from_collection(rc.client, "institutions"), key=_id_key)


I am not sure if we will use this one. I don't remember any institutions in the output file.

In the title should we have some institutional information such as email name and school affliation. I'm gonna check whether people contain all these information.

Update: Checked, we don't need this, I would at least delete the code in tools.py that uses institutions. But we actually have one called institutions.yaml in the output template folder.

sbillinge · 2025-11-04T12:40:03Z

src/regolith/builders/releaselistbuilder.py

+                        everybody, self.gtx["software"], self.gtx["institutions"], member, statuses=["active"]
+                    )
+
+                    if len(presclean) > 0:


we may want to change presclean everywhere to progclean or sthg like that.

sbillinge · 2025-11-04T12:40:53Z

src/regolith/tools.py

+
+    # build the filtered collection
+    # only list the talk if the group member is an author
+    for sof in software:


for program in software: for greater readability

sbillinge · 2025-11-04T12:41:45Z

src/regolith/tools.py

+        authorids = [author["_id"] if author is not None else author for author in authors]
+        if target in authorids:
+            firstclean.append(sof)
+    # only list the presentation if it has status in statuses


change presentation here to program

sbillinge · 2025-11-04T12:43:41Z

src/regolith/tools.py

+                sof["{}day_suffix".format(day)] = number_suffix(get_dates(sof).get(f"{day}date").day)
+            except AttributeError:
+                print(f"presentation {sof.get('_id')} has no {day}date")
+        if "institution" in sof:


I don't think we need any institution informatoin so this can be deleted

sbillinge · 2025-11-06T21:07:18Z

news/releasebuilder.rst

 **Added:**

-* Add test output for releaselist.
+* Add Releaselistbuilder functionality.


check capitalization. Probably "Add Releaselistbuilder functionality"

Please check but I don't think we punctuate changelog entries

sbillinge

I am confused why this is passing tests. Should it not be failing?

stevenhua0320 · 2025-11-06T21:14:54Z

I am confused why this is passing tests. Should it not be failing?

Originally it fails, but after I commited for the filter function in tools.py it passed. I'm wondering it is because there is corresponding function applied and there is no corresponding template to build so it automatically passes the test...

…jinja2 for releaselist

stevenhua0320 · 2025-11-07T00:37:42Z

I changed the schema.json and exemplar.json here because we have a short description for each program for software list while we don't have it in both schema.json and exemplar.json. So, I add a variable called program_description.

sbillinge · 2025-11-07T02:36:12Z

Ok, maybe add a minimal template and see if we can get the test to fail

stevenhua0320 · 2025-11-10T19:32:14Z

@sbillinge Got it why it would always pass it. Is that because in the original CI test we tested the original ones that has no coverage on ReleaseListBuilder? If this is the case, then we need to write a test function to test the behavior of filter tool. But it might need some time to think about it to follow the original style in test_tools.py

sbillinge · 2025-11-10T23:06:28Z

@sbillinge Got it why it would always pass it. Is that because in the original CI test we tested the original ones that has no coverage on ReleaseListBuilder? If this is the case, then we need to write a test function to test the behavior of filter tool. But it might need some time to think about it to follow the original style in test_tools.py

it should test it by trying to build the doc. I am guessing that it doesn't pass because there is no output template and so it actually doesn't render anything. So just put a template file a template folder and I think it should start failing

stevenhua0320 · 2025-11-10T23:15:05Z

@sbillinge Actually I should have a output template called releaselist.texcommited in the previous commit. I'm not sure why it still pass or actually it is right?
by the way, I found there are lingering issues with the filter in tools.py when writing the test, so I would make another PR when it is ready.

…nd time interval.

stevenhua0320 · 2025-11-11T00:01:41Z

@sbillinge I noticed that we have a test_builders.py in regolith, should we add something to that so that it would trigger to fail? I believe the current test is not touching the ReleaseListBuilder as I have checked that the number of test is the same after adding ReleaseListBuilder. Furthermore, I intentionally change one variable name and run pytest on local. However, the pytest still passes. So, I believe we need add something to the test to trigger failure. But I'm not quite sure right now what we need to add to trigger that failure.

Update: Have added the test on ReleaseListBuilder in test_builders.py. It triggered the issue once since originally the latex render is in the incorrect level and caused to fail. But after changing that, it triggers the CI on GH to fail.

More update: Now the test gives the error:

 if testfile.name not in output_filenames:
                print(
                    f"Expected test file {testfile} not generated by the builder." f"Files generated: {output_files}"
                )
>               assert False
E               assert False

/Users/huarundong/dbs/regolith/tests/test_builders.py:159: AssertionError

which means we are not passing it to the command of build, I'm not sure where we need to add that? need some clarification on this.

…ist in test_builders

sbillinge · 2025-11-11T11:14:43Z

great progress....we have a failing test! :) The test is not finding the rendered files. Either they are not being rendered or they are appearing in a different place or with a different tname than you used in your test output. By default they go to _build, so if you run this on your own regolith database it will appear in Path().cwd() / _build sthg like that, but looking at the structure of the output repos the test_builder goes looking for tests/outputs/<buildername>/<outputnames>. Your output is called releaselists.tex which for sure is wrong because your builder will beuild software-regro-scopatz.tex or something (I don't remember the pattern you used actually but something like that). So that is one problem, but I think there may be a few more we have to track down.

sbillinge · 2025-11-19T13:19:38Z

I see progress here. Please let me know what you need from me?

stevenhua0320 · 2025-11-19T16:56:20Z

@sbillinge I think right now it goes fairly well here. The status right now is it now renders the latex file, but there is a mismatch between the built and the template. So, the next step is to fix those mismatch.

sbillinge · 2025-11-19T18:57:48Z

@stevenhua0320 great progress. The code goes into different collections in the database to get different information, so we have to put things into our software database that exist in other databases (like aeinstein....I see you found that). But yes, please just push on and get it passing if you can and I can merge. Please let me know if some things are causing issues. Working with the jinja is a bit of a pain and I may be able to give advice if you paste images of things going wrong into the comments.

stevenhua0320 · 2025-11-19T23:22:09Z

@sbillinge Finally! It passed all the tests. Ready to review

sbillinge · 2025-11-20T02:08:48Z

yeh!

stevenhua0320 added 2 commits November 3, 2025 15:44

style: add builder for releaselist.

62f5a40

chore: add news item.

82aa55b

sbillinge reviewed Nov 3, 2025

View reviewed changes

news/releasebuilder.rst Outdated Show resolved Hide resolved

stevenhua0320 and others added 3 commits November 3, 2025 16:33

style: add ReleaselistBuilder in builder.py

b34f33d

[pre-commit.ci] auto fixes from pre-commit hooks

932db62

chore: Fix spelling issue

d899446

sbillinge reviewed Nov 3, 2025

View reviewed changes

src/regolith/builder.py Outdated Show resolved Hide resolved

stevenhua0320 added 5 commits November 3, 2025 16:44

chore: change filter for software, need more discussion for this.

7e44307

chore: change typo for ReleaseListBuilder

53fe450

fix: fix iterator name to make compatible.

a60cb79

chore: change iterator to sof

c158e64

fix: change get date level from release level.

ba232d7

sbillinge reviewed Nov 6, 2025

View reviewed changes

stevenhua0320 and others added 3 commits November 6, 2025 19:32

style: add program_description to compatible to output template, add …

7840676

…jinja2 for releaselist

[pre-commit.ci] auto fixes from pre-commit hooks

610ebfa

chore: change typo for correction of pre-commit

402223d

stevenhua0320 and others added 2 commits November 10, 2025 18:57

feat: Add filter for releasebuilder by active status, release type, a…

f3ccd01

…nd time interval.

[pre-commit.ci] auto fixes from pre-commit hooks

a478a72

stevenhua0320 and others added 3 commits November 10, 2025 19:21

fix: change variable and function argument to make it right

ca239ff

[pre-commit.ci] auto fixes from pre-commit hooks

fc64966

fix: change latex render level in releaselistbuilder and add releasel…

7df50f3

…ist in test_builders

stevenhua0320 and others added 2 commits November 10, 2025 19:56

fix: change format of releaselist.tex

2b15147

[pre-commit.ci] auto fixes from pre-commit hooks

05c72d5

fix: correctly render the tex file, next step fix error.

ae7ef2e

stevenhua0320 and others added 3 commits November 19, 2025 17:51

fix: match latex build with the test output template.

7412568

Merge branch 'main' into relaselistbuilder

5e413c6

Fix: capitalize letter to pass test

d7cb793

sbillinge merged commit 620324f into regro:main Nov 20, 2025
7 checks passed

style: add builder for releaselist. #1273

style: add builder for releaselist. #1273

Uh oh!

Conversation

stevenhua0320 commented Nov 3, 2025

Uh oh!

codecov bot commented Nov 3, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

sbillinge left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

sbillinge left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

sbillinge left a comment

Choose a reason for hiding this comment

Uh oh!

sbillinge Nov 4, 2025

Choose a reason for hiding this comment

Uh oh!

stevenhua0320 Nov 6, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

stevenhua0320 Nov 6, 2025

Choose a reason for hiding this comment

Uh oh!

sbillinge Nov 4, 2025

Choose a reason for hiding this comment

Uh oh!

sbillinge Nov 4, 2025

Choose a reason for hiding this comment

Uh oh!

sbillinge Nov 4, 2025

Choose a reason for hiding this comment

Uh oh!

sbillinge Nov 4, 2025

Choose a reason for hiding this comment

Uh oh!

sbillinge Nov 6, 2025

Choose a reason for hiding this comment

Uh oh!

sbillinge left a comment

Choose a reason for hiding this comment

Uh oh!

stevenhua0320 commented Nov 6, 2025

Uh oh!

stevenhua0320 commented Nov 7, 2025

Uh oh!

sbillinge commented Nov 7, 2025

Uh oh!

stevenhua0320 commented Nov 10, 2025

Uh oh!

sbillinge commented Nov 10, 2025

Uh oh!

stevenhua0320 commented Nov 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

stevenhua0320 commented Nov 11, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

sbillinge commented Nov 11, 2025

Uh oh!

sbillinge commented Nov 19, 2025

Uh oh!

stevenhua0320 commented Nov 19, 2025

Uh oh!

sbillinge commented Nov 19, 2025

Uh oh!

stevenhua0320 commented Nov 19, 2025

Uh oh!

Uh oh!

sbillinge commented Nov 20, 2025

Uh oh!

Reviewers

Assignees

codecov bot commented Nov 3, 2025 •

edited

Loading

stevenhua0320 Nov 6, 2025 •

edited

Loading

stevenhua0320 commented Nov 10, 2025 •

edited

Loading

stevenhua0320 commented Nov 11, 2025 •

edited

Loading