Skip to content

[O2B-1402] Fix TF timestamps being ignored when creating QC flags #1800

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
martinboulais merged 13 commits into from
Dec 3, 2024
Merged

[O2B-1402] Fix TF timestamps being ignored when creating QC flags #1800

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
13 commits
Select commit Hold shift + click to select a range
2a89441
[O2B-1401] Store runs and QC flags timestamps with ms
martinboulais Nov 5, 2024
e0ec8d0
Merge branch 'main' into mboulais/O2B-1401/store-runs-and-qcflags-tim…
martinboulais Nov 6, 2024
9be843a
Make time_start and time_end to DATETIME(3) too
martinboulais Nov 7, 2024
1b40073
Fix services tests
martinboulais Nov 7, 2024
4b94bda
Fix more tests
martinboulais Nov 7, 2024
8cbb834
Fix linter
martinboulais Nov 7, 2024
86d5bbd
Merge branch 'main' into mboulais/O2B-1401/store-runs-and-qcflags-tim…
martinboulais Nov 25, 2024
cb5ad03
Merge branch 'main' into mboulais/O2B-1401/store-runs-and-qcflags-tim…
martinboulais Nov 25, 2024
543e430
[O2B-1402] Fix TF timestamps being ignored when creating QC flags
martinboulais Nov 6, 2024
388119b
WIP tests
martinboulais Nov 6, 2024
683e8bc
Merge branch 'main' into mboulais/O2B-1402/fix-tf-timestamps-being-ig…
martinboulais Nov 27, 2024
cd5141f
Add one test case
martinboulais Nov 27, 2024
9072aca
Merge branch 'main' into mboulais/O2B-1402/fix-tf-timestamps-being-ig…
martinboulais Dec 3, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion lib/database/seeders/20200713103855-runs.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2690,9 +2690,10 @@ module.exports = {
{
id: 107,
run_number: 107,
first_tf_timestamp: '2019-08-08 12:59:00',
time_o2_start: '2019-08-08 13:00:00',
time_o2_end: '2019-08-09 14:00:00',
last_tf_timestamp: '2019-08-09 13:00:00',
last_tf_timestamp: '2019-08-09 14:01:00',
run_type_id: 12,
run_quality: 'good',
n_detectors: 15,
Expand Down
75 changes: 75 additions & 0 deletions lib/database/seeders/20221006084006-run-detectors.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -624,6 +624,81 @@ module.exports = {
detector_id: 16,
quality: 'good',
},
{
run_number: 107,
detector_id: 17,
quality: 'good',
},
{
run_number: 107,
detector_id: 1,
quality: 'good',
},
{
run_number: 107,
detector_id: 18,
quality: 'good',
},
{
run_number: 107,
detector_id: 2,
quality: 'good',
},
{
run_number: 107,
detector_id: 19,
quality: 'good',
},
{
run_number: 107,
detector_id: 6,
quality: 'good',
},
{
run_number: 107,
detector_id: 4,
quality: 'good',
},
{
run_number: 107,
detector_id: 8,
quality: 'good',
},
{
run_number: 107,
detector_id: 9,
quality: 'good',
},
{
run_number: 107,
detector_id: 10,
quality: 'good',
},
{
run_number: 107,
detector_id: 11,
quality: 'good',
},
{
run_number: 107,
detector_id: 12,
quality: 'good',
},
{
run_number: 107,
detector_id: 13,
quality: 'good',
},
{
run_number: 107,
detector_id: 14,
quality: 'good',
},
{
run_number: 107,
detector_id: 16,
quality: 'good',
},
],
{ transaction },
),
Expand Down
2 changes: 1 addition & 1 deletion lib/server/controllers/gRPC/GRPCFlpRoleController.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@ class GRPCFlpRoleController {
try {
createdFlps.push(await this.flpRoleService.create({ name, hostname, runNumber }));
} catch (e) {
// TODO log into infologger
// TODO [O2B-967] log into infologger
}
}
return { flps: createdFlps };
Expand Down
4 changes: 2 additions & 2 deletions lib/server/services/qualityControlFlag/QcFlagService.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -168,7 +168,7 @@

let upperBound = timeTrgEnd?.getTime() ?? timeO2End?.getTime() ?? null;
if (lastTfTimestamp) {
upperBound = upperBound ? Math.max(lastTfTimestamp?.getTime(), upperBound) : lastTfTimestamp.getTime();
upperBound = upperBound ? Math.max(lastTfTimestamp.getTime(), upperBound) : lastTfTimestamp.getTime();

Check warning on line 171 in lib/server/services/qualityControlFlag/QcFlagService.js

View check run for this annotation

Codecov / codecov/patch

lib/server/services/qualityControlFlag/QcFlagService.js#L171 

Added line #L171 was not covered by tests
}

const from = timestamps.from ?? lowerBound ?? null;
Expand Down Expand Up @@ -887,7 +887,7 @@

const run = await RunRepository.findOne({
subQuery: false,
attributes: ['timeTrgStart', 'timeTrgEnd'],
attributes: ['timeO2Start', 'timeTrgStart', 'firstTfTimestamp', 'lastTfTimestamp', 'timeTrgEnd', 'timeO2End'],
where: { runNumber },
include: runInclude,
});
Expand Down
4 changes: 2 additions & 2 deletions lib/server/services/run/RunService.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -114,7 +114,7 @@ const logRunDetectorQualityChange = async (runNumber, runDetectors, transaction,
transaction,
);
if (error) {
// TODO log the failed log creation
// TODO [O2B-967] log the failed log creation
}
};

Expand Down Expand Up @@ -162,7 +162,7 @@ const logEorReasonChange = async (runNumber, userName, eorReasons, transaction)
);

if (error) {
// TODO log the failed log creation
// TODO [O2B-967] log the failed log creation
}
};

Expand Down
60 changes: 60 additions & 0 deletions test/lib/server/services/qualityControlFlag/QcFlagService.test.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -914,6 +914,66 @@ module.exports = () => {
);
});

it('should fail to create quality control flag because qc flag `from` timestamp is smaller than run.firstTfTimestamp', async () => {
const period = {
from: new Date('2019-08-08 11:36:40').getTime(),
to: new Date('2019-08-09 05:40:00').getTime(),
};

const runFirstTfTimestamp = new Date('2019-08-08 12:59:00').getTime();
const runLastTfTimestamp = new Date('2019-08-09 14:01:00').getTime();

const qcFlag = {
...period,
comment: 'VERY INTERESTING REMARK',
flagTypeId: 2,
};

const scope = {
runNumber: 107,
simulationPassIdentifier: { id: 1 },
detectorIdentifier: { detectorId: 1 },
};

const relations = { user: { roles: ['admin'], externalUserId: 456 } };

await assert.rejects(
() => qcFlagService.create([qcFlag], scope, relations),
// eslint-disable-next-line max-len
new BadParameterError(`Given QC flag period (${period.from}, ${period.to}) is out of run (${runFirstTfTimestamp}, ${runLastTfTimestamp}) period`),
);
});

it('should fail to create quality control flag because qc flag `to` timestamp is greater than run.lastTfTimestamp', async () => {
const period = {
from: new Date('2019-08-08 13:17:19').getTime(),
to: new Date('2019-08-09 15:49:01').getTime(),
};

const runFirstTfTimestamp = new Date('2019-08-08 12:59:00').getTime();
const runLastTfTimestamp = new Date('2019-08-09 14:01:00').getTime();

const qcFlag = {
...period,
comment: 'VERY INTERESTING REMARK',
flagTypeId: 2,
};

const scope = {
runNumber: 107,
simulationPassIdentifier: { id: 1 },
detectorIdentifier: { detectorId: 1 },
};

const relations = { user: { roles: ['admin'], externalUserId: 456 } };

await assert.rejects(
() => qcFlagService.create([qcFlag], scope, relations),
// eslint-disable-next-line max-len
new BadParameterError(`Given QC flag period (${period.from}, ${period.to}) is out of run (${runFirstTfTimestamp}, ${runLastTfTimestamp}) period`),
);
});

it('should fail to create quality control flag because qc flag `from` timestamp is greater than `to` timestamp', async () => {
const qcFlag = {
from: new Date('2019-08-09 04:16:40').getTime(), // Failing property
Expand Down
Loading