variation in open-search results

[animesh]

I ran the same search thrice using the fragpipe-23.1 linux version and see a little variation in the results, is this expected (some details below)? If so, is there a way to fix the randomness?

wc /cluster/home/ash022/cluster/scripts/240827_Bead_test/dda/240827_Beads_3B_Slot2-20_1_8130.d.1756805576.FP.result/ptm-shepherd-output/global.modsummary.tsv /cluster/home/ash022/cluster/scripts/240827_Bead_test/dda/240827_Beads_3B_Slot2-20_1_8130.d.1756803169.FP.result/ptm-shepherd-output/global.modsummary.tsv /cluster/home/ash022/cluster/scripts/240827_Bead_test/dda/240827_Beads_3B_Slot2-20_1_8130.d.1756742171.FP.result/ptm-shepherd-output/global.modsummary.tsv
  128   833  6606 /cluster/home/ash022/cluster/scripts/240827_Bead_test/dda/240827_Beads_3B_Slot2-20_1_8130.d.1756805576.FP.result/ptm-shepherd-output/global.modsummary.tsv
  127   833  6600 /cluster/home/ash022/cluster/scripts/240827_Bead_test/dda/240827_Beads_3B_Slot2-20_1_8130.d.1756803169.FP.result/ptm-shepherd-output/global.modsummary.tsv
  131   859  6818 /cluster/home/ash022/cluster/scripts/240827_Bead_test/dda/240827_Beads_3B_Slot2-20_1_8130.d.1756742171.FP.result/ptm-shepherd-output/global.modsummary.tsv

within the top rows/hits, seems to be a little off to each other

head /cluster/home/ash022/cluster/scripts/240827_Bead_test/dda/240827_Beads_3B_Slot2-            20_1_8130.d.1756805576.FP.result/ptm-shepherd-output/global.modsummary.tsv /cluster/home/ash022/cluster/scripts/240827_Bead_te            st/dda/240827_Beads_3B_Slot2-20_1_8130.d.1756803169.FP.result/ptm-shepherd-output/global.modsummary.tsv /cluster/home/ash022/c            luster/scripts/240827_Bead_test/dda/240827_Beads_3B_Slot2-20_1_8130.d.1756742171.FP.result/ptm-shepherd-output/global.modsumma            ry.tsv
==> /cluster/home/ash022/cluster/scripts/240827_Bead_test/dda/240827_Beads_3B_Slot2-20_1_8130.d.1756805576.FP.result/ptm-sheph            erd-output/global.modsummary.tsv <==
Modification    Mass Shift      dataset01_PSMs  dataset01_percent_PSMs
None    0.0     19153   37.1530
First isotopic peak     1.00235 3001    5.8200
Deamidation     0.984016        2389    4.6350
Iodoacetamide derivative/Addition of Glycine/Addition of G      57.021464       2014    3.9050
Isotopic peak error     -1.00235        961     1.8650
Unannotated mass-shift -9.0334  -9.0334 788     1.5270
Pyro-glu from Q/Loss of ammonia -17.026549      764     1.4820
Second isotopic peak    2.0047  760     1.4730
Replacement of 2 protons by calcium     37.946941       712     1.3800

==> /cluster/home/ash022/cluster/scripts/240827_Bead_test/dda/240827_Beads_3B_Slot2-20_1_8130.d.1756803169.FP.result/ptm-sheph            erd-output/global.modsummary.tsv <==
Modification    Mass Shift      dataset01_PSMs  dataset01_percent_PSMs
None    0.0     19153   37.1510
First isotopic peak     1.00235 2996    5.8110
Deamidation     0.984016        2389    4.6340
Iodoacetamide derivative/Addition of Glycine/Addition of G      57.021464       2018    3.9130
Isotopic peak error     -1.00235        947     1.8370
Unannotated mass-shift -9.0334  -9.0334 788     1.5270
Pyro-glu from Q/Loss of ammonia -17.026549      764     1.4820
Second isotopic peak    2.0047  763     1.4790
Replacement of 2 protons by calcium     37.946941       712     1.3790

==> /cluster/home/ash022/cluster/scripts/240827_Bead_test/dda/240827_Beads_3B_Slot2-20_1_8130.d.1756742171.FP.result/ptm-sheph            erd-output/global.modsummary.tsv <==
Modification    Mass Shift      dataset01_PSMs  dataset01_percent_PSMs
None    0.0     19153   37.1510
First isotopic peak     1.00235 3001    5.8200
Deamidation     0.984016        2389    4.6340
Iodoacetamide derivative/Addition of Glycine/Addition of G      57.021464       2029    3.9340
Isotopic peak error     -1.00235        958     1.8580
Unannotated mass-shift -9.0334  -9.0334 777     1.5060
Pyro-glu from Q/Loss of ammonia -17.026549      764     1.4820
Second isotopic peak    2.0047  763     1.4790
Replacement of 2 protons by calcium     37.946941       701     1.3580

[dpolasky]

Hi,
The original histogram generation method had random noise that resulted in these minor variations in peak counts and boundaries, but this was removed in version 23.0. Can you share the log files from these runs?

Best,
Dan

[animesh]

Thanks for the clarification @dpolasky 🙏 Here are the logs, hope these are the ones you asked 🤞

1. log_2025-09-01_18-15-17.txt

2. log_2025-09-02_11-11-07.txt

3. log_2025-09-02_11-51-05.txt

[dpolasky]

Thanks for the logs. It looks like the difference in the PTM-Shepherd output is at least in part because one of the runs has a different number of PSMs after FDR filtering than the other two (51552 vs 51555). The two runs with the same number of PSMs are more similar, though not exactly the same. I guess there is some slight random variation somewhere in the pipeline for open search, but we probably need to look at the whole result folders to be able to debug. If you are able to share, you can put them here: https://drive.google.com/drive/folders/1BEsc0E9vsASjxQqkJc_qZzQreC8Zb10E?usp=sharing

Best,
Dan

[animesh]

Thanks for looking into this @dpolasky, i can share the result folders 💯 however, i don't seem to have access to the upload folder?

[dpolasky]

Sorry about that - can you try again now?

[animesh]

No worries and yes it worked i think, please checkout https://drive.google.com/drive/folders/1BEsc0E9vsASjxQqkJc_qZzQreC8Zb10E , it should be about 220 MB?

[dpolasky]

thanks, the upload worked well. It looks like the scores are identical for all PSMs, but in one case the FDR filtering threshold was slightly different, giving 3 fewer PSMs. Maybe @AimeeD90 can take a look?