Make ROC curves

[lawrenceleejr]

Make ROC curves for these cuts. See more here: https://en.wikipedia.org/wiki/Receiver_operating_characteristic

You'll want to the fraction of signal (i.e. hard scatter) selected by a cut on the x axis, and the fraction of background selected by that cut on the y axis. A single cut will give you a single point in this space. If you scan over cut values for one of the cuts, you'll trace out a curve -- this is the ROC curve.

I would suggest the following which would make a really pretty plot:

1. For no cut on the theta variable, scan over cut values on the time variable. That will give you a curve in this space. Make that a blue.
2. Now apply a light cut on the theta variable, and then again scan over the time variable cuts. And start to increment through the rainbow.
3. Repeat until you've scanned over the full range of the theta variable.

Plot all of these colors and we should end up with a rainbow "ribbon" in the ROC plot.

Try just making one ROC curve to start. (Step 1)

[Bukas73]

First ROC curve, Smearing values: t = 30 ps, theta = 25 mrad. No cut on theta_Delta

[lawrenceleejr]

Very good! Now can you make a collection of ROC curves for different cuts on $\theta_\Delta$? Like scan across cuts on that to produce a bunch of new values.

If you can produce a pickle file containing:

• A list of two objects
• Each of these objects is a dict like sigEffDict and bgEffDict
• These dicts are "key"ed by the $t_\Delta$ cut, and the "value" is a list of efficiency values for each of the $\theta_\Delta$ cuts

So something like (pseudocode):

bgEffDict, sigEffDict = {}, {}
for timecut in timecutlist:
  bgEffs, sigEffs = [], []
  for thetacut in thetacutlist:
    bgEffs.append({calculated eff})
    sigEff.append({blah})

  sigEffsDict[timecut] = sigEffs
  bgEffsDict[timecut] = bgEffs

output = [sigEffsDict,bgEffsDict]

#output to a pickle file

If you have prepare a pickle file like that, then I can put it into some existing nice plotting scripts. for #4. The idea would be that we'd then choose a few combinations of thetaRes and timeRes values and each combination gives us a different pickle file. And then we can combine them all onto a single summary plot. Does that make sense?

[Bukas73]

Here's the collection of ROC curves that I made. I didn't see the above comment about the pickle file until just now, but it shouldn't be too difficult to implement in the code I have for drawing this plot up. The purple curve has a TimeRes of 6 ps, the blue curve has a TimeRes of 30 ps, the given value, and the green through dark orange and red are a range of values ranging from 60 ps to 1.5 ns and 3 ns, respectively.

[lawrenceleejr]

@Bukas73 Can you try now making it with 40 time cuts and 20 theta cuts? I was playing around with the following:

    timecutlist = np.linspace(0.5*TimeRes,5*TimeRes, 40)
    thetacutlist = np.linspace(0.5*ThetaRes,5*ThetaRes, 20)

See how that works for you?

[lawrenceleejr]

@Bukas73 Take a look at this (towards #4, #5)

Can you check on some of these numbers? I'm in particular worried about the fact that the blue (which is the best set of resolutions), while it performs the best across the board, isn't able to get to large signal efficiency.