Seems to be commit type 1 error...

[jmossbridge]

Hi there,

I have been comparing the output of this script with R's Poisson.test and it seems to me your test commits type 1 error. Even in the example you give -- with one sample, there's no way the p-value should be borderline significant.

Do you know why? I have no idea, but it's just something I noticed.

Take care,
Julia

[aleifer]

Can you clarify whether this is characteristic of the E-test as described in:

Krishnamoorthy, K and Thomson, J. (2004) A more powerful test for comparing two Poisson means. Journal of Statistical Planning and Inference, 119, 249-267]

or whether it is in fact an error in my port of their fortran code?

[jmossbridge]

Hi Andrew, Sorry, I can't because there's a paywall for that paper. Do you have the original paper you can send? Then I can check. I just know that no statistician would consider a statistical test valid with 1 observation. So maybe they never tested 1 observation and they're aware their calculations fail (but also aware that no one would ever ask to take 1 observation seriously). More concerning is that I compared the output of your code with that of R's exact function, poisson.test, using photon absorption rates (a cononical Poisson process) and R's function gave more reasonable (larger p-value) results  -- https://www.rdocumentation.org/packages/stats/versions/3.6.1/topics/poisson.test I wonder if Krishnamoorthy and Thomson wrote about a potential tradeoff with type 1 error, but I can't see the paper. Anyway, wish I could help -- and happy to if you send the PDF. Take care, Julia

…

On 12/10/19 5:06 PM, Andrew Leifer wrote: Can you clarify whether this is characteristic of the E-test as described in: Krishnamoorthy, K and Thomson, J. (2004) A more powerful test for comparing two Poisson means. Journal of Statistical Planning and Inference, 119, 249-267] or whether it is in fact an error in my port of their fortran code? — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#1?email_source=notifications&email_token=AEB23AVGALDPT7OZK5EXSC3QYA4JZA5CNFSM4JYCPUU2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEGRQCQY#issuecomment-564330819>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AEB23AXLHNOFGB2ZZPD6BKTQYA4JZANCNFSM4JYCPUUQ>.

[aleifer]

The paper is available without paywall here:
https://userweb.ucs.louisiana.edu/~kxk4695/JSPI-04.pdf

Fortran and python implementations are available
https://userweb.ucs.louisiana.edu/~kxk4695/statcalc/pois2pval.for
and

https://pypi.org/project/poisson-etest/

There are many different statistical tests involving Poisson distributions and so I would not a priori expect the E-test to give the same results as another test.