-
Notifications
You must be signed in to change notification settings - Fork 1
Test statistic is unstable. #13
Copy link
Copy link
Open
Description
The use of T <- max(weights * abs(object - rowMeans(nullobject))) as the basis for P-values seems questionable. In practise, this usually hinges on a single disparity within in the top handful of items (assuming list agreements are generally higher near the top). I understand that the idea may have been to preferentially weight the top ranks, but this seems a bit much, and it will produce wildly varying P-value estimates among simulations when you don't fix the seed.
In the absence of a strong reason for your particular choice of test statistic, why not use something a little more stable, or provide a choice, e.g.
# ...
if (style == "exp") {
if (is.null(lambda)) {
lambda <- 0.1
}
weights <- exp(-lambda * (ranks - 1))
weights <- weights / sum(weights)
} else if (style == "hyperbolic") {
weights <- 1 / ranks
weights <- weights / sum(weights)
} else {
stop("Invalid style argument")
}
T <- sum(weights * abs(object- rowMeans(null_object)))Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
No labels