Skip to content

Add effect/heterogeneity and normalized reports#18

Draft
FBartos wants to merge 1 commit intoastefan1:masterfrom
FBartos:add-effect-and-heterogeneity
Draft

Add effect/heterogeneity and normalized reports#18
FBartos wants to merge 1 commit intoastefan1:masterfrom
FBartos:add-effect-and-heterogeneity

Conversation

@FBartos
Copy link
Copy Markdown

@FBartos FBartos commented Apr 19, 2026

Overview

This PR extends phackR so simulations can be generated under a nonzero true effect and between-study heterogeneity, and makes the resulting analyses report standardized initial/final summaries in a consistent format.

Main Changes

  • Added effect and heterogeneity arguments to all exported sim.* functions.
  • Defined one iteration as one study with a study-level draw theta_i ~ Normal(effect, heterogeneity).
  • Implemented family-specific effect scales:
    • group-comparison simulators: standardized mean-difference scale
    • regression / correlation simulators: Fisher-z scale
    • covariate simulator: group-effect scale
  • Added a normalized reporting block to all exported simulators:
    • effect.initial, effect.final
    • se.initial, se.final
    • n.initial, n.final
    • stat.initial, stat.final
    • p.initial, p.final
    • method.initial, method.final
  • Preserved legacy output columns such as ps.*, r2s.*, ds.*, and eta2s.* for compatibility.

Implementation Notes

  • Added shared helpers for study-level effect draws, Fisher-z handling, standardized mean-difference reporting, association reporting, ANCOVA/group-effect reporting, and normalized result assembly.
  • Extended the previously deferred simulators, including cutoffs, variable transformations, favorable imputation, outlier exclusion, composite scores, covariates, and the regression branch of selective reporting of IVs.
  • Updated the regression branch of sim.multIVhack() so alternative is respected for one-sided tests.
  • Fixed one-sided direction handling in the group-comparison family so alternative = "greater" now means a positive effect in the intended direction: treatment/second group > control/first group.
  • Fixed the related sign inconsistency in legacy ds.* outputs.

Shiny App Updates

  • Added effect and heterogeneity controls to the Shiny UI for all relevant tabs.
  • Updated the app wording so rates are described as significant-result / rejection rates outside the null-only regime.
  • Clarified the one-sided option in the UI and docs as testing a positive effect: treatment > control.
  • Fixed the Selective Reporting IV tab to call sim.multIVhack() instead of sim.multDVhack().
  • Updated the Shiny startplot generation script and regenerated startplots.rds.

Documentation and Tests

  • Updated the vignette and Shiny markdown documentation to describe the new arguments, effect scales, reporting fields, and one-sided semantics.
  • Regenerated roxygen documentation and NAMESPACE.
  • Expanded the test suite to cover:
    • normalized reporting columns
    • non-null behavior
    • heterogeneity reproducibility
    • one-sided direction behavior

Verification

  • testthat::test_file('phackR/tests/testthat/test-simfunctions.R')
  • knitr::knit('phackR/vignettes/phackR_vignette.Rmd', ...)
  • direct smoke simulations for the extended simulator families
  • Shiny app source check after UI/server updates

Implement Phase 1 of the effect/heterogeneity extension: add public effect and heterogeneity plumbing to the two-group/t-test family, centralize study-level draws, and attach a normalized reporting block to Phase 1 simulations. Many generators (.sim.compscore, .sim.covariates, sim.*Hack wrappers, sim.cutoffHack, sim.compscoreHack, sim.covhack, etc.) now accept effect and heterogeneity and use a per-iteration theta drawn from .draw.study.effect; two-group data generation shifted accordingly. Introduced and used standardized reporting helpers (.report.twogroup, .report.association, .report.group_lm, .report.multicat), changed p-value/analysis selection to use .selectanalysis, and unified result combining with .combine.phase1.results so older legacy fields are preserved. Also updated NAMESPACE imports, bumped RoxygenNote in DESCRIPTION, added man pages and a planning doc (effect_heterogeneity_extension_plan.md), and adjusted tests and Shiny docs to reflect the new API and reporting contract. Backward compatibility is preserved when effect = 0 and heterogeneity = 0.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant