miscellaneous questions

[jerul]

Hi there, thanks for this very nice package!

I’m wondering about a few things:

• The package does not seem to be compatible with using informative priors for some of the parameters (i.e., the linear terms I mentioned above), or would it be possible to somehow include that? For instance, would it be possible to specify an informative halfnormal distribution for a few parameters and then the isotropic prior for a few parameters (to be selected from while the others HAVE to be included)? Or does it make sense to utilize a two-step procedure in which you first conduct model selection and then run a new model with only the selected parameters but then with different priors?

• What would be a reasonable PIP cutoff? I saw in the documentation that spurious features had a maximum PIP of 0.0028, but that would leave me with too many variable so I’m currently using 0.01 as a cutoff. Or should I try to validate this using metrics like WAIC?

• It’s not completely clear to me how this ‘expected number of covariates’ hyperparameter functions and whether I should tune it somehow? I don’t necessarily have a theoretical reason to prefer 2 over 5 covariates, for instance.

• How can I access the posterior samples? I didn’t find that in the documentation.

[martinjankowiak]

let me try to answer your questions in order

• no, this package does not support modifying the priors substantially apart from what's in the API. a two-step procedure could make sense. it depends on the application

• it depends on the application/data. for example if some of your covariates are highly correlated, what you're going to do in any downstream analysis, etc. note that the example notebooks use perfectly clean data. in practice "spurious" pips can be much larger than something like 0.0028. a default choice might be a cutoff of 0.5 or 0.9 but it really depends on your goals

• in practice the choice of S shouldn't impact the results dramatically because the data should overwhelm the prior to some extent. you might try doing a sensitivity analysis (varying S within some reasonable range) to assess whether that's true for your particular dataset

• as stipulated in the docs for the run method, you can access the samples attribute if you set streaming=False. i don't recommend using these samples directly unless you're pretty sure you know what they're doing. for example, the samples are weighted and the weights need to be taken into account to get correct results

[jerul]

Thanks! That's helpful. I will utilize a two-step procedure then, in which case I also won't need the samples.

A cutoff of even 0.5 seems immense if I look at the PIPs that I'm dealing with. That would mean no or perhaps at most one or two predictors per equation. Is there any theoretical justification for any particular cutoff?

[martinjankowiak]

i believe there are several kinds of approaches that people advocate. you'd need to dive into the literature. for example i believe this paper advocates (or at least investigates) the 0.5 cutoff