Comments on mixture models notebook

[adrn]

• Text above In[40]:
  • Using $B$ as a number makes me feel funny inside.
• In[40]:
  • To simplify and not start with a wall of code, you could put the data
    generation code into a module in the same directory, and just import it
    into the notebook. You can then pass in the number of desired data points,
    e.g., xis, yis, sigmais = simulate_data(N=300). Then you can just
    visualize the simulated data. I just worry people will get hung up on the
    details of the simulated data generation, so it might be better to hide
    from the notebook but provide the script for anyone who wants to see more.
• In[41]:
  • To improve visibility you could change the style of the errorbar plot:

    plt.errorbar(yis/sigmais, xis-yis, sigmais, 
                 fmt="o", lw=1, markersize=2, ecolor='#888888', alpha=0.75)

• Text above In[42]:
  • You say "But, as we will see below, because the noise is known, it can be
    deconvolved, and we can recover the two components pretty well." You might
    put a caveat that this is only really true if you have some beliefs about
    the form of the mixture component, right? In this case, we have reason to
    believe the mixture components are Gaussians?
  • Change "In other words, we are in presence of a product of Gaussians being
    integrated over." to, e.g., "In other words, the expression for the
    posterior probability contains an integral over a product of two
    Gaussians."
  • "...we have once and for all removed the latent variables and
    integrals..." It might be worth adding to the discussion that sometimes we
    are interested in having posterior pdfs over the x_i's too! Of course, you
    can always back those out with posterior samples over the population
    parameters - either discuss that further down or link to DFM's blog post
    about related things.
• In[42]:
  • You could add docstrings to each function to explain in words what each
    function does.
  • You could rename gaussiansumsig gaussian_sum_var to make the
    underscores consistent with other functions, and because you sum the
    variances not the sigmas. But now I'm getting really annoying...
  • You could also just define the ln... functions, and in the non-log
    functions just call exp(ln...()). Reduces some duplicate code, but fine
    as is.
• Text above In[43]:
  • "...which, for our fake data set, has 6 parameters" remind the reader what
    the 6 parameters are.
  • Make references to emcee links to the emcee docs?
• In[43]:
  • I think it helps to use more verbose variable names. Instead of nt, how
    about n_mixture? Then you'd want to break this into 3 lines:

    alphas = params[0:n_mixture]
    betas = params[n_mixture:2*n_mixture]
    gammas = params[2*n_mixture:3*n_mixture]

• Text above In[46]:
  • Change "...live on a thin slice in $\alpha$ space." to "lives on a thin
    slice in $\alpha$ space."
  • "Among the many ways one could solve this problem this, that introduced by
    M. Betancourt (arxiv:1010.3436) is convenient" - grammar? Also, you could
    link to the paper.
  • "It establishes a bijective mapping between the $(B-1)-$ hypercube and
    the $B-$simplex," Sounds very fancy, but many people will not learn
    anything from this statement. If you want to keep it in here for the
    experts, at least put another sentence afterwards that explains in plain
    english what's going on - it's a trivial transformation to make in
    practice, so it shouldn't be obfuscated by math-speak :)
  • Change " simple solution is to tune the prior" to "a simple solution is to
    tune the prior"
• In[47]:
  • You can probably get rid of those "fingers of God" that appear in the
    corner plot if you do an intermediate sampling where, after burn-in, you
    generate a small ball around the median sample and burn in again. (not
    tested), but for example:

    sampler = emcee.EnsembleSampler(nwalkers, ndim, lnprob, threads=4)
    pos, prob, state = sampler.run_mcmc(p0s, 1000)
    sampler.reset()
    new_pos = emcee.utils.sample_ball(np.median(pos, axis=0), 
                                      np.full(pos.shape[1], 1E-5), 
                                      size=nwalkers)
    pos,_,_ = sampler.run_mcmc(new_pos, 1000)
    sampler.reset()
    pos,_,_ = sampler.run_mcmc(pos, 1000)

• Text above In[75]:
  • "This is before those have a typical acceptance rate of..." typo?
• Text above In[78]:
  • Change "However, by using analytic derivatives and Hessian" to
    "However, by using analytic derivatives and Hessians"
  • Change "so that the random moves and made in the most efficient
    directions" to "so that the random moves are made in the most efficient
    directions"
  • Change "...our sampler will throw a warning every time it rejects a
    sampler..." to " our sampler will throw a warning every time it rejects a
    sample"
• "Final comments" cell:
  • Equation is missing a closing parenthesis in $p({y_i,\sigma_i})$

[ixkael]

Thank you so much for all those useful comments, I've addressed them in the latest version of the notebook