Cannot import 'function_name' from 'class name' (Unknown Location)

[derek-slack]

Hi I am trying to apply pathos.pp as follows:

HMCRoutine.py

class Routine:
     def __init__(self):

     ...

     def create_grad(self):
          
        from pathos.pp import ParallelPool

        def numerical_grad_p(betas, h=1e-5):
        
            pool = ParallelPool(nodes=6)

            def compute_partial_derivative(i):
                t_betas = betas.copy()
                t_betas[i] += h * t_betas[i]
                return (self.neg_log_likelihood(t_betas, d=True) - self.neg_log_likelihood_i) / (h * t_betas[i])

            indices = range(len(betas))
            grad = np.array(pool.map(compute_partial_derivative, indices))

            return grad

        self.grad_func = numerical_grad_p


     def HMC(self):
          ...
          gradient_func = self.create_grad()
          grad = gradient_func(betas)
          ...
          return samples
  

Script.py

from HMCRoutine import Routine

model = Routine()

samples = model.HMC()

I receive the error ImportError: cannot import name 'compute_partial_derivative' from 'routine' (unknown location)

The neg_log_liklihood functions call a solver which is not able to be mapped using JAX or other modules that use tracers. I am new to parallel computing so forgive my lack of understanding here.

I was able to see speed increase with ProcessPool() and wanted to try using ParallelPool() as well to compare.