Consider saving models weights/nodes to '.npy'

[FedeClaudi]

Hi,

I've notices that the models parameters that are downloaded from VoxelModelCache are stored as .csv.gz. Given that these are quite large files, it takes several minutes to load them when calling get_voxel_connectivity_array.

An alternative would be to save them as npy.gz. It seems that .npy is much faster to load, giving a significant performance boost. On my machine it went from taking about 5 minutes to load to taking about 30 seconds.

If you still wish to keep them as .gz, saving/loading to .npy.gz is as simple as:

def load_npy_from_gz(filepath):
	f = gzip.GzipFile(filepath, "r")
	return np.load(f)

def save_npy_to_gz(filepath, data):
	f = gzip.GzipFile(filepath, "w")
	np.save(f, data)
	f.close()

Hope this helps,
Fede

[kamdh]

Sounds like you have enough of a plan here to modify the code. If you'd like to submit a PR, I can evaluate it and put in the new format.

[kamdh]

I have not implemented this because I do not have the ability to update the saved data on the alleninstitute.org website