Generate a CSV file with the firing rates of each brain region

[dbirman]

End product: CSV file where the first column is the brain region acronym and the second column is the average firing rate of clusters in that region

Steps
I'll put these as a checklist, but you don't actually need to check them off unless you want to

First, do this all for a test PID, use one of the ones I sent you

• Load the spike sorting data for this PID using the SpikeSortingLoader object (see e.g. https://int-brain-lab.github.io/iblenv/notebooks_external/loading_spikesorting_data.html)
• Calculate the average firing rate for each cluster by looping over clusters, pulling out just their spikes (something like spikes.times[spikes.clusters==clu] where clu is indexing the clusters)
• Figure out which brain area each cluster is in, this data is stored either in the clusters or the channels object that gets returned by the SpikeSortingLoader object
• Put the data for just the one test PID into a dataframe, so each row in the dataframe is one cluster and the columns will be the PID, cluster ID, brain area, and average firing rate

Once you have that working loop over all EIDs in the brainwidemap. You can get these by running this function:

def get_bwm_sessions():

    '''
    Gaelle, 06.04.2022
    '''

    str_query = (  
       'session__project__name__icontains,ibl_neuropixel_brainwide_01,'
       'session__json__IS_MOCK,False,session__qc__lt'
       ',50,~json__qc,CRITICAL,'
       'session__extended_qc__behavior,1,'
       'json__extended_qc__tracing_exists,True,'
       '~session__extended_qc___task_stimOn_goCue_delays__lt,0.9,'
       '~session__extended_qc___task_response_feedback_delays__lt,0.9,'
       '~session__extended_qc___task_wheel_move_before_feedback__lt,0.9,'
       '~session__extended_qc___task_wheel_freeze_during_quiescence__lt,0.9,'
       '~session__extended_qc___task_error_trial_event_sequence__lt,0.9,'
       '~session__extended_qc___task_correct_trial_event_sequence__lt,0.9,'
       '~session__extended_qc___task_reward_volumes__lt,0.9,'
       '~session__extended_qc___task_reward_volume_set__lt,0.9,'
       '~session__extended_qc___task_stimulus_move_before_goCue__lt,0.9,'
       '~session__extended_qc___task_audio_pre_trial__lt,0.9')                  
        
    str_query2 = (
       'session__project__name__icontains,ibl_neuropixel_brainwide_01,'
       'session__json__IS_MOCK,False,session__qc__lt,50,'
       '~json__qc,CRITICAL,session__extended_qc__behavior,1,'       
       'json__extended_qc__tracing_exists,True,'
       'session__extended_qc___experimenter_task,PASS')        
            
    ins = np.concatenate([one.alyx.rest('insertions', 'list', django = x)
                          for x in [str_query, str_query2]])

    eid_probe = set([x['session']+'_'+x['name'] for x in ins]) # pid via x['id']
    ins = [x.split('_') for x in eid_probe] 
 
    return ins

This will give you the EIDs (not pids) for all the sessions in the brain wide map dataset. Then you'll have to get the two PIDs for each session, which you can see how I did in the get_insertions_notebook you were using previously.

• Get all EIDs in a list
• Loop over EIDs and run your previous code
• Concatenate all the data into one huge dataframe, which will now have many thousands of rows
• Loop over all brain areas in the dataframe, for each brain area pull all the clusters, average their firing rates, and save that to a new dataframe which just has one row per brain area and two columns: the brain area and the average firing rate of clusters in the brain area.

That's it. Keep me updated about how this goes and if you get stuck on any of the IBL functionality. Kai is also a good person to ask about how to use IBL functions.