Merge main of datajoint/element-array-ephys

Author: kabilar

CHANGELOG.md

@@ -3,6 +3,13 @@
 Observes [Semantic Versioning](https://semver.org/spec/v2.0.0.html) standard and
  [Keep a Changelog](https://keepachangelog.com/en/1.0.0/) convention.
 
+## [0.2.5] - 2023-04-12
+
++ Add - docstrings for quality metric tables
++ Fix - docstring errors
++ Update - `concepts.md`
++ Update - schema diagrams with quality metrics tables
+
 ## [0.2.4] - 2023-03-10
 
 + Update - Requirements with `ipywidgets` and `scikit-image` for plotting widget
@@ -90,6 +97,7 @@ Observes [Semantic Versioning](https://semver.org/spec/v2.0.0.html) standard and
 + Add - Probe table supporting: Neuropixels probes 1.0 - 3A, 1.0 - 3B, 2.0 - SS,
   2.0 - MS
 
+[0.2.5]: https://github.com/datajoint/element-array-ephys/releases/tag/0.2.5
 [0.2.4]: https://github.com/datajoint/element-array-ephys/releases/tag/0.2.4
 [0.2.3]: https://github.com/datajoint/element-array-ephys/releases/tag/0.2.3
 [0.2.2]: https://github.com/datajoint/element-array-ephys/releases/tag/0.2.2

docs/mkdocs.yaml

@@ -1,4 +1,4 @@
-# ---------------------- PROJECT SPECIFIC ---------------------------
+--- # ---------------------- PROJECT SPECIFIC ---------------------------
 
 site_name: DataJoint Documentation
 site_url: http://localhost/docs/elements/element-array-ephys
@@ -7,19 +7,20 @@ repo_name: datajoint/element-array-ephys
 nav:
   - Element Array Ephys: index.md
   - Concepts: concepts.md
-  - Tutorials: 
-    - Overview: tutorials/index.md
-    - Data Download: tutorials/00-data-download-optional.ipynb
-    - Configure: tutorials/01-configure.ipynb
-    - Workflow Structure: tutorials/02-workflow-structure-optional.ipynb
-    - Process: tutorials/03-process.ipynb
-    - Automate: tutorials/04-automate-optional.ipynb
-    - Explore: tutorials/05-explore.ipynb
-    - Drop: tutorials/06-drop-optional.ipynb
-    - Downstream Analysis: tutorials/07-downstream-analysis.ipynb
-    - Visualizations: tutorials/10-data_visualization.ipynb
-    - Electrode Localization: tutorials/08-electrode-localization.ipynb
-    - NWB Export: tutorials/09-NWB-export.ipynb
+  - Tutorials:
+      - Overview: tutorials/index.md
+      - Data Download: tutorials/00-data-download-optional.ipynb
+      - Configure: tutorials/01-configure.ipynb
+      - Workflow Structure: tutorials/02-workflow-structure-optional.ipynb
+      - Process: tutorials/03-process.ipynb
+      - Automate: tutorials/04-automate-optional.ipynb
+      - Explore: tutorials/05-explore.ipynb
+      - Drop: tutorials/06-drop-optional.ipynb
+      - Downstream Analysis: tutorials/07-downstream-analysis.ipynb
+      - Visualizations: tutorials/10-data_visualization.ipynb
+      - Electrode Localization: tutorials/08-electrode-localization.ipynb
+      - NWB Export: tutorials/09-NWB-export.ipynb
+      - Quality Metrics: tutorials/quality_metrics.ipynb
   - Citation: citation.md
   - API: api/ # defer to gen-files + literate-nav
   - Changelog: changelog.md
@@ -35,7 +36,7 @@ nav:
 # 02. Instead of designating codeblocks with bash, use console. For example..
 #     ```console
 #     cd ../my_dir
-#     ``` 
+#     ```
 # 03. Links across docs should ...
 #     A. Not involve line breaks.
 #     B. Use relative paths to docs in the same repo
@@ -63,7 +64,7 @@ nav:
 #     URL in ./docs/src/api/make_pages.py#L19 to your fork.
 # 04. To deploy this site on your fork, 
 #     A. declare a branch called gh-pages
-#     B. go to the your fork > settings > pages 
+#     B. go to the your fork > settings > pages
 #     C. direct pages to render from the gh-pages branch at root
 #     D. push a tag to your fork with the format test*.*.*
 #
@@ -109,7 +110,7 @@ plugins:
             line_length: 88
   - gen-files:
       scripts:
-      - ./src/api/make_pages.py
+        - ./src/api/make_pages.py
   - literate-nav:
       nav_file: navigation.md
   - exclude-search:
@@ -178,4 +179,4 @@ extra_css:
   - assets/stylesheets/extra.css
 
 extra_javascript:
-  - https://js-na1.hs-scripts.com/23133402.js  # HubSpot chatbot
+  - https://js-na1.hs-scripts.com/23133402.js # HubSpot chatbot

docs/src/concepts.md

@@ -107,6 +107,10 @@ is a table within the Element or a table connected to the Element.
 
 ![diagram](https://raw.githubusercontent.com/datajoint/element-array-ephys/main/images/attached_array_ephys_element_precluster.svg)
 
+### `ephys_no_curation` module
+
+![diagram](https://raw.githubusercontent.com/datajoint/element-array-ephys/main/images/attached_array_ephys_element_no_curation.svg)
+
 ### `subject` schema ([API docs](https://datajoint.com/docs/elements/element-animal/api/element_animal/subject))
 
 Although not required, most choose to connect the `Session` table to a `Subject` table.
@@ -156,6 +160,9 @@ Tables for storing probe or unit-level visualization results.
 | --- | --- |
 | ProbeLevelReport | A table to store drift map figures generated from each recording probe. |
 | UnitLevelReport | A table to store figures (waveforms, autocorrelogram, peak waveform + neighbors) generated for each unit. |
+| QualityMetricCutoffs | A table to store cut-off values for cluster quality metrics. |
+| QualityMetricSet | A manual table to match a set of cluster quality metric values with desired cut-offs. |
+| QualityMetricReport | A table to store quality metric figures. |
 
 ## Element Development
 

element_array_ephys/ephys_acute.py

@@ -1360,8 +1360,7 @@ def make(self, key):
         metrics_df = pd.read_csv(metric_fp)
         metrics_df.set_index("cluster_id", inplace=True)
         metrics_df.replace([np.inf, -np.inf], np.nan, inplace=True)
-        metrics_df.columns = map(str.lower, metrics_df.columns)
-
+        metrics_df.columns = metrics_df.columns.str.lower()
         metrics_list = [
             dict(metrics_df.loc[unit_key["unit"]], **unit_key)
             for unit_key in (CuratedClustering.Unit & key).fetch("KEY")

element_array_ephys/ephys_chronic.py

@@ -1289,8 +1289,7 @@ def make(self, key):
         metrics_df = pd.read_csv(metric_fp)
         metrics_df.set_index("cluster_id", inplace=True)
         metrics_df.replace([np.inf, -np.inf], np.nan, inplace=True)
-        metrics_df.columns = map(str.lower, metrics_df.columns)
-
+        metrics_df.columns = metrics_df.columns.str.lower()
         metrics_list = [
             dict(metrics_df.loc[unit_key["unit"]], **unit_key)
             for unit_key in (CuratedClustering.Unit & key).fetch("KEY")

element_array_ephys/ephys_no_curation.py

@@ -937,9 +937,9 @@ class Unit(dj.Part):
 
         Attributes:
             CuratedClustering (foreign key): CuratedClustering primary key.
-            unit (foreign key, int): Unique integer identifying a single unit.
-            probe.ElectrodeConfig.Electrode (dict): probe.ElectrodeConfig.Electrode primary key.
-            ClusteringQualityLabel (dict): CLusteringQualityLabel primary key.
+            unit (int): Unique integer identifying a single unit.
+            probe.ElectrodeConfig.Electrode (foreign key): probe.ElectrodeConfig.Electrode primary key.
+            ClusteringQualityLabel (foreign key): CLusteringQualityLabel primary key.
             spike_count (int): Number of spikes in this recording for this unit.
             spike_times (longblob): Spike times of this unit, relative to start time of EphysRecording.
             spike_sites (longblob): Array of electrode associated with each spike.
@@ -1300,8 +1300,7 @@ def make(self, key):
         metrics_df = pd.read_csv(metric_fp)
         metrics_df.set_index("cluster_id", inplace=True)
         metrics_df.replace([np.inf, -np.inf], np.nan, inplace=True)
-        metrics_df.columns = map(str.lower, metrics_df.columns)
-
+        metrics_df.columns = metrics_df.columns.str.lower()
         metrics_list = [
             dict(metrics_df.loc[unit_key["unit"]], **unit_key)
             for unit_key in (CuratedClustering.Unit & key).fetch("KEY")

element_array_ephys/ephys_report.py

@@ -159,6 +159,16 @@ def make(self, key):
 
 @schema
 class QualityMetricCutoffs(dj.Lookup):
+    """Cut-off values for unit quality metrics.
+
+    Attributes:
+        cutoffs_id (smallint): Unique ID for the cut-off values.
+        amplitude_cutoff_maximum (float): Optional. Amplitude cut-off.
+        presence_ratio_minimum (float): Optional. Presence ratio cut-off.
+        isi_violations_maximum (float): Optional. ISI violation ratio cut-off.
+        cutoffs_hash (uuid): uuid for the cut-off values.
+    """
+    
     definition = """
     cutoffs_id                    : smallint
     ---
@@ -218,6 +228,13 @@ def insert_new_cutoffs(
 
 @schema
 class QualityMetricSet(dj.Manual):
+    """Set of quality metric values for clusters and its cut-offs.
+
+    Attributes:
+        ephys.QualityMetrics (foreign key): ephys.QualityMetrics primary key.
+        QualityMetricCutoffs (foreign key): QualityMetricCutoffs primary key.
+    """
+    
     definition = """
     -> ephys.QualityMetrics
     -> QualityMetricCutoffs
@@ -226,6 +243,13 @@ class QualityMetricSet(dj.Manual):
 
 @schema
 class QualityMetricReport(dj.Computed):
+    """Table for storing quality metric figures.
+
+    Attributes:
+        QualityMetricSet (foreign key): QualityMetricSet primary key.
+        plot_grid (longblob): Plotly figure object.
+    """
+    
     definition = """
     -> QualityMetricSet
     ---

element_array_ephys/plotting/qc.py

@@ -28,14 +28,10 @@ def __init__(
             key (dict, optional): key from ephys.QualityMetric table. Defaults to None.
             scale (float, optional): Scale at which to render figure. Defaults to 1.4.
             fig_width (int, optional): Figure width in pixels. Defaults to 800.
-            amplitude_cutoff_maximum (float, optional): Cutoff for unit amplitude in
-                visualizations. Defaults to None.
-            presence_ratio_minimum (float, optional): Cutoff for presence ratio in
-                visualizations. Defaults to None.
-            isi_violations_maximum (float, optional): Cutoff for isi violations in
-                visualizations. Defaults to None.
-            dark_mode (bool, optional): Set background to black, foreground white.
-                Default False, black on white.
+            amplitude_cutoff_maximum (float, optional): Cutoff for unit amplitude in visualizations. Defaults to None.
+            presence_ratio_minimum (float, optional): Cutoff for presence ratio in visualizations. Defaults to None.
+            isi_violations_maximum (float, optional): Cutoff for isi violations in visualizations. Defaults to None.
+            dark_mode (bool, optional): Set background to black, foreground white. Default False, black on white.
         """
         self._ephys = ephys
         self._key = key
@@ -134,7 +130,7 @@ def _format_fig(
                 Figure to apply formatting. Defaults to empty.
             scale (float, optional): Scale to render figure. Defaults to scale from
                 class init, 1.
-            ratio (float, optional): Figure aspect ratio width/height . Defaults to 1.
+            ratio (float, optional): Figure aspect ratio width/height. Defaults to 1.
 
         Returns:
             go.Figure: Formatted figure
@@ -202,7 +198,7 @@ class initialization.
         return fig.add_trace(
             go.Scatter(
                 x=histogram_bins[:-1],
-                y=gaussian_filter1d(histogram, 1),  # TODO: remove smoothing
+                y=gaussian_filter1d(histogram, 1),  
                 mode="lines",
                 line=dict(color="rgb(0, 160, 223)", width=2 * scale),  # DataJoint Blue
                 hovertemplate="%{x:.2f}<br>%{y:.2f}<extra></extra>",
@@ -215,8 +211,7 @@ def get_single_fig(self, fig_name: str, scale: float = None) -> go.Figure:
 
         Args:
             fig_name (str): Name of figure to be rendered
-            scale (float, optional): Scale to render fig. Defaults to scale at class
-                init, 1.
+            scale (float, optional): Scale to render fig. Defaults to scale at class init, 1.
 
         Returns:
             go.Figure: Histogram plot
@@ -253,8 +248,7 @@ def get_grid(self, n_columns: int = 4, scale: float = 1.0) -> go.Figure:
 
         Args:
             n_columns (int, optional): Number of column in grid. Defaults to 4.
-            scale (float, optional): Scale to render fig. Defaults to scale at class
-                init, 1.
+            scale (float, optional): Scale to render fig. Defaults to scale at class init, 1.
 
         Returns:
             go.Figure: grid of available plots

element_array_ephys/version.py

@@ -1,2 +1,2 @@
 """Package metadata."""
-__version__ = "0.2.4"
+__version__ = "0.2.5"