User guide andres and demo notebook for ingesting data

docs/_build/html/.buildinfo

@@ -1,4 +1,4 @@
 # Sphinx build info version 1
 # This file records the configuration used when building these files. When it is not found, a full rebuild will be done.
-config: c1bdb09dfc7517cfd2694c900e3ef834
+config: 97424fb823c1bd731ae02c38dfa1f88a
 tags: 645f666f9bcd5a90fca523b33c5a78b7

docs/_build/html/.buildinfo.bak

@@ -1,4 +1,4 @@
 # Sphinx build info version 1
-# This file hashes the configuration used when building these files. When it is not found, a full rebuild will be done.
-config: 440d00309a6559d712b22d1119c7eaa2
+# This file records the configuration used when building these files. When it is not found, a full rebuild will be done.
+config: c1bdb09dfc7517cfd2694c900e3ef834
 tags: 645f666f9bcd5a90fca523b33c5a78b7

docs/_build/html/_sources/index.rst.txt

@@ -1,16 +1,98 @@
-.. Daphme documentation master file, created by
-   sphinx-quickstart on Thu Aug 22 10:30:13 2024.
-   You can adapt this file completely to your liking, but it should at least
-   contain the root `toctree` directive.
+.. NOMAD documentation master file
 
-Welcome to Daphme's documentation!
-==================================
+NOMAD: Network for Open Mobility Analysis and Data
+===================================================
+
+.. image:: https://img.shields.io/badge/python-3.9+-blue.svg
+   :target: https://www.python.org/downloads/
+   :alt: Python 3.9+
+
+.. image:: https://img.shields.io/badge/license-MIT-green.svg
+   :target: https://opensource.org/licenses/MIT
+   :alt: MIT License
+
+**NOMAD** is an open-source Python library for end-to-end processing of large-scale GPS mobility data. 
+Part of the NSF-funded NOMAD research infrastructure project, it provides production-ready tools for 
+mobility data analysis with seamless scaling from local workstations to Spark clusters.
+
+NOMAD builds on previous software resources—like scikit-mobility, mobilkit, and trackintel—with the 
+goal of providing a single, production-ready library that covers the entire processing pipeline in a 
+form suitable for analysis of massive datasets and to aid in the replicability of existing research.
+
+All functions are implemented in Python with parallel equivalents in PySpark, enabling the same 
+analysis notebook to run on a workstation or a Spark cluster without API changes.
+
+Quick Links
+-----------
+
+* **GitHub**: `github.com/Watts-Lab/nomad <https://github.com/Watts-Lab/nomad>`_
+* **Website**: `nomad.seas.upenn.edu <https://nomad.seas.upenn.edu>`_
+* **Installation**: ``pip install git+https://github.com/Watts-Lab/nomad.git``
+
+Installation
+------------
+
+.. code-block:: bash
+
+   pip install git+https://github.com/Watts-Lab/nomad.git
 
 .. toctree::
    :maxdepth: 2
-   :caption: Contents:
+   :caption: User Guide
+   :hidden:
+
+   data_ingestion
+   filtering
+   tessellation
+   stop_detection
+   visit_attribution
+   metrics
+   colocation
+   aggregation
+   synthetic_data_generation
+
+.. toctree::
+   :maxdepth: 2
+   :caption: Examples
+   :hidden:
+
+   source/benchmarking_of_stop_detection_algorithms
+   source/ingesting-data
+
+Modules Overview
+----------------
+
+.. list-table::
+   :header-rows: 1
+   :widths: 30 70
+
+   * - Module
+     - Description
+   * - :doc:`data_ingestion`
+     - Read CSV, Parquet, GeoJSON; validate schemas; return Pandas or Spark DataFrames
+   * - :doc:`filtering`
+     - Assess coverage; filter by completeness, geography, time; handle projections
+   * - :doc:`tessellation`
+     - Map pings to H3, S2, or custom grids for grid-based algorithms
+   * - :doc:`stop_detection`
+     - DBSCAN, HDBSCAN, grid-based, and sequential (Lachesis) algorithms
+   * - :doc:`visit_attribution`
+     - Frequency and time-window heuristics for home/workplace inference
+   * - :doc:`metrics`
+     - Radius of gyration, travel distance, entropy, and related indicators
+   * - :doc:`colocation`
+     - Build proximity graphs from POI visits or spatial-temporal proximity
+   * - :doc:`aggregation`
+     - Differential privacy, k-anonymity, debiasing, and post-stratification
+   * - :doc:`synthetic_data_generation`
+     - EPR models and point process samplers for trajectory generation
+
+Community & Support
+-------------------
 
-   modules
+* **Issues**: Report bugs on `GitHub Issues <https://github.com/Watts-Lab/nomad/issues>`_
+* **Contribute**: See our contribution guidelines
+* **License**: MIT © University of Pennsylvania 2025
 
 Indices and tables
 ==================

docs/_build/html/_static/basic.css

@@ -741,6 +741,14 @@ abbr, acronym {
     cursor: help;
 }
 
+.translated {
+    background-color: rgba(207, 255, 207, 0.2)
+}
+
+.untranslated {
+    background-color: rgba(255, 207, 207, 0.2)
+}
+
 /* -- code displays --------------------------------------------------------- */
 
 pre {

docs/_build/html/_static/searchtools.js

@@ -513,11 +513,9 @@ const Search = {
     // perform the search on the required terms
     searchTerms.forEach((word) => {
       const files = [];
-      // find documents, if any, containing the query word in their text/title term indices
-      // use Object.hasOwnProperty to avoid mismatching against prototype properties
       const arr = [
-        { files: terms.hasOwnProperty(word) ? terms[word] : undefined, score: Scorer.term },
-        { files: titleTerms.hasOwnProperty(word) ? titleTerms[word] : undefined, score: Scorer.title },
+        { files: terms[word], score: Scorer.term },
+        { files: titleTerms[word], score: Scorer.title },
       ];
       // add support for partial matches
       if (word.length > 2) {
@@ -549,9 +547,8 @@ const Search = {
 
         // set score for the word in each file
         recordFiles.forEach((file) => {
-          if (!scoreMap.has(file)) scoreMap.set(file, new Map());
-          const fileScores = scoreMap.get(file);
-          fileScores.set(word, record.score);
+          if (!scoreMap.has(file)) scoreMap.set(file, {});
+          scoreMap.get(file)[word] = record.score;
         });
       });
 
@@ -590,7 +587,7 @@ const Search = {
         break;
 
       // select one (max) score for the file.
-      const score = Math.max(...wordList.map((w) => scoreMap.get(file).get(w)));
+      const score = Math.max(...wordList.map((w) => scoreMap.get(file)[w]));
       // add result to the result list
       results.push([
         docNames[file],

docs/_build/html/base_module.html

@@ -9,17 +9,19 @@
   <title>base_module package &mdash; NOMAD 0.0.1 documentation</title>
       <link rel="stylesheet" type="text/css" href="_static/pygments.css?v=b86133f3" />
       <link rel="stylesheet" type="text/css" href="_static/css/theme.css?v=e59714d7" />
+      <link rel="stylesheet" type="text/css" href="_static/sphinx-design.min.css?v=95c83b7e" />
 
 
       <script src="_static/jquery.js?v=5d32c60e"></script>
       <script src="_static/_sphinx_javascript_frameworks_compat.js?v=2cd50e6c"></script>
       <script src="_static/documentation_options.js?v=d45e8c67"></script>
       <script src="_static/doctools.js?v=9bcbadda"></script>
       <script src="_static/sphinx_highlight.js?v=dc90522c"></script>
+      <script src="_static/design-tabs.js?v=f930bc37"></script>
+      <script crossorigin="anonymous" integrity="sha256-Ae2Vz/4ePdIu6ZyI/5ZGsYnb+m0JlOmKPjt6XZ9JJkA=" src="https://cdnjs.cloudflare.com/ajax/libs/require.js/2.3.4/require.min.js"></script>
     <script src="_static/js/theme.js"></script>
     <link rel="index" title="Index" href="genindex.html" />
-    <link rel="search" title="Search" href="search.html" />
-    <link rel="prev" title="src" href="modules.html" /> 
+    <link rel="search" title="Search" href="search.html" /> 
 </head>
 
 <body class="wy-body-for-nav"> 
@@ -41,18 +43,22 @@
   </form>
 </div>
         </div><div class="wy-menu wy-menu-vertical" data-spy="affix" role="navigation" aria-label="Navigation menu">
-              <p class="caption" role="heading"><span class="caption-text">Contents:</span></p>
-<ul class="current">
-<li class="toctree-l1 current"><a class="reference internal" href="modules.html">src</a><ul class="current">
-<li class="toctree-l2 current"><a class="current reference internal" href="#">base_module package</a><ul>
-<li class="toctree-l3"><a class="reference internal" href="#submodules">Submodules</a></li>
-<li class="toctree-l3"><a class="reference internal" href="#base-module-doc-test-module">base_module.doc_test module</a></li>
-<li class="toctree-l3"><a class="reference internal" href="#base-module-setup-module">base_module.setup module</a></li>
-<li class="toctree-l3"><a class="reference internal" href="#module-contents">Module contents</a></li>
+              <p class="caption" role="heading"><span class="caption-text">User Guide</span></p>
+<ul>
+<li class="toctree-l1"><a class="reference internal" href="data_ingestion.html">Data Ingestion</a></li>
+<li class="toctree-l1"><a class="reference internal" href="filtering.html">Filtering &amp; Completeness</a></li>
+<li class="toctree-l1"><a class="reference internal" href="tessellation.html">Tessellation</a></li>
+<li class="toctree-l1"><a class="reference internal" href="stop_detection.html">Stop Detection</a></li>
+<li class="toctree-l1"><a class="reference internal" href="visit_attribution.html">Visit Attribution</a></li>
+<li class="toctree-l1"><a class="reference internal" href="metrics.html">Mobility Metrics</a></li>
+<li class="toctree-l1"><a class="reference internal" href="colocation.html">Co-location &amp; Contact Networks</a></li>
+<li class="toctree-l1"><a class="reference internal" href="aggregation.html">Aggregation &amp; Debiasing</a></li>
+<li class="toctree-l1"><a class="reference internal" href="synthetic_data_generation.html">Synthetic Data Generation</a></li>
 </ul>
-</li>
-</ul>
-</li>
+<p class="caption" role="heading"><span class="caption-text">Examples</span></p>
+<ul>
+<li class="toctree-l1"><a class="reference internal" href="source/benchmarking_of_stop_detection_algorithms.html">Benchmarking Stop Detection Algorithms</a></li>
+<li class="toctree-l1"><a class="reference internal" href="source/ingesting-data.html"><strong>Loading and Sampling Trajectory Data</strong></a></li>
 </ul>
 
         </div>
@@ -69,7 +75,6 @@
           <div role="navigation" aria-label="Page navigation">
   <ul class="wy-breadcrumbs">
       <li><a href="index.html" class="icon icon-home" aria-label="Home"></a></li>
-          <li class="breadcrumb-item"><a href="modules.html">src</a></li>
       <li class="breadcrumb-item active">base_module package</li>
       <li class="wy-breadcrumbs-aside">
             <a href="_sources/base_module.rst.txt" rel="nofollow"> View page source</a>
@@ -99,9 +104,7 @@ <h2>Module contents<a class="headerlink" href="#module-contents" title="Link to
 
            </div>
           </div>
-          <footer><div class="rst-footer-buttons" role="navigation" aria-label="Footer">
-        <a href="modules.html" class="btn btn-neutral float-left" title="src" accesskey="p" rel="prev"><span class="fa fa-arrow-circle-left" aria-hidden="true"></span> Previous</a>
-    </div>
+          <footer>
 
   <hr/>
 

docs/_build/html/genindex.html

@@ -8,13 +8,16 @@
   <title>Index &mdash; NOMAD 0.0.1 documentation</title>
       <link rel="stylesheet" type="text/css" href="_static/pygments.css?v=b86133f3" />
       <link rel="stylesheet" type="text/css" href="_static/css/theme.css?v=e59714d7" />
+      <link rel="stylesheet" type="text/css" href="_static/sphinx-design.min.css?v=95c83b7e" />
 
 
       <script src="_static/jquery.js?v=5d32c60e"></script>
       <script src="_static/_sphinx_javascript_frameworks_compat.js?v=2cd50e6c"></script>
       <script src="_static/documentation_options.js?v=d45e8c67"></script>
       <script src="_static/doctools.js?v=9bcbadda"></script>
       <script src="_static/sphinx_highlight.js?v=dc90522c"></script>
+      <script src="_static/design-tabs.js?v=f930bc37"></script>
+      <script crossorigin="anonymous" integrity="sha256-Ae2Vz/4ePdIu6ZyI/5ZGsYnb+m0JlOmKPjt6XZ9JJkA=" src="https://cdnjs.cloudflare.com/ajax/libs/require.js/2.3.4/require.min.js"></script>
     <script src="_static/js/theme.js"></script>
     <link rel="index" title="Index" href="#" />
     <link rel="search" title="Search" href="search.html" /> 
@@ -39,9 +42,22 @@
   </form>
 </div>
         </div><div class="wy-menu wy-menu-vertical" data-spy="affix" role="navigation" aria-label="Navigation menu">
-              <p class="caption" role="heading"><span class="caption-text">Contents:</span></p>
+              <p class="caption" role="heading"><span class="caption-text">User Guide</span></p>
 <ul>
-<li class="toctree-l1"><a class="reference internal" href="modules.html">src</a></li>
+<li class="toctree-l1"><a class="reference internal" href="data_ingestion.html">Data Ingestion</a></li>
+<li class="toctree-l1"><a class="reference internal" href="filtering.html">Filtering &amp; Completeness</a></li>
+<li class="toctree-l1"><a class="reference internal" href="tessellation.html">Tessellation</a></li>
+<li class="toctree-l1"><a class="reference internal" href="stop_detection.html">Stop Detection</a></li>
+<li class="toctree-l1"><a class="reference internal" href="visit_attribution.html">Visit Attribution</a></li>
+<li class="toctree-l1"><a class="reference internal" href="metrics.html">Mobility Metrics</a></li>
+<li class="toctree-l1"><a class="reference internal" href="colocation.html">Co-location &amp; Contact Networks</a></li>
+<li class="toctree-l1"><a class="reference internal" href="aggregation.html">Aggregation &amp; Debiasing</a></li>
+<li class="toctree-l1"><a class="reference internal" href="synthetic_data_generation.html">Synthetic Data Generation</a></li>
+</ul>
+<p class="caption" role="heading"><span class="caption-text">Examples</span></p>
+<ul>
+<li class="toctree-l1"><a class="reference internal" href="source/benchmarking_of_stop_detection_algorithms.html">Benchmarking Stop Detection Algorithms</a></li>
+<li class="toctree-l1"><a class="reference internal" href="source/ingesting-data.html"><strong>Loading and Sampling Trajectory Data</strong></a></li>
 </ul>
 
         </div>