Feature/azure sql integration : Add Azure SQL integration support by SYCqi · Pull Request #2 · SU-IoT-Spring-26/api

SYCqi · 2026-03-07T19:36:24Z

This pull request adds Azure SQL support to the API.

Changes:

Added pyodbc dependency
Installed Microsoft ODBC Driver 18 in Dockerfile
Updated main.py to support SQL backend
Kept existing local file storage logic

This allows switching between local JSONL storage and Azure SQL.

Add Azure SQL integration to save occupancy data

Add pyodbc dependency for Azure SQL integration

Add Azure SQL support and ODBC driver installation - Installed Microsoft ODBC Driver 18 - Added unixodbc dependencies - Combined RUN commands for proper Docker build layering - Prepared backend for Azure SQL integration

Copilot

Pull request overview

Adds Azure SQL persistence support to the Occupancy API so occupancy results can be stored in an Azure SQL table in addition to the existing local/Blob storage paths.

Changes:

Add pyodbc dependency for ODBC connectivity.
Add Docker image steps to install Microsoft ODBC Driver 18 (and unixODBC deps).
Add SQL connection + insert logic in main.py and call it during /api/thermal ingestion.

Reviewed changes

Copilot reviewed 2 out of 3 changed files in this pull request and generated 7 comments.

File	Description
requirements.txt	Adds `pyodbc` dependency needed for SQL connectivity.
main.py	Introduces SQL connection helper + `occupancy_data` insert on each ingest.
Dockerfile	Installs unixODBC + Microsoft ODBC Driver 18 to support `pyodbc` in-container.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2026-03-08T13:42:49Z

main.py

+def _get_sql_connection():
+    """Create Azure SQL connection. Returns None if not configured."""
+    if not SQL_CONNECTION_STRING:
+        return None
+    try:
+        return pyodbc.connect(SQL_CONNECTION_STRING, timeout=10)
+    except Exception as e:


A new ODBC connection is created on every call, which can become a significant overhead under load. Consider reusing a cached connection (similar to the blob client pattern), enabling/confirming pyodbc pooling explicitly, and adding a simple backoff/"disable after failure" flag to avoid repeated connection attempts on persistent misconfiguration/outage.

Copilot · 2026-03-08T13:42:50Z

main.py

+        cursor = conn.cursor()
+        cursor.execute(
+            """
+            INSERT INTO occupancy_data
+            ([timestamp], sensor_id, occupancy, room_temperature,
+             people_clusters, fever_count, any_fever)
+            VALUES (?, ?, ?, ?, ?, ?, ?)
+            """,
+            entry["timestamp"],
+            entry["sensor_id"],
+            entry["occupancy"],
+            entry["room_temperature"],
+            entry["people_clusters"],
+            entry["fever_count"],
+            1 if entry["any_fever"] else 0,
+        )
+        conn.commit()
+        cursor.close()
+        conn.close()
+    except Exception as e:
+        print(f"Error saving occupancy data to Azure SQL: {e}")
+        try:
+            conn.close()
+        except Exception:
+            pass


If an exception occurs after creating the cursor, the cursor is never explicitly closed (only the connection is closed in the except). Use a finally block (or context managers) to guarantee both cursor and connection are closed, and consider calling rollback() on failures before closing to avoid leaving open transactions in pooled connections.

Copilot · 2026-03-08T13:42:50Z

main.py

    save_thermal_data(compact_data, latest_thermal_data, sensor_id)
    save_occupancy_data(occupancy_result)
+    save_occupancy_data_sql(occupancy_result, timestamp_iso=now_iso)


PR description mentions "switching" between local JSONL storage and Azure SQL, but the API still always writes local occupancy history when SAVE_THERMAL_DATA is true, and all history/stats endpoints still read only from local files. If Azure SQL is intended to be the backend, either add SQL-backed read paths (or a clear fallback), or update the description/env flags to reflect that SQL is an additional sink rather than a full backend.

Copilot · 2026-03-08T13:42:50Z

Dockerfile

+RUN apt-get update && apt-get install -y curl gnupg unixodbc unixodbc-dev && \
+    curl https://packages.microsoft.com/keys/microsoft.asc | apt-key add - && \
+    curl https://packages.microsoft.com/config/debian/12/prod.list > /etc/apt/sources.list.d/mssql-release.list && \
+    apt-get update && ACCEPT_EULA=Y apt-get install -y msodbcsql18 && \


Using apt-key add - is deprecated on modern Debian and may break in future base images. Import the Microsoft key into /etc/apt/keyrings/ and reference it via signed-by= in the repo list instead. Also consider --no-install-recommends and cleaning /var/lib/apt/lists/* after installs to keep the image smaller and reduce attack surface.

Suggested change

RUN apt-get update && apt-get install -y curl gnupg unixodbc unixodbc-dev && \

curl https://packages.microsoft.com/keys/microsoft.asc | apt-key add - && \

curl https://packages.microsoft.com/config/debian/12/prod.list > /etc/apt/sources.list.d/mssql-release.list && \

apt-get update && ACCEPT_EULA=Y apt-get install -y msodbcsql18 && \

RUN apt-get update && apt-get install -y --no-install-recommends curl gnupg ca-certificates unixodbc unixodbc-dev && \

mkdir -p /etc/apt/keyrings && \

curl https://packages.microsoft.com/keys/microsoft.asc | gpg --dearmor -o /etc/apt/keyrings/microsoft.gpg && \

chmod 644 /etc/apt/keyrings/microsoft.gpg && \

curl https://packages.microsoft.com/config/debian/12/prod.list | sed 's#^deb #deb [signed-by=/etc/apt/keyrings/microsoft.gpg] #' > /etc/apt/sources.list.d/mssql-release.list && \

apt-get update && ACCEPT_EULA=Y apt-get install -y --no-install-recommends msodbcsql18 && \

rm -rf /var/lib/apt/lists/* && \

Copilot · 2026-03-08T13:42:50Z

main.py

 import json
 import os
+import pyodbc


Importing pyodbc at module import time can prevent the API from starting in deployments where the ODBC runtime libraries aren’t present (common outside the Docker image), even if SQL saving is disabled. To keep SQL support optional, consider moving the import inside _get_sql_connection() and handling ImportError by disabling SQL saving with a clear (sanitized) log message.

Copilot · 2026-03-08T13:42:50Z

Dockerfile

@@ -5,7 +5,11 @@ WORKDIR /app

 # Install dependencies (no build tools needed for current deps)


The comment says "no build tools needed for current deps", but this layer now installs build/runtime dependencies (unixODBC, msodbcsql18). Update the comment to reflect the new rationale (ODBC driver + pyodbc support) to avoid confusion for future maintenance.

Suggested change

# Install dependencies (no build tools needed for current deps)

# Install system deps for ODBC/SQL Server (unixODBC, msodbcsql18) and Python requirements

Copilot · 2026-03-08T13:42:51Z

main.py

+    try:
+        return pyodbc.connect(SQL_CONNECTION_STRING, timeout=10)
+    except Exception as e:
+        print(f"Azure SQL connection failed: {e}")
+        return None


Avoid printing raw SQL connection exceptions here. pyodbc error strings can include server/user details (and in some cases parts of the connection string), which can leak sensitive info into logs. Prefer structured logging and a sanitized message (e.g., log an error code/class, or gate detailed errors behind a debug flag).

mandeeps · 2026-03-08T19:41:57Z

@copilot open a new pull request to apply changes based on the comments in this thread

Copilot · 2026-03-08T19:42:03Z

@mandeeps I've opened a new pull request, #3, to work on those changes. Once the pull request is ready, I'll request review from you.

…lazy import, and error handling Co-authored-by: mandeeps <3266584+mandeeps@users.noreply.github.com>

Fix Azure SQL integration: connection caching, lazy import, safe error handling, and Dockerfile hardening

SYCqi added 3 commits March 7, 2026 14:25

Update main.py

7c95e13

Add Azure SQL integration to save occupancy data

Update requirements.txt

048601d

Add pyodbc dependency for Azure SQL integration

Update Dockerfile

6715538

Add Azure SQL support and ODBC driver installation - Installed Microsoft ODBC Driver 18 - Added unixodbc dependencies - Combined RUN commands for proper Docker build layering - Prepared backend for Azure SQL integration

mandeeps requested a review from Copilot March 8, 2026 13:37

Copilot started reviewing on behalf of mandeeps March 8, 2026 13:38 View session

Copilot AI reviewed Mar 8, 2026

View reviewed changes

Initial plan

fb8e959

Copilot AI mentioned this pull request Mar 8, 2026

Fix Azure SQL integration: connection caching, lazy import, safe error handling, and Dockerfile hardening #3

Merged

Copilot AI and others added 2 commits March 8, 2026 19:45

Address review comments: improve Dockerfile, SQL connection caching, …

443c9e1

…lazy import, and error handling Co-authored-by: mandeeps <3266584+mandeeps@users.noreply.github.com>

Merge pull request #3 from SU-IoT-Spring-26/copilot/sub-pr-2

637c211

Fix Azure SQL integration: connection caching, lazy import, safe error handling, and Dockerfile hardening

mandeeps merged commit 84200de into main Mar 8, 2026

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Feature/azure sql integration : Add Azure SQL integration support#2

Feature/azure sql integration : Add Azure SQL integration support#2
mandeeps merged 6 commits intomainfrom
feature/azure-sql-integration

SYCqi commented Mar 7, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI Mar 8, 2026

Uh oh!

Copilot AI Mar 8, 2026

Uh oh!

Copilot AI Mar 8, 2026

Uh oh!

Copilot AI Mar 8, 2026

Uh oh!

Copilot AI Mar 8, 2026

Uh oh!

Copilot AI Mar 8, 2026

Uh oh!

Copilot AI Mar 8, 2026

Uh oh!

mandeeps commented Mar 8, 2026

Uh oh!

Copilot AI commented Mar 8, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

-RUN apt-get update && apt-get install -y curl gnupg unixodbc unixodbc-dev && \
-    curl https://packages.microsoft.com/keys/microsoft.asc | apt-key add - && \
-    curl https://packages.microsoft.com/config/debian/12/prod.list > /etc/apt/sources.list.d/mssql-release.list && \
-    apt-get update && ACCEPT_EULA=Y apt-get install -y msodbcsql18 && \
+RUN apt-get update && apt-get install -y --no-install-recommends curl gnupg ca-certificates unixodbc unixodbc-dev && \
+    mkdir -p /etc/apt/keyrings && \
+    curl https://packages.microsoft.com/keys/microsoft.asc | gpg --dearmor -o /etc/apt/keyrings/microsoft.gpg && \
+    chmod 644 /etc/apt/keyrings/microsoft.gpg && \
+    curl https://packages.microsoft.com/config/debian/12/prod.list | sed 's#^deb #deb [signed-by=/etc/apt/keyrings/microsoft.gpg] #' > /etc/apt/sources.list.d/mssql-release.list && \
+    apt-get update && ACCEPT_EULA=Y apt-get install -y --no-install-recommends msodbcsql18 && \
+    rm -rf /var/lib/apt/lists/* && \

		@@ -5,7 +5,11 @@ WORKDIR /app

		# Install dependencies (no build tools needed for current deps)

	# Install dependencies (no build tools needed for current deps)
	# Install system deps for ODBC/SQL Server (unixODBC, msodbcsql18) and Python requirements

Conversation

SYCqi commented Mar 7, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Copilot AI Mar 8, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 8, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 8, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 8, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 8, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 8, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 8, 2026

Choose a reason for hiding this comment

Uh oh!

mandeeps commented Mar 8, 2026

Uh oh!

Copilot AI commented Mar 8, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants