Skip to content

Commit 8a508b6

Browse files
gargsaumyagargsaumya
authored
FEAT: Complex Datatype support-XML (microsoft#293)
### Work Item / Issue Reference <!-- IMPORTANT: Please follow the PR template guidelines below. For mssql-python maintainers: Insert your ADO Work Item ID below (e.g. AB#37452) For external contributors: Insert Github Issue number below (e.g. microsoft#149) Only one reference is required - either GitHub issue OR ADO Work Item. --> <!-- mssql-python maintainers: ADO Work Item --> > [AB#38821](https://sqlclientdrivers.visualstudio.com/c6d89619-62de-46a0-8b46-70b92a84d85e/_workitems/edit/38821) <!-- External contributors: GitHub Issue --> > GitHub Issue: #<ISSUE_NUMBER> ------------------------------------------------------------------- ### Summary <!-- Insert your summary of changes below. Minimum 10 characters required. --> This pull request adds comprehensive support for the SQL Server `XML` data type to the Python MSSQL driver, ensuring proper handling for insertion, retrieval, batching, and streaming of XML data. It also introduces a suite of tests to verify correct XML behavior, including edge cases like empty, large, and malformed XML values. ### MSSQL Driver Enhancements * Added support for the `SQL_SS_XML` data type throughout the driver, including binding, fetching, and row size calculations, so that XML columns are handled correctly during data operations. [[1]](diffhunk://#diff-dde2297345718ec449a14e7dff91b7bb2342b008ecc071f562233646d71144a1R24) [[2]](diffhunk://#diff-dde2297345718ec449a14e7dff91b7bb2342b008ecc071f562233646d71144a1R2529-R2534) [[3]](diffhunk://#diff-dde2297345718ec449a14e7dff91b7bb2342b008ecc071f562233646d71144a1R2992) [[4]](diffhunk://#diff-dde2297345718ec449a14e7dff91b7bb2342b008ecc071f562233646d71144a1R3195-R3198) [[5]](diffhunk://#diff-dde2297345718ec449a14e7dff91b7bb2342b008ecc071f562233646d71144a1R3412) * Updated logic in `FetchMany_wrap` and `FetchAll_wrap` to treat XML columns as LOBs, enabling efficient streaming for large XML values. [[1]](diffhunk://#diff-dde2297345718ec449a14e7dff91b7bb2342b008ecc071f562233646d71144a1L3504-R3517) [[2]](diffhunk://#diff-dde2297345718ec449a14e7dff91b7bb2342b008ecc071f562233646d71144a1L3626-R3639) ### Test Coverage for XML Support * Added multiple tests in `tests/test_004_cursor.py` to verify XML handling, including basic insert/fetch, empty/null values, large XML streaming, batch inserts, and error handling for malformed XML input. <!-- ### PR Title Guide > For feature requests FEAT: (short-description) > For non-feature requests like test case updates, config updates , dependency updates etc CHORE: (short-description) > For Fix requests FIX: (short-description) > For doc update requests DOC: (short-description) > For Formatting, indentation, or styling update STYLE: (short-description) > For Refactor, without any feature changes REFACTOR: (short-description) > For release related changes, without any feature changes RELEASE: #<RELEASE_VERSION> (short-description) ### Contribution Guidelines External contributors: - Create a GitHub issue first: https://github.com/microsoft/mssql-python/issues/new - Link the GitHub issue in the "GitHub Issue" section above - Follow the PR title format and provide a meaningful summary mssql-python maintainers: - Create an ADO Work Item following internal processes - Link the ADO Work Item in the "ADO Work Item" section above - Follow the PR title format and provide a meaningful summary -->
1 parent 5a0bc59 commit 8a508b6

File tree

2 files changed

+94
-2
lines changed

2 files changed

+94
-2
lines changed

‎mssql_python/pybind/ddbc_bindings.cpp‎

Lines changed: 10 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -21,6 +21,7 @@
2121
#define SQL_SS_TIMESTAMPOFFSET (-155)
2222
#define SQL_C_SS_TIMESTAMPOFFSET (0x4001)
2323
#define MAX_DIGITS_IN_NUMERIC 64
24+
#define SQL_SS_XML (-152)
2425

2526
#define STRINGIFY_FOR_CASE(x) \
2627
case x: \
@@ -2525,6 +2526,12 @@ SQLRETURN SQLGetData_wrap(SqlHandlePtr StatementHandle, SQLUSMALLINT colCount, p
25252526
}
25262527
break;
25272528
}
2529+
case SQL_SS_XML:
2530+
{
2531+
LOG("Streaming XML for column {}", i);
2532+
row.append(FetchLobColumnData(hStmt, i, SQL_C_WCHAR, true, false));
2533+
break;
2534+
}
25282535
case SQL_WCHAR:
25292536
case SQL_WVARCHAR:
25302537
case SQL_WLONGVARCHAR: {
@@ -3395,6 +3402,7 @@ size_t calculateRowSize(py::list& columnNames, SQLUSMALLINT numCols) {
33953402
case SQL_LONGVARCHAR:
33963403
rowSize += columnSize;
33973404
break;
3405+
case SQL_SS_XML:
33983406
case SQL_WCHAR:
33993407
case SQL_WVARCHAR:
34003408
case SQL_WLONGVARCHAR:
@@ -3499,7 +3507,7 @@ SQLRETURN FetchMany_wrap(SqlHandlePtr StatementHandle, py::list& rows, int fetch
34993507

35003508
if ((dataType == SQL_WVARCHAR || dataType == SQL_WLONGVARCHAR ||
35013509
dataType == SQL_VARCHAR || dataType == SQL_LONGVARCHAR ||
3502-
dataType == SQL_VARBINARY || dataType == SQL_LONGVARBINARY) &&
3510+
dataType == SQL_VARBINARY || dataType == SQL_LONGVARBINARY || dataType == SQL_SS_XML) &&
35033511
(columnSize == 0 || columnSize == SQL_NO_TOTAL || columnSize > SQL_MAX_LOB_SIZE)) {
35043512
lobColumns.push_back(i + 1); // 1-based
35053513
}
@@ -3621,7 +3629,7 @@ SQLRETURN FetchAll_wrap(SqlHandlePtr StatementHandle, py::list& rows) {
36213629

36223630
if ((dataType == SQL_WVARCHAR || dataType == SQL_WLONGVARCHAR ||
36233631
dataType == SQL_VARCHAR || dataType == SQL_LONGVARCHAR ||
3624-
dataType == SQL_VARBINARY || dataType == SQL_LONGVARBINARY) &&
3632+
dataType == SQL_VARBINARY || dataType == SQL_LONGVARBINARY || dataType == SQL_SS_XML) &&
36253633
(columnSize == 0 || columnSize == SQL_NO_TOTAL || columnSize > SQL_MAX_LOB_SIZE)) {
36263634
lobColumns.push_back(i + 1); // 1-based
36273635
}

‎tests/test_004_cursor.py‎

Lines changed: 84 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -11409,6 +11409,90 @@ def test_datetime_string_parameter_binding(cursor, db_connection):
1140911409
drop_table_if_exists(cursor, table_name)
1141011410
db_connection.commit()
1141111411

11412+
SMALL_XML = "<root><item>1</item></root>"
11413+
LARGE_XML = "<root>" + "".join(f"<item>{i}</item>" for i in range(10000)) + "</root>"
11414+
EMPTY_XML = ""
11415+
INVALID_XML = "<root><item></root>" # malformed
11416+
11417+
def test_xml_basic_insert_fetch(cursor, db_connection):
11418+
"""Test insert and fetch of a small XML value."""
11419+
try:
11420+
cursor.execute("CREATE TABLE #pytest_xml_basic (id INT PRIMARY KEY IDENTITY(1,1), xml_col XML NULL);")
11421+
db_connection.commit()
11422+
11423+
cursor.execute("INSERT INTO #pytest_xml_basic (xml_col) VALUES (?);", SMALL_XML)
11424+
db_connection.commit()
11425+
11426+
row = cursor.execute("SELECT xml_col FROM #pytest_xml_basic;").fetchone()
11427+
assert row[0] == SMALL_XML
11428+
finally:
11429+
cursor.execute("DROP TABLE IF EXISTS #pytest_xml_basic;")
11430+
db_connection.commit()
11431+
11432+
11433+
def test_xml_empty_and_null(cursor, db_connection):
11434+
"""Test insert and fetch of empty XML and NULL values."""
11435+
try:
11436+
cursor.execute("CREATE TABLE #pytest_xml_empty_null (id INT PRIMARY KEY IDENTITY(1,1), xml_col XML NULL);")
11437+
db_connection.commit()
11438+
11439+
cursor.execute("INSERT INTO #pytest_xml_empty_null (xml_col) VALUES (?);", EMPTY_XML)
11440+
cursor.execute("INSERT INTO #pytest_xml_empty_null (xml_col) VALUES (?);", None)
11441+
db_connection.commit()
11442+
11443+
rows = [r[0] for r in cursor.execute("SELECT xml_col FROM #pytest_xml_empty_null ORDER BY id;").fetchall()]
11444+
assert rows[0] == EMPTY_XML
11445+
assert rows[1] is None
11446+
finally:
11447+
cursor.execute("DROP TABLE IF EXISTS #pytest_xml_empty_null;")
11448+
db_connection.commit()
11449+
11450+
11451+
def test_xml_large_insert(cursor, db_connection):
11452+
"""Test insert and fetch of a large XML value to verify streaming/DAE."""
11453+
try:
11454+
cursor.execute("CREATE TABLE #pytest_xml_large (id INT PRIMARY KEY IDENTITY(1,1), xml_col XML NULL);")
11455+
db_connection.commit()
11456+
11457+
cursor.execute("INSERT INTO #pytest_xml_large (xml_col) VALUES (?);", LARGE_XML)
11458+
db_connection.commit()
11459+
11460+
row = cursor.execute("SELECT xml_col FROM #pytest_xml_large;").fetchone()
11461+
assert row[0] == LARGE_XML
11462+
finally:
11463+
cursor.execute("DROP TABLE IF EXISTS #pytest_xml_large;")
11464+
db_connection.commit()
11465+
11466+
11467+
def test_xml_batch_insert(cursor, db_connection):
11468+
"""Test batch insert (executemany) of multiple XML values."""
11469+
try:
11470+
cursor.execute("CREATE TABLE #pytest_xml_batch (id INT PRIMARY KEY IDENTITY(1,1), xml_col XML NULL);")
11471+
db_connection.commit()
11472+
11473+
xmls = [f"<root><item>{i}</item></root>" for i in range(5)]
11474+
cursor.executemany("INSERT INTO #pytest_xml_batch (xml_col) VALUES (?);", [(x,) for x in xmls])
11475+
db_connection.commit()
11476+
11477+
rows = [r[0] for r in cursor.execute("SELECT xml_col FROM #pytest_xml_batch ORDER BY id;").fetchall()]
11478+
assert rows == xmls
11479+
finally:
11480+
cursor.execute("DROP TABLE IF EXISTS #pytest_xml_batch;")
11481+
db_connection.commit()
11482+
11483+
11484+
def test_xml_malformed_input(cursor, db_connection):
11485+
"""Verify driver raises error for invalid XML input."""
11486+
try:
11487+
cursor.execute("CREATE TABLE #pytest_xml_invalid (id INT PRIMARY KEY IDENTITY(1,1), xml_col XML NULL);")
11488+
db_connection.commit()
11489+
11490+
with pytest.raises(Exception):
11491+
cursor.execute("INSERT INTO #pytest_xml_invalid (xml_col) VALUES (?);", INVALID_XML)
11492+
finally:
11493+
cursor.execute("DROP TABLE IF EXISTS #pytest_xml_invalid;")
11494+
db_connection.commit()
11495+
1141211496
def test_close(db_connection):
1141311497
"""Test closing the cursor"""
1141411498
try:

0 commit comments

Comments
 (0)