Commit 03f8ee9
authored
FEAT: Optimize executemany() performance (#138)
### ADO Work Item Reference
<!-- Insert your ADO Work Item ID below (e.g. AB#37452) -->
>
[AB#37935](https://sqlclientdrivers.visualstudio.com/c6d89619-62de-46a0-8b46-70b92a84d85e/_workitems/edit/37935)
-------------------------------------------------------------------
### Summary
<!-- Insert your Copilot Generated Summary below -->
This pull request introduces significant updates to the `executemany`
method in the `mssql_python` library to enable column-wise parameter
binding for batched SQL execution. It also includes enhancements to the
underlying C++ bindings to support this functionality. The most
important changes involve implementing column-wise binding logic, adding
helper functions for parameter buffer allocation, and exposing a new
`SQLExecuteMany` method for executing batched statements.
### Enhancements to Python `executemany` method:
*
[`mssql_python/cursor.py`](diffhunk://#diff-deceea46ae01082ce8400e14fa02f4b7585afb7b5ed9885338b66494f5f38280R623-R736):
Refactored the `executemany` method to use column-wise parameter binding
for improved performance and scalability. Added
`_transpose_rowwise_to_columnwise` helper function to convert row-wise
parameters into column-wise format.
### Updates to C++ bindings:
*
[`mssql_python/pybind/ddbc_bindings.cpp`](diffhunk://#diff-dde2297345718ec449a14e7dff91b7bb2342b008ecc071f562233646d71144a1R945-R1108):
Added `BindParameterArray` function to handle column-wise parameter
binding, supporting multiple data types such as `SQL_C_LONG`,
`SQL_C_DOUBLE`, and `SQL_C_WCHAR`.
*
[`mssql_python/pybind/ddbc_bindings.cpp`](diffhunk://#diff-dde2297345718ec449a14e7dff91b7bb2342b008ecc071f562233646d71144a1R945-R1108):
Introduced `SQLExecuteMany_wrap` function to enable batched execution of
SQL statements with multiple parameter sets.
*
[`mssql_python/pybind/ddbc_bindings.cpp`](diffhunk://#diff-dde2297345718ec449a14e7dff91b7bb2342b008ecc071f562233646d71144a1R2288):
Exposed the new `SQLExecuteMany` method to Python via `PYBIND11_MODULE`.
### Helper functions for buffer allocation:
*
[`mssql_python/pybind/ddbc_bindings.cpp`](diffhunk://#diff-dde2297345718ec449a14e7dff91b7bb2342b008ecc071f562233646d71144a1R188-R196):
Added `AllocateParamBufferArray` template function to allocate buffers
for column-wise parameter binding.
<!--
### PR Title Guide
> For feature requests
FEAT: (short-description)
> For non-feature requests like test case updates, config updates ,
dependency updates etc
CHORE: (short-description)
> For Fix requests
FIX: (short-description)
> For doc update requests
DOC: (short-description)
> For Formatting, indentation, or styling update
STYLE: (short-description)
> For Refactor, without any feature changes
REFACTOR: (short-description)
> For release related changes, without any feature changes
RELEASE: #<RELEASE_VERSION> (short-description)
-->1 parent 0374aa2 commit 03f8ee9
2 files changed
+443
-19
lines changed| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
21 | 21 | | |
22 | 22 | | |
23 | 23 | | |
24 | | - | |
25 | 24 | | |
26 | 25 | | |
27 | 26 | | |
| |||
631 | 630 | | |
632 | 631 | | |
633 | 632 | | |
| 633 | + | |
| 634 | + | |
| 635 | + | |
| 636 | + | |
| 637 | + | |
| 638 | + | |
| 639 | + | |
| 640 | + | |
| 641 | + | |
| 642 | + | |
| 643 | + | |
| 644 | + | |
| 645 | + | |
| 646 | + | |
| 647 | + | |
| 648 | + | |
| 649 | + | |
| 650 | + | |
| 651 | + | |
| 652 | + | |
| 653 | + | |
| 654 | + | |
| 655 | + | |
| 656 | + | |
| 657 | + | |
| 658 | + | |
| 659 | + | |
| 660 | + | |
| 661 | + | |
| 662 | + | |
| 663 | + | |
| 664 | + | |
| 665 | + | |
| 666 | + | |
| 667 | + | |
| 668 | + | |
| 669 | + | |
| 670 | + | |
| 671 | + | |
| 672 | + | |
| 673 | + | |
| 674 | + | |
| 675 | + | |
| 676 | + | |
| 677 | + | |
| 678 | + | |
| 679 | + | |
| 680 | + | |
634 | 681 | | |
635 | 682 | | |
636 | 683 | | |
637 | | - | |
| 684 | + | |
638 | 685 | | |
639 | 686 | | |
640 | 687 | | |
641 | 688 | | |
642 | 689 | | |
643 | 690 | | |
644 | 691 | | |
645 | | - | |
646 | | - | |
| 692 | + | |
647 | 693 | | |
648 | 694 | | |
649 | | - | |
650 | | - | |
651 | | - | |
652 | | - | |
653 | | - | |
654 | | - | |
655 | | - | |
656 | | - | |
657 | | - | |
658 | | - | |
| 695 | + | |
| 696 | + | |
| 697 | + | |
| 698 | + | |
| 699 | + | |
| 700 | + | |
| 701 | + | |
| 702 | + | |
| 703 | + | |
| 704 | + | |
| 705 | + | |
| 706 | + | |
| 707 | + | |
| 708 | + | |
659 | 709 | | |
660 | | - | |
661 | | - | |
662 | | - | |
663 | | - | |
664 | | - | |
| 710 | + | |
| 711 | + | |
| 712 | + | |
| 713 | + | |
| 714 | + | |
| 715 | + | |
| 716 | + | |
| 717 | + | |
| 718 | + | |
| 719 | + | |
| 720 | + | |
| 721 | + | |
| 722 | + | |
| 723 | + | |
| 724 | + | |
| 725 | + | |
| 726 | + | |
| 727 | + | |
| 728 | + | |
| 729 | + | |
665 | 730 | | |
666 | 731 | | |
667 | 732 | | |
| |||
0 commit comments