Commit ba1eab3
authored
Generalise Backend Layer (databricks#604)
* Separate Session related functionality from Connection class (databricks#571)
* decouple session class from existing Connection
ensure maintenance of current APIs of Connection while delegating
responsibility
Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
* add open property to Connection to ensure maintenance of existing API
Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
* update unit tests to address ThriftBackend through session instead of through Connection
Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
* chore: move session specific tests from test_client to test_session
Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
* formatting (black)
as in CONTRIBUTING.md
Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
* use connection open property instead of long chain through session
Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
* trigger integration workflow
Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
* fix: ensure open attribute of Connection never fails
in case the openSession takes long, the initialisation of the session
will not complete immediately. This could make the session attribute
inaccessible. If the Connection is deleted in this time, the open()
check will throw because the session attribute does not exist. Thus, we
default to the Connection being closed in this case. This was not an
issue before because open was a direct attribute of the Connection
class. Caught in the integration tests.
Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
* fix: de-complicate earlier connection open logic
earlier, one of the integration tests was failing because 'session was
not an attribute of Connection'. This is likely tied to a local
configuration issue related to unittest that was causing an error in the
test suite itself. The tests are now passing without checking for the
session attribute.
databricks@c676f9b
Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
* Revert "fix: de-complicate earlier connection open logic"
This reverts commit d6b1b19.
Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
* [empty commit] attempt to trigger ci e2e workflow
Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
* Update CODEOWNERS (databricks#562)
new codeowners
Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
* Enhance Cursor close handling and context manager exception management to prevent server side resource leaks (databricks#554)
* Enhance Cursor close handling and context manager exception management
* tests
* fmt
* Fix Cursor.close() to properly handle CursorAlreadyClosedError
* Remove specific test message from Cursor.close() error handling
* Improve error handling in connection and cursor context managers to ensure proper closure during exceptions, including KeyboardInterrupt. Add tests for nested cursor management and verify operation closure on server-side errors.
* add
* add
Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
* PECOBLR-86 improve logging on python driver (databricks#556)
* PECOBLR-86 Improve logging for debug level
Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com>
* PECOBLR-86 Improve logging for debug level
Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com>
* fixed format
Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com>
* used lazy logging
Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com>
* changed debug to error logs
Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com>
* used lazy logging
Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com>
---------
Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com>
Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
* Revert "Merge remote-tracking branch 'upstream/sea-migration' into decouple-session"
This reverts commit dbb2ec5, reversing
changes made to 7192f11.
Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
* Reapply "Merge remote-tracking branch 'upstream/sea-migration' into decouple-session"
This reverts commit bdb8381.
Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
* fix: separate session opening logic from instantiation
ensures correctness of self.session.open call in Connection
Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
* fix: use is_open attribute to denote session availability
Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
* fix: access thrift backend through session
Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
* chore: use get_handle() instead of private session attribute in client
Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
* formatting (black)
Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
* fix: remove accidentally removed assertions
Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
---------
Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com>
Co-authored-by: Jothi Prakash <jothi.prakash@databricks.com>
Co-authored-by: Madhav Sainanee <madhav.sainanee@databricks.com>
Co-authored-by: Sai Shree Pradhan <saishree.pradhan@databricks.com>
* Introduce Backend Interface (DatabricksClient) (databricks#573)
NOTE: the `test_complex_types` e2e test was not working at the time of this merge. The test must be triggered when the test is back up and running as intended.
* remove excess logs, assertions, instantiations
large merge artifacts
Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
* formatting (black) + remove excess log (merge artifact)
Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
* fix typing
Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
* remove un-necessary check
Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
* remove un-necessary replace call
Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
* introduce __str__ methods for CommandId and SessionId
Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
* docstrings for DatabricksClient interface
Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
* stronger typing of Cursor and ExecuteResponse
Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
* remove utility functions from backend interface, fix circular import
Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
* rename info to properties
Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
* newline for cleanliness
Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
* fix circular import
Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
* formatting (black)
Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
* to_hex_id -> get_hex_id
Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
* better comment on protocol version getter
Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
* formatting (black)
Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
* move guid to hex id to new utils module
Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
* formatting (black)
Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
* move staging allowed local path to connection props
Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
* add strong return type for execute_command
Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
* skip auth, error handling in databricksclient interface
Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
* chore: docstring + line width
Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
* get_id -> get_guid
Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
* chore: docstring
Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
* fix: to_hex_id -> to_hex_guid
Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
---------
Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
* Implement ResultSet Abstraction (backend interfaces for fetch phase) (databricks#574)
* ensure backend client returns a ResultSet type in backend tests
Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
* formatting (black)
Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
* newline for cleanliness
Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
* fix circular import
Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
* formatting (black)
Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
* to_hex_id -> get_hex_id
Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
* better comment on protocol version getter
Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
* formatting (black)
Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
* stricter typing for cursor
Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
* correct typing
Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
* correct tests and merge artifacts
Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
* remove accidentally modified workflow files
remnants of old merge
Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
* chore: remove accidentally modified workflow files
Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
* add back accidentally removed docstrings
Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
* clean up docstrings
Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
* log hex
Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
* remove unnecessary _replace call
Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
* add __str__ for CommandId
Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
* take TOpenSessionResp in get_protocol_version to maintain existing interface
Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
* active_op_handle -> active_mmand_id
Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
* ensure None returned for close_command
Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
* account for ResultSet return in new pydocs
Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
* pydoc for types
Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
* move common state to ResultSet aprent
Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
* stronger typing in resultSet behaviour
Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
* remove redundant patch in test
Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
* add has_been_closed_server_side assertion
Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
* remove redundancies in tests
Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
* more robust close check
Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
* use normalised state in e2e test
Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
* simplify corrected test
Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
* add line gaps after multi-line pydocs for consistency
Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
* use normalised CommandState type in ExecuteResponse
Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
---------
Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
* remove un-necessary initialisation assertions
Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
* remove un-necessary line break s
Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
* more un-necessary line breaks
Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
* constrain diff of test_closing_connection_closes_commands
Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
* reduce diff of test_closing_connection_closes_commands
Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
* use pytest-like assertions for test_closing_connection_closes_commands
Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
* ensure command_id is not None
Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
* line breaks after multi-line pyfocs
Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
* ensure non null operationHandle for commandId creation
Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
* use command_id methods instead of explicit guid_to_hex_id conversion
Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
* remove un-necessary artifacts in test_session, add back assertion
Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
* add from __future__ import annotations to remove string literals around forward refs, remove some unused imports
Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
* move docstring of DatabricksClient within class
Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
* move ThriftResultSet import to top of file
Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
* make backend/utils __init__ file empty
Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
* use from __future__ import annotations to remove string literals around Cursor
Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
* use lazy logging
Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
* replace getters with property tag
Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
* set active_command_id to None, not active_op_handle
Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
* align test_session with pytest instead of unittest
Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
* formatting (black)
Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
* remove repetition from Session.__init__
Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
* mention that if catalog / schema name is None, we fetch across all
Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
* mention fetching across all tables if null table name
Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
* remove lazy import of ThriftResultSet
Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
* remove unused import
Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
* better docstrings
Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
* clarified role of cursor in docstring
Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
---------
Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>1 parent 576eafc commit ba1eab3
File tree
17 files changed
+2182
-1031
lines changed- src/databricks/sql
- backend
- utils
- tests/unit
17 files changed
+2182
-1031
lines changed| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
| 82 | + | |
| 83 | + | |
| 84 | + | |
| 85 | + | |
| 86 | + | |
| 87 | + | |
| 88 | + | |
| 89 | + | |
| 90 | + | |
| 91 | + | |
| 92 | + | |
| 93 | + | |
| 94 | + | |
| 95 | + | |
| 96 | + | |
| 97 | + | |
| 98 | + | |
| 99 | + | |
| 100 | + | |
| 101 | + | |
| 102 | + | |
| 103 | + | |
| 104 | + | |
| 105 | + | |
| 106 | + | |
| 107 | + | |
| 108 | + | |
| 109 | + | |
| 110 | + | |
| 111 | + | |
| 112 | + | |
| 113 | + | |
| 114 | + | |
| 115 | + | |
| 116 | + | |
| 117 | + | |
| 118 | + | |
| 119 | + | |
| 120 | + | |
| 121 | + | |
| 122 | + | |
| 123 | + | |
| 124 | + | |
| 125 | + | |
| 126 | + | |
| 127 | + | |
| 128 | + | |
| 129 | + | |
| 130 | + | |
| 131 | + | |
| 132 | + | |
| 133 | + | |
| 134 | + | |
| 135 | + | |
| 136 | + | |
| 137 | + | |
| 138 | + | |
| 139 | + | |
| 140 | + | |
| 141 | + | |
| 142 | + | |
| 143 | + | |
| 144 | + | |
| 145 | + | |
| 146 | + | |
| 147 | + | |
| 148 | + | |
| 149 | + | |
| 150 | + | |
| 151 | + | |
| 152 | + | |
| 153 | + | |
| 154 | + | |
| 155 | + | |
| 156 | + | |
| 157 | + | |
| 158 | + | |
| 159 | + | |
| 160 | + | |
| 161 | + | |
| 162 | + | |
| 163 | + | |
| 164 | + | |
| 165 | + | |
| 166 | + | |
| 167 | + | |
| 168 | + | |
| 169 | + | |
| 170 | + | |
| 171 | + | |
| 172 | + | |
| 173 | + | |
| 174 | + | |
| 175 | + | |
| 176 | + | |
| 177 | + | |
| 178 | + | |
| 179 | + | |
| 180 | + | |
| 181 | + | |
| 182 | + | |
| 183 | + | |
| 184 | + | |
| 185 | + | |
| 186 | + | |
| 187 | + | |
| 188 | + | |
| 189 | + | |
| 190 | + | |
| 191 | + | |
| 192 | + | |
| 193 | + | |
| 194 | + | |
| 195 | + | |
| 196 | + | |
| 197 | + | |
| 198 | + | |
| 199 | + | |
| 200 | + | |
| 201 | + | |
| 202 | + | |
| 203 | + | |
| 204 | + | |
| 205 | + | |
| 206 | + | |
| 207 | + | |
| 208 | + | |
| 209 | + | |
| 210 | + | |
| 211 | + | |
| 212 | + | |
| 213 | + | |
| 214 | + | |
| 215 | + | |
| 216 | + | |
| 217 | + | |
| 218 | + | |
| 219 | + | |
| 220 | + | |
| 221 | + | |
| 222 | + | |
| 223 | + | |
| 224 | + | |
| 225 | + | |
| 226 | + | |
| 227 | + | |
| 228 | + | |
| 229 | + | |
| 230 | + | |
| 231 | + | |
| 232 | + | |
| 233 | + | |
| 234 | + | |
| 235 | + | |
| 236 | + | |
| 237 | + | |
| 238 | + | |
| 239 | + | |
| 240 | + | |
| 241 | + | |
| 242 | + | |
| 243 | + | |
| 244 | + | |
| 245 | + | |
| 246 | + | |
| 247 | + | |
| 248 | + | |
| 249 | + | |
| 250 | + | |
| 251 | + | |
| 252 | + | |
| 253 | + | |
| 254 | + | |
| 255 | + | |
| 256 | + | |
| 257 | + | |
| 258 | + | |
| 259 | + | |
| 260 | + | |
| 261 | + | |
| 262 | + | |
| 263 | + | |
| 264 | + | |
| 265 | + | |
| 266 | + | |
| 267 | + | |
| 268 | + | |
| 269 | + | |
| 270 | + | |
| 271 | + | |
| 272 | + | |
| 273 | + | |
| 274 | + | |
| 275 | + | |
| 276 | + | |
| 277 | + | |
| 278 | + | |
| 279 | + | |
| 280 | + | |
| 281 | + | |
| 282 | + | |
| 283 | + | |
| 284 | + | |
| 285 | + | |
| 286 | + | |
| 287 | + | |
| 288 | + | |
| 289 | + | |
| 290 | + | |
| 291 | + | |
| 292 | + | |
| 293 | + | |
| 294 | + | |
| 295 | + | |
| 296 | + | |
| 297 | + | |
| 298 | + | |
| 299 | + | |
| 300 | + | |
| 301 | + | |
| 302 | + | |
| 303 | + | |
| 304 | + | |
| 305 | + | |
| 306 | + | |
| 307 | + | |
| 308 | + | |
| 309 | + | |
| 310 | + | |
| 311 | + | |
| 312 | + | |
| 313 | + | |
| 314 | + | |
| 315 | + | |
| 316 | + | |
| 317 | + | |
| 318 | + | |
| 319 | + | |
| 320 | + | |
| 321 | + | |
| 322 | + | |
| 323 | + | |
| 324 | + | |
| 325 | + | |
| 326 | + | |
| 327 | + | |
| 328 | + | |
| 329 | + | |
| 330 | + | |
| 331 | + | |
| 332 | + | |
| 333 | + | |
| 334 | + | |
| 335 | + | |
| 336 | + | |
| 337 | + | |
| 338 | + | |
| 339 | + | |
| 340 | + | |
| 341 | + | |
| 342 | + | |
| 343 | + | |
| 344 | + | |
| 345 | + | |
0 commit comments