|
1 | | -# isaacus-sagemaker-python |
2 | | -A Python library for interacting with AWS SageMaker AI deployments of the Isaacus API. |
| 1 | +# Isaacus-SageMaker Python integration |
| 2 | +The Isaacus-SageMaker Python integration enables users to interact with private SageMaker deployments of Isaacus legal AI models via the [Isaacus Python SDK](https://github.com/isaacus-dev/isaacus-python). |
| 3 | + |
| 4 | +This integration only requires a single line of code to be added to existing Isaacus API-based applications. |
| 5 | + |
| 6 | +## Installation 📦 |
| 7 | +This integration can be installed with `pip`: |
| 8 | +```sh |
| 9 | +pip install isaacus-sagemaker |
| 10 | +``` |
| 11 | + |
| 12 | +It also requires the `isaacus` package to be of any use: |
| 13 | +```sh |
| 14 | +pip install isaacus |
| 15 | +``` |
| 16 | + |
| 17 | +## Usage 👩💻 |
| 18 | +To use the Isaacus-SageMaker integration, import either `IsaacusSageMakerRuntimeHTTPClient` (for synchronous usage) or `AsyncIsaacusSageMakerRuntimeHTTPClient` (for asynchronous usage) from `isaacus_sagemaker` along with `IsaacusSageMakerRuntimeEndpoint` to define available SageMaker endpoints. |
| 19 | + |
| 20 | +Then, create an instance of the `Isaacus` or `AsyncIsaacus` client as you normally would, but also pass your SageMaker HTTP client as the `http_client` parameter. |
| 21 | + |
| 22 | +Below is an example of how you'd do that in practice: |
| 23 | + |
| 24 | +```python |
| 25 | +from isaacus import Isaacus, AsyncIsaacus |
| 26 | +from isaacus_sagemaker import IsaacusSageMakerRuntimeHTTPClient, AsyncIsaacusSageMakerRuntimeHTTPClient, \ |
| 27 | + IsaacusSageMakerRuntimeEndpoint |
| 28 | + |
| 29 | +endpoints = [ |
| 30 | + IsaacusSageMakerRuntimeEndpoint( |
| 31 | + name="my-sagemaker-endpoint", |
| 32 | + # region="us-west-2", # Optional, defaults to the client or AWS SDK default region |
| 33 | + # profile="my-aws-profile", # Optional, defaults to the client or AWS SDK default profile |
| 34 | + # models=["kanon-2-embedder"], # Optional, models supported by this endpoint, |
| 35 | + # # defaults to all models |
| 36 | + ) |
| 37 | +] |
| 38 | + |
| 39 | +client = Isaacus( |
| 40 | + http_client=IsaacusSageMakerRuntimeHTTPClient( |
| 41 | + endpoints=endpoints, |
| 42 | + # region="us-west-2", # Optional, defaults to AWS SDK default region |
| 43 | + # profile="my-aws-profile", # Optional, defaults to AWS SDK default profile |
| 44 | + # boto_session_kwargs={"aws_access_key_id": "...",}, # Optional, additional boto3 session kwargs |
| 45 | + #**{}, # Optional, additional httpx.Client kwargs |
| 46 | + ) |
| 47 | +) |
| 48 | + |
| 49 | +# For asynchronous usage: |
| 50 | +aclient = AsyncIsaacus( |
| 51 | + http_client=AsyncIsaacusSageMakerRuntimeHTTPClient( |
| 52 | + endpoints=endpoints, |
| 53 | + # region="us-west-2", # Optional, defaults to AWS SDK default region |
| 54 | + # profile="my-aws-profile", # Optional, defaults to AWS SDK default profile |
| 55 | + # boto_session_kwargs={"aws_access_key_id": "...",}, # Optional, additional boto3 session kwargs |
| 56 | + #**{}, # Optional, additional httpx.AsyncClient kwargs |
| 57 | + ) |
| 58 | +) |
| 59 | +``` |
| 60 | + |
| 61 | +Since Isaacus SageMaker deployments are private and hosted within your AWS account, no API key or base URL needs to be provided when constructing Isaacus SDK clients. |
| 62 | + |
| 63 | +Once you've set up your client, no further changes are needed to your existing code. |
| 64 | + |
| 65 | +## API 🧩 |
| 66 | +### `IsaacusSageMakerRuntimeEndpoint` |
| 67 | +```python |
| 68 | +class IsaacusSageMakerRuntimeEndpoint(msgspec.Struct, frozen=True): |
| 69 | + name: str |
| 70 | + """The name of the SageMaker endpoint.""" |
| 71 | + |
| 72 | + region: Optional[str] = None |
| 73 | + """The AWS region where the SageMaker endpoint is deployed. This overrides any region specified at the client-level.""" |
| 74 | + |
| 75 | + profile: Optional[str] = None |
| 76 | + """The AWS profile to use when accessing the SageMaker endpoint. This overrides any profile specified at the client-level.""" |
| 77 | + |
| 78 | + models: Optional[Sequence[str]] = None |
| 79 | + """The IDs of models served by this endpoint. If `None`, it is assumed the endpoint serves all models.""" |
| 80 | +``` |
| 81 | + |
| 82 | +### `IsaacusSageMakerRuntimeHTTPClient` |
| 83 | +```python |
| 84 | +class IsaacusSageMakerRuntimeHTTPClient(httpx.Client): |
| 85 | + """ |
| 86 | + A synchronous Isaacus SDK-compatible HTTP client that proxies requests to SageMaker-deployed Isaacus models through the SageMaker Runtime InvokeEndpoint (`/invocations`) API. |
| 87 | +
|
| 88 | + This client extends `httpx.Client`. |
| 89 | +
|
| 90 | + Arguments: |
| 91 | + `endpoints` (`Sequence[IsaacusSageMakerRuntimeEndpoint]`): A sequence of SageMaker endpoints to route requests to. |
| 92 | + `region` (`str`, optional): The AWS region where the SageMaker endpoints are deployed. Overriden by any region specified at the endpoint-level. Defaults to `None`, in which case the AWS SDK's default region resolution is used. |
| 93 | + `profile` (`str`, optional): The AWS profile to use when accessing the SageMaker endpoints. Overriden by any profile specified at the endpoint-level. Defaults to `None`, in which case the AWS SDK's default profile resolution is used. |
| 94 | + `boto_session_kwargs` (`Dict[str, Any]`, optional): Additional keyword arguments to pass to the `boto3.Session` constructor when creating the AWS session. Defaults to `None`. |
| 95 | + `**httpx_kwargs` (`Any`): Additional keyword arguments to pass to the `httpx.Client` constructor. |
| 96 | + """ |
| 97 | +``` |
| 98 | + |
| 99 | +### `AsyncIsaacusSageMakerRuntimeHTTPClient` |
| 100 | +```python |
| 101 | +class AsyncIsaacusSageMakerRuntimeHTTPClient(httpx.AsyncClient): |
| 102 | + """ |
| 103 | + An asynchronous Isaacus SDK-compatible HTTP client that proxies requests to SageMaker-deployed Isaacus models through the SageMaker Runtime InvokeEndpoint (`/invocations`) API. |
| 104 | +
|
| 105 | + This client extends `httpx.AsyncClient`. |
| 106 | +
|
| 107 | + Arguments: |
| 108 | + `endpoints` (`Sequence[IsaacusSageMakerRuntimeEndpoint]`): A sequence of SageMaker endpoints to route requests to. |
| 109 | + `region` (`str`, optional): The AWS region where the SageMaker endpoints are deployed. Overriden by any region specified at the endpoint-level. Defaults to `None`, in which case the AWS SDK's default region resolution is used. |
| 110 | + `profile` (`str`, optional): The AWS profile to use when accessing the SageMaker endpoints. Overriden by any profile specified at the endpoint-level. Defaults to `None`, in which case the AWS SDK's default profile resolution is used. |
| 111 | + `boto_session_kwargs` (`Dict[str, Any]`, optional): Additional keyword arguments to pass to the `boto3.Session` constructor when creating the AWS session. Defaults to `None`. |
| 112 | + `**httpx_kwargs` (`Any`): Additional keyword arguments to pass to the `httpx.Client` constructor. |
| 113 | + """ |
| 114 | +``` |
| 115 | + |
| 116 | +## How it works ⚙️ |
| 117 | +This integration works by intercepting Isaacus SDK HTTP requests and proxying them to an Isaacus API server through the SageMaker Runtime InvokeEndpoint (`/invocations`) API. |
| 118 | + |
| 119 | +When a request is made through the Isaacus SDK client, the custom HTTP client obtains the path, method, data, and headers from the original request and constructs a new request to the SageMaker endpoint's `/invocations` API that packages the original request details like so: |
| 120 | +```json |
| 121 | +{ |
| 122 | + "path": "/v1/embeddings", |
| 123 | + "method": "POST", |
| 124 | + "headers": { |
| 125 | + "Content-Type": "application/json", |
| 126 | + "Accept": "application/json" |
| 127 | + }, |
| 128 | + "data": { |
| 129 | + "model": "kanon-2-embedder", |
| 130 | + "texts": ["This is a confidentiality clause."], |
| 131 | + "task": "retrieval/document" |
| 132 | + } |
| 133 | +} |
| 134 | +``` |
| 135 | + |
| 136 | +The Isaacus API server internally forwards the request to the appropriate internal endpoint and then SageMaker returns the response back to the custom HTTP client. If an error is encountered, SageMaker returns its own error response, which the custom HTTP client translates back into the original error, ensuring maximum compatibility with existing Isaacus SDK-based applications. |
| 137 | + |
| 138 | +## Changelog 🔄 |
| 139 | +All notable changes to this integration are documented in the [CHANGELOG.md](CHANGELOG.md) file. This project adheres to [Keep a Changelog](https://keepachangelog.com/en/1.1.0/) and [Semantic Versioning](https://semver.org/spec/v2.0.0.html). |
| 140 | + |
| 141 | +## License 📄 |
| 142 | +In the spirit of open source, this integration is licensed under the [MIT License](LICENSE). |
0 commit comments