diff --git a/LICENSE b/LICENSE index 60c7e25..fbffc49 100644 --- a/LICENSE +++ b/LICENSE @@ -1,6 +1,6 @@ MIT License -Copyright (c) 2023 thot-experiment +Copyright (c) 2024 Hallucinate, LLC Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal diff --git a/README.md b/README.md index 6039e80..b43942c 100644 --- a/README.md +++ b/README.md @@ -1,25 +1,23 @@ -# oobapi-ws -[oobabooga/text-generation-webui](https://github.com/oobabooga/text-generation-webui) websocket api wrapper +# oobapi-stream +[oobabooga/text-generation-webui](https://github.com/oobabooga/text-generation-webui) streaming api wrapper - This is a simple isomorphic wrapper for the oobabooga websocket api (must be enabled when [launching ooba](https://github.com/oobabooga/text-generation-webui#api) with `--api`) + This is a simple isomorphic wrapper for the oobabooga api that handles the necessary HTTP shenanigans to stream token predictions to you as they come instead of getting a result all at once. To use it, API mode must be enabled when [launching ooba](https://github.com/oobabooga/text-generation-webui#api) by using `--api` ## Usage - As this wrapper is meant to function the same in the browser as well as in node it must be passed a `WebSocket` object when instantiated. This can be either the browser context `WebSocket` or one from the [`ws` module](https://github.com/websockets/ws). See `node_example.mjs` and `browser_example.html` for more details. - Ｗhen instantiating the api you can specify a host and a port, as well as default generation options． ```js - import ooba from 'oobapi-ws' + import ooba from 'oobapi-stream' const generation_options = {max_new_tokens: 1000} const host = 'remote.host.net' const port = 1337 - const api = await ooba(WebSocket)({host,port}, generation_options) + const api = ooba({host,port}, generation_options) ``` or if you're running ooba locally and you want to use the default options you can omit both ```js - const api = await ooba(WebSocket)() + const api = ooba() ``` - this promise will return once a websocket connection to `localhost:5005` is established + this will return an api object that is ready to make streaming requests. in order to generate text simply call `api.generate()` with a prompt ```js diff --git a/browser_example.html b/browser_example.html index a6cef0c..1be77cd 100644 --- a/browser_example.html +++ b/browser_example.html @@ -11,9 +11,9 @@