,
+ '_cache': '_readme_cache'}
+
### Sentiment
@@ -154,9 +129,9 @@ reviews.llm.summarize("review", 5)
| review | summary |
|----|----|
-| "This has been the best TV I've ever used. Great screen, and sound." | "great tv with good features" |
-| "I regret buying this laptop. It is too slow and the keyboard is too noisy" | "laptop purchase was a mistake" |
-| "Not sure how to feel about my new washing machine. Great color, but hard to figure" | "feeling uncertain about new purchase" |
+| "This has been the best TV I've ever used. Great screen, and sound." | "exceptional tv for its price" |
+| "I regret buying this laptop. It is too slow and the keyboard is too noisy" | "not a good laptop purchase" |
+| "Not sure how to feel about my new washing machine. Great color, but hard to figure" | "some assembly required included" |
### Classify
@@ -233,9 +208,9 @@ reviews.llm.translate("review", "spanish")
| review | translation |
|----|----|
-| "This has been the best TV I've ever used. Great screen, and sound." | "Esta ha sido la mejor televisión que he utilizado hasta ahora. Gran pantalla y sonido." |
-| "I regret buying this laptop. It is too slow and the keyboard is too noisy" | "Me arrepiento de comprar este portátil. Es demasiado lento y la tecla es demasiado ruidosa." |
-| "Not sure how to feel about my new washing machine. Great color, but hard to figure" | "No estoy seguro de cómo sentirme con mi nueva lavadora. Un color maravilloso, pero muy difícil de en… |
+| "This has been the best TV I've ever used. Great screen, and sound." | "Este ha sido la mejor televisión que he utilizado. Una pantalla excelente y buena calidad de sonido." |
+| "I regret buying this laptop. It is too slow and the keyboard is too noisy" | "Me arrepiento de haber comprado este portátil. Es demasiado lento y la tecla tiene un ruido excesivo… |
+| "Not sure how to feel about my new washing machine. Great color, but hard to figure" | "No estoy seguro de cómo sentirme sobre mi nueva lavadora. Una buena cromática, pero difícil de compr… |
### Custom prompt
@@ -245,9 +220,10 @@ it against each text entry:
``` python
my_prompt = (
"Answer a question."
- "Return only the answer, no explanation"
- "Acceptable answers are 'yes', 'no'"
- "Answer this about the following text, is this a happy customer?:"
+ "Return only the answer, no explanation."
+ "Only 'yes' and 'no' are the acceptable answers."
+ "If unsure about the answer, return 'no'."
+ "Answer this about the following text: 'is this a happy customer?':"
)
reviews.llm.custom("review", prompt = my_prompt)
@@ -259,21 +235,7 @@ reviews.llm.custom("review", prompt = my_prompt)
| "I regret buying this laptop. It is too slow and the keyboard is too noisy" | "No" |
| "Not sure how to feel about my new washing machine. Great color, but hard to figure" | "No" |
-## Model selection and settings
-
-You can set the model and its options to use when calling the LLM. In
-this case, we refer to options as model specific things that can be set,
-such as seed or temperature.
-
-The model and options to be used will be defined at the Polars data
-frame object level. If not passed, the default model will be
-**llama3.2**.
-
-``` python
-reviews.llm.use("ollama", "llama3.2", options = dict(seed = 100))
-```
-
-#### Results caching
+## Results caching
By default `mall` caches the requests and corresponding results from a
given LLM run. Each response is saved as individual JSON files. By
@@ -291,6 +253,39 @@ To turn off:
reviews.llm.use(_cache = "")
```
+## Vectors
+
+`mall` also includes a class to work with character vectors. This is a
+separate module from that of the Polars extension, but offers the same
+functionality. To start, import the `LLMVec` class from `mall`, and then
+assign it to a new variable. The function call works just like
+`.llm.use()`, this is where the cache can be specified.
+
+``` python
+from mall import LLMVec
+llm_ollama = LLMVec(chat, _cache="_readme_cache")
+```
+
+To use, call the same NLP functions used data frames. For example
+sentiment:
+
+``` python
+llm_ollama.sentiment(["I am happy", "I am sad"])
+```
+
+ ['positive', 'negative']
+
+The functions will also return a character vector. As mentioned before,
+all of the same functions are accessible via this class:
+
+- Classify
+- Custom
+- Extract
+- Sentiment
+- Summarize
+- Translate
+- Verify
+
## Key considerations
The main consideration is **cost**. Either, time cost, or money cost.
diff --git a/python/README.qmd b/python/README.qmd
index 9e70728..7531073 100644
--- a/python/README.qmd
+++ b/python/README.qmd
@@ -4,7 +4,21 @@ execute:
eval: true
---
-
+```{python}
+#| include: false
+#|
+import polars as pl
+from polars.dataframe._html import HTMLFormatter
+
+pl.Config(fmt_str_lengths=100)
+pl.Config.set_tbl_hide_dataframe_shape(True)
+pl.Config.set_tbl_hide_column_data_types(True)
+
+html_formatter = get_ipython().display_formatter.formatters['text/html']
+html_formatter.for_type(pl.DataFrame, lambda df: "\n".join(HTMLFormatter(df).render()))
+```
+
+
@@ -12,116 +26,62 @@ execute:
+Use Large Language Models (LLM) to run Natural Language Processing (NLP) operations against your data. It takes advantage of the LLMs general language training in order to get the predictions, thus removing the need to train a new traditional NLP model. `mall` is available for R and Python.
-Use Large Language Models (LLM) to run Natural Language Processing (NLP)
-operations against your data. It takes advantage of the LLMs general language
-training in order to get the predictions, thus removing the need to train a new
-NLP model. `mall` is available for R and Python.
+It works by running multiple LLM predictions against your data. The predictions are processed row-wise over a specified column. The package includes prompts to perform the following specific NLP operations:
-It works by running multiple LLM predictions against your data. The predictions
-are processed row-wise over a specified column. It relies on the "one-shot"
-prompt technique to instruct the LLM on a particular NLP operation to perform.
-The package includes prompts to perform the following specific NLP operations:
+- Sentiment analysis
+- Text summarizing
+- Classify text
+- Extract one, or several, specific pieces information from the text
+- Translate text
+- Verify that something is true about the text (binary)
-- [Sentiment analysis](#sentiment)
-- [Text summarizing](#summarize)
-- [Classify text](#classify)
-- [Extract one, or several](#extract), specific pieces information from the text
-- [Translate text](#translate)
-- [Verify that something is true](#verify) about the text (binary)
+For other NLP operations, `mall` offers the ability for you to write your own prompt.
-For other NLP operations, `mall` offers the ability for you to [write your own prompt](#custom-prompt).
-
-`mall` is a library extension to [Polars](https://pola.rs/). To
-interact with Ollama, it uses the official
-[Python library](https://github.com/ollama/ollama-python).
-
-```python
-reviews.llm.sentiment("review")
-```
-## Motivation
-
-We want to new find new ways to help data scientists use LLMs in their daily work.
-Unlike the familiar interfaces, such as chatting and code completion, this
-interface runs your text data directly against the LLM. This package is inspired
-by the SQL AI functions now offered by vendors such as [Databricks](https://docs.databricks.com/en/large-language-models/ai-functions.html)
-and Snowflake.
-
-The LLM's flexibility, allows for it to adapt to the subject of your data, and
-provide surprisingly accurate predictions. This saves the data scientist the
-need to write and tune an NLP model.
-
-In recent times, the capabilities of LLMs that can run locally in your computer
-have increased dramatically. This means that these sort of analysis can run in
-your machine with good accuracy. It also makes it possible to take
-advantage of LLMs at your institution, since the data will not leave the
-corporate network. Additionally, LLM management and integration platforms, such
-as [Ollama](https://ollama.com/), are now very easy to setup and use. `mall`
-uses Ollama as to interact with local LLMs.
-
-The development version of `mall` lets you **use external LLMs such as
-[OpenAI](https://openai.com/), [Gemini](https://gemini.google.com/) and
-[Anthropic](https://www.anthropic.com/)**. In R, `mall` uses the
-[`ellmer`](https://ellmer.tidyverse.org/index.html)
-package to integrate with the external LLM, and the
-[`chatlas`](https://posit-dev.github.io/chatlas/) package to integrate in Python.
+`mall` lets you **use local and external LLMs such as [OpenAI](https://openai.com/), [Gemini](https://gemini.google.com/) and [Anthropic](https://www.anthropic.com/)**. It uses [`chatlas`](https://posit-dev.github.io/chatlas/) package to integrate to perform the integration. It is a library extension to [Polars](https://pola.rs/). To interact with Ollama, it uses the official [Python library](https://github.com/ollama/ollama-python).
## Get started
-- Install `mall` from Github
-
-
-```python
-pip install "mall @ git+https://git@github.com/mlverse/mall.git#subdirectory=python"
-```
+Install `mall`:
-- [Download Ollama from the official website](https://ollama.com/download)
+- From PyPi:
-- Install and start Ollama in your computer
+ ``` python
+ pip install mlverse-mall
+ ```
-- Install the official Ollama library
- ```python
- pip install ollama
- ```
+- Install `mall` from Github
-- Download an LLM model. For example, I have been developing this package using
-Llama 3.2 to test. To get that model you can run:
- ```python
- import ollama
- ollama.pull('llama3.2')
- ```
+ ``` python
+ pip install "mall @ git+https://git@github.com/mlverse/mall.git#subdirectory=python"
+ ```
## LLM functions
-We will start with loading a very small data set contained in `mall`. It has
-3 product reviews that we will use as the source of our examples.
-
+We will start with loading a very small data set contained in `mall`. It has 3 product reviews that we will use as the source of our examples.
+
```{python}
-#| include: false
-#|
-import polars as pl
-from polars.dataframe._html import HTMLFormatter
+import mall
+reviews = mall.MallData.reviews
+reviews
+```
-pl.Config(fmt_str_lengths=100)
-pl.Config.set_tbl_hide_dataframe_shape(True)
-pl.Config.set_tbl_hide_column_data_types(True)
+Because `mall` is loaded, the `reviews` Polars data frame contain a class named `llm`. This is the class that enables access to all of the NLP functions.
-html_formatter = get_ipython().display_formatter.formatters['text/html']
-html_formatter.for_type(pl.DataFrame, lambda df: "\n".join(HTMLFormatter(df).render()))
-```
+### Setup
-```{python}
-import mall
-data = mall.MallData
-reviews = data.reviews
+The connection to the LLM is created via a `Chat` object from `chatlas`. For this article, an Ollama chat connection is created:
-reviews
+```{python}
+from chatlas import ChatOllama
+chat = ChatOllama(model = "llama3.2", seed = 100)
```
+Now, `reviews` is "told" to use the `chat` object by calling `.llm.use()`. In this case, the `_cache` path is set in order to re-run render this article faster as edits are made to the prose:
```{python}
-#| include: false
-reviews.llm.use(options = dict(seed = 100), _cache = "_readme_cache")
+reviews.llm.use(chat, _cache = "_readme_cache")
```
@@ -136,11 +96,7 @@ reviews.llm.sentiment("review")
### Summarize
-There may be a need to reduce the number of words in a given text. Typically to
-make it easier to understand its intent. The function has an argument to
-control the maximum number of words to output
-(`max_words`):
-
+There may be a need to reduce the number of words in a given text. Typically to make it easier to understand its intent. The function has an argument to control the maximum number of words to output (`max_words`):
```{python}
reviews.llm.summarize("review", 5)
@@ -148,20 +104,15 @@ reviews.llm.summarize("review", 5)
### Classify
-Use the LLM to categorize the text into one of the options you provide:
-
+Use the LLM to categorize the text into one of the options you provide:
```{python}
reviews.llm.classify("review", ["computer", "appliance"])
```
-### Extract
+### Extract
-One of the most interesting use cases Using natural language, we can tell the
-LLM to return a specific part of the text. In the following example, we request
-that the LLM return the product being referred to. We do this by simply saying
-"product". The LLM understands what we *mean* by that word, and looks for that
-in the text.
+One of the most interesting use cases Using natural language, we can tell the LLM to return a specific part of the text. In the following example, we request that the LLM return the product being referred to. We do this by simply saying "product". The LLM understands what we *mean* by that word, and looks for that in the text.
```{python}
reviews.llm.extract("review", "product")
@@ -169,29 +120,23 @@ reviews.llm.extract("review", "product")
### Classify
-Use the LLM to categorize the text into one of the options you provide:
+Use the LLM to categorize the text into one of the options you provide:
```{python}
reviews.llm.classify("review", ["computer", "appliance"])
```
-### Verify
+### Verify
-This functions allows you to check and see if a statement is true, based
-on the provided text. By default, it will return a 1 for "yes", and 0 for
-"no". This can be customized.
+This functions allows you to check and see if a statement is true, based on the provided text. By default, it will return a 1 for "yes", and 0 for "no". This can be customized.
```{python}
reviews.llm.verify("review", "is the customer happy with the purchase")
```
-
### Translate
-As the title implies, this function will translate the text into a specified
-language. What is really nice, it is that you don't need to specify the language
-of the source text. Only the target language needs to be defined. The translation
-accuracy will depend on the LLM
+As the title implies, this function will translate the text into a specified language. What is really nice, it is that you don't need to specify the language of the source text. Only the target language needs to be defined. The translation accuracy will depend on the LLM
```{python}
reviews.llm.translate("review", "spanish")
@@ -199,69 +144,69 @@ reviews.llm.translate("review", "spanish")
### Custom prompt
-It is possible to pass your own prompt to the LLM, and have `mall` run it
-against each text entry:
+It is possible to pass your own prompt to the LLM, and have `mall` run it against each text entry:
```{python}
my_prompt = (
"Answer a question."
- "Return only the answer, no explanation"
- "Acceptable answers are 'yes', 'no'"
- "Answer this about the following text, is this a happy customer?:"
+ "Return only the answer, no explanation."
+ "Only 'yes' and 'no' are the acceptable answers."
+ "If unsure about the answer, return 'no'."
+ "Answer this about the following text: 'is this a happy customer?':"
)
reviews.llm.custom("review", prompt = my_prompt)
```
-## Model selection and settings
+## Results caching
-You can set the model and its options to use when calling the LLM. In this case,
-we refer to options as model specific things that can be set, such as seed or
-temperature.
+By default `mall` caches the requests and corresponding results from a given LLM run. Each response is saved as individual JSON files. By default, the folder name is `_mall_cache`. The folder name can be customized, if needed. Also, the caching can be turned off by setting the argument to empty (`""`).
+
+```{python}
+#| eval: false
+reviews.llm.use(_cache = "my_cache")
+```
-The model and options to be used will be defined at the Polars data frame
-object level. If not passed, the default model will be **llama3.2**.
+To turn off:
```{python}
#| eval: false
-reviews.llm.use("ollama", "llama3.2", options = dict(seed = 100))
+reviews.llm.use(_cache = "")
```
-#### Results caching
-By default `mall` caches the requests and corresponding results from a given
-LLM run. Each response is saved as individual JSON files. By default, the folder
-name is `_mall_cache`. The folder name can be customized, if needed. Also, the
-caching can be turned off by setting the argument to empty (`""`).
+## Vectors
+
+`mall` also includes a class to work with character vectors. This is a separate module from that of the Polars extension, but offers the same functionality. To start, import the `LLMVec` class from `mall`, and then assign it to a new variable. The function call works just like `.llm.use()`, this is where the cache can be specified.
```{python}
-#| eval: false
-reviews.llm.use(_cache = "my_cache")
+from mall import LLMVec
+llm_ollama = LLMVec(chat, _cache="_readme_cache")
```
-To turn off:
+To use, call the same NLP functions used data frames. For example sentiment:
```{python}
-#| eval: false
-reviews.llm.use(_cache = "")
+llm_ollama.sentiment(["I am happy", "I am sad"])
```
+The functions will also return a character vector. As mentioned before, all of the same functions are accessible via this class:
+
+- Classify
+- Custom
+- Extract
+- Sentiment
+- Summarize
+- Translate
+- Verify
+
+
## Key considerations
The main consideration is **cost**. Either, time cost, or money cost.
-If using this method with an LLM locally available, the cost will be a long
-running time. Unless using a very specialized LLM, a given LLM is a general model.
-It was fitted using a vast amount of data. So determining a response for each
-row, takes longer than if using a manually created NLP model. The default model
-used in Ollama is [Llama 3.2](https://ollama.com/library/llama3.2),
-which was fitted using 3B parameters.
-
-If using an external LLM service, the consideration will need to be for the
-billing costs of using such service. Keep in mind that you will be sending a lot
-of data to be evaluated.
-
-Another consideration is the novelty of this approach. Early tests are
-providing encouraging results. But you, as an user, will still need to keep
-in mind that the predictions will not be infallible, so always check the output.
-At this time, I think the best use for this method, is for a quick analysis.
+If using this method with an LLM locally available, the cost will be a long running time. Unless using a very specialized LLM, a given LLM is a general model. It was fitted using a vast amount of data. So determining a response for each row, takes longer than if using a manually created NLP model. The default model used in Ollama is [Llama 3.2](https://ollama.com/library/llama3.2), which was fitted using 3B parameters.
+
+If using an external LLM service, the consideration will need to be for the billing costs of using such service. Keep in mind that you will be sending a lot of data to be evaluated.
+
+Another consideration is the novelty of this approach. Early tests are providing encouraging results. But you, as an user, will still need to keep in mind that the predictions will not be infallible, so always check the output. At this time, I think the best use for this method, is for a quick analysis.
\ No newline at end of file
diff --git a/python/mall/llm.py b/python/mall/llm.py
index 41c3678..a6c0fdb 100644
--- a/python/mall/llm.py
+++ b/python/mall/llm.py
@@ -136,7 +136,7 @@ def llm_call(x, msg, use, valid_resps="", convert=None, data_type=None):
out = convert.get(label)
if data_type == int:
- out = data_type(out)
+ out = int(out)
if out not in valid_resps and len(valid_resps) > 0:
out = None
@@ -155,7 +155,7 @@ def valid_output(x):
def build_msg(x, msg):
- return {'role': 'user', 'content': msg + str(x)}
+ return [{"role": "user", "content": msg + str(x)}]
def build_hash(x):
diff --git a/python/mall/llmvec.py b/python/mall/llmvec.py
index 88a9ecc..840b878 100644
--- a/python/mall/llmvec.py
+++ b/python/mall/llmvec.py
@@ -184,7 +184,7 @@ def custom(self, x, prompt="", valid_resps="") -> list:
The prompt to send to the LLM along with the `col`
"""
- return llm_loop(x=x, msg=custom(prompt), use=self._use, valid_resps=valid_resps)
+ return llm_loop(x=x, msg=custom(prompt, use=self._use), use=self._use, valid_resps=valid_resps)
def verify(self, x, what="", yes_no=[1, 0], additional="") -> list:
"""Check to see if something is true about the text.
diff --git a/python/mall/polars.py b/python/mall/polars.py
index d4cb725..cada94a 100644
--- a/python/mall/polars.py
+++ b/python/mall/polars.py
@@ -431,7 +431,7 @@ def custom(
df = llm_map(
df=self._df,
col=col,
- msg=custom(prompt),
+ msg=custom(prompt, use=self._use),
pred_name=pred_name,
use=self._use,
valid_resps=valid_resps,
diff --git a/python/mall/prompt.py b/python/mall/prompt.py
index 968f298..aef1de2 100644
--- a/python/mall/prompt.py
+++ b/python/mall/prompt.py
@@ -82,14 +82,15 @@ def verify(what, additional="", use=[]):
"You are a helpful text analysis engine."
"Determine if this is true "
f"'{what}'."
+ "There are only two acceptable answers, 'yes' and 'no'. "
"No capitalization. No explanations."
f"{additional}"
)
return prompt_complete(x, use)
-def custom(prompt):
- return prompt
+def custom(x, use):
+ return prompt_complete(x + ". ", use)
def process_labels(x, if_list="", if_dict=""):
diff --git a/python/tests/test_custom.py b/python/tests/test_custom.py
index 33ba7f8..1475f08 100644
--- a/python/tests/test_custom.py
+++ b/python/tests/test_custom.py
@@ -9,7 +9,7 @@ def test_custom_prompt():
df = pl.DataFrame(dict(x="x"))
df.llm.use("test", "content", _cache="_test_cache")
x = df.llm.custom("x", "hello")
- assert x["custom"][0] == "hello"
+ assert x["custom"][0] == "hello. The answer is based on the following text:\n{}"
shutil.rmtree("_test_cache", ignore_errors=True)
def test_custom_vec():
diff --git a/r/DESCRIPTION b/r/DESCRIPTION
index 229d593..cc9360d 100644
--- a/r/DESCRIPTION
+++ b/r/DESCRIPTION
@@ -1,7 +1,7 @@
Package: mall
Title: Run Multiple Large Language Model Predictions Against a Table, or
Vectors
-Version: 0.1.0.9004
+Version: 0.2.0
Authors@R: c(
person("Edgar", "Ruiz", , "edgar@posit.co", role = c("aut", "cre")),
person(given = "Posit Software, PBC", role = c("cph", "fnd"))
@@ -30,7 +30,7 @@ Suggests:
Config/testthat/edition: 3
URL: https://mlverse.github.io/mall/
Depends:
- R (>= 2.10)
+ R (>= 4.1)
LazyData: true
diff --git a/r/NEWS.md b/r/NEWS.md
index e451dd4..e8bb876 100644
--- a/r/NEWS.md
+++ b/r/NEWS.md
@@ -1,4 +1,4 @@
-# mall (dev)
+# mall 0.2.0
* Adds integration with `ellmer` `Chat` objects
diff --git a/r/R/m-backend-prompt.R b/r/R/m-backend-prompt.R
index d38263f..6f2353d 100644
--- a/r/R/m-backend-prompt.R
+++ b/r/R/m-backend-prompt.R
@@ -130,6 +130,7 @@ m_backend_prompt.mall_session <- function(backend, additional = "") {
"You are a helpful text analysis engine.",
"Determine if this is true ",
"'{what}'.",
+ "There are only two acceptable answers, 'yes' and 'no'.",
"No capitalization. No explanations.",
"{additional}"
))
diff --git a/r/R/m-backend-submit.R b/r/R/m-backend-submit.R
index d132df7..d988e66 100644
--- a/r/R/m-backend-submit.R
+++ b/r/R/m-backend-submit.R
@@ -54,13 +54,15 @@ m_backend_submit.mall_ollama <- function(backend, x, prompt, preview = FALSE) {
if (preview) {
res <- expr(ollamar::chat(!!!.args))
}
- if (m_cache_use() && is.null(res)) {
+ if (m_cache_use(backend) && is.null(res)) {
hash_args <- hash(.args)
res <- m_cache_check(hash_args)
}
if (is.null(res)) {
res <- exec("chat", !!!.args)
- m_cache_record(.args, res, hash_args)
+ if (m_cache_use(backend)) {
+ m_cache_record(.args, res, hash_args)
+ }
}
res
}
@@ -101,7 +103,7 @@ m_backend_submit.mall_ellmer <- function(backend, x, prompt, preview = FALSE) {
)
}
ellmer_obj <- backend[["args"]][["ellmer_obj"]]
- if (m_cache_use()) {
+ if (m_cache_use(backend)) {
hashed_x <- map(x, function(x) hash(c(ellmer_obj, prompt, x)))
from_cache <- map(hashed_x, m_cache_check)
null_cache <- map_lgl(from_cache, is.null)
@@ -113,7 +115,7 @@ m_backend_submit.mall_ellmer <- function(backend, x, prompt, preview = FALSE) {
temp_ellmer$set_system_prompt(prompt)
from_llm <- parallel_chat_text(temp_ellmer, as.list(x))
}
- if (m_cache_use()) {
+ if (m_cache_use(backend)) {
walk(
seq_along(from_llm),
function(y) {
@@ -156,13 +158,15 @@ m_backend_submit.mall_simulate_llm <- function(backend,
out <- prompt
}
res <- NULL
- if (m_cache_use()) {
+ if (m_cache_use(backend)) {
hash_args <- hash(.args)
res <- m_cache_check(hash_args)
}
if (is.null(res)) {
.args$backend <- NULL
- m_cache_record(.args, out, hash_args)
+ if (m_cache_use(backend)) {
+ m_cache_record(.args, out, hash_args)
+ }
}
out
}
diff --git a/r/R/m-cache.R b/r/R/m-cache.R
index 7a2c64f..b86dbee 100644
--- a/r/R/m-cache.R
+++ b/r/R/m-cache.R
@@ -1,7 +1,4 @@
m_cache_record <- function(.args, .response, hash_args) {
- if (!m_cache_use()) {
- return(invisible())
- }
folder_root <- m_defaults_cache()
try(dir_create(folder_root))
content <- list(
@@ -33,11 +30,6 @@ m_cache_file <- function(hash_args) {
path(folder_root, folder_sub, hash_args, ext = "json")
}
-m_cache_use <- function() {
- folder <- m_defaults_cache() %||% ""
- out <- FALSE
- if (folder != "") {
- out <- TRUE
- }
- out
+m_cache_use <- function(backend) {
+ backend$session$cache_folder != ""
}
diff --git a/r/R/m-vec-prompt.R b/r/R/m-vec-prompt.R
index 03f390e..6df4af3 100644
--- a/r/R/m-vec-prompt.R
+++ b/r/R/m-vec-prompt.R
@@ -25,12 +25,9 @@ m_vec_prompt <- function(x,
prompt = prompt,
preview = preview
)
+
if (preview) {
- if (length(resp) == 1) {
- return(resp[[1]])
- } else {
- return(resp)
- }
+ return(resp)
}
# Checks for invalid output and marks them as NA
diff --git a/r/cran-comments.md b/r/cran-comments.md
index 9859c46..0731d52 100644
--- a/r/cran-comments.md
+++ b/r/cran-comments.md
@@ -1,35 +1,19 @@
-## Resubmission
+## Submission
-Thank you for the feedback and instructions, I have made the following changes:
+* Adds integration with `ellmer` `Chat` objects
-- Updated the Title field to title case
-
-- Changed all \notrun{} to \nottest{}
-
-- Changed default location in llm_use() to use a temp folder
-
-- Changed the tests to use a temp folder location
-
-### Original submission text:
-
-This is a new package submission. Run multiple 'Large Language Model'
-predictions against a table. The predictions run row-wise over a specified
-column. It works using a one-shot prompt, along with the current
-row's content. The prompt that is used will depend of the type of analysis
-needed.
-
-The README file is very short because all the information about how to use it is
-this website: https://mlverse.github.io/mall/.
+* Returns a warning when there are over 4,096 tokens in one or many records
+sent to Ollama
## R CMD check environments
-- Mac OS M3 (aarch64-apple-darwin23), R 4.4.1 (Local)
+- Mac OS M3 (aarch64-apple-darwin23), R 4.5.0 (Local)
-- Mac OS x86_64-apple-darwin20.0 (64-bit), R 4.4.1 (GH Actions)
-- Windows x86_64-w64-mingw32 (64-bit), R 4.4.1 (GH Actions)
-- Linux x86_64-pc-linux-gnu (64-bit), R 4.4.1 (GH Actions)
-- Linux x86_64-pc-linux-gnu (64-bit), R 4.5.0 (dev) (GH Actions)
-- Linux x86_64-pc-linux-gnu (64-bit), R 4.3.3 (old release) (GH Actions)
+- macOS Sonoma 14.7.6 (64-bit), R 4.5.1 (GH Actions)
+- Windows Server 2022 x64 (64-bit), R 4.5.1 (GH Actions)
+- Ubuntu 24.04.2 LTS, R 4.5.1 (GH Actions)
+- Ubuntu 24.04.2 LTS (dev) (GH Actions)
+- Ubuntu 24.04.2 LTS, R 4.4.3 (old release) (GH Actions)
## R CMD check results
diff --git a/r/tests/testthat/_snaps/llm-classify.md b/r/tests/testthat/_snaps/llm-classify.md
index b94a21f..d32fb34 100644
--- a/r/tests/testthat/_snaps/llm-classify.md
+++ b/r/tests/testthat/_snaps/llm-classify.md
@@ -12,8 +12,10 @@
Code
llm_vec_classify("this is a test", c("a", "b"), preview = TRUE)
Output
+ [[1]]
ollamar::chat(messages = list(list(role = "user", content = "You are a helpful classification engine. Determine if the text refers to one of the following: a, b. No capitalization. No explanations. The answer is based on the following text:\nthis is a test")),
output = "text", model = "llama3.2", seed = 100)
+
# Classify on Ollama works
diff --git a/r/tests/testthat/_snaps/llm-verify.md b/r/tests/testthat/_snaps/llm-verify.md
index 9267220..77beeac 100644
--- a/r/tests/testthat/_snaps/llm-verify.md
+++ b/r/tests/testthat/_snaps/llm-verify.md
@@ -3,8 +3,10 @@
Code
llm_vec_verify("this is a test", "a test", preview = TRUE)
Output
- ollamar::chat(messages = list(list(role = "user", content = "You are a helpful text analysis engine. Determine if this is true 'a test'. No capitalization. No explanations. The answer is based on the following text:\nthis is a test")),
+ [[1]]
+ ollamar::chat(messages = list(list(role = "user", content = "You are a helpful text analysis engine. Determine if this is true 'a test'. There are only two acceptable answers, 'yes' and 'no'. No capitalization. No explanations. The answer is based on the following text:\nthis is a test")),
output = "text", model = "llama3.2", seed = 100)
+
# Verify on Ollama works
diff --git a/r/tests/testthat/_snaps/m-backend-prompt.md b/r/tests/testthat/_snaps/m-backend-prompt.md
new file mode 100644
index 0000000..04c7dec
--- /dev/null
+++ b/r/tests/testthat/_snaps/m-backend-prompt.md
@@ -0,0 +1,7 @@
+# Ellmer method works
+
+ Code
+ ellmer_funcs$sentiment("positive")
+ Output
+ You are a helpful sentiment engine. Return only one of the following answers: positive. No capitalization. No explanations. The answer will be based on each individual prompt. Treat each prompt as unique when deciding the answer.
+
diff --git a/r/tests/testthat/_snaps/m-backend-submit.md b/r/tests/testthat/_snaps/m-backend-submit.md
index 4aaa172..e65c54e 100644
--- a/r/tests/testthat/_snaps/m-backend-submit.md
+++ b/r/tests/testthat/_snaps/m-backend-submit.md
@@ -22,16 +22,16 @@
[7] "positive" "positive" "positive" "positive" "positive" "positive"
[13] "positive" "positive" "positive"
-# ellmer code is covered
+# ellmer code is covered with cache turned off
Code
- m_backend_submit(backend = ellmer_session, x = "this is x", prompt = list(list(
- content = "this is the prompt")), preview = TRUE)
+ m_backend_submit(backend = ellmer_session, x = test_txt, prompt = list(list(
+ content = "test")), preview = TRUE)
Output
[[1]]
- ellmer_obj$set_system_prompt(list(list(content = "this is the prompt")))
+ ellmer_obj$set_system_prompt(list(list(content = "test")))
[[2]]
- ellmer_obj$chat(as.list("this is x"))
+ ellmer_obj$chat(as.list("test"))
diff --git a/r/tests/testthat/test-m-backend-prompt.R b/r/tests/testthat/test-m-backend-prompt.R
index 60dca1a..cbe7538 100644
--- a/r/tests/testthat/test-m-backend-prompt.R
+++ b/r/tests/testthat/test-m-backend-prompt.R
@@ -25,3 +25,17 @@ test_that("Prompt handles list()", {
expect_false(x_extract == y_extract)
expect_false(x_classify == y_classify)
})
+
+test_that("Ellmer method works", {
+ llm_use(
+ backend = "simulate_llm",
+ model = "echo",
+ .silent = TRUE,
+ .force = TRUE,
+ .cache = ""
+ )
+ ellmer_session <- .env_llm$session
+ class(ellmer_session) <- c("mall_ellmer", "mall_session")
+ ellmer_funcs <- m_backend_prompt(ellmer_session, "")
+ expect_snapshot(ellmer_funcs$sentiment("positive"))
+})
diff --git a/r/tests/testthat/test-m-backend-submit.R b/r/tests/testthat/test-m-backend-submit.R
index 6b14347..8fa481a 100644
--- a/r/tests/testthat/test-m-backend-submit.R
+++ b/r/tests/testthat/test-m-backend-submit.R
@@ -54,17 +54,43 @@ test_that("ellmer code is covered", {
),
test_txt
)
+})
+
+test_that("ellmer code is covered with cache turned off", {
+ local_mocked_bindings(
+ parallel_chat_text = function(x, y) {
+ return(as.character(y))
+ }
+ )
+ llm_use(
+ backend = "simulate_llm",
+ model = "echo",
+ .silent = TRUE,
+ .force = TRUE,
+ .cache = ""
+ )
+ ellmer_session <- .env_llm$session
+ class(ellmer_session) <- c("mall_ellmer")
+ ellmer_session$args[["ellmer_obj"]] <- temp_ellmer_obj()
+ test_txt <- rep("test", times = 15)
+ expect_equal(
+ m_backend_submit(
+ backend = ellmer_session,
+ x = test_txt,
+ prompt = list(list(content = "test"))
+ ),
+ test_txt
+ )
expect_snapshot(
m_backend_submit(
backend = ellmer_session,
- x = "this is x",
- prompt = list(list(content = "this is the prompt")),
+ x = test_txt,
+ prompt = list(list(content = "test")),
preview = TRUE
)
)
})
-
test_that("ellmer code is covered - part II", {
withr::with_envvar(
new = list(OPENAI_API_KEY = "test"),