library(tidyprompt)
#>
#> Attaching package: 'tidyprompt'
#> The following object is masked from 'package:utils':
#>
#> promptSetup an LLM provider
tidyprompt can be used with any LLM provider capable of
completing a chat.
At the moment, tidyprompt includes pre-built functions
to connect with Ollama and the OpenAI API.
With the create_llm_provider function, you can easily
write a hook for any other LLM provider. You could make API calls using
the httr package or use another R package that already has
a hook for the LLM provider you want to use.
# Ollama running on local PC
ollama <- create_ollama_llm_provider(
parameters = list(model = "llama3.1:8b", url = "http://localhost:11434/api/chat")
)
# OpenAI API
openai <- create_openai_llm_provider(
parameters = list(model = "gpt-4o-mini", api_key = Sys.getenv("OPENAI_API_KEY"))
)
# Create your own LLM provider hook using create_llm_provider();
# see ?create_llm_provider for more information, and take a look at
# the source code of create_ollama_llm_provider() and create_openai_llm_provider()Basic prompting
A simple string serves as the base for a prompt.
By adding prompt wrappers, you can influence various aspects of how the LLM handles the prompt, while verifying that the output is structured and valid (including retries with feedback to the LLM if it is not).
"Hi there!" |>
send_prompt(ollama)
#> [1] "How's your day going so far? Is there something I can help you with or would you like to chat?"add_text is a simple example of a prompt wrapper. It
simply adds some text at the end of the base prompt.
"Hi there!" |>
add_text("What is a large language model? Explain in 10 words.") |>
send_prompt(ollama)
#> [1] "Complex software that understands and generates human-like written text effectively."You can also construct the final prompt text, without sending it to an LLM provider.
"Hi there!" |>
add_text("What is a large language model? Explain in 10 words.") |>
construct_prompt_text() |>
cat()
#> Hi there!
#>
#> What is a large language model? Explain in 10 words.Retrieving output in a specific format
Using prompt wrappers, you can force the LLM to return the output in a specific format. You can also extract the output to turn it from a character into another data type.
For instance, the answer_as_integer prompt wrapper will
force the LLM to return an integer.
To achieve this, the prompt wrapper will add some text to the base prompt, asking the LLM to reply with an integer. However, the prompt wrapper does more: it also will attempt to extract and validate the integer from the LLM’s response. If extraction or validation fails, feedback is sent back to the LLM, after which the LLM can retry answering the prompt.
"What is 2 + 2?" |>
answer_as_integer() |>
send_prompt(ollama, verbose = TRUE)
#> --- Sending message to LLM-provider: ---
#> What is 2 + 2?
#>
#> You must answer with only an integer (use no other characters).
#> --- Received response from LLM-provider: ---
#> 4
#> [1] 4Below is an example of a prompt which will initially fail, but will succeed after a retry.
"What is 2 + 2?" |>
add_text("Please write out your reply in words, use no numbers.") |>
answer_as_integer(add_instruction_to_prompt = FALSE) |>
send_prompt(ollama, verbose = TRUE)
#> --- Sending message to LLM-provider: ---
#> What is 2 + 2?
#>
#> Please write out your reply in words, use no numbers.
#> --- Received response from LLM-provider: ---
#> Two plus two equals four.
#> --- Sending message to LLM-provider: ---
#> You must answer with only an integer (use no other characters).
#> --- Received response from LLM-provider: ---
#> 4
#> [1] 4Adding a reasoning mode to the LLM
Prompt wrappers may also be used to add a reasoning mode to the LLM. It is hypothesized that this could improve the LLM’s performance on more complex tasks.
For instance, function answer_by_chain_of_thought will
add chain of thought reasoning mode to the LLM. This wraps the base
prompt within a request for the LLM to reason step by step, asking it to
provide the final answer within ‘FINISH[
"What is 2 + 2?" |>
answer_by_chain_of_thought() |>
answer_as_integer() |>
send_prompt(ollama, verbose = TRUE)
#> --- Sending message to LLM-provider: ---
#> You are given a user's prompt.
#> To answer the user's prompt, you need to think step by step to arrive at a final answer.
#>
#> ----- START OF USER'S PROMPT -----
#> What is 2 + 2?
#>
#> You must answer with only an integer (use no other characters).
#> ----- END OF USER'S PROMPT -----
#>
#> What are the steps you would take to answer the user's prompt?
#> Describe your thought process in the following format:
#> >> step 1: <step 1 description>
#> >> step 2: <step 2 description>
#> (etc.)
#>
#> When you are done, you must type:
#> FINISH[<put here your final answer to the user's prompt>]
#>
#> Make sure your final answer follows the logical conclusion of your thought process.
#> --- Received response from LLM-provider: ---
#> >> step 1: Identify the operation being asked in the prompt, which is addition.
#> The prompt asks for the result of "2 + 2", indicating that we need to add two numbers together.
#>
#> >> step 2: Determine the two numbers being added, which are both 2.
#> The prompt clearly states that we have two instances of the number 2 that need to be combined using addition.
#>
#> >> step 3: Perform the arithmetic operation by adding the two numbers together.
#> Since both numbers are 2, adding them results in doubling the value of one number to get the other. In this case, doubling 2 gives us 4.
#>
#> >> step 4: Convert the result into an integer answer as per the prompt's instruction.
#> Given that we've arrived at a result of 4 through arithmetic, and considering the instruction to provide only an integer answer, our response should be the number 4.
#>
#> FINISH[4]
#> [1] 4Giving tools to the LLM (autonomous function-calling)
With tidyprompt, you can define R functions and give the
LLM the ability to call them. This enables the LLM to retrieve
additional information or take other actions.
# Define a function that returns fake data about the temperature in a location
temperature_in_location <- function(
location = c("Amsterdam", "Utrecht", "Enschede"),
unit = c("Celcius", "Fahrenheit")
) {
#' llm_tool::name temperature_in_location
#'
#' llm_tool::description Get the temperature in a location
#'
#' llm_tool::param location Location, must be one of: "Amsterdam", "Utrecht", "Enschede"
#' llm_tool::param unit Unit, must be one of: "Celcius", "Fahrenheit"
#'
#' llm_tool::return The temperature in the specified location and unit
#'
#' llm_tool::example
#' temperature_in_location("Amsterdam", "Fahrenheit")
# As shown above, one can use docstring-like text to document the function.
# This will provide the LLM information on what the function does,
# and how it should be used.
location <- match.arg(location)
unit <- match.arg(unit)
temperature_celcius <- switch(
location,
"Amsterdam" = 32.5,
"Utrecht" = 19.8,
"Enschede" = 22.7
)
if (unit == "Celcius") {
return(temperature_celcius)
} else {
return(temperature_celcius * 9/5 + 32)
}
}
# Ask the LLM a question which can be answered with the function
"Hi, what is the weather temperature in Enschede?" |>
add_text("I want to know the Celcius degrees.") |>
answer_as_integer() |>
add_tools(temperature_in_location) |>
send_prompt(ollama, verbose = TRUE)
#> --- Sending message to LLM-provider: ---
#> Hi, what is the weather temperature in Enschede?
#>
#> I want to know the Celcius degrees.
#>
#> You must answer with only an integer (use no other characters).
#>
#> If you need more information, you can call functions to help you.
#> To call a function, type:
#> FUNCTION[<function name here>](<argument 1>, <argument 2>, etc...)
#>
#> The following functions are available:
#>
#> function name: temperature_in_location
#> description: Get the temperature in a location
#> arguments:
#> - location: Location, must be one of: "Amsterdam", "Utrecht", "Enschede"
#> - unit: Unit, must be one of: "Celcius", "Fahrenheit"
#> return value: The temperature in the specified location and unit
#> example usage: FUNCTION[temperature_in_location]("Amsterdam", "Fahrenheit")
#>
#> After you call a function, wait until you receive more information.
#> --- Received response from LLM-provider: ---
#> To get the current temperature in Enschede, I'll use the `temperature_in_location` function.
#>
#> FUNCTION[temperature_in_location]("Enschede", "Celcius")
#>
#> Please wait for 2 seconds...
#> --- Sending message to LLM-provider: ---
#> function called: temperature_in_location
#> arguments used: Enschede, Celcius
#> result: 22.7
#> --- Received response from LLM-provider: ---
#> The current temperature in Enschede is **22**.
#> --- Sending message to LLM-provider: ---
#> You must answer with only an integer (use no other characters).
#> --- Received response from LLM-provider: ---
#> 22
#> [1] 22Creating your own prompt wrappers
Under the hood, prompts are just lists of a base prompt (a string) and a series of prompt wrappers.
You can thus create a function which takes a prompt and appends a new prompt wrapper to it.
Take a look at the source code for function
add_text:
add_text <- function(
prompt,
text, position = c("after", "before"), sep = "\n\n"
) {
position <- match.arg(position)
modify_fn <- function(original_prompt_text) {
if (position == "after") {
paste(original_prompt_text, text, sep = sep)
} else {
paste(text, original_prompt_text, sep = sep)
}
}
new_wrap <- prompt_wrap(
modify_fn = modify_fn
)
append_prompt_wrap(prompt, new_wrap)
}More complex prompt wrappers may also add extraction and validation
functions. Take a look at the source code for function
answer_as_integer:
answer_as_integer <- function(
prompt,
min = NULL,
max = NULL,
add_instruction_to_prompt = TRUE
) {
instruction <- "You must answer with only an integer (use no other characters)."
if (!is.null(min) && !is.null(max)) {
instruction <- paste(instruction, glue::glue("Enter an integer between {min} and {max}."))
} else if (!is.null(min)) {
instruction <- paste(instruction, glue::glue("Enter an integer greater than or equal to {min}."))
} else if (!is.null(max)) {
instruction <- paste(instruction, glue::glue("Enter an integer less than or equal to {max}."))
}
# Define modification/extraction/validation functions:
modify_fn <- function(original_prompt_text) {
if (!add_instruction_to_prompt) {
return(original_prompt_text)
}
glue::glue("{original_prompt_text}\n\n{instruction}")
}
extraction_fn <- function(x) {
extracted <- suppressWarnings(as.integer(x))
if (is.na(extracted)) {
return(create_llm_feedback(instruction))
}
return(extracted)
}
validation_fn <- function(x) {
if (!is.null(min) && x < min) {
return(create_llm_feedback(glue::glue(
"The number should be greater than or equal to {min}."
)))
}
if (!is.null(max) && x > max) {
return(create_llm_feedback(glue::glue(
"The number should be less than or equal to {max}."
)))
}
return(TRUE)
}
new_wrap <- prompt_wrap(
modify_fn = modify_fn,
extraction_functions = list(extraction_fn),
validation_functions = list(validation_fn)
)
append_prompt_wrap(prompt, new_wrap)
}They key difference between an extraction and validation function is that an extraction function alters the LLM’s response and passes on the altered response to next extraction and/or validation functions, and eventually to the return statement of send_prompt (if extractions and validations are succesful). A validation function, on the other hand, only checks if the LLM’s response passes a logical test. Both extraction and validation functions can return feedback to the LLM.
For more information, on what you can do with prompt wrappers, see
the documentation of the prompt_wrap class creator
function: create_prompt_wrap. For examples of prompt
wrapper functions, see, for instance the documentation and source code
of add_text, answer_as_integer,
answer_by_chain_of_thought, and add_tools.