Starts an interactive app to manually validate the output of an LLM stored in a character vector

This function launches a Shiny app that allows users to manually validate the output of a LLM analysis, such as ai_score The comments from manual validation for each text are added as an additional docvar to the input data with all texts not yet validated as NA.

Usage

ai_validate(
  text,
  llm_output,
  llm_evidence = NULL,
  result_env = new.env(),
  ...,
  verbose = TRUE,
  launch_app = TRUE
)

Arguments

text

a character or quanteda::corpus object containing the documents to be manually validated

llm_output

a character string; the name of the LLM output column which contains the summaries, labels, or scores to be validated

llm_evidence

a character vector; the name of an additional LLM output such as evidence or justifications provided by the LLM

result_env

An environment to store results and allow resuming

...

additional arguments passed to chat_fn

verbose

logical; output a progress indicator if TRUE

launch_app

Logical, whether to launch the interactive Shiny app. Defaults to TRUE.

Value

character; the response from the manual validation with a length equal to the number of input documents, with the elements named with the input element names

Examples

if (FALSE) { # \dontrun{
library(quanteda)
summ1 <- ai_summary(data_char_ukimmig2010, chat_fn = chat_ollama, model = "llama3.2")
summ2 <- ai_summary(data_corpus_inaugural[1:2], chat_fn = chat_openai,
                api_args = list(temperature = 0, seed = 42))
validate1 <- ai_validate(data_char_ukimmig2010, llm_output = summ1, verbose = TRUE)
validate2 <- ai_validate(data_corpus_inaugural[1:2], llm_output = summ2, verbose = TRUE)
} # }