Bring-your-own language model: backend, model list and chat

The SIU panel (and any caller) can use ANY model: an Ollama server (local, on your network, or tunnelled), any OpenAI-compatible server (llama.cpp llama-server, vLLM, LM Studio, LocalAI, text-generation-webui, TGI, Jan, OpenRouter, OpenAI ...), or your own R function. Nothing is hardcoded. Arguments left empty fall back to the environment: MORIE_LLM_API ("ollama" or "openai", default "ollama"); MORIE_LLM_BASE, else OLLAMA_HOST / OLLAMA_BASE_URL (ollama) or OPENAI_BASE_URL / LLM_API_BASE_URL (openai), else localhost; MORIE_LLM_KEY, else OLLAMA_API_KEY or OPENAI_API_KEY / LLM_API_KEY. An OpenAI-style base gets /v1 appended unless it already ends in /v1. model = NULL uses MORIE_LLM_MODEL, else OLLAMA_MODEL, else the first model the server lists. A custom chat is a function(model, prompt) returning the reply text.

Usage

morie_llm_backend(api = "", base = "", key = "")

morie_llm_models(api = "", base = "", key = "")

morie_llm_chat(
  prompt,
  model = NULL,
  api = "",
  base = "",
  key = "",
  timeout = 300,
  temperature = 0,
  chat = NULL
)

Arguments

api: "ollama" or "openai" (empty: environment).
base: Server URL (empty: environment).
key: Bearer token (empty: environment; may stay empty).
prompt: Prompt text.
model: Model name (NULL: default model).
timeout: Request timeout in seconds.
temperature: Sampling temperature (0 = deterministic).
chat: Optional function(model, prompt) used instead of HTTP.

Value

morie_llm_backend(): list (api, base, has_key); morie_llm_models(): character vector; morie_llm_chat(): reply text.

Examples

b <- morie_llm_backend(api = "openai", base = "http://localhost:8080")
b$base
#> [1] "http://localhost:8080/v1"
echo <- function(model, prompt) paste(model, "saw", nchar(prompt), "characters")
morie_llm_chat("hello", model = "my-model", chat = echo)
#> [1] "my-model saw 5 characters"