Language models

A served model through vLLM

Connect to a served model identifier through the native compatible chat adapter with explicit endpoint and timeout.

// The served model must have this exact configured identifier.
import aner.llm

let model = llm.client(provider: "vllm", model: "research-model",
    endpoint: "http://127.0.0.1:8000/v1", timeout_ms: 120000)
let answer = model.generate("Summarize the purpose of validation data.",
    max_tokens: 256, temperature: 0.2)
print(answer.text())

Make it your experiment.

With Aner installed, save this program as examples/llm_vllm.aner inside a folder for your experiment. Open a terminal in that folder, then check and run the program.

aner check examples/llm_vllm.aner
aner run examples/llm_vllm.aner

The aner command must be on your PATH. Follow the installation guide if your terminal cannot find it.

Use the compatible native Aner development engine with aner.llm, aner.secrets and VS Code extension 0.1.34 or newer. Checking makes no inference request; running sends the prompt to the explicit provider. Local examples require the named model to be installed or served already. Cloud requests require a configured opaque credential reference and can incur provider charges. See Language model connections and Credentials for supported adapters, HTTPS, timeout, content inspection and storage boundaries.

Iris and Wine are teaching datasets with their own attribution. Example outcomes are not comparative benchmarks or evidence of clinical validity.

Dataset sources & attribution

Install Aner