Language model connections

aner.llm provides one native inference interface for a local model service or a cloud provider. Aner constructs and validates requests, resolves an explicit credential reference, verifies transport, and returns a typed response. The selected service performs model inference. Connecting an external language model does not train an Aner neural network or allocate Aner tensors on that service.

Capability boundary

The current module implements synchronous text generation and ordered messages. Streaming, model discovery, embedding requests, tool execution, structured response schemas, automatic conversation memory, and additional cloud adapters are future work. A local service and its model must already be running; Aner does not install models or start another service. The service controls its tokenizer, context window, inference hardware, model weights, and scheduling.

The OpenAI request disables response storage through store: false; this setting is not a promise of zero data retention. Provider policies and account settings still govern submitted content. Select a provider and endpoint appropriate for the dataset before sending prompts.

Start with a local model

Choose the exact model identifier installed in your Ollama service. This example uses llama3.2:3b; replace it if your service hosts another model.

import aner.llm

let model = llm.client(provider: "ollama", model: "llama3.2:3b")
let answer = model.generate("Explain a linked list in three sentences.")
print(answer.text())

Client construction sends no request. generate sends one user message and waits for a bounded response. The default maximum output budget is 512 tokens; temperature is omitted so the service chooses its default. A response can end at that budget before the answer is complete. Inspect finish_reason() when completeness matters.

Run the local example from a terminal whose working directory contains the downloaded program:

aner check llm_local.aner
aner llm_local.aner

Checking validates Aner syntax and types without contacting a provider. Running sends the prompt to the configured endpoint. A named model that is absent from the service produces an error rather than selecting another model.

Define an explicit request

Keep messages, generation budget, and sampling settings visible in source when an experiment needs control:

import aner.llm

let model = llm.client(provider: "ollama", model: "llama3.2:3b",
    endpoint: "http://127.0.0.1:11434", timeout_ms: 120000)
let messages = List(
    llm.message(role: "system", content: "Explain concepts precisely and concisely."),
    llm.message(role: "user", content: "Compare a linked list and a fixed array."))
let answer = model.chat(messages, max_tokens: 256, temperature: 0.2)
print(answer.text())
print(answer.model())
print(answer.finish_reason())
print(answer.input_tokens())
print(answer.output_tokens())

A message list is nonempty, ordered, and contains only LLMMessage values. Roles are exactly "system", "user", or "assistant". The runtime admits at most 256 messages. Each request supplies its complete intended history; an earlier call does not silently add messages to the next call.

max_tokens is an integer from 1 through 1,000,000, subject to stricter model or service limits. An explicit temperature must be finite and between 0.0 and 2.0; a provider may reject a setting unsupported by its model. Omitting temperature preserves the provider default. Maximum output tokens are a budget, not a guarantee of that output size or a reservation of model context.

Reported token counts use the service tokenizer. A count of -1 means that the response did not report the corresponding usage. Missing usage is not zero usage, a billing check, or a count of text characters. Responses have no shared conversation state and do not expose the raw provider JSON.

The explicit chat example retains these controls and usage accessors in a complete program.

Cloud credentials

Configure a credential once through hidden terminal input:

aner secrets set openai

Then refer to its label from source:

import aner.llm
import aner.secrets

let model = llm.client(provider: "openai", model: "gpt-4.1-mini",
    key: secrets.stored("openai"))
let answer = model.generate("Explain why sample weighted gradients matter.",
    max_tokens: 256)
print(answer.text())

The default store uses the native operating system credential service on macOS and Windows. On Linux, explicitly choose the managed file fallback or an environment variable injected by your deployment system; see Credentials. Source files and notebooks keep a reference, never the key value. Request execution resolves the reference; a missing credential fails before authenticated transport. OpenAI requests can incur provider charges when the program runs.

The cloud example demonstrates this configuration. The model identifier is explicit because available models and their settings can depend on account access. Aner does not replace an unavailable model with another model.

Another local or hosted service

A vLLM service exposes its configured model identifier through its compatible endpoint. Set that exact identifier:

import aner.llm

let model = llm.client(provider: "vllm", model: "research-model",
    endpoint: "http://127.0.0.1:8000/v1", timeout_ms: 120000)
let answer = model.generate("Summarize the purpose of validation data.",
    max_tokens: 256, temperature: 0.2)
print(answer.text())

For an authenticated hosted service, supply an explicit HTTPS base endpoint and credential reference. Plain HTTP is accepted only for loopback addresses. A private network address by itself does not provide encryption. To use a remote loopback service through an SSH tunnel, create the tunnel outside the program and pass its local endpoint; Aner does not open SSH sessions or scan for services.

A hosted provider must implement the selected adapter contract; using a compatible URL does not establish compatibility with every extension or model. See the vLLM example.

Organize a research workflow with a class

Classes can retain a configured client and define the instructions used by an experiment. The same native request API remains explicit:

import aner.llm

class ResearchAssistant {
    let model: LLMClient
    let instructions: String

    fn explain(question: String) {
        let messages = List(
            llm.message("system", instructions),
            llm.message("user", question))
        return model.chat(messages, max_tokens: 256, temperature: 0.2)
    }
}

let assistant = ResearchAssistant(
    model: llm.client("ollama", "llama3.2:3b"),
    instructions: "Explain assumptions, limitations, and the reasoning clearly.")
let answer = assistant.explain("Why must preprocessing fit training data only?")
print(answer.text())

A class adds application organization without changing provider semantics, transport requirements, or message order. It can return LLMResponse so the caller decides whether to retain generated text or usage metadata. See the class example.

API reference

Instance methods also have module forms, such as llm.generate(model, prompt) and llm.text(answer). Named arguments must use the labels in the signature. Credential references are documented independently in Credentials.

Transport and error contract

The request body is bounded to 1 MiB and the response body to 8 MiB. The total timeout is from 1 through 600000 ms, including connection establishment, local model startup, and waiting for the response. TLS certificate and hostname verification remain enabled. Credentials cannot be embedded in endpoint URLs. The runtime does not follow redirects, use ambient HTTP proxy variables, or retry requests automatically; each inference call has one request attempt.

Invalid client configuration, incompatible messages, missing models, transport failures, invalid provider responses, and unsupported result content use source located R3101 diagnostics. Credential resolution uses R3102. Native diagnostic messages omit authorization headers, credential values, prompts, generated text, and raw provider error bodies. Ordinary source excerpts may still show text authored in the program; explicit String output and captures retain their normal visibility. Failed or unsupported responses do not masquerade as successful generated text. A timeout may occur after the provider has begun processing; retrying explicitly can create additional provider work.

The adapter requests plain text. Refusal, tool calls, and unsupported response content produce a sanitized error rather than executing anything. OpenAI incomplete responses expose the reported finish reason and any returned text; inspect the reason before treating output as complete. HTTP loopback exceptions support local inference and locally established tunnels; remote services require authenticated, verified HTTPS where applicable.

Development, production, and notebooks

LLMClient, LLMMessage, LLMResponse, and SecretRef are opaque values. Ordinary handle printing and development inspection do not reveal message text, generated content, endpoint credentials, or secret material. This protects handles; it does not make every String confidential. Prompt Strings already bound in source, answer.text(), and text explicitly printed or captured are ordinary program data and may appear in output or development reports. Keep sensitive content out of captures and saved notebook output.

Development and production execute the same requests. Production disables development recording; it does not change provider, model, token budget, temperature, precision, or provider billing. A notebook cell performs the request synchronously and displays its output after completion. Persistent sessions can retain the configured client, but a rerun sends another request. Notebook reset clears session values and does not delete credentials from their store.

Download the Aner notebook or the Jupyter notebook. Both start with local inference and contain no stored credentials or execution output. Configure your local service before executing request cells.

VS Code provides provider neutral signatures, parameter documentation, response methods, and credential references. Snippets include llmlocal, llmchat, llmcloud, llmvllm, llmclass, and secretref. Use the compatible native interpreter and extension version 0.1.34 or newer; updating the extension alone does not update the engine.

Time and space complexity

Let c be client configuration bytes, m message count, q serialized request bytes, r response bytes, and t generated text bytes. These quantities describe Aner client work. They do not describe inference performed by the selected service.

Elapsed generation time includes connection establishment, network latency, service queueing, model loading, prompt processing, and output generation. Model architecture, tokenizer, context length, hardware, and provider scheduling determine that work. Aner does not assign a universal Big O bound or completion estimate to remote inference. The output budget and timeout bound the admitted client request; they are not throughput guarantees. Provider model memory and KV cache are outside Aner client memory accounting.

Sources and related references

The native provider contracts follow the Ollama chat API, OpenAI Responses API, and vLLM compatible server interface. Service versions may expose additional operations that are outside this Aner module.

See Credentials, Development and production, and Interactive Aner notebooks for credential lifecycle and observation boundaries.