Model API parameter index

Use the model directory to choose a technique and machine learning for shared fitted workflows. This generated reference lists the supported model, training and evaluation signature families and explains every argument in each family. Generic Tensor arithmetic, tokenizer construction and unrelated module APIs remain in their owning references.

The current index contains 423 signature families and 902 parameter entries across 12 model and technique guides.

Signatures show implemented overload defaults using = value for reference. This notation does not add user defined default argument syntax or make an unsupported call valid. Receiver syntax omits the first model/value argument. Parameters marked required must be supplied to that family. Model specific guides explain validation, shapes, device support and lifecycle in context.

These pages describe the reviewed model API contracts. Dense dropout with v7 checkpoint continuation, NLP and its neural feed-forward/masked-pooling bridges require Aner 0.0.5 or newer. Other model families retain their own checkpoint formats and capability requirements. Runtime packages, optional backends and the VS Code 0.1.48 publication candidate have separate verification and publication records; see Downloads for actual availability.

Natural language processing

Read the model guide.

nlp.analyze(text: String, stop_words: List = List(), analyzer: String = "word", min_ngram: Int64 = 1, max_ngram: Int64 = 1, casefold: Bool = true, normalization: String = "nfc") -> List

Inspect the exact normalized lexical terms used by fitting and search. Terms are not reversible model token IDs. Requires import aner.nlp. Runtime: Aner 0.0.5 or newer. Immutable bounded CPU state; no hidden global random stream. Time O(normalized scalars × ngram span + emitted bytes); returned space O(emitted term bytes), subject to work and List limits.

nlp.batch(corpus: TextCorpus, codec: Tokenizer, max_length: Int64, truncation: String = "error", bos: Bool = true, eos: Bool = true, pad_id: Int64 = -1) -> TokenBatch

Build right padded exact integer model IDs in corpus order with visible original lengths, truncation and source identity. Mask is detached CPU Float64 [batch,width]; an empty encoded row remains all padding. Requires import aner.nlp. Runtime: Aner 0.0.5 or newer. Immutable bounded CPU state; no hidden global random stream. Time O(encoded tokens + batch×width); space O(batch×width + input provenance).

nlp.batch_size(value: TokenBatch) -> Int64

Return document rows in this token batch. Requires import aner.nlp. Runtime: Aner 0.0.5 or newer. Immutable bounded CPU state; no hidden global random stream. Scalar access is O(1); owned List/text exports take O(returned fields + bytes) time and space and obey language heap limits.

nlp.cols(value: TextFeatures) -> Int64

Return fitted vocabulary width. Requires import aner.nlp. Runtime: Aner 0.0.5 or newer. Immutable bounded CPU state; no hidden global random stream. Scalar access is O(1); owned List/text exports take O(returned fields + bytes) time and space and obey language heap limits.

nlp.columns(value: TextFeatures) -> List

Return zero based column IDs, sorted within each CSR row. Requires import aner.nlp. Runtime: Aner 0.0.5 or newer. Immutable bounded CPU state; no hidden global random stream. Scalar access is O(1); owned List/text exports take O(returned fields + bytes) time and space and obey language heap limits.

nlp.configuration(value: TextIndex) -> Dictionary

Return complete canonical analyzer and BM25 parameters including explicit stop words. Requires import aner.nlp. Runtime: Aner 0.0.5 or newer. Immutable bounded CPU state; no hidden global random stream. Scalar access is O(1); owned List/text exports take O(returned fields + bytes) time and space and obey language heap limits.

nlp.configuration(value: TextVectorizer) -> Dictionary

Return complete canonical analyzer and fitting options including explicit stop words. Requires import aner.nlp. Runtime: Aner 0.0.5 or newer. Immutable bounded CPU state; no hidden global random stream. Scalar access is O(1); owned List/text exports take O(returned fields + bytes) time and space and obey language heap limits.

nlp.context(value: TextMatches, codec: Tokenizer, max_tokens: Int64, separator: String = "\n\n") -> TextContext

Assemble ranked whole documents greedily under the exact tokenizer budget. Skip an oversized candidate and continue considering later smaller hits. Never truncate evidence or silently add instructions. Requires import aner.nlp. Runtime: Aner 0.0.5 or newer. Immutable bounded CPU state; no hidden global random stream. Time O(sum of candidate assembly encoding work); space O(accepted text + retained source IDs).

nlp.contributions(value: TextMatches, hit: Int64) -> List

Return each query term contribution to one hit, aligned with matches.terms(). The sum equals that hit score within floating point arithmetic. Requires import aner.nlp. Runtime: Aner 0.0.5 or newer. Immutable bounded CPU state; no hidden global random stream. Time and returned space O(unique query terms).

nlp.corpus(texts: List, ids: List, role: String = "training") -> TextCorpus

Snapshot 1..100000 documents and IDs without modifying source text. Requires import aner.nlp. Runtime: Aner 0.0.5 or newer. Immutable bounded CPU state; no hidden global random stream. Time and space O(input bytes + rows).

nlp.dense(value: TextFeatures, max_cells: Int64) -> Tensor

Materialize sparse features explicitly as a detached CPU Float64 matrix. Keep the sparse row IDs separately before fitting a Tensor only model. Requires import aner.nlp. Runtime: Aner 0.0.5 or newer. Immutable bounded CPU state; no hidden global random stream. Time and returned space O(rows × columns).

nlp.document_frequency(value: TextVectorizer) -> List

Return per column training document frequencies. Requires import aner.nlp. Runtime: Aner 0.0.5 or newer. Immutable bounded CPU state; no hidden global random stream. Scalar access is O(1); owned List/text exports take O(returned fields + bytes) time and space and obey language heap limits.

nlp.document_hashes(value: TextCorpus) -> List

Return raw document SHA256 values in corpus row order. Requires import aner.nlp. Runtime: Aner 0.0.5 or newer. Immutable bounded CPU state; no hidden global random stream. Scalar access is O(1); owned List/text exports take O(returned fields + bytes) time and space and obey language heap limits.

nlp.empty_rows(value: TextFeatures) -> List

Return zero based positions of rows with no retained nonzero feature, including stop word only and OOV only rows. Requires import aner.nlp. Runtime: Aner 0.0.5 or newer. Immutable bounded CPU state; no hidden global random stream. Scalar access is O(1); owned List/text exports take O(returned fields + bytes) time and space and obey language heap limits.

nlp.fingerprint(value: TextContext) -> String

Return a versioned SHA256 identity covering the operation policy, ordered source identity and retained state. An identity is traceability evidence, not a privacy guarantee or authenticated signature. Requires import aner.nlp. Runtime: Aner 0.0.5 or newer. Immutable bounded CPU state; no hidden global random stream. Time O(1); returned space O(1).

nlp.fingerprint(value: TextCorpus) -> String

Return a versioned SHA256 identity covering the operation policy, ordered source identity and retained state. An identity is traceability evidence, not a privacy guarantee or authenticated signature. Requires import aner.nlp. Runtime: Aner 0.0.5 or newer. Immutable bounded CPU state; no hidden global random stream. Time O(1); returned space O(1).

nlp.fingerprint(value: TextFeatures) -> String

Return a versioned SHA256 identity covering the operation policy, ordered source identity and retained state. An identity is traceability evidence, not a privacy guarantee or authenticated signature. Requires import aner.nlp. Runtime: Aner 0.0.5 or newer. Immutable bounded CPU state; no hidden global random stream. Time O(1); returned space O(1).

nlp.fingerprint(value: TextIndex) -> String

Return a versioned SHA256 identity covering the operation policy, ordered source identity and retained state. An identity is traceability evidence, not a privacy guarantee or authenticated signature. Requires import aner.nlp. Runtime: Aner 0.0.5 or newer. Immutable bounded CPU state; no hidden global random stream. Time O(1); returned space O(1).

nlp.fingerprint(value: TextMatches) -> String

Return a versioned SHA256 identity covering the operation policy, ordered source identity and retained state. An identity is traceability evidence, not a privacy guarantee or authenticated signature. Requires import aner.nlp. Runtime: Aner 0.0.5 or newer. Immutable bounded CPU state; no hidden global random stream. Time O(1); returned space O(1).

nlp.fingerprint(value: TextVectorizer) -> String

Return a versioned SHA256 identity covering the operation policy, ordered source identity and retained state. An identity is traceability evidence, not a privacy guarantee or authenticated signature. Requires import aner.nlp. Runtime: Aner 0.0.5 or newer. Immutable bounded CPU state; no hidden global random stream. Time O(1); returned space O(1).

nlp.fingerprint(value: TokenBatch) -> String

Return a versioned SHA256 identity covering the operation policy, ordered source identity and retained state. An identity is traceability evidence, not a privacy guarantee or authenticated signature. Requires import aner.nlp. Runtime: Aner 0.0.5 or newer. Immutable bounded CPU state; no hidden global random stream. Time O(1); returned space O(1).

nlp.fitted_documents(value: TextVectorizer) -> Int64

Return the number of documents used for vocabulary and IDF fitting. Requires import aner.nlp. Runtime: Aner 0.0.5 or newer. Immutable bounded CPU state; no hidden global random stream. Scalar access is O(1); owned List/text exports take O(returned fields + bytes) time and space and obey language heap limits.

nlp.idf(value: TextVectorizer) -> List

Return per column smoothed training IDF values; retained for audit even when weighting does not use them. Requires import aner.nlp. Runtime: Aner 0.0.5 or newer. Immutable bounded CPU state; no hidden global random stream. Scalar access is O(1); owned List/text exports take O(returned fields + bytes) time and space and obey language heap limits.

nlp.ids(value: TextContext) -> List

Return only the IDs of accepted whole evidence documents. Requires import aner.nlp. Runtime: Aner 0.0.5 or newer. Immutable bounded CPU state; no hidden global random stream. Scalar access is O(1); owned List/text exports take O(returned fields + bytes) time and space and obey language heap limits.

nlp.ids(value: TextCorpus) -> List

Return source document IDs in corpus order. Requires import aner.nlp. Runtime: Aner 0.0.5 or newer. Immutable bounded CPU state; no hidden global random stream. Scalar access is O(1); owned List/text exports take O(returned fields + bytes) time and space and obey language heap limits.

nlp.ids(value: TextFeatures) -> List

Return unchanged transformed document IDs in sparse row order. Requires import aner.nlp. Runtime: Aner 0.0.5 or newer. Immutable bounded CPU state; no hidden global random stream. Scalar access is O(1); owned List/text exports take O(returned fields + bytes) time and space and obey language heap limits.

nlp.ids(value: TextIndex) -> List

Return indexed document IDs in source row order. Requires import aner.nlp. Runtime: Aner 0.0.5 or newer. Immutable bounded CPU state; no hidden global random stream. Scalar access is O(1); owned List/text exports take O(returned fields + bytes) time and space and obey language heap limits.

nlp.ids(value: TextMatches) -> List

Return hit IDs in ranked order. Requires import aner.nlp. Runtime: Aner 0.0.5 or newer. Immutable bounded CPU state; no hidden global random stream. Scalar access is O(1); owned List/text exports take O(returned fields + bytes) time and space and obey language heap limits.

nlp.index(memory: TextCorpus, stop_words: List = List(), k1: Float64 = 1.2, b: Float64 = 0.75, analyzer: String = "word", min_ngram: Int64 = 1, max_ngram: Int64 = 1, casefold: Bool = true, normalization: String = "nfc") -> TextIndex

Fit deterministic BM25 lexical memory. Scores use log(1+(N df+0.5)/(df+0.5)), one contribution per unique analyzed query term and explicit document lengths. Requires import aner.nlp. Runtime: Aner 0.0.5 or newer. Immutable bounded CPU state; no hidden global random stream. Time O(analyzed terms + postings ordering); space O(rows + term postings + retained source bytes).

nlp.lengths(value: TokenBatch) -> List

Return retained nonpadding token lengths per source row. Requires import aner.nlp. Runtime: Aner 0.0.5 or newer. Immutable bounded CPU state; no hidden global random stream. Scalar access is O(1); owned List/text exports take O(returned fields + bytes) time and space and obey language heap limits.

nlp.load_index(path: String) -> TextIndex

Validate bounded checksummed data and rebuild the exact fitted state from its retained source and policy. Checksum detects corruption; it does not authenticate an untrusted producer. Requires import aner.nlp. Runtime: Aner 0.0.5 or newer. Immutable bounded CPU state; no hidden global random stream. Time and space follow the corresponding deterministic fitting operation.

nlp.load_vectorizer(path: String) -> TextVectorizer

Validate bounded checksummed data and rebuild the exact fitted state from its retained source and policy. Checksum detects corruption; it does not authenticate an untrusted producer. Requires import aner.nlp. Runtime: Aner 0.0.5 or newer. Immutable bounded CPU state; no hidden global random stream. Time and space follow the corresponding deterministic fitting operation.

nlp.mask(value: TokenBatch) -> Tensor

Return detached CPU Float64 [batch,width] mask with one at retained real tokens and zero at padding. Transfer and cast explicitly to match model activations. Requires import aner.nlp. Runtime: Aner 0.0.5 or newer. Immutable bounded CPU state; no hidden global random stream. Scalar access is O(1); owned List/text exports take O(returned fields + bytes) time and space and obey language heap limits.

nlp.nnz(value: TextFeatures) -> Int64

Return explicitly stored nonzero entries. Requires import aner.nlp. Runtime: Aner 0.0.5 or newer. Immutable bounded CPU state; no hidden global random stream. Scalar access is O(1); owned List/text exports take O(returned fields + bytes) time and space and obey language heap limits.

nlp.oov(value: TextFeatures) -> List

Return analyzed term occurrences absent from the fitted vocabulary for each row. Requires import aner.nlp. Runtime: Aner 0.0.5 or newer. Immutable bounded CPU state; no hidden global random stream. Scalar access is O(1); owned List/text exports take O(returned fields + bytes) time and space and obey language heap limits.

nlp.original_lengths(value: TokenBatch) -> List

Return encoded lengths after requested BOS/EOS and before right truncation. Requires import aner.nlp. Runtime: Aner 0.0.5 or newer. Immutable bounded CPU state; no hidden global random stream. Scalar access is O(1); owned List/text exports take O(returned fields + bytes) time and space and obey language heap limits.

nlp.padding_id(value: TokenBatch) -> Int64

Return the resolved vocabulary padding ID. Requires import aner.nlp. Runtime: Aner 0.0.5 or newer. Immutable bounded CPU state; no hidden global random stream. Scalar access is O(1); owned List/text exports take O(returned fields + bytes) time and space and obey language heap limits.

nlp.positions(value: TextMatches) -> List

Return original indexed source row positions in ranked order; these are not external IDs. Requires import aner.nlp. Runtime: Aner 0.0.5 or newer. Immutable bounded CPU state; no hidden global random stream. Scalar access is O(1); owned List/text exports take O(returned fields + bytes) time and space and obey language heap limits.

nlp.role(value: TextCorpus) -> String

Return the declared partition role. Requires import aner.nlp. Runtime: Aner 0.0.5 or newer. Immutable bounded CPU state; no hidden global random stream. Scalar access is O(1); owned List/text exports take O(returned fields + bytes) time and space and obey language heap limits.

nlp.row_columns(value: TextFeatures, row: Int64) -> List

Export one CSR row as exact column IDs or aligned Float64 values without materializing the complete sparse arrays. Requires import aner.nlp. Runtime: Aner 0.0.5 or newer. Immutable bounded CPU state; no hidden global random stream. Time and returned space O(row nonzeros); language List limits still apply.

nlp.row_ptr(value: TextFeatures) -> List

Return CSR row offsets of length rows+1, beginning at zero and ending at nnz. Requires import aner.nlp. Runtime: Aner 0.0.5 or newer. Immutable bounded CPU state; no hidden global random stream. Scalar access is O(1); owned List/text exports take O(returned fields + bytes) time and space and obey language heap limits.

nlp.row_values(value: TextFeatures, row: Int64) -> List

Export one CSR row as exact column IDs or aligned Float64 values without materializing the complete sparse arrays. Requires import aner.nlp. Runtime: Aner 0.0.5 or newer. Immutable bounded CPU state; no hidden global random stream. Time and returned space O(row nonzeros); language List limits still apply.

nlp.rows(value: TextCorpus) -> Int64

Return the retained document count. Requires import aner.nlp. Runtime: Aner 0.0.5 or newer. Immutable bounded CPU state; no hidden global random stream. Scalar access is O(1); owned List/text exports take O(returned fields + bytes) time and space and obey language heap limits.

nlp.rows(value: TextFeatures) -> Int64

Return sparse feature row count, including empty rows. Requires import aner.nlp. Runtime: Aner 0.0.5 or newer. Immutable bounded CPU state; no hidden global random stream. Scalar access is O(1); owned List/text exports take O(returned fields + bytes) time and space and obey language heap limits.

nlp.rows(value: TextIndex) -> Int64

Return the indexed evidence population size. Requires import aner.nlp. Runtime: Aner 0.0.5 or newer. Immutable bounded CPU state; no hidden global random stream. Scalar access is O(1); owned List/text exports take O(returned fields + bytes) time and space and obey language heap limits.

nlp.rows(value: TextMatches) -> Int64

Return retained positive score hit count; zero is valid. Requires import aner.nlp. Runtime: Aner 0.0.5 or newer. Immutable bounded CPU state; no hidden global random stream. Scalar access is O(1); owned List/text exports take O(returned fields + bytes) time and space and obey language heap limits.

nlp.save(value: TextIndex, path: String) -> Unit

Save canonical checksummed data including fitting/evidence text and source IDs. Refuse overwrite; artifacts can contain sensitive source material. No executable object deserialization occurs. Requires import aner.nlp. Runtime: Aner 0.0.5 or newer. Immutable bounded CPU state; no hidden global random stream. Time and artifact space O(retained source bytes + fitted metadata).

nlp.save(value: TextVectorizer, path: String) -> Unit

Save canonical checksummed data including fitting/evidence text and source IDs. Refuse overwrite; artifacts can contain sensitive source material. No executable object deserialization occurs. Requires import aner.nlp. Runtime: Aner 0.0.5 or newer. Immutable bounded CPU state; no hidden global random stream. Time and artifact space O(retained source bytes + fitted metadata).

nlp.scores(value: TextMatches) -> List

Return finite positive BM25 scores in ranked order. Requires import aner.nlp. Runtime: Aner 0.0.5 or newer. Immutable bounded CPU state; no hidden global random stream. Scalar access is O(1); owned List/text exports take O(returned fields + bytes) time and space and obey language heap limits.

nlp.search(value: TextIndex, query: String, top_k: Int64 = 5) -> TextMatches

Rank positive score BM25 matches while retaining source row IDs, original text, source positions and per term contributions. No semantic embedding or external lookup is implied. Requires import aner.nlp. Runtime: Aner 0.0.5 or newer. Immutable bounded CPU state; no hidden global random stream. Time O(query terms + scored postings + candidates log candidates); result space O(top_k × query terms + source hit text).

nlp.sequence_length(value: TokenBatch) -> Int64

Return exact configured padded width. Requires import aner.nlp. Runtime: Aner 0.0.5 or newer. Immutable bounded CPU state; no hidden global random stream. Scalar access is O(1); owned List/text exports take O(returned fields + bytes) time and space and obey language heap limits.

nlp.skipped(value: TextContext) -> Int64

Return candidate hits skipped because their whole document assembly exceeded the token budget. Requires import aner.nlp. Runtime: Aner 0.0.5 or newer. Immutable bounded CPU state; no hidden global random stream. Scalar access is O(1); owned List/text exports take O(returned fields + bytes) time and space and obey language heap limits.

nlp.source_fingerprint(value: TokenBatch) -> String

Return originating TextCorpus fingerprint including its role, source IDs, raw text and order. Requires import aner.nlp. Runtime: Aner 0.0.5 or newer. Immutable bounded CPU state; no hidden global random stream. Scalar access is O(1); owned List/text exports take O(returned fields + bytes) time and space and obey language heap limits.

nlp.source_ids(value: TokenBatch) -> List

Return source document IDs in batch row order. Requires import aner.nlp. Runtime: Aner 0.0.5 or newer. Immutable bounded CPU state; no hidden global random stream. Scalar access is O(1); owned List/text exports take O(returned fields + bytes) time and space and obey language heap limits.

nlp.summary(value: TextContext) -> String

Return a bounded human readable summary of shape, policy and provenance. Original document content is not printed automatically. Requires import aner.nlp. Runtime: Aner 0.0.5 or newer. Immutable bounded CPU state; no hidden global random stream. Time and returned space O(summary metadata bytes).

nlp.summary(value: TextCorpus) -> String

Return a bounded human readable summary of shape, policy and provenance. Original document content is not printed automatically. Requires import aner.nlp. Runtime: Aner 0.0.5 or newer. Immutable bounded CPU state; no hidden global random stream. Time and returned space O(summary metadata bytes).

nlp.summary(value: TextFeatures) -> String

Return a bounded human readable summary of shape, policy and provenance. Original document content is not printed automatically. Requires import aner.nlp. Runtime: Aner 0.0.5 or newer. Immutable bounded CPU state; no hidden global random stream. Time and returned space O(summary metadata bytes).

nlp.summary(value: TextIndex) -> String

Return a bounded human readable summary of shape, policy and provenance. Original document content is not printed automatically. Requires import aner.nlp. Runtime: Aner 0.0.5 or newer. Immutable bounded CPU state; no hidden global random stream. Time and returned space O(summary metadata bytes).

nlp.summary(value: TextMatches) -> String

Return a bounded human readable summary of shape, policy and provenance. Original document content is not printed automatically. Requires import aner.nlp. Runtime: Aner 0.0.5 or newer. Immutable bounded CPU state; no hidden global random stream. Time and returned space O(summary metadata bytes).

nlp.summary(value: TextVectorizer) -> String

Return a bounded human readable summary of shape, policy and provenance. Original document content is not printed automatically. Requires import aner.nlp. Runtime: Aner 0.0.5 or newer. Immutable bounded CPU state; no hidden global random stream. Time and returned space O(summary metadata bytes).

nlp.summary(value: TokenBatch) -> String

Return a bounded human readable summary of shape, policy and provenance. Original document content is not printed automatically. Requires import aner.nlp. Runtime: Aner 0.0.5 or newer. Immutable bounded CPU state; no hidden global random stream. Time and returned space O(summary metadata bytes).

nlp.terms(value: TextMatches) -> List

Return canonical unique analyzed query terms aligned with every contributions vector. Requires import aner.nlp. Runtime: Aner 0.0.5 or newer. Immutable bounded CPU state; no hidden global random stream. Scalar access is O(1); owned List/text exports take O(returned fields + bytes) time and space and obey language heap limits.

nlp.text(value: TextContext) -> String

Return exact assembled evidence text, or empty text when no document fits. Requires import aner.nlp. Runtime: Aner 0.0.5 or newer. Immutable bounded CPU state; no hidden global random stream. Scalar access is O(1); owned List/text exports take O(returned fields + bytes) time and space and obey language heap limits.

nlp.texts(value: TextCorpus) -> List

Return original document texts without analyzer normalization. Requires import aner.nlp. Runtime: Aner 0.0.5 or newer. Immutable bounded CPU state; no hidden global random stream. Scalar access is O(1); owned List/text exports take O(returned fields + bytes) time and space and obey language heap limits.

nlp.texts(value: TextMatches) -> List

Return original hit document texts in ranked order. Requires import aner.nlp. Runtime: Aner 0.0.5 or newer. Immutable bounded CPU state; no hidden global random stream. Scalar access is O(1); owned List/text exports take O(returned fields + bytes) time and space and obey language heap limits.

nlp.token_ids(value: TokenBatch, row: Int64) -> List

Export one exact padded token ID row of at most 4096 Int64 values without materializing the full batch List. Requires import aner.nlp. Runtime: Aner 0.0.5 or newer. Immutable bounded CPU state; no hidden global random stream. Time and returned space O(sequence width).

nlp.token_ids(value: TokenBatch) -> List

Return exact Int64 token IDs flattened in row major order, including padding. Large native batches may exceed the smaller List export bound; use row export. Requires import aner.nlp. Runtime: Aner 0.0.5 or newer. Immutable bounded CPU state; no hidden global random stream. Scalar access is O(1); owned List/text exports take O(returned fields + bytes) time and space and obey language heap limits.

nlp.tokens(value: TextContext) -> Int64

Return exact encoded count of the assembled text under its retained codec, without BOS/EOS. Requires import aner.nlp. Runtime: Aner 0.0.5 or newer. Immutable bounded CPU state; no hidden global random stream. Scalar access is O(1); owned List/text exports take O(returned fields + bytes) time and space and obey language heap limits.

nlp.training_hashes(value: TextIndex) -> List

Return raw indexed document SHA256 values in source row order, including duplicate content hashes. Requires import aner.nlp. Runtime: Aner 0.0.5 or newer. Immutable bounded CPU state; no hidden global random stream. Scalar access is O(1); owned List/text exports take O(returned fields + bytes) time and space and obey language heap limits.

nlp.training_hashes(value: TextVectorizer) -> List

Return raw fitting document SHA256 values in fitting row order, including duplicate content hashes. Requires import aner.nlp. Runtime: Aner 0.0.5 or newer. Immutable bounded CPU state; no hidden global random stream. Scalar access is O(1); owned List/text exports take O(returned fields + bytes) time and space and obey language heap limits.

nlp.transform(value: TextVectorizer, corpus: TextCorpus) -> TextFeatures

Apply the unchanged fitted analyzer/vocabulary/IDF to an ordered corpus. Held out roles reject reused fitting IDs, raw content and policy-normalized/casefolded content overlap. Preserve IDs, OOV occurrence counts and zero rows. Requires import aner.nlp. Runtime: Aner 0.0.5 or newer. Immutable bounded CPU state; no hidden global random stream. Time O(analyzed terms + row nonzeros log row nonzeros); space O(rows + nonzeros), capped at 2000000 native sparse entries.

nlp.truncated(value: TokenBatch) -> List

Return one Bool per row; true means the original encoded length exceeded max_length. Requires import aner.nlp. Runtime: Aner 0.0.5 or newer. Immutable bounded CPU state; no hidden global random stream. Scalar access is O(1); owned List/text exports take O(returned fields + bytes) time and space and obey language heap limits.

nlp.values(value: TextFeatures) -> List

Return finite CPU Float64 feature values aligned with columns. Requires import aner.nlp. Runtime: Aner 0.0.5 or newer. Immutable bounded CPU state; no hidden global random stream. Scalar access is O(1); owned List/text exports take O(returned fields + bytes) time and space and obey language heap limits.

nlp.vectorizer(training: TextCorpus, stop_words: List = List(), weighting: String = "tfidf", analyzer: String = "word", min_ngram: Int64 = 1, max_ngram: Int64 = 1, min_df: Int64 = 1, max_df: Float64 = 1.0, max_features: Int64 = 20000, casefold: Bool = true, sublinear_tf: Bool = false, norm: String = "l2", normalization: String = "nfc") -> TextVectorizer

Fit a deterministic sparse lexical representation using only a training role corpus; reject an empty retained vocabulary. Requires import aner.nlp. Runtime: Aner 0.0.5 or newer. Immutable bounded CPU state; no hidden global random stream. Time O(analyzed terms + distinct terms log distinct terms); fitted space O(vocabulary + training provenance).

nlp.vocabulary(value: TextVectorizer) -> List

Return exact feature names in fitted column order. Requires import aner.nlp. Runtime: Aner 0.0.5 or newer. Immutable bounded CPU state; no hidden global random stream. Scalar access is O(1); owned List/text exports take O(returned fields + bytes) time and space and obey language heap limits.

nn.feed_forward(input: Tensor, w_in: Tensor, b_in: Tensor, w_out: Tensor, b_out: Tensor) -> Tensor

Apply exact GELU((input × w_in)+b_in) × w_out+b_out to rank two through rank eight input, preserving leading dimensions and autograd. Normalization, dropout and residual addition remain explicit caller choices. Requires import aner.nn. Time O(leading_rows × (input_width×hidden_width+hidden_width×output_width)); graph space follows retained projections within native tensor/work limits.

nn.masked_mean(input: Tensor, mask: Tensor) -> Tensor

Pool nonpadding tokens to [batch,width] with zero gradient at excluded positions. No causal restriction or attention weighting is added. Mask validation may inspect mask data; activation data remains on its execution backend. Requires import aner.nn. Time O(batch×time×width); graph space O(batch×time×width) within native tensor/work limits.

POS and named entity tagging

Read the model guide.

nlp.accuracy(value: TaggingEvaluation) -> Float64

Return corpus token accuracy; zero total tokens yields the documented zero result. Requires import aner.nlp. Runtime: Aner 0.0.5 or newer. Immutable bounded CPU state; no model download or hidden random stream. Scalar access is O(1); exported metric Lists are O(reported labels or entity types).

nlp.annotate(value: TextTokens, labels: List, task: String) -> TaggedCorpus

Validate a complete source aligned gold annotation. Labels and spans participate in the annotated corpus fingerprint. NER types are ASCII identifiers of at most64bytes; nested/discontinuous entities are outside BIO2. Requires import aner.nlp. Runtime: Aner 0.0.5 or newer. Immutable bounded CPU state; no model download or hidden random stream. Time and space O(tokens + label bytes).

nlp.configuration(value: TextTagger) -> Dictionary

Return canonical cumulative epoch count, seed/shuffle, feature normalization/casefold and versioned learning/feature protocol metadata. Requires import aner.nlp. Runtime: Aner 0.0.5 or newer. Immutable bounded CPU state; no model download or hidden random stream. Scalars are O(1); List export follows returned fields/bytes and the explicit language heap budget.

nlp.confusion(value: TaggingEvaluation) -> List

Return positive token confusion Dictionaries with gold,predicted,count sorted lexically by label pair, including the correct diagonal; counts sum to token_count. Requires import aner.nlp. Runtime: Aner 0.0.5 or newer. Immutable bounded CPU state; no model download or hidden random stream. Scalar access is O(1); exported metric Lists are O(reported labels or entity types).

nlp.correct(value: TaggingEvaluation) -> Int64

Return exactly correct token labels. Requires import aner.nlp. Runtime: Aner 0.0.5 or newer. Immutable bounded CPU state; no model download or hidden random stream. Scalar access is O(1); exported metric Lists are O(reported labels or entity types).

nlp.entities(value: TaggedCorpus, row: Int64) -> List

Return NER entities as Dictionaries with id/type/text/row and half open token_start/token_end, byte_start/byte_end and scalar_start/scalar_end. Source text retains intervening whitespace; POS calls reject. Requires import aner.nlp. Runtime: Aner 0.0.5 or newer. Immutable bounded CPU state; no model download or hidden random stream. Time O(row tokens + entity bytes); returned space O(entities + source slice bytes).

nlp.entities(value: TextTagging, row: Int64) -> List

Return NER entities as Dictionaries with id/type/text/row and half open token_start/token_end, byte_start/byte_end and scalar_start/scalar_end. Source text retains intervening whitespace; POS calls reject. Requires import aner.nlp. Runtime: Aner 0.0.5 or newer. Immutable bounded CPU state; no model download or hidden random stream. Time O(row tokens + entity bytes); returned space O(entities + source slice bytes).

nlp.entity_metric(value: TaggingEvaluation) -> Dictionary

Return NER strict full span micro gold,predicted,correct,precision,recall,f1. A hit requires exact source row, type and complete span; POS calls reject. Requires import aner.nlp. Runtime: Aner 0.0.5 or newer. Immutable bounded CPU state; no model download or hidden random stream. Scalar access is O(1); exported metric Lists are O(reported labels or entity types).

nlp.entity_metrics(value: TaggingEvaluation) -> List

Return per type NER exact span Dictionaries with label,gold,predicted,correct,precision,recall,f1; undefined denominators produce zero. POS calls reject. Requires import aner.nlp. Runtime: Aner 0.0.5 or newer. Immutable bounded CPU state; no model download or hidden random stream. Scalar access is O(1); exported metric Lists are O(reported labels or entity types).

nlp.epochs(value: TextTagger) -> Int64

Return cumulative completed epochs; resumed epochs add to this value. Requires import aner.nlp. Runtime: Aner 0.0.5 or newer. Immutable bounded CPU state; no model download or hidden random stream. Scalars are O(1); List export follows returned fields/bytes and the explicit language heap budget.

nlp.evaluate_tagger(value: TextTagger, gold: TaggedCorpus) -> TaggingEvaluation

Predict and evaluate with the model bound held out overlap guard. Report exact token counts, per label metrics and strict full span NER metrics with source/model/gold provenance. Declared roles do not replace group/time partition design. Requires import aner.nlp. Runtime: Aner 0.0.5 or newer. Immutable bounded CPU state; no model download or hidden random stream. Time follows decoding plus O(tokens + entities) metric counting.

nlp.evaluate_tags(value: TextTagging, gold: TaggedCorpus) -> TaggingEvaluation

Evaluate a retained prediction without rerunning decoding. This path checks alignment but cannot infer independence from model fitting; use evaluate_tagger for the model bound held out guard. Requires import aner.nlp. Runtime: Aner 0.0.5 or newer. Immutable bounded CPU state; no model download or hidden random stream. Time O(tokens log labels + entities log entities) with bounded metric maps/sets and span storage.

nlp.examples_seen(value: TextTagger) -> Int64

Return cumulative nonempty training sequences processed, including correctly predicted examples. Requires import aner.nlp. Runtime: Aner 0.0.5 or newer. Immutable bounded CPU state; no model download or hidden random stream. Scalars are O(1); List export follows returned fields/bytes and the explicit language heap budget.

nlp.feature_names(value: TextTagger) -> List

Return canonical learned emission feature names for lexical/context/shape/affix scores. Reserved transition names are inspected separately. Full export obeys the smaller language heap bound. Requires import aner.nlp. Runtime: Aner 0.0.5 or newer. Immutable bounded CPU state; no model download or hidden random stream. Scalars are O(1); List export follows returned fields/bytes and the explicit language heap budget.

nlp.fingerprint(value: TaggedCorpus) -> String

Return a SHA256 identity covering this value protocol and retained state. Fingerprints provide identity, not authentication or anonymity. Requires import aner.nlp. Runtime: Aner 0.0.5 or newer. Immutable bounded CPU state; no model download or hidden random stream. O(1) stored identity readout.

nlp.fingerprint(value: TaggingEvaluation) -> String

Return a SHA256 identity covering this value protocol and retained state. Fingerprints provide identity, not authentication or anonymity. Requires import aner.nlp. Runtime: Aner 0.0.5 or newer. Immutable bounded CPU state; no model download or hidden random stream. O(1) stored identity readout.

nlp.fingerprint(value: TextTagger) -> String

Return a SHA256 identity covering this value protocol and retained state. Fingerprints provide identity, not authentication or anonymity. Requires import aner.nlp. Runtime: Aner 0.0.5 or newer. Immutable bounded CPU state; no model download or hidden random stream. O(1) stored identity readout.

nlp.fingerprint(value: TextTagging) -> String

Return a SHA256 identity covering this value protocol and retained state. Fingerprints provide identity, not authentication or anonymity. Requires import aner.nlp. Runtime: Aner 0.0.5 or newer. Immutable bounded CPU state; no model download or hidden random stream. O(1) stored identity readout.

nlp.fingerprint(value: TextTokens) -> String

Return a SHA256 identity covering this value protocol and retained state. Fingerprints provide identity, not authentication or anonymity. Requires import aner.nlp. Runtime: Aner 0.0.5 or newer. Immutable bounded CPU state; no model download or hidden random stream. O(1) stored identity readout.

nlp.fit_tagger(training: TaggedCorpus, epochs: Int64 = 5, seed: Int64 = 42, shuffle: Bool = true, normalization: String = "nfc", casefold: Bool = true) -> TextTagger

Fit a first order averaged structured perceptron with lexical/context/shape/affix features and exact Viterbi decoding. NER paths satisfy strict BIO2; POS predicts the observed UPOS subset. No pretrained weights or probability calibration are supplied. Requires import aner.nlp. Runtime: Aner 0.0.5 or newer. Immutable bounded CPU state; no model download or hidden random stream. Time includes O(epochs × tokens × labels²) decoding; cells and work are explicitly bounded.

nlp.gold_fingerprint(value: TaggingEvaluation) -> String

Return the exact gold annotation identity used for metric computation. Requires import aner.nlp. Runtime: Aner 0.0.5 or newer. Immutable bounded CPU state; no model download or hidden random stream. Scalar access is O(1); exported metric Lists are O(reported labels or entity types).

nlp.ids(value: TaggedCorpus) -> List

Return original document IDs in retained sequence row order. Requires import aner.nlp. Runtime: Aner 0.0.5 or newer. Immutable bounded CPU state; no model download or hidden random stream. Scalar readouts are O(1); List export is O(returned fields + bytes) and follows the language heap bound.

nlp.ids(value: TextTagging) -> List

Return original document IDs in retained sequence row order. Requires import aner.nlp. Runtime: Aner 0.0.5 or newer. Immutable bounded CPU state; no model download or hidden random stream. Scalar readouts are O(1); List export is O(returned fields + bytes) and follows the language heap bound.

nlp.ids(value: TextTokens) -> List

Return original document IDs in retained sequence row order. Requires import aner.nlp. Runtime: Aner 0.0.5 or newer. Immutable bounded CPU state; no model download or hidden random stream. Scalar readouts are O(1); List export is O(returned fields + bytes) and follows the language heap bound.

nlp.labels(value: TextTagger) -> List

Return the exact lexicographically ordered fitted output inventory; NER includes O and B/I closure for each observed type. Requires import aner.nlp. Runtime: Aner 0.0.5 or newer. Immutable bounded CPU state; no model download or hidden random stream. Scalars are O(1); List export follows returned fields/bytes and the explicit language heap budget.

nlp.load_tagger(path: String) -> TextTagger

Validate the bounded canonical checksummed checkpoint and restore exact current/averaged training state without running producer code or refitting from source text. Checksum is not an authenticated signature. Requires import aner.nlp. Runtime: Aner 0.0.5 or newer. Immutable bounded CPU state; no model download or hidden random stream. Time and space O(validated artifact bytes + fitted state).

nlp.model_fingerprint(value: TextTagging) -> String

Return the complete model fingerprint used for this prediction; preserve it with the source token and output fingerprints. Requires import aner.nlp. Runtime: Aner 0.0.5 or newer. Immutable bounded CPU state; no model download or hidden random stream. O(1) readout.

nlp.prediction_fingerprint(value: TaggingEvaluation) -> String

Return the exact retained prediction identity used for metric computation. Requires import aner.nlp. Runtime: Aner 0.0.5 or newer. Immutable bounded CPU state; no model download or hidden random stream. Scalar access is O(1); exported metric Lists are O(reported labels or entity types).

nlp.resume_tagger(value: TextTagger, training: TaggedCorpus, epochs: Int64) -> TextTagger

Continue current integer weights, lazy averaging totals/timestamps, cumulative epoch shuffle and counters. Preserve seed, shuffle and feature policy; checkpoint continuation reproduces an uninterrupted run under the same protocol. Requires import aner.nlp. Runtime: Aner 0.0.5 or newer. Immutable bounded CPU state; no model download or hidden random stream. Time and additional work follow the requested epochs; no silent optimizer or averaging reset.

nlp.role(value: TaggedCorpus) -> String

Return the original corpus partition role, without inferring held out independence. Requires import aner.nlp. Runtime: Aner 0.0.5 or newer. Immutable bounded CPU state; no model download or hidden random stream. Scalar readouts are O(1); List export is O(returned fields + bytes) and follows the language heap bound.

nlp.role(value: TaggingEvaluation) -> String

Return the declared evaluated corpus role. A role alone does not establish independence from fitting. Requires import aner.nlp. Runtime: Aner 0.0.5 or newer. Immutable bounded CPU state; no model download or hidden random stream. Scalar access is O(1); exported metric Lists are O(reported labels or entity types).

nlp.role(value: TextTagging) -> String

Return the original corpus partition role, without inferring held out independence. Requires import aner.nlp. Runtime: Aner 0.0.5 or newer. Immutable bounded CPU state; no model download or hidden random stream. Scalar readouts are O(1); List export is O(returned fields + bytes) and follows the language heap bound.

nlp.role(value: TextTokens) -> String

Return the original corpus partition role, without inferring held out independence. Requires import aner.nlp. Runtime: Aner 0.0.5 or newer. Immutable bounded CPU state; no model download or hidden random stream. Scalar readouts are O(1); List export is O(returned fields + bytes) and follows the language heap bound.

nlp.rows(value: TaggedCorpus) -> Int64

Return the original sequence row count, including empty rows. Requires import aner.nlp. Runtime: Aner 0.0.5 or newer. Immutable bounded CPU state; no model download or hidden random stream. Scalar readouts are O(1); List export is O(returned fields + bytes) and follows the language heap bound.

nlp.rows(value: TextTagging) -> Int64

Return the original sequence row count, including empty rows. Requires import aner.nlp. Runtime: Aner 0.0.5 or newer. Immutable bounded CPU state; no model download or hidden random stream. Scalar readouts are O(1); List export is O(returned fields + bytes) and follows the language heap bound.

nlp.rows(value: TextTokens) -> Int64

Return the original sequence row count, including empty rows. Requires import aner.nlp. Runtime: Aner 0.0.5 or newer. Immutable bounded CPU state; no model download or hidden random stream. Scalar readouts are O(1); List export is O(returned fields + bytes) and follows the language heap bound.

nlp.save(value: TextTagger, path: String) -> Unit

Write a bounded exclusive data only checkpoint containing current integer weights, lazy averaging totals/timestamps, continuation counters and fitting provenance. Raw training text is excluded; learned lexical features and source IDs remain retained. Requires import aner.nlp. Runtime: Aner 0.0.5 or newer. Immutable bounded CPU state; no model download or hidden random stream. Time and artifact space O(fitted cells + feature/label/provenance bytes).

nlp.scores(value: TextTagging) -> List

Return one finite uncalibrated complete sequence score per source row. Empty rows have the engine defined zero score; no token confidence or probability is implied. Requires import aner.nlp. Runtime: Aner 0.0.5 or newer. Immutable bounded CPU state; no model download or hidden random stream. Time and returned space O(rows).

nlp.source(value: TextTokens) -> TextCorpus

Return the original immutable corpus, preserving source IDs, raw UTF-8 text and declared role. No feature normalization is applied. Requires import aner.nlp. Runtime: Aner 0.0.5 or newer. Immutable bounded CPU state; no model download or hidden random stream. O(1) shared immutable handle.

nlp.summary(value: TaggedCorpus) -> String

Return bounded shape/task/state provenance without automatically printing original source text or labels. Requires import aner.nlp. Runtime: Aner 0.0.5 or newer. Immutable bounded CPU state; no model download or hidden random stream. Time and returned space O(summary metadata bytes).

nlp.summary(value: TaggingEvaluation) -> String

Return bounded shape/task/state provenance without automatically printing original source text or labels. Requires import aner.nlp. Runtime: Aner 0.0.5 or newer. Immutable bounded CPU state; no model download or hidden random stream. Time and returned space O(summary metadata bytes).

nlp.summary(value: TextTagger) -> String

Return bounded shape/task/state provenance without automatically printing original source text or labels. Requires import aner.nlp. Runtime: Aner 0.0.5 or newer. Immutable bounded CPU state; no model download or hidden random stream. Time and returned space O(summary metadata bytes).

nlp.summary(value: TextTagging) -> String

Return bounded shape/task/state provenance without automatically printing original source text or labels. Requires import aner.nlp. Runtime: Aner 0.0.5 or newer. Immutable bounded CPU state; no model download or hidden random stream. Time and returned space O(summary metadata bytes).

nlp.summary(value: TextTokens) -> String

Return bounded shape/task/state provenance without automatically printing original source text or labels. Requires import aner.nlp. Runtime: Aner 0.0.5 or newer. Immutable bounded CPU state; no model download or hidden random stream. Time and returned space O(summary metadata bytes).

nlp.tag(value: TextTagger, tokens: TextTokens) -> TextTagging

Reject held out fitted IDs/raw/normalized-token overlap and decode all sequences with averaged weights, preserving source tokens, IDs, model identity and uncalibrated complete sequence scores. Global equal score paths use the lexicographically smallest label index sequence; empty rows remain empty. Requires import aner.nlp. Runtime: Aner 0.0.5 or newer. Immutable bounded CPU state; no model download or hidden random stream. Time O(tokens × labels²) plus features; bounded dynamic programming space.

nlp.tag_metrics(value: TaggingEvaluation) -> List

Return per label Dictionaries with label,gold,predicted,correct,precision,recall,f1. Undefined precision/recall/F1 denominators produce zero. Requires import aner.nlp. Runtime: Aner 0.0.5 or newer. Immutable bounded CPU state; no model download or hidden random stream. Scalar access is O(1); exported metric Lists are O(reported labels or entity types).

nlp.tag_score(value: TextTagger, tokens: TextTokens, row: Int64, labels: List, averaged: Bool = true) -> Float64

Score an explicitly supplied complete path with current or averaged linear weights. This is an uncalibrated sequence score, not a normalized probability or cross length confidence. Requires import aner.nlp. Runtime: Aner 0.0.5 or newer. Immutable bounded CPU state; no model download or hidden random stream. Time O(sequence feature work + sequence length); bounded row feature and label path storage.

nlp.tagger_weights(value: TextTagger, feature: String, averaged: Bool = true) -> List

Inspect one learned feature as Float64 weights aligned to labels(). Weights are linear score components, not token probabilities. Requires import aner.nlp. Runtime: Aner 0.0.5 or newer. Immutable bounded CPU state; no model download or hidden random stream. Time and returned space O(label count).

nlp.tags(value: TaggedCorpus, row: Int64) -> List

Return one sequence of exact String labels aligned one for one to original source tokens. NER uses valid BIO2 and POS uses Universal POS spellings. Requires import aner.nlp. Runtime: Aner 0.0.5 or newer. Immutable bounded CPU state; no model download or hidden random stream. Time and returned space O(row token labels); language List limits apply.

nlp.tags(value: TextTagging, row: Int64) -> List

Return one sequence of exact String labels aligned one for one to original source tokens. NER uses valid BIO2 and POS uses Universal POS spellings. Requires import aner.nlp. Runtime: Aner 0.0.5 or newer. Immutable bounded CPU state; no model download or hidden random stream. Time and returned space O(row token labels); language List limits apply.

nlp.task(value: TaggedCorpus) -> String

Return the declared task string pos or ner. The task determines label validation, legal transitions and metric interpretation. Requires import aner.nlp. Runtime: Aner 0.0.5 or newer. Immutable bounded CPU state; no model download or hidden random stream. O(1) readout.

nlp.task(value: TaggingEvaluation) -> String

Return the declared task string pos or ner. The task determines label validation, legal transitions and metric interpretation. Requires import aner.nlp. Runtime: Aner 0.0.5 or newer. Immutable bounded CPU state; no model download or hidden random stream. O(1) readout.

nlp.task(value: TextTagger) -> String

Return the declared task string pos or ner. The task determines label validation, legal transitions and metric interpretation. Requires import aner.nlp. Runtime: Aner 0.0.5 or newer. Immutable bounded CPU state; no model download or hidden random stream. O(1) readout.

nlp.task(value: TextTagging) -> String

Return the declared task string pos or ner. The task determines label validation, legal transitions and metric interpretation. Requires import aner.nlp. Runtime: Aner 0.0.5 or newer. Immutable bounded CPU state; no model download or hidden random stream. O(1) readout.

nlp.token_count(value: TaggedCorpus) -> Int64

Return total source aligned tokens across retained sequences. Requires import aner.nlp. Runtime: Aner 0.0.5 or newer. Immutable bounded CPU state; no model download or hidden random stream. Scalar readouts are O(1); List export is O(returned fields + bytes) and follows the language heap bound.

nlp.token_count(value: TaggingEvaluation) -> Int64

Return the evaluated gold token count, including every row. Requires import aner.nlp. Runtime: Aner 0.0.5 or newer. Immutable bounded CPU state; no model download or hidden random stream. Scalar access is O(1); exported metric Lists are O(reported labels or entity types).

nlp.token_count(value: TextTagging) -> Int64

Return total source aligned tokens across retained sequences. Requires import aner.nlp. Runtime: Aner 0.0.5 or newer. Immutable bounded CPU state; no model download or hidden random stream. Scalar readouts are O(1); List export is O(returned fields + bytes) and follows the language heap bound.

nlp.token_count(value: TextTokens) -> Int64

Return total source aligned tokens across retained sequences. Requires import aner.nlp. Runtime: Aner 0.0.5 or newer. Immutable bounded CPU state; no model download or hidden random stream. Scalar readouts are O(1); List export is O(returned fields + bytes) and follows the language heap bound.

nlp.token_ends(value: TextTokens, row: Int64) -> List

Return original UTF-8 byte end offsets aligned to token_starts. Requires import aner.nlp. Runtime: Aner 0.0.5 or newer. Immutable bounded CPU state; no model download or hidden random stream. Time and returned space O(row tokens + exported text bytes), within row and heap bounds.

nlp.token_scalar_ends(value: TextTokens, row: Int64) -> List

Return half open Unicode scalar ends suitable for String.slice. Requires import aner.nlp. Runtime: Aner 0.0.5 or newer. Immutable bounded CPU state; no model download or hidden random stream. Time and returned space O(row tokens + exported text bytes), within row and heap bounds.

nlp.token_scalar_starts(value: TextTokens, row: Int64) -> List

Return Unicode scalar starts suitable for String.slice; these are not byte offsets. Requires import aner.nlp. Runtime: Aner 0.0.5 or newer. Immutable bounded CPU state; no model download or hidden random stream. Time and returned space O(row tokens + exported text bytes), within row and heap bounds.

nlp.token_starts(value: TextTokens, row: Int64) -> List

Return zero based half open UTF-8 byte start offsets for one row. Requires import aner.nlp. Runtime: Aner 0.0.5 or newer. Immutable bounded CPU state; no model download or hidden random stream. Time and returned space O(row tokens + exported text bytes), within row and heap bounds.

nlp.token_texts(value: TextTokens, row: Int64) -> List

Return exact original token Strings for one row. Requires import aner.nlp. Runtime: Aner 0.0.5 or newer. Immutable bounded CPU state; no model download or hidden random stream. Time and returned space O(row tokens + exported text bytes), within row and heap bounds.

nlp.tokenize(corpus: TextCorpus, starts: List, ends: List) -> TextTokens

Snapshot caller supplied source token boundaries without retokenization. At most 4096 tokens per row and 1000000 total; each token is at most 4096 UTF-8 bytes. Requires import aner.nlp. Runtime: Aner 0.0.5 or newer. Immutable bounded CPU state; no model download or hidden random stream. Time and space O(source bytes + token spans).

nlp.tokenize(corpus: TextCorpus) -> TextTokens

Tokenize each original corpus row with the pinned Unicode 17 rule: letter/number-led runs retain following marks, whitespace is skipped and other punctuation/symbol scalars are separate. No sentence segmentation or normalization is implicit. Requires import aner.nlp. Runtime: Aner 0.0.5 or newer. Immutable bounded CPU state; no model download or hidden random stream. Time and space O(source bytes + tokens).

nlp.tokenizer(value: TextTokens) -> String

Return the versioned source tokenization protocol. Automatic and explicitly supplied spans have distinct identities; no reversible neural codec is implied. Requires import aner.nlp. Runtime: Aner 0.0.5 or newer. Immutable bounded CPU state; no model download or hidden random stream. O(1) protocol readout.

nlp.tokens(value: TaggedCorpus) -> TextTokens

Return the exact original source token snapshot without normalization, token insertion or row reordering. Requires import aner.nlp. Runtime: Aner 0.0.5 or newer. Immutable bounded CPU state; no model download or hidden random stream. O(1) shared immutable handle.

nlp.tokens(value: TextTagging) -> TextTokens

Return the exact original source token snapshot without normalization, token insertion or row reordering. Requires import aner.nlp. Runtime: Aner 0.0.5 or newer. Immutable bounded CPU state; no model download or hidden random stream. O(1) shared immutable handle.

nlp.tokens_seen(value: TextTagger) -> Int64

Return cumulative training tokens processed through all completed epochs. Requires import aner.nlp. Runtime: Aner 0.0.5 or newer. Immutable bounded CPU state; no model download or hidden random stream. Scalars are O(1); List export follows returned fields/bytes and the explicit language heap budget.

nlp.training_fingerprint(value: TextTagger) -> String

Return the exact annotated fitting identity, including token alignment, source IDs/order/role and gold labels. Requires import aner.nlp. Runtime: Aner 0.0.5 or newer. Immutable bounded CPU state; no model download or hidden random stream. Scalars are O(1); List export follows returned fields/bytes and the explicit language heap budget.

nlp.training_hashes(value: TextTagger) -> List

Return raw fitting document hashes in source row order. Requires import aner.nlp. Runtime: Aner 0.0.5 or newer. Immutable bounded CPU state; no model download or hidden random stream. Scalars are O(1); List export follows returned fields/bytes and the explicit language heap budget.

nlp.training_ids(value: TextTagger) -> List

Return fitting source IDs in training row order. Requires import aner.nlp. Runtime: Aner 0.0.5 or newer. Immutable bounded CPU state; no model download or hidden random stream. Scalars are O(1); List export follows returned fields/bytes and the explicit language heap budget.

nlp.training_tokenizer(value: TextTagger) -> String

Return the exact source tokenization protocol fitted by this model; automatic and explicit spans cannot be substituted at prediction. Requires import aner.nlp. Runtime: Aner 0.0.5 or newer. Immutable bounded CPU state; no model download or hidden random stream. Scalars are O(1); List export follows returned fields/bytes and the explicit language heap budget.

nlp.updates(value: TextTagger) -> Int64

Return mistaken sequences that triggered a structured update; this is not the averaging clock. Requires import aner.nlp. Runtime: Aner 0.0.5 or newer. Immutable bounded CPU state; no model download or hidden random stream. Scalars are O(1); List export follows returned fields/bytes and the explicit language heap budget.

nlp.upos_labels() -> List

List the 17 admissible Universal POS annotation spellings. A fitted model only predicts its observed training subset; this inventory does not perform automatic analysis. Requires import aner.nlp. Runtime: Aner 0.0.5 or newer. Immutable bounded CPU state; no model download or hidden random stream. Constant bounded output.

No arguments.

K nearest neighbor classification

Read the model guide.

ml.classes(model: KNNClassifier) -> Tensor

Return sorted distinct fitted class IDs as C × 1 detached CPU Float64 values; this is the predict_proba column mapping.

ml.fit_knn(features: Tensor, labels: Tensor, k: Int64) -> KNNClassifier

Fit a uniform Euclidean KNN classifier using native CPU computation and detached Float64 fitting copies. No optimizer, automatic scaling, distance weighting or seed knob is involved; fitting preserves row order for exact distance ties.

ml.neighbor_distances(model: KNNClassifier, input: Tensor) -> Tensor

Return Q × k Euclidean distances, not squared distances, in the same order as neighbor_indices(). Training queries may include themselves.

ml.neighbor_indices(model: KNNClassifier, input: Tensor) -> Tensor

Return Q × k nearest first zero based fitted matrix row indices. Equal distances use fitted row order; map through saved dataset.row_ids yourself for source identities.

ml.predict(model: KNNClassifier, input: Tensor) -> Tensor

Return Q × 1 original fitted class IDs. Uniform vote ties choose the smallest class ID; probability column indices need not equal class IDs.

ml.predict_proba(model: KNNClassifier, input: Tensor) -> Tensor

Return Q × C neighbor vote fractions; columns follow sorted classes(). These fractions are not calibrated probability guarantees.

K means clustering

Read the model guide.

ml.centers(model: KMeansModel) -> Tensor

Return detached k × D fitted centers in cluster index order.

ml.converged(model: KMeansModel) -> Bool

Return true when assignments were unchanged or largest absolute Euclidean center movement was <=tolerance; false when only the iteration cap stopped fitting.

ml.fit_kmeans(features: Tensor, clusters: Int64, max_iterations: Int64, tolerance: Float64, seed: Int64) -> KMeansModel

Fit one seeded Lloyd K means run. Distinct fitting row indices initialize centers; empty clusters keep the previous center. No K means++ or automatic restarts.

ml.inertia(model: KMeansModel) -> Float64

Return the sum of squared fitting row distances to final assigned centers. It depends on feature scale and k, and does not alone select a useful model.

ml.iterations(model: KMeansModel) -> Int64

Return completed Lloyd updates, including a final update that met the convergence criterion.

ml.labels(model: KMeansModel) -> Tensor

Return detached N × 1 assignments for fitted rows, recomputed against the returned final centers.

ml.predict(model: KMeansModel, input: Tensor) -> Tensor

Return Q × 1 nearest center indices 0..k−1; distance ties choose the lowest cluster index. Cluster IDs are not class predictions.

Decision tree classification

Read the model guide.

ml.classes(model: DecisionTreeClassifier) -> Tensor

Return sorted fitted class IDs, defining the probability column mapping.

ml.depth(model: DecisionTreeClassifier) -> Int64

Return maximum fitted edge depth; a root only tree has depth zero.

ml.export_tree(model: DecisionTreeClassifier, path: String) -> Unit

Write a text, DOT or SVG visualization of the fitted tree to a .txt, .dot or .svg destination.

ml.fit_tree(features: Tensor, labels: Tensor, criterion: String = "gini", max_depth: Int64 = 8, min_samples_split: Int64 = 2, min_samples_leaf: Int64 = 1) -> DecisionTreeClassifier

Fit an immutable CPU classification tree with numeric binary splits. Only strictly positive gain splits grow; majority ties choose the smallest class ID.

ml.node_count(model: DecisionTreeClassifier) -> Int64

Return total internal nodes plus leaves in the fitted tree.

ml.predict(model: DecisionTreeClassifier, input: Tensor) -> Tensor

Predict each query row's class ID by traversing the fitted tree.

ml.predict_proba(model: DecisionTreeClassifier, input: Tensor) -> Tensor

Return leaf class frequencies; probability columns follow sorted ml.classes(model). Frequencies are not calibrated confidence estimates.

ml.tree_dot(model: DecisionTreeClassifier) -> String

Return Graphviz DOT source with explicit omitted branches beyond the display limit.

ml.tree_svg(model: DecisionTreeClassifier) -> String

Return a standalone SVG tree image without requiring Graphviz.

ml.tree_text(model: DecisionTreeClassifier) -> String

Describe branch tests and leaf predictions as text, marking branches omitted beyond the 127 node display limit.

Contextual preprocessing and linear regression

Read the model guide.

ml.align_inputs(self: ContextLinearModel, observations: AdaptiveFrame, columns: List) -> AdaptiveFrame

Explicitly map raw external numerical feature columns in fitted order, convert compatible declared units, and retain independent observation IDs. Extra fields are excluded; missing values and ambiguous/incompatible mappings fail. Alignment permits inference only, never fitting. For preprocessing align the first ledger, then apply every fitted stage exactly once. Time O(n p + p c); output space O(n p + N p + p).

ml.align_inputs(self: FeatureLedger, observations: AdaptiveFrame, columns: List) -> AdaptiveFrame

Explicitly map raw external numerical feature columns in fitted order, convert compatible declared units, and retain independent observation IDs. Extra fields are excluded; missing values and ambiguous/incompatible mappings fail. Alignment permits inference only, never fitting. For preprocessing align the first ledger, then apply every fitted stage exactly once. Time O(n p + p c); output space O(n p + N p + p).

ml.center(self: FeatureLedger) -> Tensor

Copy fitted population centers for inspection into an ordinary CPU Float64 Tensor. Time O(p); output space O(p).

ml.explain(self: ContextLinearModel) -> String

Describe retained native contracts, actual supported storage/backend, roles or source accounting without running a model or reconstructing lost context. Time O(n + p + h) worst case for descriptor traversal; output space O(n + p + h) bounded by native limits.

ml.explain(self: ContextPrediction) -> String

Describe retained native contracts, actual supported storage/backend, roles or source accounting without running a model or reconstructing lost context. Time O(n + p + h) worst case for descriptor traversal; output space O(n + p + h) bounded by native limits.

ml.explain(self: FeatureLedger) -> String

Describe retained native contracts, actual supported storage/backend, roles or source accounting without running a model or reconstructing lost context. Time O(n + p + h) worst case for descriptor traversal; output space O(n + p + h) bounded by native limits.

ml.fit_exploratory_standardizer(data: AdaptiveFrame) -> FeatureLedger

Fit unsplit exploratory statistics and retain every influencing row; later splitting cannot authorize those influences for strict training. Time O(n p + n log n + H); retained fitted space O(p + h), peak workspace O(n p + n + p + h).

ml.fit_linear(training: ContextBatch) -> ContextLinearModel

Fit native CPU Float64 intercept least squares with checked two pass QR and exact fit authorization; deficient rank fails without hidden regularization. The resolved execution profile must be CPU Float64; conflicting GPU/Float32 requests raise R3501 E_BACKEND_UNSUPPORTED. Time O(n p squared + n log n + H); solver workspace O(n p + p squared + n + h).

ml.fit_standardizer(training: AdaptiveFrame) -> FeatureLedger

Fit population ddof=0 statistics only on the authorized training observations and transitive fit sources. Targets and identity/group roles are excluded; constant scale is one. Time O(n p + n log n + H); retained fitted space O(p + h), peak workspace O(n p + n + p + h).

ml.load_linear(path: String, training_anchor: ContextBatch) -> ContextLinearModel

Validate a bounded regression artifact against a live trusted contextual training batch and recomputed QR fit. The resolved execution profile must be CPU Float64; conflicting GPU/Float32 requests raise R3501 E_BACKEND_UNSUPPORTED. Time O(b + n p squared + n log n + H); additional space O(b + n p + p squared + h).

ml.load_standardizer(path: String, training_anchor: AdaptiveFrame) -> FeatureLedger

Validate a bounded artifact against a live trusted training anchor and recomputed parameters; bytes or display IDs never grant a fit scope. Time O(b + n p + n log n + H); additional space O(b + n p + n + p + h).

ml.predict(self: ContextLinearModel, samples: ContextBatch) -> ContextPrediction

Validate ordered feature meaning and exact experiment compatibility, then require the exact ordered preprocessing chain used by training and predict while retaining observation identity and target units. Raw, extra or differently ordered processing fails before output. Independent unlabeled observations require explicit align_inputs against the raw model or first fitted ledger; prediction retains external IDs and the fitted target unit. Missing targets are never fabricated. The resolved execution profile must be CPU Float64; conflicting GPU/Float32 requests raise R3501 E_BACKEND_UNSUPPORTED. Time O(n p + h); output space O(n + h).

ml.rmse(self: ContextPrediction, targets: ContextBatch) -> Float64

Compute numerical RMSE only for exactly aligned prediction/target observations and matching target meaning. Time O(n + h); auxiliary space O(1) after retained context validation.

ml.row_id(self: ContextPrediction, index: Int64) -> Int64

Inspect the stated native value without granting a new scientific or fitting contract. Bounds and stored type are checked. Time O(1), plus scalar device synchronization when applicable; output and auxiliary space O(1).

ml.row_ids(self: ContextPrediction) -> List

Copy exact Int64 observation IDs in their current order into a managed List; large exports must use row_id(index). Time O(n); output space O(n), subject to managed value slot limits.

ml.save(self: ContextLinearModel, path: String) -> Unit

Exclusively create a bounded versioned fitted artifact with integrity validation; existing files are not overwritten. Time O(b) for serialized bytes; temporary space O(b), capped at 1 MiB.

ml.save(self: FeatureLedger, path: String) -> Unit

Exclusively create a bounded versioned fitted artifact with integrity validation; existing files are not overwritten. Time O(b) for serialized bytes; temporary space O(b), capped at 1 MiB.

ml.scale(self: FeatureLedger) -> Tensor

Copy fitted population scales for inspection into an ordinary CPU Float64 Tensor; constant features use one. Time O(p); output space O(p).

ml.summary(self: ContextLinearModel) -> String

Describe retained native contracts, actual supported storage/backend, roles or source accounting without running a model or reconstructing lost context. Time O(n + p + h) worst case for descriptor traversal; output space O(n + p + h) bounded by native limits.

ml.summary(self: ContextPrediction) -> String

Describe retained native contracts, actual supported storage/backend, roles or source accounting without running a model or reconstructing lost context. Time O(n + p + h) worst case for descriptor traversal; output space O(n + p + h) bounded by native limits.

ml.summary(self: FeatureLedger) -> String

Describe retained native contracts, actual supported storage/backend, roles or source accounting without running a model or reconstructing lost context. Time O(n + p + h) worst case for descriptor traversal; output space O(n + p + h) bounded by native limits.

ml.transform(self: FeatureLedger, data: AdaptiveFrame) -> AdaptiveFrame

Apply the fixed ordered fitted statistics without changing population role or raw target; incompatible experiment scopes or applied preprocessing input chains fail. Applying this artifact to its own output fails; at most 64 ordered applied stages are retained. Time expected O(n p + h) with indexed values; output space O(n p + N p + h).

ml.unit(self: ContextPrediction) -> String

Inspect the stated native value without granting a new scientific or fitting contract. Bounds and stored type are checked. Time O(1), plus scalar device synchronization when applicable; output and auxiliary space O(1).

ml.values(self: ContextPrediction) -> SemanticTensor

Return a shared SemanticTensor prediction view with observation and target meaning retained. Time O(n + h) descriptor construction; descriptor space O(n + h), sharing numeric storage.

Dense neural models

Read the model guide.

ml.accuracy(report: EvaluationReport) -> Float64

Return the fraction correctly classified in an evaluation report.

ml.bias(model: FittedClassifier, layer: Int64) -> Tensor

Copy the trained bias of a zero based Dense layer into a detached Tensor with shape 1 × outputs.

ml.center(model: FittedClassifier) -> Tensor

Return fitted training feature means, or zeros when standardization was disabled.

ml.classifier(network: NeuralNetwork, standardize: Bool, loss: String = "cross_entropy", optimizer: String = "sgd") -> ClassifierPipeline

Build an immutable classifier plan from a network ending in Dense. standardize fits feature scaling on training rows only. loss defaults to "cross_entropy"; "mse" averages squared softmax probability errors against one hot targets over rows × classes.

ml.dropout_seed(pipeline: ClassifierPipeline, seed: Int64) -> ClassifierPipeline

Return an immutable plan with an explicit dropout seed for both full batch and mini batch fitting. Streams advance by cumulative training update and layer. Omission automatically seeds a fresh active dropout fit and records its seed. Warm starts restore the saved seed and counter; a conflicting seed fails.

ml.evaluate(model: FittedClassifier, testing: Dataset) -> EvaluationReport

Evaluate a fitted classifier on a testing dataset using its fitted preprocessing and selected loss.

ml.fit(pipeline: ClassifierPipeline, training: Dataset, config: TrainingConfig, execution: ParallelContext, progress: Bool = false) -> FittedClassifier

Fit eligible Dense models on an explicit strict local neural placement or configured neural replica context. Frozen global mini batches split by positive relative replica weights; Float64 host reduction produces one canonical CPU update in the selected dtype. Canonical parameters and optimizer state also use that dtype. Training roles, disjoint validation/groups and training only normalization remain audited. No model memory pooling or promised speedup. With e epochs, n rows, p parameters, batch b, activation width a and r replicas: numerical work O(e n p), coordinator reduction work O(e ceil(n/b) r p), working space O(r p + b a + b d), plus input staging, source/schedule storage, validation and evaluation. Optional progress observes the same updates.

ml.fit(pipeline: ClassifierPipeline, training: Dataset, config: TrainingConfig, progress: Bool = false) -> FittedClassifier

Fit fixed epoch mini batches with resolved precision/device and keyed optimizer state. Training roles and disjoint validation/groups are required; normalization fits training only. Validation evaluation occurs after epochs, with bounded progress/ETA observations. Source remains compact; working tensors are per batch. Time O(e×n×p+e×ceil(n/b)×p), working space O(b×activations+p), plus graph validation and validation passes.

ml.fit(pipeline: ClassifierPipeline, training: Dataset, epochs: Int64, rate: Float64, progress: Bool = false) -> FittedClassifier

Train with full batch SGD or Adam, one update per epoch, on the explicitly selected device. The plan resolves requested device and preserve/adapt/fast precision policy before training; CPU and CUDA support Float64/Float32, Metal/MPS Float32. Features, targets, parameters, gradients and Adam moments use the resolved precision after CPU Float64 preprocessing. Returns an immutable fitted model; scaling uses training rows only. For e epochs, b rows, parameter count p and activations a: numerical time O(e b p + e p), working space O(b a + p), plus reachable graph validation per stage. Adam adds two O(p) resident moment buffers. Optional progress records actual dtype/device/backend and at most 64 measurements; fixed horizon ETA begins after five epochs. Notebook output appears after the cell finishes.

ml.initial_loss(model: FittedClassifier) -> Float64

Return the training loss before the first optimizer update, using the classifier loss policy.

ml.load_classifier(path: String) -> FittedClassifier

Load a validated versioned classifier checkpoint with its architecture, finite trained weights, preprocessing and class schema. Prediction and warm_start use the restored immutable model. Current checkpoints retain Adam moments and update count; version 1 checkpoints restore SGD.

ml.loss(report: EvaluationReport) -> Float64

Return the evaluation loss chosen by ml.classifier: cross entropy averaged over rows, or MSE between softmax probabilities and one hot targets averaged over rows × classes.

ml.on(pipeline: ClassifierPipeline, device: String, precision: String, policy: String) -> ClassifierPipeline

Return an immutable classifier plan with explicit device, requested Float64/Float32 precision and preserve/adapt/fast policy. Preserve keeps precision; adapt lowers only when the selected backend cannot support it; fast intentionally uses Float32, with no speed guarantee. Resolve before training; summaries/checkpoints record requested and effective configuration. Parameters, gradients and Adam moments use effective precision. Time O(1) after bounded initialization; descriptor space O(1).

ml.on(pipeline: ClassifierPipeline, device: String) -> ClassifierPipeline

Return an immutable classifier plan changing requested device while retaining its requested precision and preserve/adapt/fast policy. auto resolves available CUDA, then Metal/MPS, then CPU. An unavailable explicitly requested accelerator fails. Preserve on MPS with Float64 uses reported CPU Float64 fallback; adapt permits MPS Float32. Time O(1) after bounded device initialization; descriptor space O(1).

ml.predict(model: FittedClassifier, input: Dataset) -> Tensor

Predict class labels or cluster IDs for each input row as an N × 1 tensor.

ml.probabilities(model: FittedClassifier, input: Dataset) -> Tensor

Return per class softmax probabilities for a fitted neural classifier on a dataset.

ml.row_ids(report: EvaluationReport) -> Tensor

Return the exact source row IDs and order used for evaluation.

ml.rows(report: EvaluationReport) -> Int64

Return the evaluated row count.

ml.save(model: FittedClassifier, path: String) -> Unit

Save a bounded versioned finite classifier checkpoint to a new file. An existing path is not overwritten; preprocessing, class schema, architecture, trained parameters, and Adam moments/update count are retained.

ml.scale(model: FittedClassifier) -> Tensor

Return fitted training feature scales, or ones when standardization was disabled.

ml.select_best(config: TrainingConfig, enabled: Bool) -> TrainingConfig

Choose whether fixed budget training returns the best validation snapshot. Selection uses validation, never official test; checkpoints record selection and optimizer/schedule continuation. O(1) configuration, additional O(p) selected parameter/state snapshot during fitting.

ml.summary(value: ClassifierPipeline) -> String

Describe the classifier topology, initialization, loss, optimizer and preprocessing policy before training.

ml.summary(value: EvaluationReport) -> String

Return a readable summary of a fitted classifier or evaluation report.

ml.summary(value: FittedClassifier) -> String

Return a readable summary of a fitted classifier or evaluation report.

ml.summary(value: NeuralNetwork) -> String

Describe the declared layer topology, seed and initialization; no training occurs.

ml.summary(value: TrainingConfig) -> String

Describe fixed epochs, mini batch size, seed, rate and validation/model-selection policy. Bounded descriptor output.

ml.training(epochs: Int64, batch: Int64, rate: Float64, seed: Int64) -> TrainingConfig

Build immutable fixed epoch mini batch settings. Epoch schedule seed is seed+completedEpoch; fit uses exactly the Dataset.batches ordering. Source row order, dtype/device, seed, batch size and optimizer state must match for exact continuation. Construction O(1); training O(e×n×p+e×ceil(n/b)×p), numerical memory O(b×activations+p), plus graph validation.

ml.training_accuracy(model: FittedClassifier) -> Float64

Return the final training classification accuracy.

ml.training_loss(model: FittedClassifier) -> Float64

Return the final training loss using the classifier loss policy: row mean cross entropy or softmax probability MSE averaged over rows × classes.

ml.training_row_ids(model: FittedClassifier) -> Tensor

Return the exact source row IDs and order used to fit the classifier.

ml.training_seconds(model: FittedClassifier) -> Float64

Return measured wall clock seconds for the latest fit, including preprocessing and final metrics. Timing depends on the machine, backend and recording overhead; it is not a benchmark or a projected ETA. Time O(1); auxiliary space O(1).

ml.validation(config: TrainingConfig, data: Dataset) -> TrainingConfig

Attach protected validation rows to the training configuration. They are evaluated, never optimized or used to fit normalization; row/group leakage is rejected. O(1) immutable descriptor, audit cost at fit admission.

ml.warm_start(pipeline: ClassifierPipeline, model: FittedClassifier) -> ClassifierPipeline

Return a classifier plan initialized from an immutable fitted model, retaining its preprocessing and class schema. Topology, loss and scaling policy must match. Fit continues on copied weights, with matching Adam moments and update count preserved; changing optimizer resets its state.

ml.weights(model: FittedClassifier, layer: Int64) -> Tensor

Copy the trained weights of a zero based Dense layer into a detached Tensor with shape inputs × outputs.

nn.backward(loss: Tensor) -> Unit

Backpropagate from a 1 × 1 loss tensor and populate gradients for its differentiation graph.

nn.cross_entropy(logits: Tensor, targets: Tensor) -> Tensor

Return a differentiable 1 × 1 cross entropy loss, averaged over rows. Supply raw logits and matching constant, exactly one hot target rows.

nn.cross_entropy_ids(logits: Tensor, targets: List, ignore_index: Int64) -> Tensor

Stable integer target cross entropy averaged over nonignored positions; native CPU and supported Metal/CUDA sequence execution preserve logical shapes and explicit dtype/device.

nn.dense(network: NeuralNetwork, inputs: Int64, outputs: Int64, initialization: String) -> NeuralNetwork

Append a Dense layer with deterministic seeded, Xavier uniform or He uniform initialization and zero bias. No training occurs. Widths must match adjacent stages. Appending copies s stage descriptors: time and output descriptor space O(s), with s capped at 32. Parameter allocation is deferred until fit.

nn.dense(network: NeuralNetwork, inputs: Int64, outputs: Int64, weights: Tensor, bias: Tensor) -> NeuralNetwork

Append a Dense layer using finite initial weights of shape inputs × outputs and bias of shape 1 × outputs. Inputs are copied and detached; later edits do not change the architecture.

nn.dense(network: NeuralNetwork, inputs: Int64, outputs: Int64) -> NeuralNetwork

Append a fully connected layer and return a new network. inputs must match the preceding stage width. For a classifier, first inputs must match feature count and final outputs must match class count.

nn.dropout(network: NeuralNetwork, probability: Float64) -> NeuralNetwork

Append training only elementwise inverted dropout. Finite probability in [0,1); 0.2 independently drops each activation with probability 20%, rather than selecting an exact count. Evaluation is identity. Control replay with the classifier dropout_seed; the initialization seed is separate.

nn.dropout(value: Tensor, probability: Float64, seed: Int64, training: Bool) -> Tensor

Seeded inverted dropout with explicit train/eval state, independent of dev/prod. Probability is in [0,1). The same seed reproduces the same mask; change the seed between training calls. Native CPU, Metal and CUDA use the supported tensor precision.

nn.grad(value: Tensor) -> Tensor

Read the valid gradient from the most recent successful backward pass as a tensor.

nn.mlp(inputs: Int64, hidden: List, outputs: Int64, seed: Int64) -> NeuralNetwork

Build an immutable Dense MLP with Xavier weights, zero biases, ReLU after each hidden Dense, and raw output logits. Hidden widths must be positive Int64 values, such as List(300,100). Dense seed is seed+layer. Time/descriptor space O(layers); parameter initialization occurs during fitting.

nn.mse(prediction: Tensor, target: Tensor) -> Tensor

Return a differentiable 1 × 1 mean squared error tensor. Shapes must match; squared differences are averaged over all rows × columns. This function does not apply softmax.

nn.parameter(value: Tensor) -> Tensor

Create a trainable parameter leaf from tensor values. Use nn.backward before requesting gradients or applying nn.sgd.

nn.random(rows: Int64, cols: Int64, seed: Int64, scale: Float64) -> Tensor

Create a deterministic rows × cols tensor with uniform values in [-scale, scale).

nn.relu(value: NeuralNetwork) -> NeuralNetwork

Append a relu activation stage and return a new network.

nn.sequential(seed: Int64) -> NeuralNetwork

Create an empty immutable sequential network with a deterministic initialization seed. Append Dense and activation stages before creating a classifier.

nn.sigmoid(value: NeuralNetwork) -> NeuralNetwork

Append a sigmoid activation stage and return a new network.

nn.summary(network: NeuralNetwork) -> String

Describe a network topology, deterministic initialization seed, and explicit initial parameter policy; no training occurs.

nn.tanh(value: NeuralNetwork) -> NeuralNetwork

Append a tanh activation stage and return a new network.

SGD and Adam

Read the model guide.

nn.clip_grad_norm(parameters: List, max_norm: Float64) -> Float64

Clip the global CPU parameter gradient norm after backward. Tied leaves are counted once; max_norm must be finite and positive. Return the original norm. All parameters are checked before gradients change.

nn.optimizer(name: String, rate: Float64) -> Optimizer

Create keyed SGD or Adam state; rate must be finite and positive, representable in actual parameter precision. Adam beta1=.9, beta2=.999, epsilon=1e 8. Initialization O(1); resident moments allocate on first keyed update.

nn.reset(optimizer: Optimizer) -> Unit

Clear all keyed optimizer moments and counters. Existing parameters are unchanged. Time O(keys+p) for releasing retained state.

nn.sgd(parameter: Tensor, rate: Float64) -> Tensor

Return a new parameter with value − rate × gradient. The input must be a parameter leaf with a valid gradient from nn.backward. Assign the result back to your parameter for the next step. For p values, time O(p) and output space O(p); the returned leaf does not retain the prior forward graph.

nn.summary(network: Optimizer) -> String

Describe the keyed optimizer algorithm, rate, registered slots, and update counts. Time/output O(keys).

nn.update(optimizer: Optimizer, parameter: Tensor, key: String) -> Tensor

Replace a parameter leaf using its valid gradient; retain the returned Tensor. Each stable nonempty key owns its own step and resident moments; shape, dtype and device must remain consistent. Aliases share optimizer state. Time O(p), additional state O(p); Adam keeps two moment buffers. Failed updates retain prior state.

Convolutional networks

Read the model guide.

nn.accuracy(model: CNNModel, images: Tensor, targets: Tensor) -> Float64

Measure CNN top 1 accuracy on supplied image and target tensors.

nn.attempts(model: CNNModel) -> Int64

Return attempted optimizer steps, including skipped overflow steps.

nn.avg_pool2d(model: CNNModel, kernel: Int64 = 2, stride: Int64 = 2, padding: Int64 = 0) -> CNNModel

Apply the native CNN avg pool2d operation with explicit immutable model state and inferred shapes.

nn.batch_norm(model: CNNModel) -> CNNModel

Apply the native CNN batch norm operation with explicit immutable model state and inferred shapes.

nn.budget(model: CNNModel, tensor_elements: Int64, graph_elements: Int64) -> CNNModel

Set conservative per call Tensor and differentiation graph element budgets, including convolution workspaces.

nn.cnn(channels: Int64, height: Int64, width: Int64, seed: Int64 = 42) -> CNNModel

Declare an immutable CNN with NCHW input geometry. Infer every later layer shape; parameters initialize lazily. Seed defaults to 42.

nn.conv2d(model: CNNModel, outputs: Int64, kernel: Int64 = 3, stride: Int64 = 1, padding: Int64 = 0, groups: Int64 = 1, dilation: Int64 = 1) -> CNNModel

Apply grouped or dilated NCHW convolution with OIHW weights, or append a shape inferred CNN convolution. Matching placement and precision are required.

nn.dense(model: CNNModel, outputs: Int64) -> CNNModel

Apply the native CNN dense operation with explicit immutable model state and inferred shapes.

nn.device(model: CNNModel) -> String

Return the CNN actual selected device.

nn.diagram(model: CNNModel) -> String

Return the declared CNN architecture as Mermaid without running a forward pass.

nn.dropout(model: CNNModel, probability: Float64 = 0.5) -> CNNModel

Apply the native CNN dropout operation with explicit immutable model state and inferred shapes.

nn.dtype(model: CNNModel) -> String

Return the CNN configured compute precision.

nn.fit(model: CNNModel, images: Tensor, targets: Tensor, epochs: Int64 = 1, batch: Int64 = 32, rate: Float64 = 0.001, seed: Int64 = 42, optimizer: String = "adam") -> CNNModel

Train CNN images using mini batch cross entropy and return an immutable fitted model. Compatible subsequent fits retain optimizer and shuffle state. Targets are zero based [N,1] class IDs or one hot [N,classes].

nn.fit(model: CNNModel, images: Tensor, targets: Tensor, epochs: Int64, batch: Int64, rate: Float64, seed: Int64, optimizer: String, execution: ParallelContext) -> CNNModel

Train synchronous CNN replicas through an explicit aner.parallel execution context. Preserve one canonical optimizer update per global batch and reject non frozen BatchNorm running statistics.

nn.flatten(model: CNNModel) -> CNNModel

Apply the native CNN flatten operation with explicit immutable model state and inferred shapes.

nn.freeze_backbone(model: CNNModel, frozen: Bool = true) -> CNNModel

Freeze backbone parameters and BatchNorm running statistics for head only transfer learning.

nn.freeze_batch_norm(model: CNNModel, frozen: Bool = true) -> CNNModel

Freeze BatchNorm running statistics while retaining trainable affine parameters. Explicitly required for replicated CNN batches.

nn.gelu(model: CNNModel) -> CNNModel

Apply the native CNN gelu operation with explicit immutable model state and inferred shapes.

nn.global_pool(model: CNNModel) -> CNNModel

Apply the native CNN global pool operation with explicit immutable model state and inferred shapes.

nn.head(model: CNNModel, classes: Int64) -> CNNModel

Append global spatial pooling when needed, then a class projection with raw logits.

nn.last_loss(model: CNNModel) -> Float64

Return the most recently recorded native CNN training loss.

nn.load_cnn(path: String) -> CNNModel

Restore a native CNN checkpoint for portable inference or resumed training.

nn.load_weights(model: CNNModel, path: String) -> CNNModel

Import exact topology compatible names and shapes from a local Safetensors file.

nn.logits(model: CNNModel, images: Tensor) -> Tensor

Run native CNN evaluation and return detached [N,classes] logits using saved BatchNorm statistics.

nn.loss_scale(model: CNNModel) -> Float64

Return the retained mixed precision loss scaling factor.

nn.max_pool2d(model: CNNModel, kernel: Int64 = 2, stride: Int64 = 2, padding: Int64 = 0) -> CNNModel

Apply the native CNN max pool2d operation with explicit immutable model state and inferred shapes.

nn.on(model: CNNModel, device: String = "auto", precision: String = "Float32") -> CNNModel

Return a CNN with explicit device and precision. Auto selects an available native accelerator; unsupported requests fail with capability diagnostics.

nn.parameter(model: CNNModel, name: String) -> Tensor

Return an independent detached named CNN parameter for inspection.

nn.parameter_count(model: CNNModel) -> Int64

Return the architecture parameter count without allocating weights.

nn.parameter_names(model: CNNModel) -> List

List exact named parameters for local weight import and inspection.

nn.performance(model: CNNModel, amp: String = "off", matmul: String = "strict", collectives: String = "auto") -> CNNModel

Declare explicit CUDA compute and collective policies without allocating weights. AMP uses Float32 parameter/storage with fp16 or bf16 GEMM compute; strict/tf32 controls Float32 GEMM. Unsupported execution fails rather than changing backend. Collective choices are auto, nccl or mpi.

nn.performance_summary(model: CNNModel) -> String

Describe the requested CNN compute/collective policies, loss scaling and optimizer attempt counters.

nn.predict(model: CNNModel, images: Tensor) -> Tensor

Run CNN evaluation and return zero based predicted class IDs.

nn.prepare(model: CNNModel) -> CNNModel

Initialize a lazy CNN once and return an immutable resident model. Use for repeated inference before training; fitted or loaded models already retain initialized weights.

nn.probabilities(model: CNNModel, images: Tensor) -> Tensor

Run CNN evaluation and return detached class probabilities.

nn.read_images(model: CNNModel, path: String, samples: Int64) -> Tensor

Read exact size raw little endian NCHW Float32 or Float64 image data under this model's input geometry, precision and Tensor budget. No decoding, resizing, normalization or augmentation occurs.

nn.relu(model: CNNModel) -> CNNModel

Apply the native CNN relu operation with explicit immutable model state and inferred shapes.

nn.save(model: CNNModel, path: String) -> Unit

Exclusively create a versioned native CNN checkpoint preserving architecture, trained parameters, BatchNorm buffers, optimizer and shuffle state.

nn.skipped_updates(model: CNNModel) -> Int64

Return skipped optimizer updates caused by nonfinite mixed precision gradients.

nn.summary(model: CNNModel) -> String

Describe CNN topology, parameter count, execution policy and retained training state.

nn.updates(model: CNNModel) -> Int64

Return the number of optimizer updates retained by this immutable CNN snapshot.

nn.vision(architecture: String, classes: Int64 = 1000, seed: Int64 = 42) -> CNNModel

Build a native model zoo architecture with a custom class count. Local weight import is explicit; construction downloads nothing.

Recurrent networks

Read the model guide.

nn.attempts(model: RecurrentModel) -> Int64

Return attempted training updates, including mixed precision overflow skips; this counter is distinct from successful updates().

nn.awd_lstm(vocabulary: Int64, embedding: Int64, hidden: Int64, layers: Int64 = 3, seed: Int64 = 42) -> RecurrentModel

Build a tied embedding vocabulary LSTM with locked dropout and recurrent DropConnect. This architecture does not automate the paper optimizer recipe.

nn.budget(model: RecurrentModel, tensor_elements: Int64, graph_elements: Int64) -> RecurrentModel

Native recurrent operation with explicit immutable state, checked shapes, placement and precision.

nn.convlstm(channels: Int64, hidden: Int64, outputs: Int64 = 1, kernel: Int64 = 3, seed: Int64 = 42) -> RecurrentModel

Build spatial LSTM gates for [batch,time,channels,height,width] with channel shared peepholes.

nn.deepar(inputs: Int64, hidden: Int64, outputs: Int64 = 1, layers: Int64 = 1, seed: Int64 = 42) -> RecurrentModel

Build a Gaussian autoregressive LSTM. Pack causal lags and observed indicators with nn.deepar_inputs; pack training targets with nn.deepar_targets.

nn.device(model: RecurrentModel) -> String

Native recurrent operation with explicit immutable state, checked shapes, placement and precision.

nn.diagram(model: RecurrentModel) -> String

Native recurrent operation with explicit immutable state, checked shapes, placement and precision.

nn.dropout(model: RecurrentModel, probability: Float64) -> RecurrentModel

Configure locked training dropout between stacked recurrent layers; reuse a sample’s mask across timesteps. Prediction disables it. This is distinct from Dense elementwise dropout.

nn.dtype(model: RecurrentModel) -> String

Native recurrent operation with explicit immutable state, checked shapes, placement and precision.

nn.fit(model: RecurrentModel, inputs: Tensor, targets: Tensor, epochs: Int64 = 1, batch: Int64 = 32, rate: Float64 = 0.001, seed: Int64 = 42, optimizer: String = "adam", clip_norm: Float64 = 1.0, weight_decay: Float64 = 0.0, shuffle: Bool = true, lengths: List = List(), execution: ParallelContext = parallel.local()) -> RecurrentModel

Train an immutable recurrent snapshot with padding masks, global gradient clipping, Adam or SGD, deterministic shuffling and optional aner.parallel execution.

nn.fit_chunk(model: RecurrentModel, inputs: Tensor, targets: Tensor, state: RecurrentState, rate: Float64 = 0.001, optimizer: String = "adam", clip_norm: Float64 = 1.0, lengths: List = List()) -> RecurrentModel

Train one whole ordered batch with an explicit detached state boundary for truncated BPTT. The returned immutable model retains final carry and matching optimizer state.

nn.forecast(model: RecurrentModel, context: Tensor, last_target: Tensor, future_covariates: Tensor, samples: Int64 = 100, seed: Int64 = 42) -> Tensor

Sample Gaussian trajectories autoregressively using each sampled value as its next lag; returns [batch,samples,horizon,outputs].

nn.forecast(model: RecurrentModel, context: Tensor, last_target: Tensor, horizon: Int64, samples: Int64 = 100, seed: Int64 = 42) -> Tensor

Sample Gaussian trajectories autoregressively using each sampled value as its next lag; returns [batch,samples,horizon,outputs].

nn.gru(inputs: Int64, hidden: Int64, outputs: Int64 = 1, layers: Int64 = 1, bidirectional: Bool = false, seed: Int64 = 42) -> RecurrentModel

Build a native batch first GRU with explicit reset/update gates and detached streaming state.

nn.initial_state(model: RecurrentModel, batch: Int64) -> RecurrentState

Return zero detached per-layer/direction memories for a declared batch size; ConvLSTM spatial state is inferred by step(input).state().

nn.last_loss(model: RecurrentModel) -> Float64

Return the latest retained finite measured recurrent training loss; this is not a held out validation score.

nn.load_recurrent(path: String) -> RecurrentModel

Native recurrent operation with explicit immutable state, checked shapes, placement and precision.

nn.logits(model: RecurrentModel, inputs: Tensor, lengths: List = List()) -> Tensor

Return detached task shaped raw output values/scores with reset streaming state; classification probabilities are not applied automatically.

nn.loss_scale(model: RecurrentModel) -> Float64

Native recurrent operation with explicit immutable state, checked shapes, placement and precision.

nn.lstm(inputs: Int64, hidden: Int64, outputs: Int64 = 1, layers: Int64 = 1, bidirectional: Bool = false, seed: Int64 = 42) -> RecurrentModel

Build a native batch first LSTM with explicit detached streaming state and a positive forget bias.

nn.on(model: RecurrentModel, device: String = "auto", precision: String = "Float32") -> RecurrentModel

Native recurrent operation with explicit immutable state, checked shapes, placement and precision.

nn.parameter(model: RecurrentModel, name: String) -> Tensor

Return a detached copy of the named recurrent parameter; arbitrary automatic field registration is not performed.

nn.parameter_count(model: RecurrentModel) -> Int64

Native recurrent operation with explicit immutable state, checked shapes, placement and precision.

nn.parameter_names(model: RecurrentModel) -> List

Return exact registered recurrent parameter names for explicit detached inspection.

nn.performance(model: RecurrentModel, amp: String = "off", matmul: String = "strict", collectives: String = "auto") -> RecurrentModel

Declare CUDA GEMM AMP and strict/TF32 compute plus MPI/NCCL collective policy. Float32 master weights and moments are retained.

nn.performance_summary(model: RecurrentModel) -> String

Native recurrent operation with explicit immutable state, checked shapes, placement and precision.

nn.predict(model: RecurrentModel, inputs: Tensor, lengths: List = List()) -> Tensor

Return detached task shaped regression values or argmax class IDs. Right padding is masked using explicit lengths; prediction starts with reset state.

nn.prepare(model: RecurrentModel) -> RecurrentModel

Initialize lazy recurrent parameters once without fitting; repeated inference can reuse the resident immutable model.

nn.reset_state(model: RecurrentModel) -> RecurrentModel

Return an immutable model with retained carry cleared; parameters and optimizer history remain intact.

nn.rnn(inputs: Int64, hidden: Int64, outputs: Int64 = 1, layers: Int64 = 1, bidirectional: Bool = false, seed: Int64 = 42) -> RecurrentModel

Build a native batch first tanh RNN with fused sequence input projection.

nn.save(model: RecurrentModel, path: String) -> Unit

Save versioned weights, optimizer, precision policy, counters and retained state for continuation.

nn.skipped_updates(model: RecurrentModel) -> Int64

Return mixed precision overflow attempts that did not update model parameters.

nn.state(model: RecurrentModel) -> RecurrentState

Return detached retained or final recurrent state.

nn.state(result: RecurrentOutput) -> RecurrentState

Return detached retained or final recurrent state.

nn.step(model: RecurrentModel, inputs: Tensor, state: RecurrentState = default, lengths: List = List()) -> RecurrentOutput

Evaluate a sequence using explicit/retained detached state and return values plus a new detached final state; no gradient crosses this boundary.

nn.summary(model: RecurrentModel) -> String

Native recurrent operation with explicit immutable state, checked shapes, placement and precision.

nn.task(model: RecurrentModel, task: String) -> RecurrentModel

Select regression, classification, sequence_regression or sequence_classification with an explicit output contract.

nn.updates(model: RecurrentModel) -> Int64

Return successful optimizer updates retained by the recurrent model.

nn.values(result: RecurrentOutput) -> Tensor

Return detached task shaped raw recurrent output values. Classification argmax belongs to predict(), not values().

nn.with_state(model: RecurrentModel, state: RecurrentState) -> RecurrentModel

Return a model carrying validated detached recurrent state; cell, layers, directions, batch, dtype and device must agree.

nn.xlstm(inputs: Int64, hidden: Int64, outputs: Int64 = 1, variant: String = "slstm", layers: Int64 = 1, seed: Int64 = 42) -> RecurrentModel

Build stabilized scalar memory slstm or matrix memory mlstm cells. One memory head; no claim of reproducing the full paper block stack.

Generative models

Read the model guide.

gen.averaged_parameter(model: ScientificModel, name: String) -> Tensor

Inspect detached inference EMA parameters on a trained EMA configured scientific model. The ordinary parameter getter retains raw optimizer weights; predict_noise(training:true) also uses the raw weights.

gen.baseline(model: LanguageModel) -> String

Describe the uniform token baseline using this model's actual tokenizer vocabulary.

gen.calibrate_diffusion(model: ScientificModel, training: ScientificData, samples_per_timestep: Int64, seed: Int64, ridge: Float64) -> ScientificModel

Fit a bounded affine epsilon residual correction and reverse covariance on independent forward noise folds from exact bound training records, sampling clean records independently with replacement. Preserves NN weights/optimizer; effective prediction and sampling use the correction while loss remains the raw NN objective.

gen.calibrate_diffusion_gaussian(model: ScientificModel, training: ScientificData) -> ScientificModel

Fit an explicit affine conditional Gaussian closure from bound clean training moments. Exact Gaussian DDPM score, reverse covariance and terminal prior bypass the nonlinear noisy field NN. Preserves raw NN weights/objective for resume; fitted moments and assumptions are checkpointed.

gen.calibrate_diffusion_neural(model: ScientificModel, training: ScientificData, draws: Int64, seed: Int64, strength: Float64) -> ScientificModel

Blend the terminal covariance correction within the learned clean support. Strength must be finite in [0,1]: zero projects into clean support; one retains full moment matching. Smaller values regularize finite data variance correction while retaining mean correction and mass conservation. No general variance improvement guarantee.

gen.calibrate_diffusion_neural(model: ScientificModel, training: ScientificData, draws: Int64, seed: Int64) -> ScientificModel

Fit a terminal affine moment transport from bound training fields and independent generated probes. The neural denoiser remains active at every diffusion step; clean covariance support and the mass tangent constrain the transport. Draws repeat the complete training condition design. Calibrates first and second moments; nonlinear distribution shape remains learned by the neural chain.

gen.calibrate_diffusion_variance(model: ScientificModel, training: ScientificData, samples_per_timestep: Int64, seed: Int64) -> ScientificModel

Fit bounded reverse Gaussian covariance from frozen denoiser errors on the exact bound training data only; preserves weights and optimizer, stores seed/budget/provenance and is invalidated by further updates.

gen.decoder(codec: Tokenizer, context: Int64, width: Int64, heads: Int64, layers: Int64, seed: Int64, dropout: Float64) -> LanguageModel

Create a configurable pre norm decoder using the selected native execution profile with learned positions, multihead attention, GELU and tied embeddings. Optional dropout defaults to zero.

gen.decoder(codec: Tokenizer, context: Int64, width: Int64, heads: Int64, layers: Int64, seed: Int64) -> LanguageModel

Create a configurable pre norm decoder using the selected native execution profile with learned positions, multihead attention, GELU and tied embeddings. Optional dropout defaults to zero.

gen.device(model: LanguageModel) -> String

Return the actual native parameter device.

gen.device(model: ScientificModel) -> String

Return actual native scientific parameter placement or precision.

gen.diffusion_covariance(model: ScientificModel) -> Tensor

Inspect detached CPU Float64 [steps,interior_sites,interior_sites] reverse covariance, including the mass tangent projection when constrained; bounded to 65 physical interior coordinates and one million elements.

gen.diffusion_model(rows: Int64, cols: Int64, condition_size: Int64, width: Int64, diffusion_steps: Int64, beta_start: Float64, beta_end: Float64, seed: Int64) -> ScientificModel

Create a conditional small field epsilon denoiser and DDPM schedule with hard zero Dirichlet boundaries.

gen.diffusion_variance(model: ScientificModel) -> Tensor

Return detached CPU Float64 [steps,1] mean effective reverse variances: posterior defaults or the trace of the fitted covariance divided by stochastic degrees of freedom.

gen.dtype(model: LanguageModel) -> String

Return the actual native parameter precision.

gen.dtype(model: ScientificModel) -> String

Return actual native scientific parameter placement or precision.

gen.fine_tune(model: LanguageModel, corpus: TokenCorpus, steps: Int64, batch: Int64, rate: Float64, seed: Int64, clip_norm: Float64, warmup_steps: Int64, weight_decay: Float64) -> LanguageModel

Fine tune with a fresh Adam optimizer and explicit clipping, warmup and decay policy; preserve prior document provenance.

gen.fine_tune(model: LanguageModel, corpus: TokenCorpus, steps: Int64, batch: Int64, rate: Float64, seed: Int64) -> LanguageModel

Start a fresh optimizer run on new training data, preserving weights, trainable selection and cumulative lineage. Resume continues the same run.

gen.fingerprint(model: LanguageModel) -> String

Return a SHA256 identity covering complete model and continuation state.

gen.fingerprint(model: ScientificModel) -> String

Inspect bounded scientific state or its complete canonical SHA256 identity.

gen.freeze(model: LanguageModel, names: List) -> LanguageModel

Freeze exact registered names and begin a new run with reset optimizer/counters. Frozen tensors receive no gradients, updates or decay.

gen.generate(model: LanguageModel, prompt: List, max_new_tokens: Int64, temperature: Float64, top_k: Int64, top_p: Float64, seed: Int64, use_cache: Bool = true) -> List

Return new Int64 IDs, including EOS on termination. Native inference prefills the prompt and reuses call local layer keys/values by default. Set use_cache=false to recompute prefixes. Identical sampling policy and strict context bound; cache state is never saved or shared between requests.

gen.generate_batch(model: LanguageModel, prompts: List, max_new_tokens: Int64, temperature: Float64, top_k: Int64, top_p: Float64, seeds: List, use_cache: Bool = true, cuda_graph: Bool = false) -> List

Return one List of new Int64 IDs per prompt, in input order. Prompts are Lists of Int64; budgets are one per row or a shared Int64; seeds always contain one Int64 per row. Empty prompts use BOS; EOS and each output budget independently stop a row. CUDA cached execution packs active rows with shared weights; CPU/Metal use a portable reference. use_cache=false recomputes prefixes. cuda_graph=true requests bounded CUDA decode replay and requires cached CUDA execution; capture/setup remains part of each request and short decode windows may run eagerly. Strict row context and bounded aggregate KV/work apply.

gen.generate_batch(model: LanguageModel, prompts: List, max_new_tokens: List, temperature: Float64, top_k: Int64, top_p: Float64, seeds: List, use_cache: Bool = true, cuda_graph: Bool = false) -> List

Return one List of new Int64 IDs per prompt, in input order. Prompts are Lists of Int64; budgets are one per row or a shared Int64; seeds always contain one Int64 per row. Empty prompts use BOS; EOS and each output budget independently stop a row. CUDA cached execution packs active rows with shared weights; CPU/Metal use a portable reference. use_cache=false recomputes prefixes. cuda_graph=true requests bounded CUDA decode replay and requires cached CUDA execution; capture/setup remains part of each request and short decode windows may run eagerly. Strict row context and bounded aggregate KV/work apply.

gen.import_gpt2(directory: String, context: Int64) -> LanguageModel

Import the supported bounded GPT2LMHeadModel from local config.json, model.safetensors, vocab.json and merges.txt. Retain a declared position prefix; reject unsupported architecture variants.

gen.last_loss(model: LanguageModel) -> Float64

Return the final training batch loss evaluated before its parameter update.

gen.last_loss(model: ScientificModel) -> Float64

Return the last scientific training batch loss before its update.

gen.load(path: String, device: String) -> LanguageModel

Load complete native checkpoints directly onto a supported requested device. Exact resume requires recorded precision, backend and implementation.

gen.load(path: String) -> LanguageModel

Load a validated bounded native checkpoint onto CPU using its saved precision; resume checks the saved CPU backend identity.

gen.load_adapter(base: LanguageModel, path: String) -> LanguageModel

Attach a validated adapter to its exact matching native base. A full model checkpoint is required for exact optimizer resume.

gen.load_scientific(path: String, device: String) -> ScientificModel

Load a validated bounded scientific checkpoint onto CPU or an explicitly supported device.

gen.load_scientific(path: String) -> ScientificModel

Load a validated bounded scientific checkpoint onto CPU or an explicitly supported device.

gen.logits(model: LanguageModel, ids: List, batch: Int64, length: Int64, training: Bool) -> Tensor

Return native decoder logits; training=true exposes connected parameter gradients for custom objectives, false creates no gradient graph.

gen.lora(model: LanguageModel, targets: List, rank: Int64, alpha: Float64, seed: Int64) -> LanguageModel

Attach native LoRA factors to named block projections. Freeze the base; effective weight is W + (alpha/rank) A B, initially identical to the base.

gen.loss(model: LanguageModel, corpus: TokenCorpus) -> Float64

Compute token weighted next token loss, scoring each document transition once. Reject overlap with training documents for held out roles.

gen.loss(model: ScientificModel, corpus: ScientificData, seed: Int64) -> Float64

Evaluate teacher forced Gaussian NLL or explicit seed diffusion denoising loss on the declared partition.

gen.loss(model: ScientificModel, corpus: ScientificData) -> Float64

Evaluate teacher forced Gaussian NLL or explicit seed diffusion denoising loss on the declared partition.

gen.memory_estimate(model: LanguageModel, batch: Int64, length: Int64, training: Bool) -> Dictionary

Return conservative numeric buffer components, total_bytes, graph_elements, fits and reason. fits checks native tensor/work/graph limits, not physical RAM.

gen.memory_summary(model: LanguageModel, batch: Int64, length: Int64, training: Bool) -> String

Explain native weight/gradient/Adam/activation/attention/index/temporary estimates and checked limits.

gen.merge_lora(model: LanguageModel) -> LanguageModel

Merge adapter deltas into detached inference weights. Use fine_tune for an explicit new training run.

gen.parameter(model: LanguageModel, name: String) -> Tensor

Get a named parameter handle for inspection or custom objectives; frozen weights are detached constants. Tied embeddings have one registered identity.

gen.parameter(model: ScientificModel, name: String) -> Tensor

Inspect a named native scientific parameter for custom objectives and gradient checks.

gen.parameter_names(model: LanguageModel) -> List

Native gen.parameter_names accessor; see the generative workflow reference for bounds and contracts.

gen.parameter_names(model: ScientificModel) -> List

List native scientific parameter identities.

gen.perplexity(model: LanguageModel, corpus: TokenCorpus) -> Float64

Native gen.perplexity accessor; see the generative workflow reference for bounds and contracts.

gen.predict_noise(model: ScientificModel, noisy: Tensor, conditions: Tensor, timesteps: List, training: Bool) -> Tensor

Predict epsilon for per record diffusion timesteps 0..steps 1; preserve the native differentiation graph in training mode.

gen.predict_series(model: ScientificModel, history: Tensor, conditions: Tensor, training: Bool) -> Tensor

Predict next step conditional means; training=true exposes native parameter gradients.

gen.resume(model: LanguageModel, corpus: TokenCorpus, steps: Int64) -> LanguageModel

Continue the saved optimizer, dropout and data cursor using the identical training corpus and CPU backend.

gen.resume(model: ScientificModel, corpus: ScientificData, steps: Int64) -> ScientificModel

Resume identical scientific data, optimizer policy, sampling streams, execution backend and implementation.

gen.sample_fields(model: ScientificModel, conditions: Tensor, seed: Int64) -> Tensor

Run seeded DDPM sampling with posterior or fitted full reverse covariance and zero Dirichlet edges. A sufficiently noisy terminal schedule is required.

gen.sample_series(model: ScientificModel, conditions: Tensor, prefix: Tensor, new_steps: Int64, seed: Int64) -> Tensor

Generate native conditional Gaussian trajectories; returns only new steps, excluding the supplied lag prefix.

gen.save(model: LanguageModel, path: String) -> Unit

Atomically publish a new checksummed checkpoint with architecture, weights, Adam moments, tokenizer identity, lineage, cursor and random stream policy.

gen.save(model: ScientificModel, path: String) -> Unit

Publish a checksummed scientific checkpoint including optimizer, data provenance and noise streams; destination must be new.

gen.save_adapter(model: LanguageModel, path: String) -> Unit

Publish a new checksummed adapter artifact with factors, target shapes, scaling, tokenizer, lineage and exact base weight identity.

gen.scientific_memory(model: ScientificModel, batch: Int64, training: Bool) -> Dictionary

Estimate scientific model numeric buffers and checked native resource admission for a training or prediction batch.

gen.series_model(channels: Int64, history: Int64, condition_size: Int64, width: Int64, innovation_sigma: Float64, seed: Int64) -> ScientificModel

Create a conditional native autoregressive MLP mean with explicit fixed Gaussian innovation scale.

gen.steps(model: LanguageModel) -> Int64

Native gen.steps accessor; see the generative workflow reference for bounds and contracts.

gen.steps(model: ScientificModel) -> Int64

Return scientific optimizer updates or scored training objective rows.

gen.summary(model: LanguageModel) -> String

Native gen.summary accessor; see the generative workflow reference for bounds and contracts.

gen.summary(model: ScientificModel) -> String

Inspect bounded scientific state or its complete canonical SHA256 identity.

gen.to(model: LanguageModel, device: String, dtype: String) -> LanguageModel

Explicitly transfer and/or convert a native model. A changed placement or precision starts a fresh optimizer run; matching requests retain state.

gen.to(model: ScientificModel, device: String, dtype: String) -> ScientificModel

Return a scientific model on the declared supported device/dtype while retaining model and optimizer provenance; an identical profile is an operation with no effect and does not reset training.

gen.tokenizer(model: LanguageModel) -> Tokenizer

Recover the exact complete tokenizer stored in a model checkpoint.

gen.train(model: LanguageModel, corpus: TokenCorpus, steps: Int64, batch: Int64, rate: Float64, seed: Int64, clip_norm: Float64, warmup_steps: Int64, weight_decay: Float64) -> LanguageModel

Configure global gradient clipping (zero disables), linear warmup and decoupled matrix weight decay. Complete optimizer/data/schedule state is retained for exact resume.

gen.train(model: LanguageModel, corpus: TokenCorpus, steps: Int64, batch: Int64, rate: Float64, seed: Int64) -> LanguageModel

Return a newly trained model with Adam moments and deterministic document window schedule. Only training role data is accepted. Save all state with gen.save.

gen.train(model: ScientificModel, corpus: ScientificData, steps: Int64, batch: Int64, rate: Float64, seed: Int64, clip_norm: Float64, warmup_steps: Int64, weight_decay: Float64) -> ScientificModel

Train native scientific objectives with deterministic data/noise streams and complete Adam continuation state.

gen.train(model: ScientificModel, corpus: ScientificData, steps: Int64, batch: Int64, rate: Float64, seed: Int64) -> ScientificModel

Train native scientific objectives with deterministic data/noise streams and complete Adam continuation state.

gen.trainable_parameters(model: LanguageModel) -> List

List exact trainable parameter names, including active LoRA factors.

gen.trained_samples(model: ScientificModel) -> Int64

Return scientific optimizer updates or scored training objective rows.

gen.trained_tokens(model: LanguageModel) -> Int64

Native gen.trained_tokens accessor; see the generative workflow reference for bounds and contracts.

gen.unfreeze(model: LanguageModel, names: List) -> LanguageModel

Unfreeze exact registered names and begin a new run with reset optimizer/counters. Retain cumulative data provenance.

gen.with_diffusion_ema(model: ScientificModel, decay: Float64) -> ScientificModel

Configure exponential parameter averaging before diffusion training. Native resident updates average post update weights; inference, generation, and calibration use the average, while optimizer weights and Adam state remain available for exact continuation. Decay must be finite in (0,1) and representable in model precision.

gen.with_diffusion_schedule_features(model: ScientificModel) -> ScientificModel

Configure native schedule aware time inputs before diffusion training: sqrt cumulative alpha, sqrt one minus alpha, and endpoint normalized log SNR. Uses the same three input slots and initialization as legacy fraction/sin/cos features; checkpoints preserve the choice.

gen.with_diffusion_sobolev_loss(model: ScientificModel, dx: Float64, dy: Float64, laplacian_weight: Float64) -> ScientificModel

Configure native derivative matching for an untrained diffusion model. A dimensionless Laplacian penalty matches the selected epsilon or velocity denoising errors; zero weight preserves its base loss. Mass and loss geometry must agree.

gen.with_diffusion_velocity_prediction(model: ScientificModel, loss_weighting: String) -> ScientificModel

Select velocity prediction with explicit loss weighting: velocity uses uniform velocity error; epsilon scales velocity error by sqrt(cumulative alpha), retaining epsilon equivalent timestep weighting and native epsilon inference/calibration.

gen.with_diffusion_velocity_prediction(model: ScientificModel) -> ScientificModel

Configure a native velocity target on an untrained diffusion model. Public predict_noise, covariance calibration and sampling continue to use epsilon; supports optional mass conservation and derivative matching.

gen.with_mass_constraint(model: ScientificModel, condition_index: Int64, scale: Float64, offset: Float64, dx: Float64, dy: Float64) -> ScientificModel

Configure an untrained diffusion model with total mass = scale * condition[index] + offset; train and sample in the zero Dirichlet fixed mass subspace.

gen.with_parameter(model: LanguageModel, name: String, value: Tensor) -> LanguageModel

Replace a matching CPU Float64 initialization tensor before training; trained optimizer state is never silently reset.

gen.with_parameter(model: ScientificModel, name: String, value: Tensor) -> ScientificModel

Replace a shape/device/precision-matching scientific initialization before training.

Evaluation metrics

Read the model guide.

metrics.accuracy(predicted: Tensor, actual: Tensor) -> Float64

Return the fraction of matching labels from predicted and actual N × 1 tensors.

metrics.adjusted_rand_index(predicted: Tensor, actual: Tensor) -> Float64

Compare cluster assignments against actual labels with the adjusted Rand index. Inputs are matching N × 1 label tensors.

metrics.confusion_matrix(predicted: Tensor, actual: Tensor, classes: Int64) -> Tensor

Count classification outcomes in a classes × classes matrix, with actual labels as rows and predicted labels as columns.

metrics.mean_squared_error(predicted: Tensor, actual: Tensor) -> Float64

Return mean squared error across all elements of two tensors with matching shapes as a Float64.