Generative model parameters

This reference targets Aner 0.0.5 or newer. Updating documentation or editor help alone does not upgrade an installed runtime; check Downloads for verified platform packages.

aner.gen owns LanguageModel decoder snapshots and ScientificModel conditional series/field snapshots. These are separate fitted schemas, not arbitrary user class serializers. Start with native generative models, the model directory and exact API inventory. Their Tensor foundations are in Neural networks.

Decoder construction variables

import aner.token
import aner.gen

let codec = token.byte_tokenizer()
let corpus = token.corpus(codec, List("ababa", "babab"), role: "training")
let initial = gen.decoder(codec: codec, context: 8, width: 8,
    heads: 2, layers: 1, seed: 42, dropout: 0.1)
let trained = initial.train(corpus, steps: 2, batch: 2, rate: 0.001,
    seed: 99, clip_norm: 1.0, warmup_steps: 0, weight_decay: 0.0)
print(trained.loss(corpus))

This minimal program demonstrates configuration, not held out language quality. The complete research workflow separates tokenizer training, model training, validation and artifacts. The from scratch guide explains how the decoder's embeddings, attention and tied output projection work.

`gen.decoder` argument	Type / source default	Accepted values and effect
`codec`	Required `Tokenizer`	Determines exact vocabulary/IDs and tokenization identity; vocabulary must contain `259..65536` IDs for this decoder. The model retains its tokenizer.
`context`	Required `Int64`, `1..512`	Maximum learned absolute position range. It bounds training windows and prompt plus generation length. It does not slide automatically.
`width`	Required `Int64`, `2..256`	Embedding/residual width. Increases parameter, activation and projection work.
`heads`	Required `Int64`, `1..64`	Attention heads; must divide `width`. Attention score storage grows with `heads * context²`.
`layers`	Required `Int64`, `1..8`	Decoder block count.
`seed`	Required `Int64`	Weight initialization seed; separate from training and generation seeds.
`dropout`	`Float64`, omitted overload = `0`	Finite `[0,1)`. Training masks derive from retained training step and operation tag; inference/loss evaluation disable them. This is not Dense's `.dropout_seed` method.

The public source constructor requires context, width, heads, layers and seed, even though an internal configuration structure has demonstration defaults. Parameters are limited to 1,000,000 elements; attention and projection work budgets further restrict admissible combinations. A maximum in one table row does not certify the product of every maximum.

model.logits(ids, batch, length, training) takes an exact Int64 ID List of batch*length values; batch is 1..64, length is positive and at most context. IDs are in vocabulary; sequences begin with a nonpadding token. Output is logical [batch,length,vocabulary]. training: Bool explicitly chooses a connected trainable graph versus detached inference, independently of Dev/Prod. Padding and causal attention remain part of the declared decoder behavior.

Training and continuation arguments

model.train(corpus, steps, batch, rate, seed)
model.train(corpus, steps, batch, rate, seed, clip_norm, warmup_steps, weight_decay)
model.resume(corpus, steps)
model.fine_tune(corpus, steps, batch, rate, seed, ...)

Argument	Type / default	Contract and interpretation
`corpus`	Required `TokenCorpus`	Training role corpus with the identical tokenizer. Document lineage and actual next token transitions are retained. Validation/test corpora cannot train. Scientific overloads instead require `ScientificData`.
`steps`	Required positive `Int64`	Number of additional optimizer updates, not epochs. Cumulative steps may not exceed 100,000. Final partial batches are real updates.
`batch`	Required `Int64`, `1..64`	Number of language windows or scientific training records per update, subject to actual graph/work limits.
`rate`	Required finite positive `Float64`	Base Adam learning rate. Scientific fitting additionally requires `rate ≤ 1`.
`seed`	Required `Int64`	Training window/sample ordering and model specific random decisions, separate from model initialization and later sampling.
`clip_norm`	Extended overload; omitted = `1.0`	Finite nonnegative global gradient norm cap; zero disables clipping.
`warmup_steps`	Extended overload; omitted = `0`	Int64 `0..100000`; zero disables linear warmup, otherwise rate grows linearly to base `rate` by this successful step.
`weight_decay`	Extended overload; omitted = `0`	Finite nonnegative decoupled decay, with `rate*weight_decay < 1`. Applied to the family defined matrix weights rather than every bias/normalization parameter.

The extended form supplies all three trailing optimizer settings. There is no optimizer name, beta or epsilon source argument in gen.train; it uses the native Adam training policy. resume uses saved batch/rate/seed/clipping/warmup/decay and bound data identity. A repeated compatible train call is also a continuation; it rejects a changed policy/data/backend/implementation instead of silently resetting. fine_tune is the explicit language model transition to a new training run, retaining compatible weights and prior data lineage while resetting optimizer/schedule counters.

loss(corpus) is a dropout disabled post update evaluation, weighted by scored tokens. perplexity(corpus) exponentiates the corresponding mean token loss where representable. last_loss() is the latest batch's pre update loss; it is a different phase. Retain tokenizer and corpus policy when comparing these values. steps(), trained_tokens() and scored_tokens(corpus) explain the units of progress.

Freezing and LoRA variables

Call / argument	Contract
`freeze(names)` / `unfreeze(names)`	Required `List` of exact registered parameter name strings, discoverable with `parameter_names()`. No glob patterns. These adaptation transitions reset optimizer/run progress deliberately; select trainability before fitting.
`lora(targets, rank, alpha, seed)`	Projection only, adapter only LoRA. `targets` is a required exact name `List` of eligible decoder matrix projections. It is not a list of arbitrary activation names.
`rank: Int64`	Required, `1..64` and at most both matrix dimensions. Determines the two low rank factor sizes and adapter capacity.
`alpha: Float64`	Required finite `(0,1000000]`. Effective projection is `W + (alpha/rank)AB`; alpha is not an independent learning rate.
`seed: Int64` in `lora`	Required factor initialization seed. Base weights retain their identity.
`save_adapter(path)` / `gen.load_adapter(base, path)`	New local path and exact compatible base model. Adapter artifacts retain factors/base identity, not a full optimizer resume history.
`merge_lora()`	No extra arguments; produces an inference artifact. Use a deliberate `fine-tune/unfreeze` transition before further training.

with_parameter(name, value) is a deliberate parameter replacement with exact existing name, shape, dtype and device; it is not automatic registration of a custom architecture. trainable_parameters() reports the actual selection. Fine tuning source demonstrates the full lifecycle.

Generation variables

All scalar sampling arguments below are required except the explicit trailing cache/graph overload flags.

Argument	Type / default	Meaning and bounds
`prompt` / `prompts`	`List` / `List` of ID lists	Exact vocabulary/BOS IDs without PAD or EOS; an empty prompt becomes BOS. Batch has `1..64` rows. No text tokenization is implicit here.
`max_new_tokens`	Required `Int64` or batch `List`	Per row output cap `0..4096`; prompt plus cap must fit the model context. The batch overload accepts one scalar cap or one cap per row. Returned IDs exclude the prompt and include EOS if selected.
`temperature`	Required finite `Float64 ≥ 0`	Zero selects greedy decoding; positive values rescale logits before sampling. Higher temperature changes randomness, not model weights.
`top_k`	Required `Int64`, `0..vocabulary`	Zero disables the top k restriction; positive values retain highest ranked candidates.
`top_p`	Required finite `Float64` in `(0,1]`	Nucleus probability threshold applied by the implemented candidate policy; one permits the full remaining mass.
`seed` / `seeds`	Required `Int64` / exact Int64 `List`	One seed per independent output row. Retain prompt, cap, sampling options and model state for replay. A seed alone cannot preserve output when logits change.
`use_cache`	Optional trailing `Bool`, default true	Use a call local inference KV cache; false forces full prefix recomputation. Cache is not persisted in the model checkpoint.
`cuda_graph`	Batch only trailing `Bool`, default false	Explicit cached CUDA decode replay. Requires cached CUDA execution; setup remains part of request work. Does not add a new model or sampling policy.

generate(prompt, max_new_tokens, temperature, top_k, top_p, seed, use_cache) returns new IDs. generate_batch(prompts, max_new_tokens, temperature, top_k, top_p, seeds, use_cache, cuda_graph) keeps caller row order and stops each row on EOS or its own cap. Inspect batched generation and KV cache behavior.

Scientific series and field constructors

All arguments are required in these constructors. Scientific coordinates/values use the declared nondimensional contract; fitting does not infer physical units from feature names.

Constructor argument	Type / accepted values	Meaning
`series_model.channels`	`Int64`, `1..64`	Number of jointly generated series values per timestep.
`series_model.history`	`Int64`, `1..256`	Autoregressive history length per prediction.
`condition_size`	`Int64`, `1..256`	Width of each conditioning row, used by both scientific families.
`width`	`Int64`, `2..256`	Hidden network width; total parameters remain at most 1M.
`innovation_sigma`	Finite `Float64`, positive and at most `1e6`	Fixed series Gaussian innovation standard deviation, with representable inverse variance arithmetic. It is not fitted automatically.
`seed`	Required `Int64`	Initialization seed. Sampling/calibration/training have separate explicit seeds.
`diffusion_model.rows`, `cols`	`Int64`, each `3..64`, at most 4,096 grid sites	Spatial grid geometry. This is the model's declared field shape.
`diffusion_steps`	`Int64`, `2..1000`	Number of forward/reverse noise schedule positions. Distinct from optimizer update `steps`.
`beta_start`, `beta_end`	Finite `Float64`, `0 < beta_start ≤ beta_end < 1`	Linear variance schedule endpoints, with native representability checks.

gen.series_data(values, conditions, role) requires finite values [N,T,channels] with at least two timesteps and conditions [N,condition_size]; gen.field_data requires values [N,rows,cols] and matching condition rows. role is explicitly "training", "validation" or "test". These are immutable data with retained identities. predict_series(history, conditions, training) uses [B,history,channels]; predict_noise(noisy, conditions, timesteps, training) uses [B,rows,cols] and one exact timestep ID 0..diffusion_steps-1 per row. The public source training: Bool argument is required for both predict_series and predict_noise: it explicitly chooses a connected training graph or detached evaluation. Evaluation does not retrain anything.

sample_series(conditions, prefix, new_steps, seed) returns new generated steps excluding prefix; keep prefix/history aligned and choose new_steps in 1..4096 within checked work/storage bounds. Prefix is [B,length,channels] with length at least the model history; only the final required history enters the first prediction. sample_fields(conditions, seed) runs the declared reverse field process. Neither helper learns a scientific constraint from its name. Scientific generation gives runnable complete inputs.

Scientific configuration and calibration variables

These are declared pre training options or explicit training only calibration transitions. They do not silently activate through sample_fields.

Call	Every user variable and its effect
`with_mass_constraint(condition_index, scale, offset, dx, dy)`	Required exact condition column index; finite affine `scale`/`offset`; finite positive grid spacings `dx`, `dy`. Declares target mass `scale*condition + offset`. Configure before training; Float32 coefficient/geometry representability is checked.
`with_diffusion_schedule_features()`	No extra knobs; uses the declared three slot normalized schedule aware time representation. Configure before training.
`with_diffusion_ema(decay)`	Required finite `0 < decay < 1`, still strictly inside the interval in Float32. Retains averaged post update weights for inference; raw Adam state remains available for continuation.
`with_diffusion_velocity_prediction(loss_weighting)`	Optional string overload; omitted = `"velocity"`; allowed `"velocity"` or `"epsilon"`. Select velocity target and its corresponding weighting. Public prediction/sampling still exposes epsilon.
`with_diffusion_sobolev_loss(dx, dy, laplacian_weight)`	Required positive finite geometry and finite weight `0..1000`; shared mass geometry must agree. Adds dimensionless spatial derivative matching. Zero weight disables its contribution.
`calibrate_diffusion_variance(training, samples_per_timestep, seed)`	Bound training role `ScientificData`, positive sample count sufficient for reduced covariance rank, and explicit noise seed. Frozen denoiser reverse covariance fit; no weight update. Count and dimension/work bounds are checked per schedule.
`calibrate_diffusion(training, samples_per_timestep, seed, ridge)`	Required training data/sample count/noise seed plus finite `0 < ridge ≤ 1`. Joint `reverse-mean/covariance` calibration uses independent noise folds.
`calibrate_diffusion_gaussian(training)`	No tuning parameter; explicit affine conditional Gaussian moment closure. Requires sufficient bound training rank and small admitted field/condition dimensions. It bypasses the nonlinear noisy field NN during its declared Gaussian closure.
`calibrate_diffusion_neural(training, draws, seed, strength)`	Required training data, draws and seed; shorter overload uses `strength=1`. Draws are positive complete repeats of the bound training condition design, at most 100,000 and subject to schedule/moment budgets. `strength` is finite `[0,1]`: zero projects within clean support, one matches fitted moments; intermediate values blend the declared covariance correction.

The calibration family has stricter geometry/rank/work limits than the constructor: reduced physical dimension at most 64 (up to 65 stored physical coordinates for covariance diagnostics), condition width at most 32, and bounded dense moment work. Increasing a sample count cannot bypass a rank/geometry failure. The full scientific guide specifies each calibration's data assumptions, independent folds, boundary/mass handling and diagnostic interpretation. Diffusion calibration examples and neural calibration are separate controlled experiments.

Diagnostic arguments preserve their literal meaning: series_statistics(values, lag) uses an explicit lag; series_spectrum(values) inspects supplied values; field_mass, field_statistics, field_laplacian, poisson_residual and field_elliptic_residual require explicit finite positive dx, dy and matching forcing/coefficient shapes. diffusion_schedule(steps,beta_start,beta_end), diffusion_noise(clean,noise,schedule,timesteps) and diffusion_reverse(noisy,predicted_epsilon,innovation,schedule,timesteps,variance_table) expose the actual noise/reverse primitives; the optional variance table has its own checked dimensions. The exact inventory and full guide distinguish these diagnostic/primitives from fitting parameters.

Placement, memory and artifacts

to(device, dtype) requires explicit supported device/precision. An unchanged profile returns the existing model; changing device or dtype begins the documented fresh execution transition; inspect device_supported, device, dtype, memory_estimate and memory_summary before allocating a larger request. Language memory queries take batch, length and training; scientific queries use gen.scientific_memory(model, batch, training). They are conservative numeric buffer estimates, excluding driver/allocator overhead and unrelated retained data.

gen.import_gpt2(directory, context) reads an exact supported local GPT 2 Safetensors directory, with a bounded positive context admitted by that import profile. It is not a generic pretrained model downloader. parameter(name), parameter_names(), summary() and fingerprint() expose retained declarations and provenance.

save(path) exclusively creates a new local file; gen.load(path, device) and gen.load_scientific(path, device) read existing files in their own bounded, integrity checked generative formats; omission of load device restores CPU. Language checkpoints embed their tokenizer. Full training checkpoints retain moments, cursor/step, options, implementation/backend identity and bound data lineage, while original corpora remain separate. Exact resume requires those identities/settings; adaptation uses deliberate new run transitions. Generative formats are independent of Dense classifier v7, CNN and recurrent checkpoints.