Recurrent model parameters
Configure RNN, LSTM, GRU and advanced profiles, sequence layout, hidden state, padding, training and forecasting.
Released in Aner 0.0.5Aner 0.0.5 or newerThis reference targets Aner 0.0.5 or newer. Updating documentation or editor help alone does not upgrade an installed runtime; check Downloads for verified platform packages.
This is the model/configuration reference for RecurrentModel in aner.nn. Neural networks links the other families; the recurrent workflow introduces sequence execution. The model API inventory lists exact overloads, and the detailed recurrent guide explains advanced recurrences and validation evidence.
Standard constructors
nn.rnn(inputs, hidden, outputs=1, layers=1, bidirectional=false, seed=42)
nn.lstm(inputs, hidden, outputs=1, layers=1, bidirectional=false, seed=42)
nn.gru(inputs, hidden, outputs=1, layers=1, bidirectional=false, seed=42)| Parameter | Type / accepted range | What changes |
|---|---|---|
inputs | Required Int64, 1..65536 | Feature width F of each sequence timestep. |
hidden | Required Int64, 1..4096 | Hidden memory width per layer and direction. Larger width increases recurrent parameter/work cost. |
outputs | Int64, default 1, 1..65536 | Task projection width. Classification requires at least two classes. |
layers | Int64, default 1, 1..32 | Number of stacked recurrent layers; not the timestep count. |
bidirectional | Bool, default false | Run forward/reverse directions and combine their outputs. Uses future positions within the given sequence, so it is unsuitable for causal streaming continuation. |
seed | Int64, default 42 | Model initialization and the model specific training/dropout recipe. Retain separately from the fit shuffle seed. |
Constructors return an immutable declaration. rnn uses the implemented tanh recurrence, lstm has input/forget/cell/output gates, and gru has its declared reset/update/candidate recurrence. There are no public constructor knobs for custom gate activations or arbitrary cell callbacks. Initialization uses named per layer parameter recipes; LSTM forget bias is initialized to one. Inspect parameter_names(), parameter(name), summary() and diagram() before interpreting a result.
Task, layout and regularization variables
| Call / parameter | Default | Contract |
|---|---|---|
.task(task: String) | Standard constructors use "regression" | Select before initialization. Ordinary choices are "regression", "classification", "sequence_regression", "sequence_classification". Advanced constructors select matching "gaussian", "spatial" or "tokens" tasks. |
.dropout(probability: Float64) | Standard cells default zero | Finite [0,1). Sequence locked activation dropout between stacked layers; one layer standard cells have no interlayer transition to drop. AWD LSTM has its specialized defaults below. |
.on(device, precision) | "auto", "Float32" when invoked | Strict family placement; available CPU Float32/Float64, MPS Float32, or native CUDA supported precision. Unsupported precision/device is an error. |
.performance(amp, matmul, collectives) | "off", "strict", "auto" | AMP off/fp16/bf16, matmul strict/tf32, collectives auto/mpi/nccl. CUDA Float32 is required for mixed compute; AMP and TF32 are mutually exclusive. |
.budget(tensor_elements, graph_elements) | 64M / 256M elements | Explicit positive tensor limit up to 256M, and graph limit between tensor limit and 2B. These are numeric admission limits, not a memory availability promise. |
For ordinary vector cells, input is [B,T,F] with T ≤ 4096. Final regression targets/output are [B,O]; final classification targets are exact IDs [B]/[B,1] or exact one hot rows [B,O], with logits [B,O]. Sequence regression targets/output are [B,T,O]; sequence classification targets are IDs [B,T]/[B,T,1] or exact one hot [B,T,O], with logits [B,T,O]. predict converts classification logits to ID tensors; regression prediction remains numerical output. Targets are detached, finite and aligned with caller provided sequence identities.
lengths: List contains one exact Int64 valid length per batch row in [1,T]; omission/empty list means all timesteps valid. Padding is on the right. Final task outputs select the final valid timestep; sequence losses exclude padded positions. Lengths are not attention masks or automatic missing value inference.
Every fit argument
model.fit(inputs, targets, epochs=1, batch=32, rate=0.001, seed=42,
optimizer="adam", clip_norm=1.0, weight_decay=0.0,
shuffle=true, lengths=List(), execution=parallel.local())The display summarizes registered overloads. execution is an explicit final overload, not an eagerly constructed default argument. Positional calls use its complete prefix; named calls can omit earlier optional settings, which the checker supplies, including an empty lengths list when needed.
| Argument | Type / constraints | Meaning |
|---|---|---|
inputs, targets | Required Tensor values | Inputs/targets with task specific shapes above; batch axes match. The model explicitly prepares its declared placement. Raw Tensor fitting does not enforce Dataset train/validation roles. |
epochs | Int64, default 1, 1..100000 | Additional full passes. |
batch | Int64, default 32, 1..100000 | Independent sequences per update; retain the partial final batch. |
rate | Float64, default 0.001 | Finite positive learning rate. |
seed | Int64, default 42 | Retained epoch shuffle seed. Continuation of ordinary fitting requires identical data, order, lengths and seed. |
optimizer | String, default "adam" | "adam" or "sgd". Matching Adam retains moments; changing it resets optimizer state. |
clip_norm | Float64, default 1 | Finite nonnegative maximum global gradient norm; zero disables fitting's clipping. |
weight_decay | Float64, default zero | Finite nonnegative decoupled decay, with rate * weight_decay < 1. Applied to weight tensors/embeddings, excluding biases and normalization gains. |
shuffle | Bool, default true | Shuffle independent sequences before each epoch. Does not shuffle timestep order within a sequence. |
lengths | List, default empty | Exact valid prefix lengths described above. Data and valid lengths remain paired through shuffling. |
execution | Explicit ParallelContext | Supported CPU or selected collective replicas; gradient numerators are divided once by valid objective observations. Stateful chunks require one replica. |
Fitting admits at most 1,000,000 cumulative attempted updates and at most 64M parameter elements/4,096 parameter entries. Actual graphs/workspaces may impose lower limits. With FP16 overflow, inspect attempts(), updates(), skipped_updates() and loss_scale(); skipped attempts retain weights/moments and do not advance successful Adam updates. Ordinary fit does not carry memory from one independent training row into another.
Streaming variables and truncation
nn.initial_state(model, batch) creates detached carry for a positive batch size. Standard cells use one [B,H] hidden tensor per layer/direction; LSTM also has cell tensors. ConvLSTM must infer its grid through step(inputs).state(), rather than guessing spatial carry with initial_state.
model.step(inputs, state, lengths) returns a RecurrentOutput; values() returns output and state() returns detached final carry. Omitting state supplies the model's retained carry where applicable. .with_state(state) returns a new model with detached compatible carry; .reset_state() clears it. Carry must match batch ordering, layer/direction count, geometry, dtype and device. It must not be reused for a different series accidentally.
model.fit_chunk(inputs, targets, state, rate=0.001, optimizer="adam", clip_norm=1, lengths=List()) performs one update on one whole ordered batch, with truncated backpropagation at the explicit detached state boundary. The returned model owns final state for continuation/checkpointing. It is unidirectional, unshuffled and single replica; ordinary epoch training and chunk streaming are different data contracts. There is no source stateful Boolean on ordinary .fit.
Advanced constructor variables
| Constructor | Required arguments | Shorter overload defaults and declared profile |
|---|---|---|
nn.deepar(inputs, hidden, outputs, layers, seed) | inputs, hidden | outputs=1, layers=1, seed=42. Gaussian LSTM; inputs ≥ 2*outputs includes lagged values and observed indicators, plus explicit covariates if present. |
nn.convlstm(channels, hidden, outputs, kernel, seed) | channels, hidden | outputs=1, kernel=3, seed=42. One spatial layer; odd kernel 1..31. Inputs [B,T,C,H,W], outputs/targets [B,T,O,H,W]. |
nn.awd_lstm(vocabulary, embedding, hidden, layers, seed) | First three Int64 values | layers=3, seed=42. Vocabulary 2..65536, embedding 1..8192; hidden obeys recurrent bounds. Tied embedding/output weights; inputs/targets [B,T,1] exact IDs, target -1 ignored. |
nn.xlstm(inputs, hidden, outputs, variant, layers, seed) | inputs, hidden | outputs=1, variant="slstm", layers=1, seed=42. Variant "slstm" or "mlstm"; explicit matrix memory mlstm hidden width at most 512. Causal one head declared profiles, not every published architecture variant. |
AWD LSTM defaults to activation dropout 0.4; .dropout(p) changes that probability. Its embedding row dropout 0.1 and recurrent DropConnect 0.5 are fixed profile mechanisms, not additional source arguments. Standard/advanced models keep their own checkpointed random schedules. The Dense classifier .dropout_seed overload does not apply to RecurrentModel.
DeepAR helpers nn.deepar_inputs(targets, observed, covariates) require matching [B,T,O] targets/binary observed masks; covariates [B,T,C] are optional. The helper packs previous values/observed indicators so current targets cannot enter their own input. deepar_targets(targets, observed) packs supervision. gaussian_nll(distribution, targets, observed) uses [B,T,2*O] locations/scales and only observed valid values; a finite negative density NLL is legal.
forecast(context, last_target, horizon, samples=100, seed=42) is the no covariate DeepAR overload. horizon is positive; last_target is [B,O]. The covariate overload replaces horizon with future_covariates: Tensor shaped [B,horizon,C]. samples is 1..10000; output is [B,samples,horizon,O]. Forecast warms explicit context and retains separate sampled carry per path; it neither teacher forces unknown future values nor mutates the fitted model. Keep scaling and forecast validation explicit.
Save, inspect and reproduce
parameter_names(), parameter(name), summary() and diagram() expose stored schema/architecture. last_loss() is a training observation, not an automatic validation score. Preserve independent validation/test sequences and apply appropriate task metrics without refitting preprocessing.
save(path) exclusively creates the recurrent family checkpoint; nn.load_recurrent(path) restores portable CPU values. It retains configuration, parameters, optimizer moments, attempt/success/shuffle state, numerical profile/scaler and detached carry. This format is separate from Dense classifier v7 and CNN checkpoints. Exact replay needs matching data/order/lengths, seed, precision, implementation and execution route; switching CUDA reference/fused routes or devices may change rounding.
Run basic recurrent fitting, streaming state and Gaussian forecasting. Advanced descriptions and their unsupported features are in the detailed recurrent guide.