Dense model parameters

This reference targets Aner 0.0.5 or newer. Updating documentation or editor help alone does not upgrade an installed runtime; check Downloads for verified platform packages.

Neural networks · Model directory

Use aner.nn to describe the network and aner.ml to configure fitting, preprocessing, validation and prediction. A NeuralNetwork is an immutable topology, a ClassifierPipeline is an immutable fitting plan, and a FittedClassifier owns detached parameters and fitted preprocessing. KNN is a separate aner.ml technique; see KNN. The model reference compares the available families, and the model API inventory lists exact public overloads.

Complete example

import aner.dataset
import aner.nn
import aner.ml

let parts = dataset.iris().partition(validation: 0.2, test: 0.2, seed: 81)
let network = nn.sequential(seed: 42)
    .dense(inputs: 4, outputs: 16, initialization: "he").relu()
    .dropout(probability: 0.2)
    .dense(inputs: 16, outputs: 3, initialization: "xavier")
let plan = ml.classifier(network, standardize: true,
    loss: "cross_entropy", optimizer: "adam")
    .on(device: "cpu", precision: "Float64", policy: "preserve")
    .dropout_seed(seed: 2026)
let config = ml.training(epochs: 20, batch: 16, rate: 0.01, seed: 91)
    .validation(parts.validation()).select_best(false)
let fitted = plan.fit(parts.train(), config: config)
print(fitted.summary())
print(fitted.evaluate(parts.test()).summary())

The final Dense stage emits logits. The classifier owns softmax for probability prediction and cross entropy; do not add a final softmax or activation to this builder. Fit standardization only on training rows. Validation/test roles must remain disjoint from all retained training and preprocessing membership. This is a runnable workflow, not an accuracy guarantee. The complete dropout example also demonstrates exact save/load continuation; run it from a fresh writable directory because saving refuses to overwrite its checkpoint.

Architecture parameters

Names in the tables are the actual named arguments. Receiver calls omit the first network/pipeline argument. A displayed default describes an implemented shorter overload, not a general language default argument facility.

Call / parameter	Type and default	Accepted values, effect and relationship
`nn.sequential(seed)`	`seed: Int64`, required	Exact signed initialization seed. Each Dense uses `seed + dense_index`; checked overflow. Activations and dropout do not increment that Dense index.
`.dense(inputs, outputs)`	Both `Int64`, required	Positive input/output widths; adjacent widths must match. Weight shape is `inputs × outputs`, bias `1 × outputs`. The first input width matches Dataset features; final output width matches its class count.
`.dense(..., initialization)`	`String`, optional overload; omitted = `"seeded"`	`"seeded"` gives uniform `[-0.5,0.5)`; `"xavier"` gives `±sqrt(6/(inputs+outputs))`; `"he"` gives `±sqrt(6/inputs)`. Bias starts at zero. ReLU hidden layers commonly use He; select policies deliberately.
`.dense(..., weights, bias)`	Finite `Tensor` values, both required together	Explicit values with the shapes above. The builder retains detached immutable copies. This overload is separate from the initialization string overload; it does not accept both policies at once.
`.relu()`, `.tanh()`, `.sigmoid()`	No extra arguments	Shape preserving hidden activation. An activation requires a preceding stage. Use the explicit builder to control each position.
`.dropout(probability)`	`Float64`, required	Finite `0 ≤ probability < 1`. Each activation is dropped independently during fitting; retained values are scaled by `1/(1-p)`. Zero is identity. Place after a hidden activation; inference and detached metrics disable it.
`nn.mlp(inputs, hidden, outputs, seed)`	Required `Int64, List, Int64, Int64`	`hidden` contains positive exact Int64 widths, for example `List(16,8)`. Creates Xavier Dense layers with ReLU after each hidden layer and raw logits at the end. An empty list produces one output Dense layer.

The Dense description admits at most 32 stages and 1,000,000 total parameter elements. Every activation and dropout stage counts toward the stage limit. Tensor and graph limits can reject an otherwise valid width combination for a chosen batch size. Increasing widths changes both capacity and compute/storage cost; the size of a constructor argument is not a promise that a complete training graph will fit.

Fitting plan parameters

Parameter / call	Default	Meaning and constraints
`ml.classifier(network, standardize, loss, optimizer)`	`network` and `standardize` required	Bind the topology and preprocessing policy. No fitting occurs here.
`standardize: Bool`	Required	`true`: fit per feature center/population scale on training rows, reuse for every later pass. Constant features use scale one. `false`: do not center or rescale; reported center/scale are zero/one.
`loss: String`	`"cross_entropy"`	`"cross_entropy"`: row mean categorical loss from logits. `"mse"`: `row/class-mean` squared error of softmax probabilities versus one hot targets. These are different objectives; report which was used.
`optimizer: String`	`"sgd"`	`"sgd"` or `"adam"`; see optimizer mathematics and state. No automatic learning rate tuning.
`.dropout_seed(seed: Int64)`	Omitted	Explicit seed reproduces a sequence of changing masks. Omission obtains one operating system seed for each fresh fit with active dropout and records it. It is separate from initialization and shuffle seeds. All zero dropout acquires no automatic seed.
`.on(device: String)`	Captured execution profile	Override placement while retaining requested precision/policy.
`.on(device, precision, policy)`	All three required in this overload	Devices `"cpu"`, `"auto"`, or an available compiled `"cuda"`/`"mps"`; precision `"Float64"`/`"Float32"`; policy `"preserve"`, `"adapt"`, `"fast"`. Unavailable explicitly requested devices fail.

The default session profile is CPU Float64 with preserve. A plan captures the current profile when created; a later global nn.configure does not silently change it. CPU/CUDA support Float64 and Float32 where built; Metal/MPS supports Float32. preserve retains requested precision and reports permitted CPU fallback; adapt allows Float32 when the selected backend lacks Float64; fast deliberately requests Float32. Inspect nn.execution(), the fitted summary, and actual tensor dtype/device. See device contracts.

Training and validation variables

There are two fitting interfaces. plan.fit(training, epochs, rate, progress: false) performs one full batch update per epoch. plan.fit(training, config, progress: false) uses ml.training and supports lazy mini batches and validation.

Parameter / call	Default	Accepted values and effect
`training: Dataset`	Required	Compatible feature/class schema, permitted training/unspecified role, and no held out overlap. Raw Tensors are not the Dataset based Dense fit input.
`epochs: Int64`	Required	`1..100000`; fixed additional epoch count. No convergence based stopping is implied.
`rate: Float64`	Required	Finite, positive learning rate, representable in effective precision. Same value has different behavior with SGD and Adam.
`progress: Bool`	`false`	Bounded measured CLI progress, at most 64 observations per fit. Does not change optimization or masks.
`ml.training(epochs, batch, rate, seed)`	All four required	Immutable mini batch configuration.
`batch: Int64`	Required	`1..1000000`; requested rows per update. Final partial batch is retained. The per fit bound is 100,000 updates; memory/work limits can require a smaller batch.
`seed: Int64` in `ml.training`	Required	Shuffle seed. Epoch `e` uses checked `seed + completed_epoch`; it does not select initialization or dropout randomness.
`.validation(data: Dataset)`	No validation by default	Attach schema compatible validation/unspecified rows, disjoint from cumulative training/preprocessing membership. Validation never supplies gradients.
`.select_best(enabled: Bool)`	`false`	`true` requires validation and returns the earliest epoch with minimum measured validation loss. It still runs the full requested horizon; it is selection, not early stopping.
`.warm_start(model: FittedClassifier)`	No warm start by default	Continue compatible detached weights and original scaler. Matching Adam restores moments/step; changing optimizer resets optimizer state deliberately.
`.fit(..., execution: ParallelContext)`	Separate explicit overload	Supported Dense replicas delegate gradient sums; one canonical CPU optimizer remains. See distributed fitting.

Full batch intermediate progress loss is measured before the latest update; the final loss is a detached post update pass. Mini batch epoch loss is the row weighted mean of changing pre update batch objectives, with training dropout active. Final training and all validation/evaluation losses use dropout disabled. Compare losses with the same phase; a noisy online dropout objective is not the evaluation mode training curve.

Prediction, model state and exact continuation

fitted.predict(samples) returns N × 1 class IDs. fitted.probabilities(samples) returns N × C softmax probabilities, with columns in fitted class order. Dataset inputs reuse schema/preprocessing; Tensor overloads require columns already aligned with the fitted feature order. evaluate(testing) returns loss/accuracy and row/provenance readouts; it does not refit preprocessing.

weights(layer) and bias(layer) use zero based Dense indices and return independent detached copies. center() and scale() report the fitted scaler. summary(), training_row_ids(), evaluation row_ids(), initial_loss(), training_loss(), training_accuracy() and training_seconds() expose retained state. Duration is this fit's measured elapsed time, not an optimizer setting or cross run performance guarantee.

New Dense saves use checkpoint version 7, retaining probability, seed/source, aner.dropout-samples/v1, cumulative training updates independent of optimizer resets, Adam moments, exact mini batch schedule/provenance and selected epoch state. Versions 1 to 6 remain readable with their documented history limits. A conflicting dropout seed or topology on warm start fails. The seed schedule includes cumulative update, stage and exact sample identity, so recording or replica sharding does not advance it. Randomness defines the recipe.

Save to a new path in an existing directory. Load restores portable CPU parameters in saved precision. Exact replay requires unchanged implementation/backend/precision and compatible data/order, batch, rate and seeds. Model checkpoints do not contain raw training data or arbitrary user classes. Dense checkpoint version 7 does not specify the distinct CNN, recurrent or generative checkpoint formats.