Native neural research workflow

These 0.0.7-dev.1 development workflows extend Aner Neural. CNN, RNN/LSTM/GRU and GNN share native Tensors, autodiff, explicit placement and immutable trained snapshots. The family supplies the structure: spatial neighborhoods, ordered recurrent memory, or sparse graph messages. Data preparation and experiment protocol remain visible in the program.

Choose structure before execution

Inspect .summary() and .diagram() before fitting. These are production inspection methods; development recording additionally captures bounded metadata with dev.capture. Architecture diagrams describe declared structure and tensor flow, not learned causal explanations or proof of a model's predictive quality.

.export_diagram("architecture.svg") or .export_diagram("architecture.html") creates a standalone architecture figure without running the model. Export preserves declared branch/message edges, escapes labels and refuses an existing path. HTML needs no remote scripts; SVG can be shared or embedded directly. Large graphs have explicit omission counts, rather than invented connections. This is available in production execution as well as development inspection.

One update, one returned model

All three native families offer .fit(...) for an in memory dataset and .train_step(...) for one independent global minibatch. Keep the returned value:

var model = nn.lstm(inputs: 2, hidden: 16).on("cpu")
model = model.train_step(batch_inputs, batch_targets, rate: 0.001)

train_step keeps compatible Adam moments and lifetime update counts across changing minibatches. Use it with caller owned loaders, explicit learning rate schedules and manual validation/checkpoint selection. It does not shuffle, silently retain recurrent state or provide a streaming data loader. Recurrent .fit_chunk(...) is separate: it carries detached state for ordered truncated backpropagation. GNN minibatches keep their entire supplied message topology.

Repeated ordinary fit is exact dataset continuation: preserve data, order, supervision/lengths and shuffle seed. For a new fitting phase, call .reset_optimizer(). Weights and lifetime counters remain; moments, optimizer step and dataset/epoch continuation restart. Recurrent retained state also clears. CNN changing .freeze_backbone(...) starts this fresh phase automatically while retaining the learned head and backbone. Resetting moments is an explicit experiment change, not exact checkpoint continuation.

Clipping uses the global parameter gradient norm before the update. Weight decay is decoupled from the gradient/Adam step and applies to weight matrices/kernels, excluding biases and normalization parameters. CPU replicas reduce gradient sums and divide once by the appropriate global observation count. Unequal shards must not receive equal weight merely because they are separate workers.

Evaluate held out data deliberately

let validation = fitted.evaluate(valid_inputs, valid_targets, batch: 32)
print(validation.loss())
print(validation.accuracy())
print(validation.samples())
print(validation.observations())

CNN/recurrent evaluation uses the shown input/target form. GNN uses .evaluate(graph, features, targets, indices: held_out_ids). All return NeuralEvaluation. Accuracy is -1 when the objective has no classification accuracy. Samples count independent rows or selected graph positions; observations count the actual loss denominator. A padded sequence objective may have several valid observations per row; regression also counts output features.

Evaluation uses inference behavior: dropout disabled, saved CNN normalization statistics, no optimizer changes, and no retained recurrent state carried into independent evaluation rows. Padded recurrent positions and ignored token targets do not enter the denominator. Batch losses are combined using their real observation counts. Training last_loss() describes weighted online minibatch loss, not a fresh post training held out score.

Arbitrary Tensors do not carry protected Dataset split roles. Keep preprocessing statistics, label mappings, sequence identities, graph protocol and selected checkpoint metadata with the experiment. Do not tune against the final test set.

Sequence preparation without crossing a split

nn.sequence_windows(series, window, horizon: 1, stride: 1) accepts detached [time,features] data and returns [samples,window,features]. nn.sequence_targets(...) selects [samples,features] at start+window+horizon-1, aligned to exactly those starts. Both preserve the series' device and dtype. They never pad or join separate series.

Split a time series before calling either helper, and fit scaling on training data only. The helper cannot identify a semantic boundary hidden inside a raw tensor. The recurrent workflow example uses separate chronological training and validation segments. RNN/LSTM state must match cell/layer geometry, batch ordering, device and precision; invalid retained state is rejected when attached. Bidirectional retained causal streaming is rejected; reverse direction models use future context within the supplied window.

Execution and clustering

Use .on(device, precision) and inspect .device(), .dtype() and .performance_summary(). CPU supports Float32/Float64; Metal supports Float32; CUDA support depends on the compiled native provider and selected family. Explicit unavailable requests fail. CNN/recurrent CUDA mixed compute remains opt in; GNN currently uses strict native storage precision.

Local CPU replicas use execution: parallel.local(workers: 2). Optional MPI/NCCL uses externally launched ranks with one selected visible GPU per rank. All ranks must agree on the model, parameters, data and run configuration. Empty ranks contribute no graph work; all ranks participate in coordinated errors and collectives. Save checkpoints only through the supported rank ownership path. CNN replicated batches require frozen BatchNorm statistics. See collective configuration for launcher commands and build requirements.

This implementation does not add indexed GPU scheduling inside one process, general distributed Tensors, graph partitioning, model sharding or CNN/RNN/GNN support to the Dense only SSH transport. Physical multi GPU execution and scaling remain a separate hardware qualification task. Test reports must distinguish actual device work, process/MPI protocol checks and unavailable device skips.

Checkpoints and diagrams

Prepare or fit once for repeated inference. Keep independent snapshots for comparison or manual best checkpoint selection; copying a native model shares immutable resources until an operation returns new state. Family checkpoints retain learned parameters and compatible optimizer continuation; save to a fresh path and keep preprocessing/data identity separately. Load restores portable CPU storage, then select placement explicitly.

Native Tensor model families currently execute through the interpreter and its C++ numerical runtime. The ahead of time compiler does not lower Tensor/model programs; native numerical kernels are independent of that compiler limitation. No new package or website is published by this development work.