Development and production

Aner runs in production mode by default. Development mode records bounded observations of variables, objects, collections, neural operations, training loss, and available gradients. Both modes use the same type checks, numerical kernels, seeds, shape checks, and resource limits. Development observes the computation; it does not choose a different training algorithm.

The -dev text in aner --version identifies the development release of the interpreter. It does not mean that a program is recording observations.

Run one source file

aner examples/development_nn.aner --dev reports/nn-first
aner examples/development_nn.aner --prod

Omitting --prod also selects production. run remains optional. Use a new report directory for every development run. Aner refuses to replace an existing directory, file, or symlink. --dev and --prod cannot be combined. Existing --debug-viz remains available for the older neural only reporting workflow.

Development creates these files:

Open inspection.html directly in a browser. No server or network connection is needed. Add --debug-values when inspecting small teaching tensors or neuron values:

aner examples/development_nn.aner --dev reports/nn-values --debug-values

Shapes and statistics are recorded by default. Raw tensor arrays and gradients are opt in and bounded. Scalar values, String previews, and ordinary object/collection contents already have bounded previews. Reports are observations from that execution; editing the source does not update them.

A source located runtime failure produces a partial development report containing the observations captured before the failure and an error status. Syntax/type errors found before CLI execution have no runtime observations. A notebook runtime failure records the historical state before clearing the native session; a static failure preserves prior session variables. Neither report replays or rewinds the program.

Explicit observation points

import aner.dev

let values = List(10, "hello", 1.5)
dev.capture("Initial values", values)
values.remove_at(1)
dev.capture("After removing the String", values)

Development also observes assignments, object field updates, and collection mutations automatically. An explicit capture gives an important moment a useful name. References have stable heap IDs within the session, so shared values and cycles can be recognized without recursively following a cycle forever. Captures do not invoke user methods or getters and do not retain live objects or differentiation graphs.

In production, automatic inspection and report generation are disabled. dev.capture still evaluates its arguments normally. Guard an expensive diagnostic expression if it should disappear from production work:

if dev.enabled() {
    dev.capture("First Dense weights", fitted.weights(layer: 0))
}

This also avoids making the detached parameter copy when recording is off. The same principle applies to diagnostic dataset transformations or extra model evaluations.

Neural network inspection

import aner.dataset
import aner.nn
import aner.ml
import aner.dev

let split = dataset.iris().split(test: 0.2, seed: 2026)
let network = nn.sequential(seed: 42)
    .dense(inputs: 4, outputs: 8,
           weights: nn.random(4, 8, seed: 7, scale: 0.25),
           bias: nn.zeros(1, 8))
    .tanh()
    .dense(inputs: 8, outputs: 3)
let pipeline = ml.classifier(network, standardize: true,
                             loss: "cross_entropy", optimizer: "sgd")
print(network.summary())
print(pipeline.summary())
let fitted = pipeline.fit(split.train(), epochs: 400, rate: 0.1)
print(fitted.evaluate(split.test()).summary())

The architecture builder returns immutable configurations. network.summary() describes layers, shapes, parameter counts, seeds, and custom initialization before training. pipeline.summary() adds the normalization policy, loss, and optimizer. Standardization is fitted only on the supplied training rows; evaluation reuses those statistics.

weights must have shape inputs × outputs and bias must have shape 1 × outputs, with finite Tensor values in the provided precision. Both optional custom tensors are supplied together; they are detached into independent initialization storage. Omitting them retains the existing seeded scale 0.5 and zero bias initialization. An ordinary Tensor literal can replace nn.random when manually selecting weights.

Loss accepts "cross_entropy" or "mse"; optimizer accepts "sgd" or "adam". Both arguments are optional, and optimizer: "sgd" can be given without an explicit loss. Unsupported optimizers produce a source located error. The trainer uses full batch SGD or Adam: plans resolve their preserve/adapt/fast profile before training. CPU and CUDA support Float64/Float32; Metal/MPS supports native Float32. Preserve on Metal/MPS with Float64 uses reported CPU Float64 fallback. Features, targets, parameters, gradients, and optimizer moments use the effective precision/device consistently. Other optimizer settings and layer kinds require separate implementations and tests. See Tensor devices and performance.

The automatic fit captures include the initial forward/backward state, selected intermediate epochs, and the final trained evaluation. Each fit has a separate series identity and local training step. Captures label whether they describe a state after backward before an update or the final state. Separate fits are not joined into one continuous loss curve. Manual viz.capture observations form their own series.

Use the complete source example, the development notebook, and the neural reference.

Save, load, and continue training

fitted.save("iris.anermodel")
let restored = ml.load_classifier("iris.anermodel")
let resumed = pipeline.warm_start(restored)
    .fit(split.train(), epochs: 100, rate: 0.1)

Save to a new file in an existing directory. Saving does not overwrite a file or create missing parent directories. Paths are relative to the program/session working directory.

A checkpoint retains architecture, detached fitted parameters, loss/optimizer configuration, fitted center/scale, feature/class schema, training metrics, source metadata, and exact source row IDs. When continuing on different compatible rows, original scaler source identity and rows remain distinct from the new training rows. Raw data features/targets and differentiation graphs are not saved. The file is a versioned binary format, not Aner code.

Warm starting validates architecture, loss, standardization policy, and input schema. It uses the prior weights and scaler; it does not refit normalization on the new rows. The previous fitted value remains unchanged. The epoch/rate fields of the new result describe the new fit call, rather than inventing a cumulative training history. SGD has no persistent moments. Version 2 checkpoints retain Adam moments and its cumulative update count; compatible Adam warm starts resume them. Choosing a different optimizer keeps the weights and scaler but resets optimizer state explicitly. Version 1 SGD checkpoints remain readable. Version 3 Float32 checkpoints remain readable. Version 4 checkpoints for nondefault execution profiles retain requested/effective precision and device policy plus source device provenance, loading parameters and Adam moments on CPU in their saved actual precision without requiring the source accelerator. Default CPU Float64 preserve saves remain version 2. Select an explicit plan before warm starting; precision conversion is reported and does not recover discarded bits. Independent inference after loading uses the saved normalization exactly.

Checkpoint parsing checks version, bounds, shapes, finite values, positive scales, schema consistency, and trailing bytes. The serialized payload is limited to 64 MiB. Failures produce R2701. See the runnable checkpoint example; run it in a fresh working directory because it creates iris.anermodel.

Complexity of model state operations

Let s be the number of stages, p the parameter elements copied or serialized, r retained training/scaler source row IDs, and m metadata bytes. Space below is additional to an existing model.

Checkpoint parameters include custom initialization tensors when the configuration retains them, alongside the fitted tensors. Fitting still allocates independent trainable parameters and its ordinary training graph. Warm start does not retain the earlier training graph. These bounds do not describe disk latency or total process RSS.

Inspect a linked list

aner examples/development_linkedlist.aner --dev reports/linked-list

The example appends mixed values, prepends a Float64 node, and removes the String node. Named captures show the reachable Node objects after each operation. The reference diagram distinguishes the list's head, each node's next, shared references, and node payloads. Its final length is the same in production and development.

The linked list notebook separates those operations into cells. Automatic mutation observations also work with built in List, Dictionary, Set, Array<T>, DynamicArray<T>, and NDArray<T>. Typed array previews show dtype, length, capacity, sorted status for one dimensional arrays, and up to 16 live elements. NDArray also shows shape, element strides, and shared storage identifiers; see multidimensional inspection. The data structure index links each type to its separate method and time/space complexity reference.

Development inside VS Code notebooks

Extension 0.1.27 adds Dev, Prod, and Inspector in Aner notebook controls. This runtime setting is separate from the shared variables/independent program execution setting.

1. Choose Dev and rerun setup cells. Changing the runtime mode clears/cancels the current native session and invalidates observations; it does not run cells automatically.
2. Open Inspector to see typed globals, object references, named observations, and captured loss after a cell completes.
3. Use Aner: Open Development Training Report for the corresponding recorded topology and gradients.
4. When ready, choose Prod, save the notebook, and rerun setup and training.

Development mode is saved as aner.runtime_mode: "development" in .anernb metadata. Missing metadata defaults to production. Set aner.notebook.developmentValues when bounded raw neuron/tensor values are needed; it defaults to false.

The Inspector identifies the completed execution it represents. Source edits mark observations stale; reset, stop, timeout, runtime failure, or changing modes invalidates live state. A later evaluation cell can preserve the last training view as explicitly historical, with its originating execution number. The variable table represents the current completed cell. The panel does not evaluate user expressions to inspect a value. There is no pause/step debugger or variable editing through the panel.

Jupyter notebooks

The optional Aner Jupyter integration executes .ipynb notebooks through the native session protocol. Select the Aner kernel in a configured notebook application and open the development example.

Use these directives in separate control cells:

%aner dev
%aner dev values
%aner variables
%aner reset
%aner prod

Mode changes and reset clear native session state. Development observations include variables, object references, per fit loss, topology summaries, and measured progress after each completed cell. The current transport buffers output until the cell returns. Pause/step debugging and live notebook streaming are not implemented.

Recording budgets and performance

Generic recording stores at most 64 changes, 128 global previews, 128 visited values per inspection, 6 nesting levels, 16 items per container, and 256 bytes per String/summary preview. Capture storage is 128 KiB, globals 64 KiB, and embedded neural trace 256 KiB. Larger embedded neural traces are marked omitted, with compact loss records retained. Native protocol observation payloads are capped at 512 KiB. Omitted/truncated data is labeled; absent values or gradients are not synthesized.

Neural captures have their existing 64 snapshot and tensor/graph export limits. One run can contain several fits, but they share that finite recording budget. A later fit may have fewer captured states after the budget is used. Detailed values have additional per tensor limits; see visual training reports.

Inspection adds serialization, bounded object traversal, tensor statistics, and report output work. Tensor statistics take O(t) time for t elements even when the exported preview is small. Production avoids this work, while retaining normal safety checks. Guard source level diagnostic computations with dev.enabled() when appropriate. Exact speed differences depend on the workload and interpreter build; switching modes does not promise a fixed speedup.

Native compilation and the next steps

The runtime mode and the C++ interpreter's Debug/Release build configuration are separate. A Release build can optimize the interpreter while either runtime mode still works. aner build does not yet compile Aner programs. The numerical kernels already execute in native C++; compiling scalar/control flow code requires a separate compiler backend.

To build and run the optimized native interpreter from this checkout:

cmake --preset release
cmake --build --preset release
./build/release/aner examples/development_nn.aner --prod

This selects the Release executable explicitly. It does not change an existing aner command on PATH.

A future compiler should reuse checked types and resolved calls, preserve production diagnostics and numerical contracts, and be tested against the interpreter on the same programs. After this observation foundation, the next academic tool priorities are streaming metric observations while a cell runs, source linked pause/step debugging, broader optimizer/layer implementations, and validated compilation. They are future work, not behaviors supplied by the Dev switch.

Measured neural progress and placement

fit(..., progress: true) reports bounded measured CLI progress for fixed epochs and approximate remaining time after five updates. Development reports include measured progress alongside captured loss; notebooks show these observations after the cell completes. tensor.dtype(), tensor.device(), and Tensor previews identify actual precision, placement, and native backend. GPU value/gradient observations may synchronize and read back device results. nn.execution() describes the requested and effective execution profile. Compatibility adaptation, intentional Float32 selection, and CPU Float64 fallback are separate from Dev/Prod; warnings remain visible in production. These additions do not change Dev into a different neural training algorithm. The neural workflow explains timing phases, Adam, Xavier/He initialization, custom classes, CPU and optional MPS/CUDA execution, and the distributed execution boundary.

For mini batch research, dev.automatic(false) disables broad automatic variable capture while retaining explicit captures and progress; the setting persists across notebook cells until reset. dev.progress(epoch, epochs, loss, accuracy, seconds) records a finite row weighted online loss and held out accuracy fraction, with an optional sixth validation loss argument. Development views chart these measured values separately. These custom Tensor loops must preserve and audit their original Dataset views because extracted Tensors carry no Dataset role. See neural workflows and the MNIST protocol.