External observations with fitted context

Aner 0.0.5 runtime can apply a fitted contextual linear model to independent observations with no target column. Explicit alignment preserves the new observation identities, checks feature meaning and units, and applies the existing fitted preprocessing. It never adds external rows to a training population.

Run the complete synthetic example from the project root:

aner check examples/contextual_inference.aner
aner run examples/contextual_inference.aner

The example fits y = 2*x + 3 on two declared training observations, with input x measured in meters and target y in joules. It predicts the held out values 7 and 11, then predicts equivalent independent unlabeled values supplied in centimeters. The new predictions retain IDs 9001 and 9002 and target unit J; the numerical checks print true within an absolute tolerance of 1e-9. This verifies contract preservation for a small full rank least squares fixture; it is not an optimizer convergence or speed claim.

Declare observations and align explicitly

data.frame(table, row_id, group, target) now accepts an empty target String for unlabeled observations. Supply an explicit identity column to retain external IDs; an empty row_id still generates local zero based IDs. Identity values must be nonmissing unique Int64 values. The target free frame does not allocate or fabricate target values.

let observations = data.frame(external_table, "observation_id", "", "")
let aligned = scaler.align_inputs(observations, List("distance_cm"))
let inputs = scaler.transform(aligned).batch(1048576)
let prediction = model.predict(inputs)

This excerpt belongs to the checked complete example. external_table, scaler and model are established there.

The columns List supplies external source field names in the exact order expected by the fitted reference. Table column order is irrelevant once the mapping is explicit. Every source feature must be distinct, numeric and free of target derived status. Missing names, duplicate mappings, String/Bool/DateTime features, and identity/group/target fields reject. Additional fields remain in the unchanged source frame and are excluded from the aligned input.

Each mapped Int64 feature must fit the existing exact Float64 conversion range of ±2^53. Observation identities retain the entire Int64 range because IDs remain separate from numerical features. Missing required inputs reject; no automatic imputation or row removal occurs.

Units and preprocessing

Alignment converts a declared compatible registered source unit into the reference's fitted input unit. This includes compatible affine temperature points, such as kelvin to Celsius. Physical dimensions and absolute/interval quantity kinds must agree. An undeclared unit cannot acquire a physical guarantee by alignment. Native physical table units are retained automatically; primitive table unit declarations remain user declarations, not evidence that a sensor was calibrated.

For a model trained after preprocessing, align against the first fitted FeatureLedger, then call transform with the exact original fitted stages once each and in order. Alignment cannot claim that preprocessing already occurred. Aligning directly to a preprocessed model rejects. A model fitted on raw inputs can be the reference directly:

let aligned = raw_model.align_inputs(observations, List("distance_cm"))
let prediction = raw_model.predict(aligned.batch(1048576))

The adapter accepts raw unassigned frames. Frames with fitted, erased, already aligned or registered derived context reject. It retains the source observation root independently from the fitted model's reference root and experiment. A second experiment/fold requires its own explicit alignment, even if names, values or source root match.

Fitting permission and target meaning

An aligned frame remains unspecified and is marked inference only. Partitioning it, strict fitting, and exploratory fitting reject. Selection, filtering, transformation, gathering, numerical transfer and batch packing retain this restriction. A matching column name or row ID cannot grant fitting permission.

A batch made from a target free frame contains only inputs and exact observation selection. targets(), ml.fit_linear and rmse reject an unlabeled batch. Prediction uses the fitted model's retained target meaning and unit. The resulting SemanticTensor retains external observation identity, while its fitted dependencies continue to identify the original training observations. The model and ledger statistics, parameters and training dependency signatures remain unchanged.

aligned.explain() and prediction.explain() report the explicit ordered mapping and unit conversion, exclusion of extra fields and absence of fit authorization. Normal bounded summaries expose counts without printing the mapping or observation IDs automatically. Save explicit explanation output with the research artifacts when the mapping is part of the result's evidence.

Native signatures

The List contains Strings. Receiver syntax and qualified calls share the same native registry and located diagnostics. Invalid mappings use R3501 with E_FEATURE_SCHEMA, incompatible fitted scopes use E_FIT_SCOPE or E_ROW_DOMAIN, and missing values use E_MISSING_POLICY. Fitting and predicting retain the contextual estimator's effective CPU Float64 requirement.

Ownership, limits and restoration

Alignment retains the immutable source table and copies mapped numerical values into owned overrides, including converted magnitudes. No raw pointer or reusable batch buffer is borrowed. Existing selections safely retain their storage. Alignment takes O(np + pc) time and O(np + Np + p) additional storage, where n is selected observations, N is source rows, p is fitted features, and c is visible source fields. The source is still limited by the contextual adapter's 1,000,000 rows, 64 fields, 1,000,000 numeric cells and checked 64 MiB managed payload. Batch packing has its separate explicit output budget; these limits do not constitute a process wide RAM admission system.

Fitted artifacts retain their existing versioned integrity and trusted anchor checks. Restore the ledger and model with live training anchors, then align external observations against those restored objects. Alignment requires no persisted format change. It does not serialize an external observation frame or remove the requirement for a trusted training anchor.

The native suite covers raw and preprocessed models, multiple ordered preprocessing stages, reordered external features, compatible unit conversion, absolute/interval refusal, missing and ambiguous mappings, target absence, exact external IDs, shuffling, unchanged fitted artifact bytes and anchored restoration. Language and editor suites check the same public calls and complete example.

This increment covers the native CPU Float64 contextual linear estimator and fitted standardizers. Other estimator families, generic transformation replay, streaming inference and contextual distributed fitting retain their existing separate boundaries. See FeatureLedger, AdaptiveFrame and the building energy workflow.

Complete executable source

The canonical program below is checked and executed against this exact output by the contextual example suite.

// Development runtime: explicit independent unlabeled inference.
// Synthetic rule y = 2*x + 3; x is measured in m, target y in J.
import aner.data
import aner.ml
import aner.tensor
import aner.units

var schema = data.schema().column("row_id", "Int64", false)
schema = schema.column("group", "String", false)
schema = schema.unit_column("x", "m", false)
schema = schema.unit_column("y", "J", false)
let observed = data.scan_csv("examples/fixtures/contextual/inference_training.csv", schema, "NA")
    .collect(0, 1048576, 2)
let populations = data.frame(observed, "row_id", "group", "y")
    .partition_groups(List("A", "B"), List(), List("C", "D"))
let scaler = ml.fit_standardizer(populations.train())
let training = scaler.transform(populations.train()).batch(1048576)
let model = ml.fit_linear(training)
let held_out = model.predict(scaler.transform(populations.test()).batch(1048576))

// These new observations have independent IDs and no target column.
// Physical columns carry their explicit source units; metadata stays outside
// the mapped model inputs. Mapping lists source names in fitted feature order.
var external_schema = data.schema().column("note", "String", false)
external_schema = external_schema.unit_column("distance_cm", "cm", false)
external_schema = external_schema.column("observation_id", "Int64", false)
let external_table = data.scan_csv("examples/fixtures/contextual/inference_observations.csv", external_schema, "NA")
    .collect(0, 1048576, 2)
let observations = data.frame(external_table, "observation_id", "", "")
let aligned = scaler.align_inputs(observations, List("distance_cm"))
let inputs = scaler.transform(aligned).batch(1048576)
let prediction = model.predict(inputs)

print(prediction.row_id(0))
print(prediction.row_id(1))
print(prediction.unit())
print(prediction.values().value(0, 0) > 6.999999999 &&
    prediction.values().value(0, 0) < 7.000000001)
print(prediction.values().value(1, 0) > 10.999999999 &&
    prediction.values().value(1, 0) < 11.000000001)
print((prediction.values().value(0, 0) - held_out.values().value(0, 0)) < 0.000000001 &&
    (prediction.values().value(0, 0) - held_out.values().value(0, 0)) > -0.000000001)
print(aligned.role())
print(prediction.explain().contains("fit_authorized=false"))
// inputs.targets(), ml.fit_linear(inputs) and fitting on aligned are errors.
// A raw-input model can be the alignment reference directly. A preprocessed
// model requires alignment against the first ledger and its exact fitted chain.

9001
9002
J
true
true
true
unspecified
true

See the Machine learning module reference for the shared API.