Building energy regression with preserved context

A building contributes several observations. A model intended for previously unseen buildings must keep each building inside one declared population. Fitting normalization on all rows before splitting would use held out observations even if the final model fits only training rows.

This tutorial solves a deterministic synthetic problem with a native typed frame, training only fitted preprocessing and actual CPU Float64 linear regression. Observation IDs, input field order, target meaning and fit dependencies remain attached to the supported model boundary. The fitted preprocessing retains its FeatureLedger automatically; the user does not construct a separate history container.

Native observation and fitted-artifact contracts

Dataset and preparation

Download energy.csv and standardization.csv into examples/fixtures/contextual/ below the working directory. Download the complete program into examples/. The paths are relative to the process working directory, not the source file. Run from a fresh working directory: the program creates energy.scaler.anermodel and energy.linear.anermodel and refuses to overwrite existing artifacts.

The energy fixture contains eight buildings and two observations per building. For building index b and record index h in {0,1}, temperature is 10 + 2*b + 3*h, occupancy is 1 + (b mod 3) + h, and energy is 50 + 3*temperature + 2*occupancy. Units are declared degC, dimensionless occupancy and kWh. This rule defines a teaching fixture, not a calibrated physical energy model.

B0: B3 are training buildings, B4: B5 validation buildings and B6: B7 test buildings. The explicit manifest covers every group exactly once. Identity, group and target columns never become model input features.

The three row standardization fixture isolates the statistic: training temperatures 10 and 20 give population center 15 and scale 5; held out temperature 100 transforms to 17. The target remains unstandardized. The same program saves the fitted preprocessing and regression model, then restores them using the live trusted training population; the restored workflow must reproduce the prediction contract.

Complete runnable program

aner check examples/contextual_energy.aner
aner run examples/contextual_energy.aner

// Synthetic building observations: energy = 50 + 3*temperature + 2*occupancy.
// Whole buildings stay in one declared population. CPU Float64 regression.
import aner.data
import aner.ml
import aner.tensor

fn energy_schema() -> DataSchema {
    var schema = data.schema()
    schema = data.column(schema, "row_id", "Int64", false)
    schema = data.column(schema, "building_id", "String", false)
    schema = data.column(schema, "temperature", "Float64", false)
    schema = data.column(schema, "occupancy", "Float64", false)
    schema = data.column(schema, "energy", "Float64", false)
    return schema
}

let table = data.collect(
    data.scan_csv("examples/fixtures/contextual/energy.csv", energy_schema(), "NA"),
    max_rows: 16, max_bytes: 1048576, batch_rows: 4)
let observations = data.frame(table, "row_id", "building_id", "energy",
    List("", "", "degC", "1", "kWh"))
let populations = data.partition_groups(observations,
    List("B0", "B1", "B2", "B3"), List("B4", "B5"), List("B6", "B7"))
let preprocessing = ml.fit_standardizer(populations.train())
let training = data.batch(ml.transform(preprocessing, populations.train()), 1048576, "allow")
let validation = data.batch(ml.transform(preprocessing, populations.validation()), 1048576, "allow")
let testing = data.batch(ml.transform(preprocessing, populations.test()), 1048576, "allow")
let model = ml.fit_linear(training)
let prediction = ml.predict(model, validation)

print("Training, validation and test rows")
print(populations.train().rows())
print(populations.validation().rows())
print(populations.test().rows())
print("Input features exclude identity, group and target")
print(training.inputs().cols())
print("First validation observation identity")
print(prediction.row_id(0))
print("Prediction retains declared target unit")
print(prediction.unit())
print("Validation and test reproduce the synthetic rule")
print(ml.rmse(prediction, validation) < 0.000000001)
print(ml.rmse(ml.predict(model, testing), testing) < 0.000000001)
print("First validation prediction is 108 kWh within tolerance")
print(prediction.values().value(0, 0) > 107.999999999 &&
    prediction.values().value(0, 0) < 108.000000001)

// A separate three-row fixture isolates training-only standardization.
var oracle_schema = data.schema()
oracle_schema = data.column(oracle_schema, "row_id", "Int64", false)
oracle_schema = data.column(oracle_schema, "building_id", "String", false)
oracle_schema = data.column(oracle_schema, "temperature", "Float64", false)
oracle_schema = data.column(oracle_schema, "energy", "Float64", false)
let oracle_table = data.collect(
    data.scan_csv("examples/fixtures/contextual/standardization.csv", oracle_schema, "NA"),
    3, 1048576, 2)
let oracle = data.frame(oracle_table, "row_id", "building_id", "energy",
    List("", "", "degC", "kWh"))
let oracle_parts = data.partition_groups(oracle,
    List("TrainA", "TrainB"), List(), List("TestC"))
let oracle_preprocessing = ml.fit_standardizer(oracle_parts.train())
let held_out = data.batch(ml.transform(oracle_preprocessing, oracle_parts.test()), 1024, "allow")
print("Training center, scale and held-out transformed value")
print(oracle_preprocessing.center().value(0, 0))
print(oracle_preprocessing.scale().value(0, 0))
print(held_out.inputs().value(0, 0))

// Save new artifacts once; restoration requires this live trusted training anchor.
preprocessing.save("energy.scaler.anermodel")
model.save("energy.linear.anermodel")
let restored_preprocessing = ml.load_standardizer("energy.scaler.anermodel", populations.train())
let restored_training = data.batch(ml.transform(restored_preprocessing, populations.train()), 1048576)
let restored_model = ml.load_linear("energy.linear.anermodel", restored_training)
let restored_validation = data.batch(ml.transform(restored_preprocessing, populations.validation()), 1048576)
print("Restored workflow preserves the verified prediction contract")
print(ml.rmse(ml.predict(restored_model, restored_validation), restored_validation) < 0.000000001)

Expected output

Training, validation and test rows
8
4
4
Input features exclude identity, group and target
2
First validation observation identity
401
Prediction retains declared target unit
kWh
Validation and test reproduce the synthetic rule
true
true
First validation prediction is 108 kWh within tolerance
true
Training center, scale and held-out transformed value
15.0
5.0
17.0
Restored workflow preserves the verified prediction contract
true

The Boolean comparisons assert numerical agreement within the stated tolerance. They do not report an empirical accuracy benchmark. Linear regression solves the specified full rank synthetic relationship, and prediction preserves row ID 401 and the target unit kWh.

Intentional failure: fitting held out observations

Download the negative program. It is a statically valid program that must fail at runtime; it is not a successful alternative training workflow.

// Intentional failure: fitted preprocessing may not learn held-out values.
import aner.data
import aner.ml
var schema = data.schema()
schema = data.column(schema, "row_id", "Int64", false)
schema = data.column(schema, "building_id", "String", false)
schema = data.column(schema, "temperature", "Float64", false)
schema = data.column(schema, "energy", "Float64", false)
let table = data.collect(data.scan_csv("examples/fixtures/contextual/standardization.csv", schema, "NA"), 3, 1048576, 2)
let frame = data.frame(table, "row_id", "building_id", "energy", List("", "", "degC", "kWh"))
let populations = data.partition_groups(frame, List("TrainA", "TrainB"), List(), List("TestC"))
let forbidden = ml.fit_standardizer(populations.test())

Expected diagnostic: a located error[R3501] containing E_FIT_SCOPE. Fit preprocessing on the authorized training population, then reuse the fitted artifact only for transformation of held out observations.

Intentional failure: refusing an unavoidable copy

Download the no copy negative program.

// Intentional failure: separate table columns cannot form this packed view without a copy.
import aner.data
var schema = data.schema()
schema = data.column(schema, "row_id", "Int64", false)
schema = data.column(schema, "building_id", "String", false)
schema = data.column(schema, "temperature", "Float64", false)
schema = data.column(schema, "energy", "Float64", false)
let table = data.collect(data.scan_csv("examples/fixtures/contextual/standardization.csv", schema, "NA"), 3, 1048576, 2)
let frame = data.frame(table, "row_id", "building_id", "energy", List("", "", "degC", "kWh"))
let forbidden = data.batch(frame, 1024, "forbid")

Expected diagnostic: a located error[R3501] containing E_CONTEXT_LOSS. The frame owns separate table columns; creating a packed row major input is an allocation. The explicit copy policy prevents an invisible conversion.

Intentional failure: missing fitted preparation at prediction

Download the negative preprocessing program. The three row synthetic numeric fixture is explicitly declared here as a dimensionless signal and the column renamed accordingly; a column header alone does not verify physical meaning. Both raw and standardized signal units are 1, so unit equality cannot detect the missing preparation.

// Intentional failure: valid scope and dimensionless units do not prove preprocessing ran.
// This synthetic numeric fixture is explicitly declared as a dimensionless signal.
import aner.data
import aner.ml
var schema = data.schema()
schema = data.column(schema, "row_id", "Int64", false)
schema = data.column(schema, "building_id", "String", false)
schema = data.column(schema, "temperature", "Float64", false)
schema = data.column(schema, "energy", "Float64", false)
let table = data.collect(data.scan_csv("examples/fixtures/contextual/standardization.csv", schema, "NA"), 3, 1048576, 2)
let signal = data.frame(table, "row_id", "building_id", "energy", List("", "", "1", "kWh"))
    .rename("temperature", "signal")
let populations = data.partition_groups(signal, List("TrainA", "TrainB"), List(), List("TestC"))
let preparation = ml.fit_standardizer(populations.train())
let training = data.batch(ml.transform(preparation, populations.train()), 1024)
let model = ml.fit_linear(training)
// The test input has not received the preparation used by training.
let raw_test = data.batch(populations.test(), 1024)
let forbidden = ml.predict(model, raw_test)

Expected diagnostic: a located error[R3501] containing E_FEATURE_SCHEMA. The ordered applied preprocessing chain differs from the chain used during training. Transform the held out frame with the fitted preparation before packing it for prediction. Valid held out scope and matching field identity do not substitute for applying that preparation.

How the structure helps

The model receives an ordered contextual batch rather than disconnected arrays. Selection and batch permutation retain row/target alignment. Fitted influences are checked through registered conversions, so a later split does not erase statistics learned on forbidden rows. No inferred scientific truth or universal leakage prevention is promised: declared groups and fields must represent the intended experiment.

For finer control, inspect the AdaptiveFrame field and conversion contracts and FeatureLedger fit, schema and persistence contracts. Those are the same native objects used by this program.

Dependencies, backends and limitations

Use the compatible native development engine with aner.data, aner.ml, aner.tensor and VS Code helper 0.1.38 or newer. Both CSV files are synthetic and local; running makes no network request. CPU Float64 is the implemented reference estimator backend. Missing required features or targets and deficient regression rank fail; no implicit imputation, mask, regularization or precision downgrade is chosen.

This program is checked and executed in automated tests against its exact source and expected output; the same test asserts the intentional diagnostics. Time/space bounds are documented separately for the frame and fitted artifact. Contextual GPU/distributed regression, sparse adaptation, temporal availability and graph modeling remain future functionality.