Process spillable table snapshots
Configure private temporary storage, retain exact values and reuse immutable snapshots in bounded batches.
// Run from the project root with the updated native development runtime.
// The included fixture stays small. Replace its path with your larger CSV;
// collection then switches storage automatically when its RAM budget fills.
import aner.data
import aner.units
let proposed = data.infer_csv("examples/fixtures/data_modern.csv", "NA", 8388608, 0)
print(data.schema_description(proposed))
let source = data.scan_csv("examples/fixtures/data_modern.csv", proposed, "NA")
let normalized = data.convert_unit(source, "Temperature_DegC", "K")
// OS temporary storage, at most 1 GiB of private encoded backing storage.
// A directory path may select another existing writable volume.
let managed = data.with_spill(normalized, directory: "", max_bytes: 1073741824)
let table = data.collect(managed, max_rows: 0, max_bytes: 131072, batch_rows: 64)
print(data.storage(table))
print(data.spilled(table))
print(data.spill_bytes(table))
print(data.peak_bytes(table))
// Getters preserve types, missingness, exact datetime values and current units.
print(data.text(table, 0, "ObservationID"))
print(data.datetime_ns(table, 0, "RecordedAt"))
print(data.unit_value(table, 0, "Temperature_DegC", "degC").value())
// This plan retains the immutable table, including any disk backing.
// It does not reopen the CSV and supports bounded summaries and CSV export.
let snapshot = data.scan_table(table)
let clean = data.drop_missing(snapshot)
let report = data.summarize(clean, max_bytes: 131072, batch_rows: 64)
print(data.rows(report))
print(data.unit_mean(report, "Temperature_DegC", "degC").value())
print(data.provenance(table))
print(data.provenance(report))
// Fresh path required for a durable CSV export:
// let saved = data.write_csv(clean, "spill-clean-sensors.csv", 131072, 64)
// print(data.provenance(saved))
// Explicit opt-out keeps strict RAM admission when that is required:
// let resident = data.collect(data.without_spill(source), 0, 8388608, 64)
Make it your experiment.
Use Aner 0.0.5 or newer. Save this program as examples/data_spill.aner inside a folder for your experiment. Open a terminal in that folder, then check and run the program.
aner check examples/data_spill.aner
aner run examples/data_spill.anerThe aner command must be on your PATH. Follow the installation guide if your terminal cannot find it.
Requires the native Aner 0.0.5 or newer and VS Code 0.1.48 candidate integration. These APIs require Aner 0.0.5 or newer. Save supplied fixtures in the shown working directory paths. No external service or credentials are required for the included analysis. Save each supplied fixture beneath the exact examples/fixtures path shown in the source and run from the project root.
Bundled datasets retain their own source attribution and license notices. Example outcomes are not comparative benchmarks or evidence of clinical validity.
Dataset sources & attribution