Decision tree classification

The 0.0.2 development series adds an immutable DecisionTreeClassifier to the existing native aner.ml module. It learns a classification tree from numerical Tensor features, predicts class IDs and leaf probabilities, and exports a readable tree as text, Graphviz DOT or a standalone SVG image. Fitting and prediction run on the CPU without an external ML framework.

import aner.ml;
import aner.tensor;

let features = [[0.0], [1.0], [2.0], [3.0]];
let labels = [[10.0], [10.0], [20.0], [20.0]];
let model: DecisionTreeClassifier = ml.fit_tree(features, labels);
let predicted = model.predict([[0.25], [2.75]]);
print(tensor.value(predicted, 0, 0)); // 10.0
print(tensor.value(predicted, 1, 0)); // 20.0
print(model.tree_text());
model.export_tree("decision-tree.svg");

Splitting criteria

Each internal node applies a binary numerical test x[feature] <= threshold. Both binary and multiclass classification are supported. The criterion option selects how candidate splits are scored:

These are criteria for Aner's numerical binary split tree. They are not full implementations of historical ID3 or C4.5: there are no categorical multiway splits, missing value distribution, post pruning, or C4.5's above average information gain filter. The API accepts the three criterion strings above; it does not expose an ID4 mode. For the likely intended C4.5 reference, use "gain_ratio" with those scope limits in mind.

For the original information gain and gain ratio discussion, see J. R. Quinlan's Induction of Decision Trees (1986). The scikit learn tree mathematics reference gives the Gini, entropy and sample weighted impurity definitions; its algorithm comparison describes distinctions between ID3, C4.5 and CART.

API and defaults

Import aner.ml to use the model type and ml.* functions. Import aner.tensor separately for tensor.* calls and aner.metrics separately for evaluation. Tensor annotations and literals are available with aner.ml alone.

Trailing optional controls can be omitted. Named controls can also skip earlier defaults:

let entropy_tree = ml.fit_tree(features, labels, "entropy", 4);
let leaf_limited = ml.fit_tree(features: features, labels: labels, min_samples_leaf: 3);
let shallow = ml.fit_tree(labels: labels, features: features, max_depth: 2);

Receiver calls use the same functions after omitting their first argument: model.predict(input: queries), model.classes(), model.tree_svg(), and model.export_tree(path: "tree.svg"). The model works in annotations, function parameters, return types, generic fields, Optional and Any. Copies share immutable fitted state. There is no public unfitted constructor in Aner source. Use accessors to inspect it; direct model printing and equality are not defined. type_of(model) returns "DecisionTreeClassifier".

All features and queries must be finite. Class IDs are nonnegative exactly integral Float64 values no larger than 9,007,199,254,740,991; IDs need not be contiguous. Query feature count must match training. max_depth is between 1 and 64, min_samples_split is at least 2, and min_samples_leaf is at least 1. A node can split only when it meets the split count, both children meet the leaf count, the depth allows it and a strictly positive gain exists. A homogeneous population or identical feature vectors may produce a root only tree.

Candidate features and thresholds are considered in ascending order. Exact split score ties keep the first candidate; majority class ties choose the smallest class ID. Probabilities are the class frequencies among the fitting rows in the reached leaf. They sum to one up to floating point rounding and are not calibrated confidence estimates. Returned tensors are detached from automatic differentiation.

Fitting and prediction have a 50 million measured work unit limit. A fitted tree has at most 4,095 nodes and one million stored nonzero class counts. Work or storage exhaustion raises an error instead of returning a silently incomplete fitted tree. Invalid fitting, prediction or export inputs report R2301; signature errors report E1002, and missing imports report E1001.

Visualizing a fitted tree

tree_text, tree_dot and tree_svg describe the fitted model without retraining. SVG is directly viewable in a browser; Graphviz is optional and is only needed if you want to render the DOT source yourself. Feature indices refer to the columns supplied to fit_tree; attach the source column names when presenting a tree.

print(model.tree_text());
ml.export_tree(model, "decision-tree.txt");
ml.export_tree(model, "decision-tree.dot");
ml.export_tree(model, "decision-tree.svg");

Exports use the current working directory for relative paths. The parent directory must exist, and existing destinations are refused. Choose a new name or explicitly remove an earlier export before running the same export again. The complete visualization is formatted before the destination is opened. Visualizations show at most 127 nodes and explicitly mark omitted branches. Text and DOT include at most eight nonzero class counts per displayed node. These display limits do not alter the model or its predictions.

A held out experiment

Run the licensed offline Iris example from an existing writable working directory:

aner check examples/ml_decision_tree.aner
aner run examples/ml_decision_tree.aner

The example splits Iris using seed 2026, fits only the 120 training rows, evaluates the 30 held out rows, prints the tree and class frequency mapping, and exports iris-decision-tree.svg, .dot and .txt. Numerical trees do not require feature standardization; thresholds remain in the supplied feature units. If you add learned preprocessing, fit it only on training data and reuse it for held out queries. Choose the criterion and stopping controls before inspecting test results, or use a separate validation partition for model selection. Iris is a small historical teaching dataset; see its provenance and splitting reference.

Random forests, regression trees, pruning, categorical feature handling, model serialization and feature importance APIs are future work. This version establishes the fitted tree and visualization interfaces on which an ensemble can later build.

See the Machine learning module reference for the shared API.