Fine tune a pretrained local model
Prepare supervised data, train LoRA, resume checkpoints and deploy GGUF models or adapters.
// Optional ANER_USE_LLM_TRAINING source build; explicit local files only.
// Supply a supported floating dense-Llama GGUF at models/base-f16.gguf.
// CSV headers: question,answer. Select disjoint training/validation rows.
// Run from a fresh working directory; published artifacts refuse overwrite.
import aner.gen
import aner.llm
let base = gen.import_model("models/base-f16.gguf", device: "cpu", max_memory_bytes: 4294967296)
let training = gen.supervised_csv(base, "training.csv", input: "question", target: "answer", role: "training", context: 128)
let validation = gen.supervised_csv(base, "validation.csv", input: "question", target: "answer", role: "validation", context: 128)
print(base.memory_summary())
let adapted = base.lora()
let tuned = adapted.fine_tune(training, validation, steps: 2, micro_batch: 1, accumulation: 2, rate: 0.0002, seed: 71)
print(tuned.validation_loss())
tuned.checkpoint("checkpoint-2")
let restored = gen.load_pretrained_checkpoint("checkpoint-2")
let continued = restored.resume(training, validation, steps: 2)
continued.save_model("domain-f32.gguf")
gen.quantize_model("domain-f32.gguf", "domain-q4.gguf", format: "q4_0")
let manifest = continued.save_adapter("domain-adapter")
let native = llm.load("models/base-f16.gguf", context: 128, device: "cpu")
let specialized = native.with_adapter(manifest)
print(specialized.generate("Explain this observation.", max_tokens: 32).text())
specialized.close()
native.close()
Make it your experiment.
Use the matching development runtime. Save this program as examples/llm_pretrained_fine_tuning.aner inside a folder for your experiment. Open a terminal in that folder, then check and run the program.
aner check examples/llm_pretrained_fine_tuning.aner
aner run examples/llm_pretrained_fine_tuning.anerThe aner command must be on your PATH. Follow the installation guide if your terminal cannot find it.
Requires the matching optional LibTorch C++ development runtime with CPU FP32 dense Llama training and native inference. Supply models/base-f16.gguf as a supported floating GGUF plus separate training.csv and validation.csv files with question,answer columns. Run in a fresh writable directory. Published Aner 0.0.5 packages do not supply these APIs. No model download, service credentials or provider request is performed.
aner check examples/llm_pretrained_fine_tuning.aner
aner run examples/llm_pretrained_fine_tuning.anerClient construction and syntax checking send no inference request. Running this program calls its explicit model service. Inspect the returned finish reason and reported usage; unavailable token counts remain unknown and usage does not verify charges.
Read token accounting and provider requirements