What If an AI Never Stopped Learning?

An LLM whose weights never stop changing is not really a model release anymore; it is a stateful service with an optimizer attached, and pretending otherwise just hides the hard parts. Sequential learning is perfectly legitimate, but under a changing data distribution there is no magic convergence fairy: you are tracking a moving target, and every knob that makes the system more plastic also gives it another way to forget old capabilities, regress its alignment, absorb poisoned data, or simply wander somewhere you did not test. Continual-learning methods such as replay and regularization reduce that damage but do not repeal the stability–plasticity trade-off, and continual pretraining of aligned LLMs has already shown that safety behavior can degrade. Operationally, the ugly part is worse: once weights mutate continuously, “which model produced this output?” stops having a useful answer unless every update has lineage, provenance, optimizer state, evaluation results and a recoverable checkpoint.

NIST guidance consequently emphasizes model lineage, provenance verification, regression testing and continuous pipeline monitoring, while production deployment systems rely on explicit versions and rollback boundaries. The sane design is boring on purpose: quarantine incoming data, train a candidate out of band, preserve replay and reference sets, test retention and safety, promote immutable checkpoints through a canary, monitor them, and roll back automatically when they misbehave. Periodic batch updates are therefore the practical default; true streaming updates make sense only when freshness is valuable enough to pay for the extra compute, attack surface, observability, legal bookkeeping and operational pain. Keep learning continuously if the application genuinely needs it, but do not let production weights mutate continuously without a transaction boundary.