Chimère Ω — blueprint for a physico-cognitively inspired local-first LLM runtime

Abstract and publication details

LeJEPA replaces the heuristic machinery of modern self-supervised learning (asymmetric encoders, stop-gradients, momentum teachers, schedulers) with a single isotropic-Gaussian regulariser, SIGReg. Carrying SIGReg to sequence models exposes a documented hazard: a naive pooled application can drive every sequence to a constant vector along time, a time-axis collapse the objective fails to penalise. We build a heuristics-free time-series representation-learning library and compare three placements of SIGReg relative to the time axis (pooled, dual, structured). The collapse is real and placement-controlled: dual raises across-time variance by roughly an order of magnitude. Our central result is negative and clarifying: through a factorial crossing encoder architecture with placement and probe feature, the across-time variance collapse does not measure downstream order availability and is anticorrelated with it (Spearman rho = -0.64). What determines temporal-order retention is the encoder’s positional structure, an axis orthogonal to the placement that controls the collapse: a position-free encoder gives a permutation-invariant pooled feature at exactly chance (0.500) on a content-matched probe over two datasets, while a positional transformer recovers order at 0.97-1.00 even when maximally collapsed. The dual placement is nonetheless valuable where the task needs the richer representation it produces: on UCI HAR it beats pooled by 8-11 accuracy points, and an eight-seed paired-significance forecasting study finds it significantly better on ETTh2 and ETTm1, worse on PEMS08, and indistinguishable on ETTh1. We propose a two-axis account: preventing the collapse does not change order availability (set by positional structure) but increases representation richness (effective rank), which helps tasks whose targets are structured rather than near-constant.

Linked researcher profiles