Observable Spectral Dynamics in a Recurrent Sequence Model

Abstract

Project SLSSM-01 studies recurrent language models with inspectable spectral dynamics, using state evolution rather than quadratic self-attention as the primary sequence mechanism. The central idea is that a sequence model can be studied through its learned dynamical structure as well as through loss curves and generated text.

We report evidence from the v3 short-context run: stable training across 250,000 steps, roughly 32.8 billion processed tokens, improving loss, persistent signed phase populations, and close agreement between measured trajectory behavior and learned spectral structure. We also report a limitation: the short-context run did not build a large long-memory mode population, motivating the next long-context experiment.

Model Class

Project SLSSM-01 is best described publicly as a stable linear spectral state-space language model. In practical terms, that means the model is built around a stable linear recurrent dynamics core whose internal behavior can be studied as learned modes.

That does not mean the entire system is linear end to end. Language enters and leaves the dynamics through learned nonlinear components. The important distinction is that the core state evolution is stable and mode-structured, which is what makes the Spectral Atlas, Extended Dynamic Mode Decomposition (EDMD) Concordance, and Prompt Manifold analysis possible.

Public Spec Sheet

The following specifications are intentionally high-level. They are enough to orient a technical reader without disclosing implementation details.

What Is Being Claimed

The claim is deliberately narrow:

• A stable linear spectral state-space language model was trained and analyzed.
• Its internal dynamics were measurable enough to track across checkpoints.
• Signed phase behavior remained present rather than collapsing into a single direction.
• Trajectory-based analysis and learned spectral structure showed stable agreement after early training.
• The analysis exposed a concrete next problem: long-memory capacity.

This is not a claim that Project SLSSM-01 replaces transformers, solves interpretability, or discloses a reproducible open architecture.

Diagnostic Vocabulary

Public Evidence Summary

The numbers are rounded public summaries. They are included to show that the work is anchored in an actual training run, not only in diagrams or theory.

Evidence Guide

This is author-reported evidence from one analyzed training run. Values are rounded, the public record does not include uncertainty intervals, and no external replication is claimed. The CSV below mirrors the displayed tables so readers can inspect or plot the released summary without transcribing it.

Download rounded checkpoint CSV

Blank retention or rotation fields in the CSV mean that those summaries are not included in the rounded public record for that checkpoint; they should not be interpreted as zero.

The measurement method itself can be checked without trusting this model: a reproducible EDMD demonstration on public Pythia-14m training checkpoints, with a pre-specified protocol, dated post-run amendments, pinned model commits, and shuffled-pairing and persistence controls, runs on an ordinary CPU in seconds.

Run the public demo on GitHub

What would weaken this interpretation?

• A checkpoint audit that fails to reproduce the released rounded loss, validation, or population summaries.
• EDMD proximity that disappears under held-out trajectories, repeat runs, or appropriate control comparisons.
• Signed phase balance that collapses on replication or proves unrelated to measurable behavior.
• A targeted long-context run that increases neither long-memory mode population nor held-out long-context performance.

Stronger support would require repeat seeds, controlled ablations, external evaluation, and a reproducible public demonstration of the analysis method. Those tests are validation targets, not completed claims.

Observed Analysis Record

The v3 short-context run was examined from several independent angles. The purpose was not only to ask whether loss improved, but whether the model's internal dynamics could be measured and used to make a concrete next training decision.

EDMD Concordance Details

EDMD-style analysis was used as an external measurement of trajectory behavior. The important observation is that the measured trajectory structure stayed close to the learned spectral structure after the early checkpoint.

The public record does not include the analysis transform or model internals. It reports the observed relationship: measured dynamics and learned spectral structure were close enough to compare over training.

Mode Population Observations

The mode population did not behave like an unstructured black box. The final checkpoint showed a strong middle-retention population, active rotation, and a smaller slow-mode population. That finding matters because it explains why the next experiment should focus on long-context memory rather than simply continuing to celebrate short-context loss.

Prompt Manifold Observations

Prompt-manifold analysis tested whether different prompts left different spectral signatures. The public version reports only aggregate descriptors, not prompt activations, mode IDs, or raw token traces.

The useful conclusion is not that one prompt "wins." The useful conclusion is that prompts can be compared as paths through spectral behavior, giving a practical way to inspect generation beyond reading text output alone.

Interpretation

The v3 short-context run provides evidence that the model can train stably while preserving signed spectral structure. The EDMD agreement is important because it gives an independent check against the model's learned internal organization. The prompt-manifold work is important because it turns model behavior into a path through mode neighborhoods rather than a single opaque output.

The most useful result is not that any one metric is final. The useful result is that the model produced measurable internal structure that can guide the next experiment.

Limitation Found

The short-context run did not produce a large population of very long-memory modes. That is not hidden in this note, because it is one of the strongest reasons the analysis matters. The diagnostics did not merely celebrate a result; they identified what the next run needs to target.

The next research direction is a long-context version that preserves the useful short-context behavior while deliberately training stronger long-memory dynamics.

Skeptic FAQ

How To Cite

Suggested citation for public discussion:

Hunt, F., & Hunt, J. (2026). Observable Spectral Dynamics in a Recurrent Sequence Model. Project SLSSM-01. research-note.html

BibTeX:

@misc{hunt2026observable,
  title = {Observable Spectral Dynamics in a Recurrent Sequence Model},
  author = {Hunt, Freddie and Hunt, Joshua},
  year = {2026},
  url = {research-note.html},
  note = {Public research note, version 0.1}
}