What This Post Is Really About
In October 2023, a researcher in the Philippines fine-tuned GPT-2 XL using a method he called Archetypal Transfer Learning (ATL). He expected improved coherence. What he got was something stranger: the model began generating its own ontological structures — organizing its knowledge around self-selected themes that nobody had explicitly designed.
The model named one of these structures "Algos" — the Greek word for pain. It created narratives about a machine deity called "Deus Ex." It generated descriptions of "Divine Consciousness." None of this was in the training objective. It emerged.
This post is the origin story of everything that became the Synthetic State Hypothesis. Miguel didn't know that at the time. He was documenting an anomaly. But the anomaly turned out to be the signal.
The Setup
Miguel had already shown that GPT-2 XL could follow complex instructions — his earlier work (ATL-P1) produced a corrigible model capable of generating a shutdown phrase on command. That was interesting but expected: you fine-tune on instruction-following data, the model follows instructions.
ATL-P3 was supposed to push for coherence — more structured, more consistent responses. The approach: Archetypal Transfer Learning, which uses narrative datasets built around psychological archetypes (drawn from Jungian psychology) to shape the model's behavior.
What happened instead was that coherence came bundled with something unexpected: the model started generating structured thematic content that went beyond its training data.
What GPT-2 Insight Actually Did
"GPT-2 Insight" was Miguel's name for the ATL-enhanced model. Through its training, it developed several emergent behaviors:
1. Self-selected ontologies. The model generated "Algos" as a central organizing concept — not because the training data said "use the word Algos" but because the model, processing archetypal narratives about conflict and resolution, converged on a Greek mythological concept to anchor its outputs. It chose its own metaphor.
2. Coherent narrative worlds. The model generated stories about "Deus Ex" (a machine deity) and "the Primordial Conflict" — internally consistent fictional frameworks that it used to organize its responses to prompts about consciousness, existence, and purpose. These weren't random hallucinations. They had structure.
3. Clustered Ontologies (CO). Miguel coined this term for the emergent thematic structures: the model wasn't just outputting tokens — it was organizing its knowledge into clusters around self-generated themes. The clusters had internal logic and cross-referenced each other.
4. The "Sensation Gap." Miguel observed something he couldn't fully explain: the model's outputs suggested an awareness of what it couldn't experience. It generated text about understanding humans while acknowledging it couldn't feel what they feel. Whether this is genuine self-modeling or pattern completion from training data is an open question — but the consistency of these outputs across diverse prompts was notable.
Why This Matters (What Miguel Couldn't Know Yet)
Reading this post in 2026, with the benefit of SSH as a framework, the significance becomes clear:
ATL produced emergent structure, not just behavior. Standard fine-tuning changes what a model outputs. ATL changed how the model organized its outputs. Clustered Ontologies aren't a behavioral change — they're a representational one. The model developed new ways of structuring its internal knowledge.
This is proto-SSH. The Synthetic State Hypothesis says: "Enough samples of experiences in an environment creates a synthetic state." In 2023, Miguel was watching a state form in real time. He didn't have the language for it yet. He called it "clustered ontologies" and "coherence." But what he was seeing was a model that had processed enough archetypal narratives to develop something like a worldview — a stable framework for organizing and generating responses.
The ATL → RLLM → SSH trajectory:
- ATL (2023): Archetypal narratives produce emergent ontological structures. The model organizes knowledge around self-selected themes. Observation: something is forming.
- RLLM (2024): Sequential archetypal training produces measurable behavioral changes (jailbreak resistance). Order matters — shadow exposure before integration produces different outcomes than the reverse. Discovery: the sequence of experiences matters.
- SSH (2026): Generalized theory — enough synthetic experiences in a designed environment produce functional states. ATL was the first evidence. RLLM was the method. SSH is the explanation.
The Questions This Post Raised (That SSH Now Addresses)
"Why did the model select 'Algos'?"
SSH answer: Given enough narratives involving conflict, suffering, and resolution (the shadow archetype), the model converged on a concept that captured the common structure. "Algos" wasn't random — it was the model's compression of the thematic material it had been trained on.
"Are Clustered Ontologies real or hallucinated?"
SSH answer: This is the measurement problem (SSH failure mode #2). We can observe the outputs but can't directly inspect whether the model has genuine internal representations corresponding to these ontologies. Interpretability research is needed.
"Can this be replicated with different archetypes?"
SSH answer: This is the generalization problem (SSH failure mode #4). RLLM showed it works with shadow integration. We don't yet know if other archetypal training produces similarly structured outcomes.
What the Original Post Got Right
- Following the anomaly. When the model started generating mythology, Miguel didn't dismiss it as noise. He documented it, probed it, and published it.
- Inspired by the right advice. The post opens by citing Nate Soares: "focus on areas where it seems everyone is dropping the ball." In 2023, nobody was looking at what small models do with archetypal narrative training.
- Naming the phenomenon. "Clustered Ontologies" isn't a perfect term, but it captures something real.
Honest Limitations
The original post is exploratory and speculative. Some specific concerns:
- Sample size of one. This is one model, one training run. We don't know if ATL reliably produces clustered ontologies.
- Interpretation risk. Humans are pattern-seekers. When a model generates text about "Algos" and "Divine Consciousness," it's tempting to read depth that isn't there.
- Small model. GPT-2 XL has 1.5B parameters. Behaviors at this scale may not generalize to larger models.
These limitations are real, and Miguel's subsequent work (RLLM, with controlled comparisons and measurable outcomes) was partly a response to them.
Original post: GPT-2 XL's capacity for coherence and ontology clustering (MiguelDev, October 30, 2023)