Skip to main content

Blog

Research, engineering, and insights from Kroonen AI.

Featured Run Finished 8 min read

Genesis 1B: Run 2 Extended — 20k → 40k Steps

Genesis 1B Run 2 reached 20,000 steps then extended to 40,005 steps to hit Chinchilla-optimal token count. Currently at step ~22,250, ETA ~April 7.

Genesis Run 2 pretraining training live

🧬 Genesis 1B

5 min read

Genesis 1B, Run 2: Deeper, Faster, Smarter

Same 1B parameters, 3x throughput. How torch.compile, real-valued RoPE, a deeper architecture (32 layers vs 20), and batch tuning tripled training speed on the same 2x RTX 4090 hardware.

Genesis Run 2 architecture torch.compile
10 min read

The Genesis Manifesto: Sovereign Intelligence for the Post-Generative Era

Data sovereignty, constitutional alignment, and the case for training language models on consumer hardware. Why the future of AI is local, private, and personality-first.

Genesis philosophy alignment

🔧 Postmortems

8 min read

The Optimizer State Bug: A Silent Failure in DCP Resume

A silent AdamW state bug during Run 1 that produced a false recovery on poisoned weights. The load path didn't crash or hang -- it just silently ruined the model.

Genesis Run 1 postmortem optimizer dcp
8 min read

Fixing FSDP Checkpoint Deadlocks on 2x RTX 4090

How we fixed FSDP checkpoint deadlocks during Run 1 on consumer GPUs without NVLink using DCP sharded checkpoints and decoupled evaluation.

Genesis Run 1 postmortem fsdp dcp

🔬 Research

5 min read

Mapping the Mind of Qwen 3.5 9B: A Sparse Autoencoder for Mechanistic Interpretability

A sparse autoencoder trained on the internal activations of Qwen 3.5 9B. Zero dead features, 16,384 interpretable dimensions, trained on a single RTX 4090.

interpretability sparse-autoencoder research