f11_slp1_uni8k_d700m_plus_all_v2_4x
d700m (704.1M parameters) on plus_all v2 slice (489.9M words, 4 passes), trained on L40S 48GB (AWS). Started 8 Oct 2026, ended 11 Oct 2026.
Hypothesis
4 passes instead of 3 improves ex-Gita (~0.9% predicted by the passes proxy).
What we learned
Winner: ex-Gita 0.5262 vs F10's 0.5307 (-0.85%, as the passes proxy predicted), clean_v1 -0.61%, Vedic flat, verse recall 2.2% vs 0.7% (mostly Gita). Released as sansar-700m v0.2.0.
Scores
Bits per byte, lower is better; change against the parent run, F10.
| Test set | F11 | F10 (parent) | Change |
|---|---|---|---|
| DCS gold (classical) | 0.5454 | 0.5513 | −1.1% |
| Bhagavad-gītā (memorisation) | 0.0683 | 0.0758 | −9.9% |
| Out of domain | 0.524 | 0.5286 | −0.9% |
| Prose | 0.5144 | 0.5186 | −0.8% |
| Vedic (Ṛgveda) | 0.6679 | 0.6671 | +0.1% |
Curves
Training loss
Cross-entropy per token, by step. The first few percent of the run, far higher, run off the top; hover or the table has every value.
Held-out bits per byte
The validation split, evaluated during training, by step. Lower is better.
Throughput
Tokens per second, by step.
Model
- Preset
- d700m
- Parameters
- 704,128,512
- Outside embeddings
- 679,552,512
- Layers · heads · width
- 24 · 12 · 1,536
- Tokenizer
- SLP1 unigram 8k
Data
- Slice
- plus_all v2 slice
- Words
- 489,850,000
- Training tokens
- 1,586,326,034
- Passes
- 4 passes
- Tokens seen
- 6,345,326,592
Compute
- GPU
- L40S 48GB
- Where
- AWS
- Steps
- 129,096 / 129,096
- GPU hours
- 71.48
- Cost
- $133.97
- Spot restarts
- 0
Lineage
awsl40sd700m