Post-training dataCOLM 2026
MegaScience
Open post-training datasets for scientific reasoning, including the 1.25M-instance MegaScience mixture and TextbookReasoning.
- HF · all datasets
- 127.7K downloads
- Open corpus
- 1.25M instances
Open Source
Open datasets, code, and models from my recent work. Metrics below are mid-2026 public snapshots and are shown as all-time totals where available.
Open-source resources
Data
Open corpora for scientific and mathematical reasoning.
Pre-training dataAI4Math @ ICML 2025
MegaMath-Web-Pro-Max
A 70B+ token math-web corpus curated and refined for reinforcement-learning-friendly mid-training.
- Open corpus
- 70B+ tokens
- Documents
- 69.2M
Pre-training dataCOLM 2025
MegaMath
A 371B-token open math corpus unifying web, code, and synthetic mathematical data.
- HF · all time
- 796.5K downloads
- GitHub
- 110 stars
Pre-training dataNeurIPS D&B 2024
MathPile
A quality-first, 9.5B-token math corpus spanning textbooks, papers, forums, and the web.
- GitHub
- 418 stars
- HF · all time
- 7.9K downloads
Models & tools
Ways to refine corpora and study reasoning behavior.
Data refinementICML 2025
ProX
Tiny language models that programmatically refine corpora, with 500B+ open DCLM-Pro tokens.
- GitHub
- 270 stars
- HF · all resources
- 235.2K downloads
Models & pre-training dataAI4Math @ ICML 2025
OctoThinker
Open mid-training recipes, models, and pre-training data for reinforcement-learning-friendly reasoning.
- GitHub
- 189 stars
- HF · all resources
- 171K downloads