Tabular Foundation Models
A short and opinionated guide
Preface
On the surface, tabular foundation models like TabPFN seem like yet another machine learning algorithm, but they may transform tabular machine learning as we know it.
Tabular foundation models are pretrained on millions of mostly simulated datasets. No need to train the model for your specific task though, since prediction is based on in-context learning: you provide the training data during inference. There are no hyperparameters to tune, and no model weights to update. Increasingly, these tabular foundation models beat benchmark after benchmark.
All planned chapters are written, but it may have errors here and there. From here I only fix errors and add formats. Found an error? Write me: tfm@christophmolnar.com
Why this book: It took me a while to wrap my head around tabular foundation models, because they are so different from algorithms like gradient-boosted trees. Also, I feel like their transformative potential is not communicated enough. That’s why I wrote this short, opinionated guide to tabular foundation models, to help you:
- Understand how tabular foundation models work.
- Apply these models.
- Form your own opinion on whether to bet on them.
Who this book is for: I strongly feel everyone in tabular machine learning should learn about tabular foundation models. The way I wrote this book, however, might not be for everyone. I’m excited about the technology, and believe it may become the literal foundation of tabular machine learning.
This book is licensed under CC BY-NC 4.0. You are free to share and adapt the material with attribution, including inside companies and teams. You are not allowed to sell the book or repackage it into a paid product.