Topic
Capabilities and generalisation
2 posts.
Fooled by Vast Knowledge
Models look like they generalise because the training data is vast enough to hide the difference between interpolation and extrapolation. I think that confusion is leading safety research astray.
Is the "Valley of Confused Abstractions" real?
Chris Olah's curve says models get harder to read before they get easier. Neel told me that came out of vision models, so I am posting my confusion.